Purpose
To identify predictors of a true scaphoid fracture among patients with radial wrist
pain following acute trauma, train 5 machine learning (ML) algorithms in predicting
scaphoid fracture probability, and design a decision rule to initiate advanced imaging
in high-risk patients.
Methods
Two prospective cohorts including 422 patients with radial wrist pain following wrist
trauma were combined. There were 117 scaphoid fractures (28%) confirmed on computed
tomography, magnetic resonance imaging, or radiographs. Eighteen fractures (15%) were
occult. Predictors of a scaphoid fracture were identified among demographics, mechanism
of injury and examination maneuvers. Five ML-algorithms were trained in calculating
scaphoid fracture probability. ML-algorithms were assessed on ability to discriminate
between patients with and without a fracture (area under the receiver operating characteristic
curve), agreement between observed and predicted probabilities (calibration), and
overall performance (Brier score). The best performing ML-algorithm was incorporated
into a probability calculator. A decision rule was proposed to initiate advanced imaging
among patients with negative radiographs.
Results
Pain over the scaphoid on ulnar deviation, sex, age, and mechanism of injury were
most strongly associated with a true scaphoid fracture. The best performing ML-algorithm
yielded an area under the receiver operating characteristic curve, calibration slope,
intercept, and Brier score of 0.77, 0.84, −0.01 and 0.159, respectively. The ML-derived
decision rule proposes to initiate advanced imaging in patients with radial-sided
wrist pain, negative radiographs, and a fracture probability of ≥10%. When applied
to our cohort, this would yield 100% sensitivity, 38% specificity, and would have
reduced the number of patients undergoing advanced imaging by 36% without missing
a fracture.
Conclusions
The ML-algorithm accurately calculated scaphoid fracture probability based on scaphoid
pain on ulnar deviation, sex, age, and mechanism of injury. The ML-decision rule may
reduce the number of patients undergoing advanced imaging by a third with a small
risk of missing a fracture. External validation is required before implementation.
Type of study/level of evidence
Diagnostic II.
Key words
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Hand SurgeryAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Comparison of CT and MRI for diagnosis of suspected scaphoid fractures.J Bone Joint Surg Am. 2011; 93: 20-28
- Clinical scaphoid score (CSS) to identify scaphoid fracture with MRI in patients with normal x-ray after a wrist trauma.Emerg Med J. 2014; 31: 659-664
- Improved detection of scaphoid fractures with high-resolution peripheral quantitative CT Compared with conventional CT.J Bone Joint Surg Am. 2020; 102: 2138-2145
- Cost-effectiveness of immediate magnetic resonance imaging in the management of patients with suspected scaphoid fracture: results from a randomized clinical trial.Value Health. 2020; 23: 1444-1452
- Controversies and best practices for acute scaphoid fracture management.J Hand Surg Eur Vol. 2018; 43: 4-12
- A comparative analysis of the accuracy, diagnostic uncertainty and cost of imaging modalities in suspected scaphoid fractures.Injury. 2008; 39: 768-774
- Diagnosis of occult scaphoid fractures: a cost-effectiveness analysis.J Bone Joint Surg Am. 2015; 97: 1860-1868
- Computed tomography versus magnetic resonance imaging versus bone scintigraphy for clinically suspected scaphoid fractures in patients with negative plain radiographs.Cochrane Database Syst Rev. 2015; : CD010023
- Imaging for suspected scaphoid fracture.J Hand Surg Am. 2008; 33: 954-957
- Management of clinical fractures of the scaphoid: results of an audit and literature review.Eur J Emerg Med. 2005; 12: 47-51
- The suspected fracture of the scaphoid: a rational approach to diagnosis.Injury. 1988; 19: 149-152
- Predictors of fracture following suspected injury to the scaphoid.J Bone Joint Surg Br. 2012; 94: 961-968
- Detecting scaphoid fractures in wrist injury: a clinical decision rule.Arch Orthop Trauma Surg. 2020; 140: 575-581
- Surgical risk is not linear: derivation and validation of a novel, user-friendly, and machine-learning-based Predictive OpTimal Trees in Emergency Surgery Risk (POTTER) calculator.Ann Surg. 2018; 268: 574-583
- Deep learning-based preoperative predictive analytics for patient-reported outcomes following lumbar diskectomy: feasibility of center-specific modeling.Spine J. 2018;
- A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models.J Clin Epidemiol. 2019; 110: 12-22
- Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks.Clin Radiol. 2018; 73: 439-445
- Is Deep learning on par with human observers for detection of radiographically visible and occult fractures of the scaphoid?.Clin Orthop Relat Res. 2020; 478: 2653-2659
- What are the applications and limitations of artificial intelligence for fracture detection and classification in orthopaedic trauma imaging? A systematic review.Clin Orthop Relat Res. 2019; 477: 2482-2491
- Machine learning-based prediction of clinical outcomes for children during emergency department triage.JAMA Netw Open. 2019; 2e186937
- Machine Learning Consortium. Artificial intelligence in orthopaedics: false hope or not? A narrative review along the line of Gartner's hype cycle.EFORT Open Rev. 2020; 5: 593-603
- Development of machine learning algorithms for prediction of discharge disposition after elective inpatient surgery for lumbar degenerative disc disorders.Neurosurg Focus. 2018; 45: E6
- Development of machine learning algorithms for prediction of 30-day mortality after surgery for spinal metastasis.Neurosurgery. 2019; 85: E83-E91
- Machine-learning-based electronic triage more accurately differentiates patients with respect to clinical outcomes compared with the emergency severity index.Ann Emerg Med. 2018; 71: 565-574 e562
- Can machine-learning techniques be used for 5-year survival prediction of patients with chondrosarcoma?.Clin Orthop Relat Res. 2018; 476: 2040-2048
- A machine learning algorithm to identify patients with tibial shaft fractures at risk for infection after operative treatment.J Bone Joint Surg Am. 2021; 103: 532-540
- The role of artificial intelligence and machine learning in predicting orthopaedic outcomes.Bone Joint J. 2019; 101-B: 1476-1478
- Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view.J Med Internet Res. 2016; 18: e323
- Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration.Ann Intern Med. 2015; 162: W1-W73
- MissForest--non-parametric missing value imputation for mixed-type data.Bioinformatics. 2012; 28: 112-118
- Feature selection with the Boruta Package.J Stat Softw. 2010; 36
- A machine learning algorithm to predict the probability of (occult) posterior malleolar fractures associated with tibial shaft fractures to guide “malleolus first” fixation.J Orthop Trauma. 2020; 34: 131-138
- Validation in prediction research: the waste by data splitting.J Clin Epidemiol. 2018; 103: 131-133
- Dermatologist-level classification of skin cancer with deep neural networks.Nature. 2017; 542: 115-118
- Data mining methods in the prediction of dementia: a real-data comparison of the accuracy, sensitivity and specificity of linear discriminant analysis, logistic regression, neural networks, support vector machines, classification trees and random forests.BMC Res Notes. 2011; 4: 299
- Assessing the performance of prediction models: a framework for traditional and novel measures.Epidemiology. 2010; 21: 128-138
- Towards better clinical prediction models: seven steps for development and an ABCD for validation.Eur Heart J. 2014; 35: 1925-1931
- Clinical diagnostic evaluation for scaphoid fractures: a systematic review and meta-analysis.J Hand Surg Am. 2014; 39: 1683-1691 e1682
- Reliability of the Southampton examination schedule for the diagnosis of upper limb disorders in the general population.Ann Rheum Dis. 2002; 61: 1103-1106
- Scaphoid Fractures - Artificial Intelligence Prediction Tool - Artificial Intelligence.https://traumaplatform-ai-prediction-tools.shinyapps.io/Scaphoid-fractures/Date accessed: January 12, 2022
Article info
Publication history
Published online: June 03, 2022
Accepted:
February 23,
2022
Received:
April 25,
2021
Footnotes
No benefits in any form have been received or will be received related directly or indirectly to the subject of this article.
Identification
Copyright
© 2022 by the American Society for Surgery of the Hand. All rights reserved.
ScienceDirect
Access this article on ScienceDirectLinked Article
- ErratumJournal of Hand SurgeryVol. 47Issue 12
- PreviewIn the article by Bulstra and Machine Learning Consortium in the August 2022 issue of The Journal of Hand Surgery (“A Machine Learning Algorithm to Estimate the Probability of a True Scaphoid Fracture After Wrist Trauma”, Vol. 47, No. 8, p. 709-718), one of the Machine Learning Consortium collaborators was listed incorrectly. “Carel (J.C.) Goslings, MD, PhD” should be “J. Carel Goslings, MD, PhD.” The online version of the article has been updated. The authors regret this error.
- Full-Text
- Preview
