Advertisement

Reliability of Radiographs and Computed Tomography in Diagnosing Scaphoid Union After Internal Fixation

Published:April 20, 2021DOI:https://doi.org/10.1016/j.jhsa.2021.03.004

      Purpose

      To evaluate the reliability of radiographs (XR) alone versus the combination of XR and computed tomography (CT) in determining scaphoid union following open reduction internal fixation (ORIF) with a headless compression screw.

      Methods

      We used our imaging database to identify 32 XR and corresponding CTs over a 6-year period (from 2012 to 2018) that were performed to evaluate scaphoid healing following ORIF. Three hand surgeons evaluated the studies to assess (1) healing, (2) if partially healed, the percentage of healing, and (3) the certainty of healing. Initially, XR were reviewed alone. Three weeks later, the same XR were reviewed with the corresponding CTs. Each reviewer performed a similar 2-stage evaluation 4 weeks later. We measured interobserver and intraobserver reliabilities using linearly weighted kappa (κ) coefficients for healing status and the percentage of healing.

      Results

      The interobserver reliability for healing (healed vs partially healed vs not healed) was moderate both with XR alone and with the combination of XR and CT. The intraobserver reliability for healing was substantial with XR alone compared to moderate with the combination of XR and CT. For the percentage of healing, both interobserver and intraobserver reliabilities were fair with XR alone or with the combination of XR and CT. Reviewers reported significantly greater certainty with the combination of XR and CT compared with XR alone.

      Conclusions

      Following ORIF, surgeons are more certain in their evaluation of scaphoid healing with the combination of CT and XR. However, the reliability of assessing scaphoid union may not be improved by the addition of CT to XR.

      Type of study/level of evidence

      Diagnostic III.

      Key words

      JHS Podcast

      July 1, 2021

      JHS Podcast Episode 64

      Dr. Graham interviews Dr. Jonas Matzon regarding his article “Reliability of Radiographs and Computed Tomography in Diagnosing Scaphoid Union After Internal Fixation”, which appears in the July 2021 issue of the Journal of Hand Surgery.

      Loading ...
      Determining union following a scaphoid fracture is challenging.
      • Dias J.J.
      Definition of union after acute fracture and surgery for fracture nonunion of the scaphoid.
      Due to the scaphoid’s size, shape, and orientation, the classic radiographic signs of union, which involve trabeculae crossing the fracture on multiple radiographic views, are unreliable.
      • Dias J.J.
      • Taylor M.
      • Thompson J.
      • Brenkel I.J.
      • Gregg P.J.
      Radiographic signs of union of scaphoid fractures. An analysis of inter-observer agreement and reproducibility.
      In order to aid in the assessment of scaphoid union, computed tomography (CT) has been increasingly utilized. For nonsurgically treated scaphoid fractures, CT has been found to be reliable in determining union.
      • Buijze G.A.
      • Wijffels M.M.
      • Guitton T.G.
      • et al.
      Interobserver reliability of computed tomography to diagnose scaphoid waist fracture union.
      ,
      • Hannemann P.F.W.
      • Brouwers L.
      • van der Zee D.
      • et al.
      Multiplanar reconstruction computed tomography for diagnosis of scaphoid waist fracture union: a prospective cohort analysis of accuracy and precision.
      However, CT is susceptible to metal artifact in the presence of implanted hardware. For fractures undergoing open reduction and internal fixation (ORIF), limited data exist to guide whether CT is useful in establishing scaphoid fracture union. Therefore, even though many hand surgeons routinely obtain CT postoperatively, it is unclear if this diagnostic modality is helpful in this setting.
      The purpose of this study was to compare the reliability of radiographs (XR) alone with that of the combination of XR and CT in determining scaphoid union following ORIF. We hypothesized that the addition of CT to XR would improve reliability.

      Materials and Methods

      An institutional (Thomas Jefferson University) review board approval was obtained prior to initiation of the study, with a waiver of informed consent per institutional protocol. We performed a search of the archives of the digital imaging software system (PACS, SECTRA) of a group of 21 fellowship-trained orthopedic hand surgeons over a 6-year period (from 2012 to 2018). We identified 421 wrist CTs, and those with a cannulated headless compression screw used for ORIF of a scaphoid fracture or a fracture nonunion were selected for review. Fixation was performed in an open (rather than percutaneous) fashion for all patients. The inclusion criteria comprised studies performed to evaluate scaphoid healing with both postoperative XR (including posteroanterior, lateral, oblique, and ulnar deviation views) and a corresponding CT available in the PACS. The exclusion criteria included incomplete XR series and/or CTs not performed using scaphoid protocol (in plane of scaphoid) with fine cuts (<1 mm). Of the 35 studies initially identified, 3 studies were excluded (2 for incomplete XR series and 1 for CT not performed using scaphoid protocol). The remaining 32 studies were included for evaluation. In this cohort, 16 surgeries were performed for acute fracture, and 16 surgeries were performed for fracture nonunion. There were no revision surgeries. Twenty-nine screws were titanium, whereas 3 screws were stainless steel.
      Three fellowship-trained, orthopedic hand surgeons (J.L.M., K.F.L., P.K.B.) reviewed the imaging studies. The studies were evaluated for (1) healing (healed vs partially healed vs not healed), (2) if partially healed, the percentage of healing (<25%, 25%–50%, 51%–75%, and 76%–99%) as per Singh et al,
      • Singh H.P.
      • Forward D.
      • Davis T.R.C.
      • Dawson J.S.
      • Oni J.A.
      • Downing N.D.
      Partial union of acute scaphoid fractures.
      and (3) reviewer certainty in the determination of healing as per (1) above (not certain at all, 25% certain, 50% certain, 75% certain, and very certain). To mimic the clinical setting, XR were first reviewed alone. Three weeks later, the same XR were reviewed with the corresponding CTs. In order to evaluate the reliability within each observer, each reviewer performed a similar but randomized 2-stage evaluation (XR evaluation followed 3 weeks later by XR and CT evaluation) 4 weeks later. By design, we did not require the observer to equate a certain percentage of healing with a determination of healed, partially healed or not healed, although it was the practice of all of the observers to consider a fracture fully healed if there was no visible fracture line.
      Measures of interobserver and intraobserver reliabilities were obtained using linearly weighted kappa (κ) coefficients for healing status and the percentage of healing. A Mann-Whitney U test for ordinal data was used to compare the certainty of healing between XR alone and the combination of XR and CT. Statistical significance was set at P < .05.
      The sample size was determined according to the methodology of Daniels et al.
      • Daniels A.M.
      • Wyers C.E.
      • Janzing H.M.J.
      • et al.
      The interobserver reliability of the diagnosis and classification of scaphoid fractures using high-resolution peripheral quantitative CT.
      ,
      • Donner A.
      • Rotondi M.A.
      Sample size requirements for interval estimation of the kappa statistic for interobserver agreement studies with a binary outcome and multiple raters.
      We considered interobserver agreement of κ0 = 0.80 between the 3 observers to be an acceptable level clinically. We estimated the expected detection of healing (π) at 0.30. The minimal acceptable limit (lower 95% confidence interval) was prespecified as substantial agreement expressed by κL = 0.60, resulting in a sample size of 31 patients.

      Results

      There were 23 men and 9 women with a mean age of 28 years (range: 15–75) at the time of surgery. The median time between the date of surgery and the date of the postoperative XR reviewed was 3.2 months (range: 1.3–19.5). The median time between the postoperative XR and the corresponding CT scan was 4.5 days (range: 1–52). Six fractures (19%) involved the proximal pole, 25 (78%) involved the waist, and 1 (3%) involved the distal pole.
      Weighted interobserver reliability κ values for healing (healed vs partially healed vs not healed) were 0.53 (range: 0.42–0.63, moderate agreement) with XR alone and 0.59 (range: 0.53–0.68, moderate) with the combination of XR and CT. When separating acute fractures from fracture nonunions, interobserver κ values for healing were 0.44 (range: 0.13–0.71, moderate agreement) and 0.51 (range: 0.47–0.55, moderate agreement), respectively, with XR alone. Interobserver κ values for healing were 0.60 (range: 0.55–0.65, moderate agreement) and 0.49 (range: 0.43–0.63, moderate agreement), respectively, with the combination of XR and CT. Similar κ values for intraobserver reliability were 0.69 (range: 0.52–0.81, substantial) with XR alone compared with 0.58 (range: 0.39–0.82, moderate) with the combination of XR and CT. When separating acute fractures from fracture nonunions, intraobserver κ values for healing were 0.55 (range: 0.40–0.99, moderate agreement) and 0.59 (range: 0.53–0.63, moderate agreement), respectively, with the combination of XR and CT.
      Weighted interobserver reliability κ values for the percentage of healing (<25%, 25%–50%, 51%–75%, and 76%–99%) were 0.31 (range: 0.31–0.32, fair) with XR alone and 0.29 (range: 0.03–0.39, fair) with the combination of XR and CT. Similar κ values for intraobserver reliability were 0.35 (range: 0.32–0.37, fair) with XR alone compared with 0.32 (range: 0.21–0.52, fair) with the combination of XR and CT.
      On a scale from 1 (not certain at all of healing) to 5 (very certain of healing), reviewers reported a median certainty of 4 (range: 1–5) based on the review of XR alone compared with 5 (range: 1–5) based on the evaluation of the combination of XR and CT. This difference was statistically significant (P < .05).

      Discussion

      In our study, we found that the surgeons were more certain in their assessment of scaphoid healing when a CT scan was reviewed in addition to XR. Despite this improvement in confidence, we found that the reliability did not improve when CT was included in the determination of healing. In fact, the intraobserver reliability for determining healing decreased from substantial with XR to moderate with the addition of CT. Therefore, while the perception among surgeons is that CT improves the reliability in assessing healing following ORIF, this does not appear to be the case.
      Overall, determining scaphoid union is challenging.
      • Dias J.J.
      Definition of union after acute fracture and surgery for fracture nonunion of the scaphoid.
      In a study of nonsurgically treated scaphoid waist fractures, Dias et al
      • Dias J.J.
      • Taylor M.
      • Thompson J.
      • Brenkel I.J.
      • Gregg P.J.
      Radiographic signs of union of scaphoid fractures. An analysis of inter-observer agreement and reproducibility.
      have reported poor agreement in assessing scaphoid union based on XR obtained 12 weeks after injury. Similarly, at 6 weeks following injury, Hannemann et al
      • Hannemann P.F.W.
      • Brouwers L.
      • Dullaert K.
      • van der Linden E.S.
      • Poeze M.
      • Brink P.R.G.
      Determining scaphoid waist fracture union by conventional radiographic examination: an analysis of reliability and validity.
      found moderate agreement using XR alone to determine the union of nonsurgically treated scaphoid fractures. In light of these data, CT is commonly performed to evaluate the scaphoid healing.
      • Hackney L.A.
      • Dodds S.D.
      Assessment of scaphoid fracture healing.
      However, few studies have discretely assessed the benefit of doing so. In a study evaluating the reliability of CT for determining union of scaphoid fractures treated nonsurgically, Hannemann et al
      • Hannemann P.F.W.
      • Brouwers L.
      • van der Zee D.
      • et al.
      Multiplanar reconstruction computed tomography for diagnosis of scaphoid waist fracture union: a prospective cohort analysis of accuracy and precision.
      found moderate interobserver reliability. Likewise, in a study of nonsurgically treated scaphoid fractures, Buijze et al
      • Buijze G.A.
      • Wijffels M.M.
      • Guitton T.G.
      • et al.
      Interobserver reliability of computed tomography to diagnose scaphoid waist fracture union.
      found interobserver reliability of CT for determining scaphoid union to be substantial. However, neither of these studies have evaluated the reliability of CT in determining union following scaphoid fracture ORIF. Furthermore, the degree of reliability required for a diagnostic test to have value remains unclear, and the “acceptable” degree of reliability may be highly dependent on the clinical setting.
      In contrast, we assessed the reliability of CT in addition to XR in determining union following scaphoid fracture ORIF, which is a relatively common clinical scenario (although performed in our study without the benefit of a clinical examination). While we hypothesized that the addition of CT to XR would improve reliability, we did not find this to be the case. It is unclear why the reliability of CT was so low in our study. Unlike the other studies in which nonsurgically treated scaphoids were assessed, we evaluated scaphoid fractures that had been fixed with a headless compression screw. The addition of the screw likely created some added difficulty in determining the presence and extent of healing. Rather than having the entire trabecular mass of the native scaphoid available to view, the metal screw occupied a substantial portion of the scaphoid. Additionally, the screw artifact may have obscured the fine detail of the surrounding bone. These factors may have contributed to the difficulty in measurement and may have increased importance in clinical situations in which artifact reduction was not obtained or was not available.
      Given that partially healed scaphoids typically progress to adequate union, determining whether a scaphoid is partially healed may be more clinically important than determining the specific degree to which it is partially healed.
      • Singh H.P.
      • Forward D.
      • Davis T.R.C.
      • Dawson J.S.
      • Oni J.A.
      • Downing N.D.
      Partial union of acute scaphoid fractures.
      Following nonsurgical treatment, Drijkoningen et al
      • Drijkoningen T.
      • Ten Berg P.W.L.
      • Guitton T.G.
      • Ring D.
      • Mudgal C.S.
      Reliability of diagnosis of partial union of scaphoid waist fractures on computed tomography.
      have reported that agreement on the degree of partial scaphoid union on CT was fair, and these investigators believe that the diagnosis of partial union of a scaphoid waist fracture on CT had limited reliability. In our study of scaphoid fractures following ORIF, the reliability of percentage of healing was fair with or without CT, thus making our conclusions similar.
      Moreover, there are limited data available to guide surgeons on when a partially healed scaphoid is healed sufficiently to permit unrestricted activity. Singh et al
      • Singh H.P.
      • Forward D.
      • Davis T.R.C.
      • Dawson J.S.
      • Oni J.A.
      • Downing N.D.
      Partial union of acute scaphoid fractures.
      have suggested that 50% to 75% healing is sufficient to progress activities, whereas other investigators have considered 50% healing to be adequate.
      • Belsky M.R.
      • Leibman M.I.
      • Ruchelsman D.E.
      Scaphoid fracture in the elite athlete.
      ,
      • Fowler J.R.
      • Hughes T.B.
      Scaphoid fractures.
      In a recent cadaveric biomechanical study, Guss et al
      • Guss M.S.
      • Mitgang J.T.
      • Sapienza A.
      Scaphoid healing required for unrestricted activity: a biomechanical cadaver model.
      have demonstrated that a 50% intact scaphoid plus headless compression screw had similar load to failure and higher stiffness than an intact scaphoid. They concluded that 50% partial healing was sufficient to allow return to unrestricted activity. There is likely some degree of partial healing whereby the strength of the scaphoid is comparable with that of the fully healed state. In our study, the addition of CT to the XR evaluation did not improve the reliability of determining the extent of partial healing.
      Overall, when reviewing XR in combination with CT, the surgeons felt more certain in their assessment of healing. This finding requires more investigation. While it may be that CT improves surgeon confidence and that this is a potential benefit of obtaining CT, there is also the possibility that it may provide a false sense of security without improving diagnostic reliability.
      There are several limitations to our study. First, given the retrospective nature of our study, we cannot confirm that metal artifact reduction sequences were specifically utilized during CT. In general, when there is hardware in place, surgeons in our practice request metal artifact reduction, and the imaging facilities in our region routinely provide it even in the absence of a physician’s request. Even so, there was likely some metal artifact that may have affected the reliability of measurement, and there may be specific image processing that would improve the reliability in this setting. Overall, the CTs had minimal artifact; thus, we doubt whether this had any substantial impact on the findings and conclusions. Second, the surgeons in our study did not have the benefit of a clinical examination or serial imaging studies to help guide the determination of healing. In the clinical setting, both of these may assist the surgeon in deciding whether a fracture is healed or healing and/or whether additional intervention may be needed.
      • Waizenegger M.
      • Barton N.J.
      • Davis T.R.
      • Wastie M.L.
      Clinical signs in scaphoid fractures.
      ,
      • Unay K.
      • Gokcen B.
      • Ozkan K.
      • Poyanli O.
      • Eceviz E.
      Examination tests predictive of bone injury in patients with clinically suspected occult scaphoid fracture.
      The reliability of the certainty or extent of healing may be influenced by the surgeon’s knowledge of the clinical examination (ie, a small persistent visible line in a patient with no point tenderness might be reproducibly categorized as healed, whereas the evaluation of the XR alone may not be). The time lapse between radiographic imaging and CT may have had an impact on reliability in some cases. We were not able to assess the extent to which this is an issue. Third, not all scaphoid fractures or nonunions received a CT to determine healing following ORIF. This may have introduced an element of selection bias given that CTs potentially were obtained in a particular situation, ie, one in which there was uncertainty with XR alone. However, we feel that this mimics the typical clinical scenario. Fourth, given that XR were reviewed prior to XR and CT, there exists the possibility of anchoring bias. However, we assumed this set-up would most closely mimic the clinical setting. In general, we suspect that CT is seldom obtained without XR. In fact, in our region, many insurance carriers do not allow for a CT to be performed without a recent XR. Fifth, due to the number of patients, we were unable to evaluate reliability in relation to fracture location. While it is conceivable that reliability may vary in relation to location, we believe that including a spectrum of manifestations is appropriate in a study of reliability. Finally, we did not measure the accuracy of the diagnostic studies. Our study did not assess whether the surgeons’ assessment of healing was correct but rather assessed whether it was reliable. Although this would be the most important determinant of the value of any radiographic assessment, there is no current gold standard test for scaphoid union following ORIF that is non-invasive.
      In conclusion, we found that the reliability of assessing scaphoid union following ORIF was not improved by the addition of CT to XR. The results of our study do not show that CT is inaccurate in determining scaphoid healing; rather, the results demonstrate that our ability to reliably interpret this study is suboptimal. Currently, we use clinical examination (presence or absence of point tenderness) and serial XR for demonstrating stable screw position and progressive radiographic healing in deciding whether a fracture has healed. We do not feel it is necessary to routinely obtain CT to evaluate scaphoid healing after ORIF; however, for those patients in whom we do obtain CT, we recognize the limitations of the information that it provides.

      References

        • Dias J.J.
        Definition of union after acute fracture and surgery for fracture nonunion of the scaphoid.
        J Hand Surg Br. 2001; 26: 321-325
        • Dias J.J.
        • Taylor M.
        • Thompson J.
        • Brenkel I.J.
        • Gregg P.J.
        Radiographic signs of union of scaphoid fractures. An analysis of inter-observer agreement and reproducibility.
        J Bone Joint Surg Br. 1988; 70: 299-301
        • Buijze G.A.
        • Wijffels M.M.
        • Guitton T.G.
        • et al.
        Interobserver reliability of computed tomography to diagnose scaphoid waist fracture union.
        J Hand Surg Am. 2012; 37: 250-254
        • Hannemann P.F.W.
        • Brouwers L.
        • van der Zee D.
        • et al.
        Multiplanar reconstruction computed tomography for diagnosis of scaphoid waist fracture union: a prospective cohort analysis of accuracy and precision.
        Skeletal Radiol. 2013; 42: 1377-1382
        • Singh H.P.
        • Forward D.
        • Davis T.R.C.
        • Dawson J.S.
        • Oni J.A.
        • Downing N.D.
        Partial union of acute scaphoid fractures.
        J Hand Surg Br. 2005; 30: 440-445
        • Daniels A.M.
        • Wyers C.E.
        • Janzing H.M.J.
        • et al.
        The interobserver reliability of the diagnosis and classification of scaphoid fractures using high-resolution peripheral quantitative CT.
        Bone Joint J. 2020; 102-B: 478-484
        • Donner A.
        • Rotondi M.A.
        Sample size requirements for interval estimation of the kappa statistic for interobserver agreement studies with a binary outcome and multiple raters.
        Int J Biostat. 2010; 6 (Article 31)
        • Hannemann P.F.W.
        • Brouwers L.
        • Dullaert K.
        • van der Linden E.S.
        • Poeze M.
        • Brink P.R.G.
        Determining scaphoid waist fracture union by conventional radiographic examination: an analysis of reliability and validity.
        Arch Orthop Trauma Surg. 2015; 135: 291-296
        • Hackney L.A.
        • Dodds S.D.
        Assessment of scaphoid fracture healing.
        Curr Rev Musculoskelet Med. 2011; 4: 16-22
        • Drijkoningen T.
        • Ten Berg P.W.L.
        • Guitton T.G.
        • Ring D.
        • Mudgal C.S.
        Reliability of diagnosis of partial union of scaphoid waist fractures on computed tomography.
        J Hand Microsurg. 2018; 10: 130-133
        • Belsky M.R.
        • Leibman M.I.
        • Ruchelsman D.E.
        Scaphoid fracture in the elite athlete.
        Hand Clin. 2012; 28 (vii): 269-278
        • Fowler J.R.
        • Hughes T.B.
        Scaphoid fractures.
        Clin Sports Med. 2015; 34: 37-50
        • Guss M.S.
        • Mitgang J.T.
        • Sapienza A.
        Scaphoid healing required for unrestricted activity: a biomechanical cadaver model.
        J Hand Surg Am. 2018; 43: 134-138
        • Waizenegger M.
        • Barton N.J.
        • Davis T.R.
        • Wastie M.L.
        Clinical signs in scaphoid fractures.
        J Hand Surg Br. 1994; 19: 743-747
        • Unay K.
        • Gokcen B.
        • Ozkan K.
        • Poyanli O.
        • Eceviz E.
        Examination tests predictive of bone injury in patients with clinically suspected occult scaphoid fracture.
        Injury. 2009; 40: 1265-1268