Advertisement

Bias in Hand Surgical Randomized Controlled Trials: Systematic Review and Meta-Epidemiological Study

Published:March 24, 2022DOI:https://doi.org/10.1016/j.jhsa.2022.01.027

      Purpose

      Inappropriately reported or conducted studies may decrease the quality of care due to under- or overestimation of the benefits or harms of interventions. Our aim was to evaluate how often hand surgical randomized controlled trials (RCTs) use and report adequate methods to ensure internal validity, and whether inadequate reporting or methods are associated with the magnitude of treatment effect estimates.

      Methods

      Data Sources were the Cochrane Central Register of Controlled Trials, MEDLINE, and Embase databases until November 2020. We included published RCTs investigating the effects of any surgical intervention in the hand and wrist region. We assessed internal validity using the Cochrane Risk of Bias (RoB) tool for 6 domains: selection, performance, detection, attrition, selective reporting, and “other” bias. We extracted the primary outcome and calculated the effect size for each study. We used mixed-effect meta-regression to assess whether the RoB modified the magnitude of the effects.

      Results

      For 207 assessed trials, the RoB was unclear or high for 72% in selection, 93% in performance, 88% in detection, 25% in attrition, 22% in selective reporting, and 34% in the “other” bias domain. Trials with a high or unclear risk of selection bias yielded 0.28 standardized mean difference (95% confidence interval, 0.02–0.55) larger effect sizes compared to studies with a low risk. Risks of bias for other domains did not modify the intervention effects. The risk for selection bias declined over time: the odds ratio for a high or unclear RoB was 0.90 (95% confidence interval, 0.85–0.95) per additional year of publication

      Conclusions

      The internal validity and credibility of hand surgical RCTs can be improved by using established methods to achieve true randomization, blinding of the participants and study personnel, publishing the trial protocol and avoiding selective reporting of the outcomes, and reporting the trial as recommended in the Consolidated Standards of Reporting Trials statement.

      Clinical relevance

      Clinicians should be aware that RCTs that do not use or report proper randomization and allocation concealment may overestimate the treatment effects.

      Key words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Hand Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Jüni P.
        • Altman D.G.
        • Egger M.
        Systematic reviews in health care: assessing the quality of controlled clinical trials.
        BMJ. 2001; 323: 42-46
        • Chalmers I.
        • Glasziou P.
        Avoidable waste in the production and reporting of research evidence.
        Lancet. 2009; 374: 86-89
        • Berkman N.D.
        • Santaguida P.L.
        • Viswanathan M.
        • Morton S.C.
        The Empirical Evidence of Bias in Trials Measuring Treatment Differences.
        Agency for Healthcare Research and Quality (US), 2014
        • Boutron I.P.M.
        • Higgins J.P.T.
        • Altman D.G.
        • Lundh A.
        • Hróbjartsson A.
        Chapter 7. Considering bias and conflicts of interest among the included studies.
        in: Higgins J.P.T. Thomas J. Chandler J. Cumpston M. Li T. Page M.J. Welch V.A. Cochrane Handbook for Systematic Reviews of Interventions. John Wiley & Sons, Hoboken, NJ2019
        • Berkman N.D.
        • Santaguida P.L.
        • Viswanathan M.
        • Morton S.C.
        The Empirical Evidence of Bias in Trials Measuring Treatment Differences. Methods Research Report. (Prepared by the RTI-UNC Evidence-based Practice Center under Contract No. 290-2007-10056-I.) AHRQ Publication No. 14-EHC050-EF.
        Agency for Healthcare Research and Quality, Rockville, MD2014
        • Sugrue C.M.
        • Joyce C.W.
        • Sugrue R.M.
        • Carroll S.M.
        Trends in the level of evidence in clinical hand surgery research.
        Hand (N Y). 2016; 11: 211-215
        • Juuso Heikkinen
        • Teemu Karjalainen
        • Rachelle Buchbinder
        • Jarkko Jokihaara
        • Soumen Das De
        • Kati Jaatinen
        A systematic review of methodological quality in hand surgical randomized controlled trials.
        • Higgins J.P.
        • Altman D.G.
        • Gøtzsche P.C.
        • et al.
        The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials.
        BMJ. 2011; 343: d5928
        • Sterne J.A.C.
        • Savović J.
        • Page M.J.
        • et al.
        RoB 2: a revised tool for assessing risk of bias in randomised trials.
        BMJ. 2019; 366: l4898
        • Higgins J.
        • Thomas J.
        • Chandler J.
        • et al.
        Cochrane Handbook for Systematic Reviews of Interventions.
        https://training.cochrane.org/handbook
        Version: Version 6.0
        Date accessed: February 9, 2022
        • Chinn S.
        A simple method for converting an odds ratio to effect size for use in meta-analysis.
        Stat Med. 2000; 19: 3127-3131
        • Lian Z.M.
        • Yang J.
        • Zhang T.L.
        • et al.
        Bridging extema/fixation combined with Kirschner-wire fixation versus volar locked plate fixation for unstable fractures of the distal radius.
        Zhongguo Zuzhi Gongcheng Yanjiu. 2016; 20: 6590-6598
        • Freedman B.
        Equipoise and the ethics of clinical research.
        N Engl J Med. 1987; 317: 141-145
        • London A.J.
        Clinical equipoise: foundational requirement or fundamental error?.
        in: Steinbock B. The Oxford Handbook of Bioethics. Oxford University Press, 2007: 571-595
        • Guyatt G.H.
        • Oxman A.D.
        • Vist G.E.
        • et al.
        GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.
        BMJ. 2008; 336: 924-926
        • Schulz K.F.
        • Chalmers I.
        • Hayes R.J.
        • Altman D.G.
        Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials.
        JAMA. 1995; 273: 408-412
        • Savović J.
        • Jones H.E.
        • Altman D.G.
        • et al.
        Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials.
        Ann Intern Med. 2012; 157: 429-438
        • Beard D.J.
        • Campbell M.K.
        • Blazeby J.M.
        • et al.
        Considerations and methods for placebo controls in surgical trials (ASPIRE guidelines).
        Lancet. 2020; 395: 828-838
        • Saltaji H.
        • Armijo-Olivo S.
        • Cummings G.G.
        • Amin M.
        • da Costa B.R.
        • Flores-Mir C.
        Influence of blinding on treatment effect size estimate in randomized controlled trials of oral health interventions.
        BMC Med Res Methodol. 2018; 18: 42
        • Saltaji H.
        • Armijo-Olivo S.
        • Cummings G.G.
        • Amin M.
        • da Costa B.R.
        • Flores-Mir C.
        Impact of selection bias on treatment effect size estimates in randomized trials of oral health interventions: a meta-epidemiological study.
        J Dent Res. 2018; 97: 5-13
        • Armijo-Olivo S.
        • Fuentes J.
        • da Costa B.R.
        • Saltaji H.
        • Ha C.
        • Cummings G.G.
        Blinding in physical therapy trials and its association with treatment effects: a meta-epidemiological study.
        Am J Phys Med Rehabil. 2017; 96: 34-44
        • Moustgaard H.
        • Clayton G.L.
        • Jones H.E.
        • et al.
        Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study.
        BMJ. 2020; 368: l6802
        • Moustgaard H.
        • Bello S.
        • Miller F.G.
        • Hróbjartsson A.
        Subjective and objective outcomes in randomized clinical trials: definitions differed in methods publications and were often absent from trial reports.
        J Clin Epidemiol. 2014; 67: 1327-1334
        • Hróbjartsson A.
        • Thomsen A.S.
        • Emanuelsson F.
        • et al.
        Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors.
        Int J Epidemiol. 2014; 43: 937-948
        • Hróbjartsson A.
        • Thomsen A.S.
        • Emanuelsson F.
        • et al.
        Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors.
        CMAJ. 2013; 185: E201-E211
        • Hróbjartsson A.
        • Thomsen A.S.
        • Emanuelsson F.
        • et al.
        Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors.
        BMJ. 2012; 344: e1119
        • Dechartres A.
        • Trinquart L.
        • Faber T.
        • Ravaud P.
        Empirical evaluation of which trial characteristics are associated with treatment effect estimates.
        J Clin Epidemiol. 2016; 77: 24-37
        • Fergusson D.
        • Laupacis A.
        • Salmi L.R.
        • McAlister F.A.
        • Huet C.
        What should be included in meta-analyses? An exploration of methodological issues using the ISPOT meta-analyses.
        Int J Technol Assess Health Care. 2000; 16: 1109-1119
        • Chan A.W.
        • Krleza-Jerić K.
        • Schmid I.
        • Altman D.G.
        Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research.
        CMAJ. 2004; 171: 735-740
        • Chan A.W.
        • Hróbjartsson A.
        • Haahr M.T.
        • Gøtzsche P.C.
        • Altman D.G.
        Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.
        JAMA. 2004; 291: 2457-2465
        • Zarin D.A.
        • Tse T.
        • Williams R.J.
        • Califf R.M.
        • Ide N.C.
        The ClinicalTrials.gov results database--update and key issues.
        N Engl J Med. 2011; 364: 852-860
        • Schulz K.F.
        • Altman D.G.
        • Moher D.
        • CONSORT Group
        CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials.
        Ann Intern Med. 2010; 152: 726-732