Scientific Article| Volume 45, ISSUE 12, P1181.e1-1181.e4, December 2020

The Reliability of Determining the Presence of Surgical Site Infection Based on Retrospective Chart Review


      Surgical site infection (SSI) can be a challenging complication after hand surgery. Retrospective studies often rely on chart review to determine presence of an SSI. The purpose of this study was to assess reliability of Centers for Disease Control and Prevention (CDC) criteria for determining an SSI as applied to a chart review. We hypothesized that interobserver and intraobserver reliability for determining an SSI using these criteria while reviewing medical record documentation would be none to minimal (κ < 0.39) based on an interpretation of Cohen’s κ statistics.


      We created and used a database of 782 patients, 48 of whom received antibiotics within 3 months of a surgical procedure of the hand. Three fellowship-trained orthopedic hand surgeons then evaluated the charts of those 48 patients, in which each reviewer determined whether an SSI was present or absent based on CDC criteria provided to the reviewers. Patients’ charts were then reassessed 1 month later by the same reviewers. Kappa statistics were calculated for each round of assessment and averaged to determine intraobserver and interobserver reliability.


      Overall κ values were 0.22 (standard error, 0.13), indicating fair reliability. Average κ value between reviewers was 0.26 (standard error, 0.13. On average, intrarater reliability was 68.7%.


      We found poor interobserver and intraobserver reliability when using CDC criteria to determine whether a patient had an SSI, based on chart review.

      Clinical relevance

      Better criteria or documentation may be needed in patients with an infection after hand surgery. Retrospective chart reviews to assess infection may be unreliable.

      Key words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Hand Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Broex E.C.
        • van Asselt A.D.
        • Bruggeman C.A.
        • van Tiel F.H.
        Surgical site infections: how high are the costs?.
        J Hosp Infect. 2009; 72: 193-201
        • de Lissovoy G.
        • Fraeman K.
        • Hutchins V.
        • Murphy D.
        • Song D.
        • Vaughn B.B.
        Surgical site infection: incidence and impact on hospital utilization and treatment costs.
        Am J Infect Control. 2009; 37: 387-397
        • Quinn A.
        • Hill A.D.
        • Humphreys H.
        Evolving issues in the prevention of surgical site infections.
        Surgeon. 2009; 7: 170-172
        • Whitehouse J.D.
        • Friedman N.D.
        • Kirkland K.B.
        • Richardson W.J.
        • Sexton D.J.
        The impact of surgical-site infections following orthopaedic surgery at a community hospital and a university hospital: adverse quality of life, excess length of stay, and extra cost.
        Infect Control Hosp Epidemiol. 2002; 23: 183-189
        • Mangram A.J.
        • Horan T.C.
        • Pearson M.L.
        • Silver L.C.
        • Jarvis W.R.
        the Hospital Infection Control Practices Advisory Committee. Guideline for prevention of surgical site infection, 1999.
        Infect Control Hosp Epidemiol. 1999; 20: 250-280
        • Menendez M.E.
        • Lu N.
        • Unizony S.
        • Choi H.K.
        • Ring D.
        Surgical site infection in hand surgery.
        Int Orthop. 2015; 39: 2191-2198
        • Werner B.C.
        • Teran V.A.
        • Deal D.N.
        Patient-related risk factors for infection following open carpal tunnel release: an analysis of over 450,000 Medicare patients.
        J Hand Surg Am. 2018; 43: 214-219
        • McHugh M.L.
        Interrater reliability: the kappa statistic.
        Biochem Med (Zagreb). 2012; 22: 276-282
        • American Academy of Orthopaedic Surgeons
        OrthoGuidelines: management of carpal tunnel syndrome.
        (Available at:)
        • Ariyan S.
        • Martin J.
        • Lal A.
        • et al.
        Antibiotic prophylaxis for preventing surgical-site infection in plastic surgery: an evidence-based consensus conference statement from the American Association of Plastic Surgeons.
        Plast Reconstr Surg. 2015; 135: 1723-1739
        • Zimlichman E.
        • Henderson D.
        • Tamir O.
        • et al.
        Health care-associated infections: a meta-analysis of costs and financial impact on the US health care system.
        JAMA Intern Med. 2013; 173: 2039-2046
        • Owens P.L.
        • Barrett M.L.
        • Raetzman S.
        • Maggard-Gibbons M.
        • Steiner C.A.
        Surgical site infections following ambulatory surgery procedures.
        JAMA. 2014; 311: 709-716
        • Milstein A.
        Ending extra payment for “never events”-stronger incentives for patients’ safety.
        N Engl J Med. 2009; 360: 2388-2390
        • Rosenthal M.B.
        Nonpayment for performance? Medicare’s new reimbursement rule.
        N Engl J Med. 2009; 357: 1573-1575
        • U.S. Department of Health and Human Services
        National Action Plan to Prevent Healthcare-Associated Infections: roadmap to elimination.
        (Available at:)
        • Ashby E.
        • Haddad F.S.
        • O’Donnell E.
        • Wilson A.P.R.
        How will surgical site infection be measured to ensure “high quality care for all”?.
        J Bone Joint Surg Br. 2010; 92: 1294-1299