Editor's Choice| Volume 45, ISSUE 12, P1105-1114, December 2020

Download started.


Establishing Validity of a Comprehensive Hand Surgical Training and Educational Platform (STEP)

Published:September 30, 2020DOI:


      The purpose of this study was to develop and perform the initial validation for the Surgical Training and Educational Platform (STEP), a cost-effective psychomotor training and assessment instrument designed to teach and evaluate fundamental skills considered critical to competency in hand surgery.


      An American Society for Surgery of the Hand (ASSH) taskforce of 13 board-certified hand surgeons developed 8 skills considered fundamental to competency in hand surgery including: (1) lag screw fixation of an oblique fracture, (2) depth of plunge during bicortical drilling, (3) central axis scaphoid fixation, (4) phalangeal fracture pinning, (5) flexor tendon repair, (6) microsurgical suturing, (7) full-thickness skin graft harvest, and (8) wrist arthroscopy. The tasks were developed from commercially available, nonclinical supplies at low cost. The startup cost for the entire system was less than $600 USD, with a cost-per-trainee-assessment of approximately $25 USD. After the tasks were finalized, 2 examiners traveled to 8 sites around the country to evaluate 93 surgeons, including 57 board-certified hand surgeons and 36 first-year orthopedic and plastic surgical residents. Scoring criteria for each task were based on a combination of time and clinically relevant quality metrics.


      Board-certified hand surgeons significantly outperformed surgical trainees on all 8 tasks, demonstrating that the STEP appropriately differentiates the skill level of expert and novice hand surgeons.


      The STEP is an effective simulator that encompasses a wide range of fundamental psychomotor skills considered critical to competency in hand surgery. Although the STEP tasks are a valuable surgical training tool, further work is required to evaluate and refine the scoring system prior to using it as a critical evaluation of performance.

      Clinical relevance

      The STEP simulator is a valuable, validated instrument for resident and fellow education and evaluation in hand surgery outside of the operating room.

      Key words

      JHS Podcast

      December 1, 2020

      JHS Podcast Episode 57

      Dr. Graham interviews Dr. Ranjan Gupta, chair of the ASSH Surgical Simulation Task Force on their article "Establishing Validity of a Comprehensive Hand Surgical Training and Educational Platform”, which appears in the December 2020 issue of the Journal of Hand Surgery.

      Loading ...
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Hand Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Bjerrum F.
        • Thomsen A.S.S.
        • Nayahangan L.J.
        • Konge L.
        Surgical simulation: current practices and future perspectives for technical skills training.
        Med Teach. 2018; 40: 668-675
        • Nousiainen M.T.
        • McQueen S.A.
        • Ferguson P.
        • et al.
        Simulation for teaching orthopaedic residents in a competency-based curriculum: do the benefits justify the increased costs?.
        Clin Orthop Relat Res. 2016; 474: 935-944
        • Coelho G.
        • Defino H.L.A.
        The role of mixed reality simulation for surgical training in spine: phase 1 validation.
        Spine (Phila Pa 1976). 2018; 43: 1609-1616
        • Jabbour N.
        • Snyderman C.H.
        The economics of surgical simulation.
        Otolaryngol Clin North Am. 2017; 50: 1029-1036
        • Atesok K.
        • Satava R.M.
        • Marsh J.L.
        • Hurwitz S.R.
        Measuring surgical skills in simulation-based training.
        J Am Acad Orthop Surg. 2017; 25: 665-672
        • Weber E.L.
        • Leland H.A.
        • Azadgoli B.
        • Minneti M.
        • Carey J.N.
        Preoperative surgical rehearsal using cadaveric fresh tissue surgical simulation increases resident operative confidence.
        Ann Transl Med. 2017; 5: 302
        • Ruder J.A.
        • Turvey B.
        • Hsu J.R.
        • Scannell B.P.
        Effectiveness of a low-cost drilling module in orthopaedic surgical simulation.
        J Surg Educ. 2017; 74: 471-476
        • Kazan R.
        • Cyr S.
        • Hemmerling T.M.
        • Lin S.J.
        • Gilardino M.S.
        The evolution of surgical simulation: the current state and future avenues for plastic surgery education.
        Plast Reconstr Surg. 2017; 139: 533e-543e
        • Tavakol M.
        • Mohagheghi M.A.
        • Dennick R.
        Assessing the skills of surgical residents using simulation.
        J Surg Educ. 2008; 65: 77-83
        • Borgersen N.J.
        • Naur T.M.H.
        • Sorensen S.M.D.
        • et al.
        Gathering validity evidence for surgical simulation: a systematic review.
        Ann Surg. 2018; 267: 1063-1068
        • Lopez G.
        • Martin D.F.
        • Wright R.
        • et al.
        Construct validity for a cost-effective arthroscopic surgery simulator for resident education.
        J Am Acad Orthop Surg. 2016; 24: 886-894
        • Lopez G.
        • Wright R.
        • Martin D.
        • Jung J.
        • Bracey D.
        • Gupta R.
        A cost-effective junior resident training and assessment simulator for orthopaedic surgical skills via fundamentals of orthopaedic surgery: AAOS exhibit selection.
        J Bone Joint Surg Am. 2015; 97: 659-666
        • Karam M.D.
        • Pedowitz R.A.
        • Natividad H.
        • Murray J.
        • Marsh J.L.
        Current and future use of surgical skills training laboratories in orthopaedic resident education: a national survey.
        J Bone Joint Surg Am. 2013; 95: e4
        • Van Nortwick S.S.
        • Lendvay T.S.
        • Jensen A.R.
        • Wright A.S.
        • Horvath K.D.
        • Kim S.
        Methodologies for establishing validity in surgical simulation studies.
        Surgery. 2010; 147: 622-630
        • Dumestre D.
        • Yeung J.K.
        • Temple-Oberle C.
        Evidence-based microsurgical skill-acquisition series part 1: validated microsurgical models—a systematic review.
        J Surg Educ. 2014; 71: 329-338
        • Dumestre D.
        • Yeung J.K.
        • Temple-Oberle C.
        Evidence-based microsurgical skills acquisition series part 2: validated assessment instruments—a systematic review.
        J Surg Educ. 2015; 72: 80-89
        • Evgeniou E.
        • Walker H.
        • Gujral S.
        The role of simulation in microsurgical training.
        J Surg Educ. 2018; 75: 171-181
        • Fried G.M.
        • Feldman L.S.
        • Vassiliou M.C.
        • et al.
        Proving the value of simulation in laparoscopic surgery.
        Ann Surg. 2004; 240: 518-525
        • McDougall E.M.
        Validation of surgical simulators.
        J Endourol. 2007; 21: 244-247
        • Howells N.R.
        • Gill H.S.
        • Carr A.J.
        • Price A.J.
        • Rees J.L.
        Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study.
        J Bone Joint Surg Br. 2008; 90: 494-499
        • Ramachandran S.
        • Ghanem A.M.
        • Myers S.R.
        Assessment of microsurgery competency—where are we now?.
        Microsurgery. 2013; 33: 406-415
        • Elliott R.M.
        • Baldwin K.D.
        • Foroohar A.
        • Levin L.S.
        The impact of residency and fellowship training on the practice of microsurgery by members of the American Society for Surgery of the Hand.
        Ann Plast Surg. 2012; 69: 451-458
        • Mehta K.
        • Pierce P.
        • Chiu D.T.
        • Thanik V.
        The effect of residency and fellowship type on hand surgery clinical practice patterns.
        Plast Reconstr Surg. 2015; 135: 179-186
        • American Educational Research Association, American Psychological Association, National Council on Measurment in Education
        Standards for Educational and Psychological Testing.
        American Educational Research Association, Washington, DC2014
        • Messick S.
        Foundations of validity: meaning and consequences in psychological assessment.
        ETS Res Rep Ser. 1993; 1993: i-i18
        • Society of American Gastrointestinal and Endoscopic Surgeons
        FLS Manual Skills Written Instructions and Performance Guidelines.
        (Available at:)
        • Peters J.H.
        • Fried G.M.
        • Swanstrom L.L.
        • et al.
        Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery.
        Surgery. 2004; 135: 21-27
        • American Board of Obstetrics and Gynecology
        ABOG Announces New Eligibility Requirement for Board Certification.
        (Available at:)
        • Bernard J.A.
        • Dattilo J.R.
        • Srikumaran U.
        • Zikria B.A.
        • Jain A.
        • LaPorte D.M.
        Reliability and validity of 3 methods of assessing orthopedic resident skill in shoulder surgery.
        J Surg Educ. 2016; 73: 1020-1025
        • Alvand A.
        • Logishetty K.
        • Middleton R.
        • et al.
        Validating a global rating scale to monitor individual resident learning curves during arthroscopic knee meniscal repair.
        Arthroscopy. 2013; 29: 906-912
        • Anderson D.D.
        • Long S.
        • Thomas G.W.
        • Putnam M.D.
        • Bechtold J.E.
        • Karam M.D.
        Objective Structured Assessments of Technical Skills (OSATS) does not assess the quality of the surgical result effectively.
        Clin Orthop Relat Res. 2016; 474: 874-881
        • Putnam M.D.
        • Kinnucan E.
        • Adams J.E.
        • Van Heest A.E.
        • Nuckley D.J.
        • Shanedling J.
        On orthopedic surgical skill prediction—the limited value of traditional testing.
        J Surg Educ. 2015; 72: 458-470
        • Thomson J.E.
        • Poudrier G.
        • Stranix J.T.
        • Motosko C.C.
        • Hazen A.
        Current status of simulation training in plastic surgery residency programs: a review.
        Arch Plast Surg. 2018; 45: 395-402
        • Kalun P.
        • Wagner N.
        • Yan J.
        • Nousiainen M.T.
        • Sonnadara R.R.
        Surgical simulation training in orthopedics: current insights.
        Adv Med Educ Pract. 2018; 9: 125-131
        • Kim-Fine S.
        • Brennand E.A.
        Surgical simulation and competency.
        Obstet Gynecol Clin North Am. 2016; 43: 575-590
        • Norman G.
        • Dore K.
        • Grierson L.
        The minimal relationship between simulation fidelity and transfer of learning.
        Med Educ. 2012; 46: 636-647
        • American Board of Surgery
        The American Board of Surgery Booklet of Information.
        (Available at:)
        • Cristancho S.M.
        • Moussa F.
        • Dubrowski A.
        A framework-based approach to designing simulation-augmented surgical education and training programs.
        Am J Surg. 2011; 202: 344-351
        • Cook D.A.
        • Hatala R.
        Validation of educational assessments: a primer for simulation and beyond.
        Adv Simul (Lond). 2016; 1: 31
        • Michelson J.D.
        • Manning L.
        Competency assessment in simulation-based procedural education.
        Am J Surg. 2008; 196: 609-615
        • Raison N.
        • Ahmed K.
        • Fossati N.
        • et al.
        Competency based training in robotic surgery: benchmark scores for virtual reality robotic simulation.
        BJU Int. 2017; 119: 804-811