Advertisement

Racial and Gender Discrimination in Hand Surgery Letters of Recommendation

Published:August 19, 2021DOI:https://doi.org/10.1016/j.jhsa.2021.07.009

      Purpose

      We sought to evaluate hand surgery applicants’ letters of recommendations to understand whether applicant and letter writer demographics contribute to racial and gender bias.

      Methods

      All applications submitted through the American Society for Surgery of the Hand match to a single institution fellowship program for the 2017 to 2019 application cycles were analyzed using validated text analysis software. Race/ethnicity information was derived from an analysis of applicant photos using the Face Secret Pro software. Primary outcome measures were differences in communal and agentic language used in letters of recommendation, stratified by both race/ethnicity and gender.

      Results

      A total of 912 letters of recommendation were analyzed for 233 applicants (51 female and 172 male). Of these, 88 were written by female letter writers and 824 were written by male letter writers. There were 8 Black, 12 Hispanic, 36 Asian, and 167 White applicants. Letter writers used more agentic language with Asian applicants and non-White applicants overall. Female letter writers used more communal terms and were not associated with applicant race or gender.

      Conclusions

      Letters of recommendation in hand surgery demonstrate disparities in language based on race and gender.

      Clinical relevance

      Alerting letter writers to the role of implicit bias will hopefully spur a discussion on tools to mitigate the use of biased language and provide a foundation for an equitable selection process. Efforts to improve policies and procedures pertaining to diversity and inclusion are paramount to ensuring that fellows more completely represent the population hand surgeons wish to serve.

      Key words

      In a survey of fellowship directors, a letter of recommendation (LOR) from a hand surgeon is one of the top 5 influential factors in hand surgery fellow selection.
      • Egro F.M.
      • Vangala S.K.
      • Nguyen V.T.
      • Spiess A.M.
      Hand surgery fellowship selection criteria: a national fellowship director survey.
      This subjective letter provides the reader with insight into an applicant’s character, operative ability, personality, and overall potential to succeed as a fellow. Implicit bias based on gender has been well documented in LORs for residency application processes in fields other than hand surgery.
      • Lin F.
      • Oh S.K.
      • Gordon L.K.
      • Pineles S.L.
      • Rosenberg J.B.
      • Tsui I.
      Gender-based differences in letters of recommendation written for ophthalmology residency applicants.
      • Filippou P.
      • Mahajan S.
      • Deal A.
      • et al.
      The presence of gender bias in letters of recommendations written for urology residency applicants.
      • Grimm L.J.
      • Redmond R.A.
      • Campbell J.C.
      • Rosette A.S.
      Gender and racial bias in radiology residency letters of recommendation.
      • Turrentine F.E.
      • Dreisbach C.N.
      • St Ivany A.R.
      • Hanks J.B.
      • Schroen A.T.
      Influence of gender on surgical residency applicants’ recommendation letters.
      Differences in both language and length of LORs have emerged as common themes in the literature. A shorter letter length has been associated with less detail, whereas a longer length is associated with a more persuasive letter.
      • Trix F.
      • Psenka C.
      Exploring the color of glass: letters of recommendation for female and male medical faculty.
      Language that is agentic, referring to an individual’s power to control their own actions and destiny, or communal, referring to traits that are shared in common by members of a group, is unconsciously or consciously used to evaluate desirable qualities of an applicant. Traditionally, agentic terms are associated with masculinity (competent, independent, decisive), while communal terms are associated with femininity (sympathetic, kind, relationship-oriented). As such, these themes are often related to gender, with women being associated with communal terms and men being associated with agentic terms.
      • Fiske S.T.
      • Cuddy A.J.C.
      • Glick P.
      • Xu J.
      A model of (often mixed) stereotype content: competence and warmth respectively follow from perceived status and competition.
      • Rosette A.S.
      • Koval C.Z.
      • Ma A.
      • Livingston R.
      Race matters for women leaders: intersectional effects on agentic deficiencies and penalties.
      • Schein V.E.
      The relationship between sex role stereotypes and requisite management characteristics.
      • Eagly A.H.
      • Karau S.J.
      Role congruity theory of prejudice toward female leaders.
      Racial and gender diversity is lacking in hand surgery training programs, with the subspecialty encompassing 27% female trainees, 3% Black trainees, 1.5% Hispanic trainees, 19% Asian trainees, and 72% White trainees.
      • Brotherton S.E.
      • Etzel S.I.
      Graduate medical education, 2018–2019.
      The Accreditation Council for Graduate Medical Education (ACGME) considers underrepresented minorities (URMs) to include Blacks, Mexican-Americans, Native Americans (American Indians, Alaska Natives, and Native Hawaiians), and mainland Puerto Ricans.
      Accreditation Council for Graduate Medical Education (ACGME)
      ACGME Common Program Requirements (Residency).
      Prior analyses of LORs in medicine evaluated ethnic and racial associations with agency and communality. These studies suggest that traditional stereotypes of URMs influence the use of agentic and communal terms.
      • Rosette A.S.
      • Leonardelli G.J.
      • Phillips K.W.
      The White standard: racial bias in leader categorization.
      An example from the studies illustrates this intersectionality. Blacks and Hispanics were viewed based on negative racial stereotypes as incompetent (communal), while simultaneously being protective and individualistic (agentic).
      • Katz D.
      • Braly K.
      Racial stereotypes of one hundred college students.
      • Niemann Y.F.
      • Jennings L.
      • Rozelle R.M.
      • Baxter J.C.
      • Sullivan E.
      Use of free responses and cluster analysis to determine stereotypes of eight groups.
      • Ghavami N.
      • Peplau L.A.
      An intersectional analysis of gender and ethnic stereotypes: testing three hypotheses.
      Overall, URMs were viewed with less agentic terms. Furthermore, the use of significant communal terms is related to a negative impact on hiring.
      • Madera J.M.
      • Hebl M.R.
      • Martin R.C.
      Gender and letters of recommendation for academia: agentic and communal differences.
      The qualitative nature of LORs lends itself to potential bias and should be further explored in surgical subspecialties.
      Given that racial and gender diversity in hand surgery is lower overall compared to the average percentage of fellows in ACGME programs, we sought to examine the potential for bias in LORs for hand surgery applicants. Our study aimed to evaluate LORs of applicants that applied to a single institution’s hand fellowship, to explore the relationship of applicant and letter writer demographics and their contributions to racial and gender bias. We hypothesized that female candidates would be associated with communal language, underrepresented minorities would have less agentic terms, and that overall female and URM candidates would have shorter chairperson lengths of letters based on themes identified in LOR literature for other specialties.

      Materials and Methods

      After University of Virginia Institutional Review Board for the Social and Behavioral Sciences approval, we analyzed LORs entered into the American Society for Surgery of the Hand match for applicants who applied to a single academic institution hand fellowship between 2017 and 2019. The information collected included the applicant’s name, gender, race, date of birth, languages spoken, medical school, residency type, US Medical Licensing Examination (USMLE) Step 1 score, Alpha Omega Alpha (AOA) status, number of LORs, number of academic works, letter writers, letter writer gender(s), letter writer academic rank(s), whether there was a letter from someone with Chairperson/Program Director status, letter body word count, and letter body contents. Race was determined using the Face Secret Pro application (2020 Zift Software LLC) to analyze the photo attached to the application. The Face Secret Pro application analyzes an applicant’s photo and provides a percentage which correlates with the primary racial appearance of the individual. If an applicant received a Face Secret Pro score of greater than a 50% match for a particular race, they were assigned to that racial category. Racial categories included White, Black, Hispanic/Latinx, or Asian. The Face Secret scores were cross-checked with known racial/ethnic backgrounds (ie, born in a Latin country or known Latin names) for errors, and 12 Hispanic/Latinx applicants were recategorized.
      After reviewing the literature, words and phrases were divided into 2 categories (agency and communality). We developed a library that was uploaded to the Linguistic Inquiry Word Count (LIWC) software (version 1.4.0; Pennebaker Conglomerates, Inc.). The LIWC is a validated, word count–based text analysis program that quantifies language metrics. Metrics include a previously validated method for conducting a psychological, social, and emotional text analysis.
      • Tausczik Y.R.
      • Pennebaker J.W.
      The psychological meaning of words: LIWC and computerized text analysis methods.
      The collected data, along with the word/phrases library (Table E1, available online on the Journal’s website at www.jhandsurg.org), were uploaded to the LIWC software for an analysis of the body of the LOR (ie, starting after “Dear Committee” and ending before “Sincerely”). The LIWC provides a single score that correlates with the instances of agentic and communal terms (ie, a higher score means more terms used). The LIWC score is not the number of instances (ie, mentioned terms 3 times therefore score equals 3), but rather a scale that converts these numbers to percentages (ie, mentioned terms 3 times in a 1000-word document, score equals 0.3%).
      Demographic variables were summarized using means and SDs for parametric data and medians and interquartile ranges for nonparametric data. Frequencies and percentages were used to summarize categorical variables. Multilevel, linear, mixed-effect models with a random intercept were used to explore the associations between agency and communality scores and, separately, applicant characteristics and letter writer characteristics. Clustering was accounted for at the subject level in all models. In order to preserve the linear model and address skewed data, log transformations were performed on the predictor variables of USMLE Step 1 score and number of academic works, as well as on the outcome variables of agency and communality scores, prior to the regression analysis. Agency and communality scores were modeled as the dependent variables in all regression analyses. The applicant characteristics modeled as independent variables included age, sex, AOA status, USMLE score, number of academic works, and facial ID score (converted to White, Black, Asian, or Hispanic/Latinx). Similarly, letter writer characteristics such as sex, academic rank (Assistant Professor, Associate Professor, etc.), and whether the letter writer held the position of Chair or Program Director were separately included as independent variables. Pearson correlation coefficients of agency and communality scores, AOA status, number of academic works, and USMLE score were generated to check for any correlation prior to running analyses. Similarly, the effect of multicollinearity was also examined using both the variance inflation factor and tolerance. Effect sizes are reported as beta estimates (BE) with 95% confidence intervals (CIs), with each BE representing the change in communality or agency score associated with each independent variable.

      Results

      For the application cycles from 2017 to 2019, 233 applicants (51 female and 172 male) applied to interview at our institution. A total of 912 letters of recommendation were submitted in support of these applicants. Of these, 88 were written by female letter writers and 824 were written by male letter writers. The analysis of race, as determined by the Face Secret Pro score and known background, revealed 8 Black applicants, 12 Hispanic/Latinx applicants, 36 Asian applicants, and 167 White applicants. There were 2 applications that had corrupt application photo files and could not be analyzed. The mean age for the applicants was 33 years (Table 1). The analysis revealed slightly lower USMLE Step 1 scores for females (240) as compared to males (246). Letters of recommendation word counts were slightly greater for female applicants (332 words) than for males (317 words).
      Table 1Description of Applicants, LORs, Letter Writers, and Linguistic Scores
      VariableAll

      Applicants
      Female ApplicantsMale

      Applicants
      Applicants, n (%)223 (100)51 (22.9)172 (77.1)
      Age, mean (SD)32.6 (2.4)32.5 (2.7)32.6 (2.3)
      USMLE Step 1 score, median (IQR)245 (235–252)240 (227–247)246 (237–253)
      Number of academic works, median (IQR)12 (7–22)13 (7–20)12 (7–24)
      AOA status, n (%)68 (30.5)14 (27.5)54 (31.4)
      Race
       Black, n (%)8 (3.6)08 (4.7)
       Hispanic, n (%)12 (5.4)2 (3.9)10 (5.8)
       Asian, n (%)36 (16.1)13 (25.5)23 (13.4)
       White, n (%)167 (74.9)36 (70.6)131 (76.2)
      Word count per LOR, median (IQR)320 (256–414)332 (264–433)317 (254–407)
      Letter writers, n (%)912 (100)214 (23.5)698 (76.5)
      Letter writer gender, n (%)
       Female88 (9.7)24 (11.3)64 (9.2)
       Male824 (90.4)189 (88.7)635 (90.8)
      Letter writer: Chair178 (19.5)47 (22.0)131 (18.8)
      Letter writer: Program Director283 (31.1)63 (29.6)220 (31.6)
      Letter writer: academic rank, n (%)
       Assistant Professor216 (23.7)40 (18.7)176 (25.2)
       Associate Professor245 (26.9)63 (29.4)182 (26.1)
       Full Professor331 (36.3)74 (34.6)257 (36.8)
       Private Attending120 (13.2)37 (17.3)83 (11.9)
      Linguistic scores
       Communality scores, median (IQR)0.5 (0.3–0.9)0.5 (0.3–1.0)0.5 (0.3–0.9)
       Agency scores, median (IQR)1.1 (0.7–1.6)1.2 (0.7–1.6)1.1 (0.7–1.6)
      IQR, interquartile range.
      Two models are reported for each outcome in Table 2. The first model includes race defined as Black, Hispanic, and Asian; and the second model includes race defined as White or non-White. Applicants’ communality scores decreased by 0.71% for every 1% increase in their USMLE Step 1 score (BE = -0.71; 95% CI, -1.12 to -0.30). Similarly, for every 1% increase in an applicant’s number of academic works, communality scores decreased by 0.03% (BE = -0.03; 95% CI, -0.05 to -0.003). Agency scores increased by 10.6% if the applicants were Asian (Model 3; BE = 0.10; 95% CI, 0.04–0.16) and by 6.3% if the applicants were non-White (Model 4; BE = 0.06; 95% CI, 0.01–0.11).
      Table 2Linear Mixed-Effect Model With Random Intercept and Subject-Level Clustering of Communal and Agentic Terms in LORs by Applicant Characteristics
      Beta estimates and 95% confidence intervals are reported.
      CharacteristicOutcome:

      Communality Score
      Outcome:

      Agency Score
      Model 1Model 2Model 3Model 4
      Age, years−0.004 (−0.013 to 0.006)−0.004 (−0.013 to 0.005)−0.0004 (−0.010 to 0.009)−0.001 (−0.011 to 0.009)
      Female vs male−0.003 (−0.054 to 0.049)−0.002 (−0.052 to 0.048)0.023 (−0.031 to 0.077)0.037 (−0.017 to 0.090)
      USMLE Step 1 score−0.712 (−1.123 to −0.302)−0.709 (−1.115 to −0.303)−0.044 (−0.476 to 0.389)−0.000034 (−0.002 to 0.002)
      Number of academic works−0.028 (−0.053 to −0.003)−0.028 (−0.053 to −0.003)−0.010 (−0.036 to 0.017)−0.00005 (−0.001 to 0.001)
      AOA status0.033 (−0.019 to 0.085)0.033 (−0.019 to 0.084)0.003 (−0.051 to 0.058)−0.001 (−0.056 to 0.054)
      Non-White vs White−0.032 (−0.081 to 0.017)0.061 (0.009–0.113)
      Black vs White−0.041 (−0.154 to 0.071)−0.070 (−0.189 to 0.049)
      Hispanic vs White−0.031 (−0.127 to 0.065)0.033 (−0.068 to 0.134)
      Asian vs White−0.030 (−0.088 to 0.028)0.101 (0.040–0.163)
      Model AIC444436593610
      AIC, akaike information criterion.
      Beta estimates and 95% confidence intervals are reported.
      When letter writer characteristics were modeled against communality and agency scores (Table 3), female letter writers were associated with a 12.9% increase in communality scores (BE = 0.12; 95% CI, 0.05–0.19), and attendings in private practice were associated with a 9.6% increase in agency scores (BE = 0.09; 95% CI, 0.02–0.16). The use of communal terms by female letter writers (compared to male letter writers) increased by 11.4% if the applicant was male and by 16.1% if the applicant was White (Table 4). Letters of recommendation were longer in length if the letter writer was a Department Chair versus non-Chair. Letters were also longer overall if the letter writer was a Program Director or held an Associate Professor position (Table 5).
      Table 3Linear Mixed-Effect Model With Random Intercept and Subject-Level Clustering of Communal and Agentic Terms in LORs by Letter Writer Characteristics
      Beta estimates and 95% confidence intervals are reported.
      FactorOutcome:

      Communality Score
      Outcome:

      Agency Score
      Letter writer female vs male0.121 (0.053 to 0.190)0.040 (−0.034 to 0.113)
      Chair−0.048 (−0.102 to 0.006)0.008 (−0.051 to 0.067)
      Program director−0.002 (−0.047 to 0.042)0.041 (−0.007 to 0.089)
      Letter writer academic rank, vs full professor
       Assistant professor−0.038 (−0.094 to 0.018)0.040 (−0.020 to 0.010)
       Associate Professor−0.038 (−0.091 to 0.016)0.030 (−0.028 to 0.087)
       Private attending0.053 (−0.014 to 0.119)0.092 (0.021 to 0.163)
      Model AIC461594
      AIC, akaike information criterion.
      Beta estimates and 95% confidence intervals are reported.
      Table 4Subgroup Analysis of the Use of Communal and Agentic Terms by Female Letter Writers Based on Applicant Demographics
      Beta estimates and 95% confidence intervals are reported.
      Applicant DemographicOutcome:

      Communality Score
      Outcome:

      Agency Score
      Female0.117 (−0.018 to 0.252)0.069 (0.564 to 0.753)
      Male0.108 (0.028 to 0.189)0.041 (−0.044 to 0.126)
      White0.149 (0.070 to 0.228)0.084 (−0.001 to 0.169)
      Black0.050 (−0.508 to 0.607)−0.134 (−0.576 to 0.308)
      Hispanic−0.067 (−0.326 to 0.193)−0.109 (−0.413 to 0.196)
      Asian0.054 (−0.117 to 0.226)−0.068 (−0.250 to 0.115)
      Beta estimates and 95% confidence intervals are reported.
      Table 5Difference in Average Word Count by Academic Rank, Chair Status, or Program Director Status
      Data were calculated using the Kruskal-Wallis test.
      Academic RankMedian (IQR)
      Chair vs nonchair
       Chair340.5 (271–454)
       Nonchair316 (252–404)
      PD vs nonPD
       PD328 (262–328)
       NonPD318 (252–404)
      Assistant professor vs Associate Professor vs professor vs private attending
       Assistant professor325 (262.5–399.5)
       Associate Professor343 (256–433)
       Professor310 (249–412)
       Private attending315 (249–388.5)
      IQR, interquartile range; PD, Program Director.
      Data were calculated using the Kruskal-Wallis test.

      Discussion

      Letters of recommendation remain highly influential in the residency and fellowship application process.
      • Egro F.M.
      • Vangala S.K.
      • Nguyen V.T.
      • Spiess A.M.
      Hand surgery fellowship selection criteria: a national fellowship director survey.
      Throughout the years, multiple specialties have analyzed LORs and discovered gender and racial differences based on language and word count.
      • Lin F.
      • Oh S.K.
      • Gordon L.K.
      • Pineles S.L.
      • Rosenberg J.B.
      • Tsui I.
      Gender-based differences in letters of recommendation written for ophthalmology residency applicants.
      • Filippou P.
      • Mahajan S.
      • Deal A.
      • et al.
      The presence of gender bias in letters of recommendations written for urology residency applicants.
      • Grimm L.J.
      • Redmond R.A.
      • Campbell J.C.
      • Rosette A.S.
      Gender and racial bias in radiology residency letters of recommendation.
      • Turrentine F.E.
      • Dreisbach C.N.
      • St Ivany A.R.
      • Hanks J.B.
      • Schroen A.T.
      Influence of gender on surgical residency applicants’ recommendation letters.
      • Trix F.
      • Psenka C.
      Exploring the color of glass: letters of recommendation for female and male medical faculty.
      Women, Black, and Hispanic surgeons are underrepresented in hand surgery and other subspecialties compared to the general population, as shown by the ACGME.
      • Brotherton S.E.
      • Etzel S.I.
      Graduate medical education, 2018–2019.
      Our review of 233 hand surgery fellowship applicants found similarities in the number of academic works and AOA status regardless of race and gender, indicating that applicants attained similar levels of achievement.
      • Lin F.
      • Oh S.K.
      • Gordon L.K.
      • Pineles S.L.
      • Rosenberg J.B.
      • Tsui I.
      Gender-based differences in letters of recommendation written for ophthalmology residency applicants.
      ,
      • Filippou P.
      • Mahajan S.
      • Deal A.
      • et al.
      The presence of gender bias in letters of recommendations written for urology residency applicants.
      However, USMLE Step 1 scores differed for gender, but not race. The literature reports conflicting findings, with some studies finding that women have slightly lower USMLE Step 1 scores, whereas in others they were equivalent.
      • Filippou P.
      • Mahajan S.
      • Deal A.
      • et al.
      The presence of gender bias in letters of recommendations written for urology residency applicants.
      ,
      • Grimm L.J.
      • Redmond R.A.
      • Campbell J.C.
      • Rosette A.S.
      Gender and racial bias in radiology residency letters of recommendation.
      Overall, USMLE Step 1 scores and the number of academic works were associated with the use of less communal terms, suggesting that a candidate’s objective data is consistent with more agentic terms (hardworking or competent). While USMLE scores provide objective data, the authors believe that the slight difference seen between genders is not meaningful.
      When considering agency, Asian and non-White applicants overall were associated with more agentic terms. Several studies highlighted the phenomenon of Asian applicants having similarities to White agentic scores over the years. Whites and Asians were described as having leadership potential and strong work ethics.
      • Grimm L.J.
      • Redmond R.A.
      • Campbell J.C.
      • Rosette A.S.
      Gender and racial bias in radiology residency letters of recommendation.
      When non-White applicants were analyzed as a single group (Black, Asian, and Hispanic), they were associated with higher agentic scores as well. Individually, however, Black applicants had lower agentic and communal scores. This suggests that the elevated agency scores in the non-White cohort may be due to the inclusion of Asian applicants. It may further suggest that Asians share more similar language associations in LORs with White applicants than with Black and Hispanic applicants. Furthermore, Asian applicants are not recognized by the ACGME as URMs.
      Interestingly, when comparing word count between academic rank, letter writers holding the position of a Chair wrote longer letters than non-Chairs (340.5 vs 316 words, respectively). Other studies previously reported that Chairs were shorter letter writers because of time constraints.
      • Turrentine F.E.
      • Dreisbach C.N.
      • St Ivany A.R.
      • Hanks J.B.
      • Schroen A.T.
      Influence of gender on surgical residency applicants’ recommendation letters.
      This difference from the literature may suggest that Chairs in smaller subspecialty fields may foster stronger professional relationships with candidates. While a median difference of 24 words may not seem significant, similar papers in the literature related longer letter length as more favorable; therefore, shorter letters can potentially introduce bias.
      • Trix F.
      • Psenka C.
      Exploring the color of glass: letters of recommendation for female and male medical faculty.
      ,
      • Staffa S.J.
      • Zurakowski D.
      Statistical power and sample size calculations: a primer for pediatric surgeons.
      • Greenburg A.G.
      • Doyle J.
      • McClure D.K.
      Letters of recommendation for surgical residencies: what they say and what they mean.
      • Judge T.A.
      • Higgins C.A.
      Affective disposition and the letter of reference.
      There was no association in the use of agentic or communal terms by Chairs.
      Previous studies indicate that physical appearance/attractiveness may contribute to bias. Interestingly, the hand surgery fellowship application uses photos with applications and does not require disclosure of race or ethnicity. Nicklin and Roch
      • Nicklin J.M.
      • Roch S.G.
      Biases influencing recommendation letter contents: physical attractiveness and gender1.
      found that the use of photos may contribute to bias toward the applicant when the applicant has a poor letter, but if the applicant is found to be attractive they are more likely to be successful in obtaining the desired position. Furthermore, Corcimaru et al
      • Corcimaru A.
      • Morrell M.C.
      • Morrell D.S.
      Do looks matter? The role of the electronic residency application service photograph in dermatology residency selection.
      identified gender, physical appearance, and smiling as being significantly associated with a more favorable Dermatology residency match outcome. As such, the role of photographs within hand surgery applications has the potential to introduce bias. We would propose consideration of a mandate of implicit bias training for all faculty participating in applicant selection and blinding applicant selection committees to applicant photographs prior to granting hand fellowship interviews.
      Letter of recommendation standardization is also a growing area of research interest, as highlighted in the orthopedic literature. Lin et al
      • Lin F.
      • Oh S.K.
      • Gordon L.K.
      • Pineles S.L.
      • Rosenberg J.B.
      • Tsui I.
      Gender-based differences in letters of recommendation written for ophthalmology residency applicants.
      suggests that standardization of LORs may decrease racial and gender bias, given findings of reduced subjectivity in standardized LORs when compared to traditional LORs. Furthermore, Kobayashi et al
      • Kobayashi A.N.
      • Sterling R.S.
      • Tackett S.A.
      • Chee B.W.
      • Laporte D.M.
      • Humbyrd C.J.
      Are there gender-based differences in language in letters of recommendation to an orthopaedic surgery residency program?.
      found that while the language used in letters for orthopedic applicants was similar overall, a more formalized approach to the LOR format is recommended to minimize gender bias. This may be an area to further explore for hand fellowship applications.
      Our study has several limitations. While the use of the LIWC software provided objective measures for agency and communal terms, it cannot take context or phrases with racial undertones into account. The Face Secret Pro application was flawed in identifying Hispanics with fair skin; therefore, those with a known Hispanic/Latinx origin (ie, born in a Latin country or known Latin names) were identified and recategorized. We acknowledge that due to this, applicants from any category could be misclassified. As it relates to the exploratory regression analyses conducted, a small sample size of minority candidates in this study may have led to the lack of associations seen between Black and Hispanic/Latinx applicants and agency/communality scores. Low numbers of female and URMs applying to the hand fellowship could also have been caused by the low percentages of females and URMs in orthopedic surgery (Black, 3.7%; Hispanic, 6%; female, 15%) and hand surgery (Black, 2.7%; Hispanic, 6%; female, 30%) in 2018 and 2019.
      • Brotherton S.E.
      • Etzel S.I.
      Graduate medical education, 2018–2019.
      Overall, the low numbers may suggest a systemic issue resulting in a small number of Black and Brown candidates. Keshinro et al
      • Keshinro A.
      • Frangos S.
      • Berman R.S.
      • et al.
      Underrepresented minorities in surgical residencies: Where are They? A Call to Action to Increase the Pipeline.
      evaluated the trends of general surgery residency applicants, matriculants, and graduates over a 13-year period and found significant disparities between each phase. These data suggest that there is need for reform that will encourage URMs to not only enter medical school, but also encourage application to competitive residencies, which may make attaining fellowship and leadership positions at the end of training more likely. With the current data, we are not able to determine the effects of LOR terminology on match rates. Finally, we could not control for physical appearance in applicant photos, which previous studies indicated may contribute to bias.
      Overall, our findings suggest that LORs in hand surgery demonstrate themes in the use of language in LORs based on race and gender. We observed no significant association between agentic or communal terms and applicant gender, but applicants with an Asian background were more associated with an increased use of agentic terms (hardworking, competent). Individually, Black applicants’ scores suggested the use of less communal and agentic terms. While future studies in a larger sample size are needed, the present study suggests that bias may be present in hand applicant LORs. Given that hand fellowship interviews will likely be held in an entirely virtual (or hybrid) format in the future, it is possible that more weight will be placed on LORs. Alerting letter writers to the role of implicit bias and providing tools to mitigate the use of biased language will help provide a foundation for a fair selection process and a foundation for more URMs to enter less diverse fields.

      Appendix A

      Table E1Listing of Terms Defining Agency, Communality, Race/Ethnicity, and Socioeconomic Status
      AgencyCommunalityRace/EthnicitySocioeconomic StatusPhrases
      Achieve
      wildcard operator.
      Affection
      wildcard operator.
      MinorityUnderserve
      wildcard operator.
      Community program
      Active
      wildcard operator.
      AgreeableAfrica
      wildcard operator.
      HardshipAcademic institution
      AggressiveAmiableLatin
      wildcard operator.
      Povert
      wildcard operator.
      Ambitio
      wildcard operator.
      CareUnderrepresentedPoor
      Analyt
      wildcard operator.
      CaringHispanicHumble
      Aspirat
      wildcard operator.
      CheerfulBlack
      AssertiveCompassion
      wildcard operator.
      AttentionConcern
      wildcard operator.
      Autonom
      wildcard operator.
      Considerat
      wildcard operator.
      Competen
      wildcard operator.
      Cooperat
      wildcard operator.
      CompetingDevot
      wildcard operator.
      Confiden
      wildcard operator.
      Eager
      Courage
      wildcard operator.
      Emotional
      DecisiveExpressive
      Dedicat
      wildcard operator.
      Feeling
      DefendFriend
      wildcard operator.
      DesireGentle
      DeterminationGood-nature
      wildcard operator.
      DeterminedGullible
      Force
      wildcard operator.
      Happy
      Goal orientedHelp
      wildcard operator.
      Goal-orientedKind
      Hard work
      wildcard operator.
      Likable
      Independen
      wildcard operator.
      Nurtur
      wildcard operator.
      IndustriousPassive
      Industrious
      wildcard operator.
      Sensitiv
      wildcard operator.
      Intelligen
      wildcard operator.
      Soft
      Lead
      wildcard operator.
      Soft spoken
      MasterSooth
      MasteredSympath
      wildcard operator.
      MasteryTender
      OutspokenTimid
      RewardsTo other
      wildcard operator.
      Self direct
      wildcard operator.
      Self motivated
      Self starter
      Self-assure
      wildcard operator.
      Self-direct
      wildcard operator.
      Self driven
      Self-driven
      Self-motivated
      Self-relian
      wildcard operator.
      Self-starterUnderstand
      wildcard operator.
      SeriousWarm
      Skill
      wildcard operator.
      Yield
      wildcard operator.
      Strong
      Tough
      Under pressure
      Work ethic
      Work-ethic
      wildcard operator.

      References

        • Egro F.M.
        • Vangala S.K.
        • Nguyen V.T.
        • Spiess A.M.
        Hand surgery fellowship selection criteria: a national fellowship director survey.
        Arch Plast Surg. 2017; 44: 428-433
        • Lin F.
        • Oh S.K.
        • Gordon L.K.
        • Pineles S.L.
        • Rosenberg J.B.
        • Tsui I.
        Gender-based differences in letters of recommendation written for ophthalmology residency applicants.
        BMC Med Educ. 2019; 19: 476
        • Filippou P.
        • Mahajan S.
        • Deal A.
        • et al.
        The presence of gender bias in letters of recommendations written for urology residency applicants.
        Urology. 2019; 134: 56-61
        • Grimm L.J.
        • Redmond R.A.
        • Campbell J.C.
        • Rosette A.S.
        Gender and racial bias in radiology residency letters of recommendation.
        J Am Coll Radiol. 2020; 17: 64-71
        • Turrentine F.E.
        • Dreisbach C.N.
        • St Ivany A.R.
        • Hanks J.B.
        • Schroen A.T.
        Influence of gender on surgical residency applicants’ recommendation letters.
        J Am Coll Surg. 2019; 228: 356-365.e3
        • Trix F.
        • Psenka C.
        Exploring the color of glass: letters of recommendation for female and male medical faculty.
        Discourse Soc. 2003; 14: 191-220
        • Fiske S.T.
        • Cuddy A.J.C.
        • Glick P.
        • Xu J.
        A model of (often mixed) stereotype content: competence and warmth respectively follow from perceived status and competition.
        J Pers Soc Psychol. 2002; 82: 878-902
        • Rosette A.S.
        • Koval C.Z.
        • Ma A.
        • Livingston R.
        Race matters for women leaders: intersectional effects on agentic deficiencies and penalties.
        Leadersh Q. 2016; 27: 429-445
        • Schein V.E.
        The relationship between sex role stereotypes and requisite management characteristics.
        J Appl Psychol. 1973; 57: 95-100
        • Eagly A.H.
        • Karau S.J.
        Role congruity theory of prejudice toward female leaders.
        Psychol Rev. 2002; 109: 573-598
        • Brotherton S.E.
        • Etzel S.I.
        Graduate medical education, 2018–2019.
        JAMA. 2019; 322: 996-1016
        • Accreditation Council for Graduate Medical Education (ACGME)
        ACGME Common Program Requirements (Residency).
        ACGME, Chicago, IL2020
        • Rosette A.S.
        • Leonardelli G.J.
        • Phillips K.W.
        The White standard: racial bias in leader categorization.
        J Appl Psychol. 2008; 93: 758-777
        • Katz D.
        • Braly K.
        Racial stereotypes of one hundred college students.
        J Abnorm Soc Psychol. 1933; 28: 280-290
        • Niemann Y.F.
        • Jennings L.
        • Rozelle R.M.
        • Baxter J.C.
        • Sullivan E.
        Use of free responses and cluster analysis to determine stereotypes of eight groups.
        Pers Soc Psychol Bull. 1994; 20: 379-390
        • Ghavami N.
        • Peplau L.A.
        An intersectional analysis of gender and ethnic stereotypes: testing three hypotheses.
        Psychol Women Q. 2013; 37: 113-127
        • Madera J.M.
        • Hebl M.R.
        • Martin R.C.
        Gender and letters of recommendation for academia: agentic and communal differences.
        J Appl Psychol. 2009; 94: 1591-1599
        • Tausczik Y.R.
        • Pennebaker J.W.
        The psychological meaning of words: LIWC and computerized text analysis methods.
        J Lang Soc Psychol. 2010; 29: 24-54
        • Staffa S.J.
        • Zurakowski D.
        Statistical power and sample size calculations: a primer for pediatric surgeons.
        J Pediatr Surg. 2020; 55: 1173-1179
        • Greenburg A.G.
        • Doyle J.
        • McClure D.K.
        Letters of recommendation for surgical residencies: what they say and what they mean.
        J Surg Res. 1994; 56: 192-198
        • Judge T.A.
        • Higgins C.A.
        Affective disposition and the letter of reference.
        Organ Behav Hum Decis Process. 1998; 75: 207-221
        • Nicklin J.M.
        • Roch S.G.
        Biases influencing recommendation letter contents: physical attractiveness and gender1.
        J Appl Soc Psychol. 2008; 38: 3053-3074
        • Corcimaru A.
        • Morrell M.C.
        • Morrell D.S.
        Do looks matter? The role of the electronic residency application service photograph in dermatology residency selection.
        Dermatol Online J. 2018; 24: 13030
        • Kobayashi A.N.
        • Sterling R.S.
        • Tackett S.A.
        • Chee B.W.
        • Laporte D.M.
        • Humbyrd C.J.
        Are there gender-based differences in language in letters of recommendation to an orthopaedic surgery residency program?.
        Clin Orthop Relat Res. 2020; 478: 1400-1408
        • Keshinro A.
        • Frangos S.
        • Berman R.S.
        • et al.
        Underrepresented minorities in surgical residencies: Where are They? A Call to Action to Increase the Pipeline.
        Ann Surg. 2020; 272: 512-520