If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Online patient educational materials have historically been written at a higher-than-recommended sixth grade reading level. The objectives of this study were to assess the readability of online hand surgery patient educational materials from the official online patient resource website of the American Society for Surgery of the Hand (ASSH) and to compare changes in the readability of the current ASSH online patient educational materials with those in 2008 and 2015.
Methods
An internet-based study of all 88 English language patient educational materials on HandCare.org, the official online patient resource website of the ASSH, was performed. The readability of each article was assessed using the Flesch reading ease formula, Flesch-Kincaid grade level, Coleman-Liau index, Gunning-Fog index, and Simple Measure of Gobbledygook grade level. To evaluate the trend in the readability of ASSH online hand surgery patient educational materials, the Flesch-Kincaid grade levels of articles published in 2020 were compared with those of data published in 2008 and 2015.
Results
The average Flesch reading ease score of the patient educational materials was 57.6, which is at the high-school reading level. The average reading grade level of patient educational materials ranged from 9.0 to 12.3 depending on the readability metric used. The average Flesch-Kincaid grade level of all the ASSH patient educational materials was 9.8 in 2020, which is significantly better than 10.4 in 2008 but significantly worse than 8.5 in 2015.
Conclusions
Online hand surgery patient educational materials continue to be written for the general public at a higher-than-recommended reading grade level. There has been no substantial improvement in the readability of online hand surgery patient educational materials since 2008.
Clinical relevance
Improvements are needed in the readability of online patient educational materials to ensure that patients with all health literacy levels are able to comprehend and benefit from health information.
Patients undergoing hand surgery commonly use the internet as a resource for health information. Nearly 50% of patients undergoing hand surgery research their condition prior to their initial consultation.
For some, the internet is a resource for self-diagnosis, whereas for others, online information may frame the understanding of their hand pathology and affect subsequent discussions and decision-making in the surgeon’s office.
Therefore, it is important that patients of all backgrounds have access to high-quality hand surgery online educational materials that are appropriately written for their health literacy level.
The National Institutes of Health, Centers for Disease Control and Prevention, and American Medical Association have recommended that health information be written at no higher than a sixth grade reading level
U.S. Department for Health and Human Services. Centers for Disease Control and Prevention Simply Put: A Guide for Creating Easy-To-Understand Materials.
; however, prior evidence has demonstrated that online patient educational materials for orthopedic surgery are consistently written at a higher-than-recommended reading level.
Similarly, in hand surgery, patient educational materials from the American Academy of Orthopaedic Surgeons, American Society for Surgery of the Hand (ASSH), and American Association for Hand Surgery have been shown to have inadequate readability.
Although the analysis of ASSH online patient educational materials in 2008, and subsequently in 2015, showed small improvements in readability over time, the materials were largely written at a higher-than-recommended reading level.
We aimed to perform a replication study of earlier works on the readability of online hand surgery patient educational materials from HandCare.org, the official online patient resource website of the ASSH, in order to determine the trend in the readability of ASSH online patient educational materials since 2008.
Our null hypothesis was that there has been no change in the readability of ASSH online patient educational materials from 2008 to 2020.
Materials and Methods
An internet-based readability study was performed without clinical subjects, and institutional review board approval was not required and not obtained. In April 2020, online hand surgery patient educational materials were collected from HandCare.org, the official online patient resource website of the ASSH. All articles written in the English language were included. The articles were copied in plain-text format and analyzed using an open-source software for the calculation of readability scores (ReadabilityFormulas.com, My Byline Media). Where applicable, the articles were categorized based on relevant anatomic location: hand and digit, wrist and forearm, elbow and arm, or shoulder and brachial plexus. Eight articles were classified by HandCare.org as topics related to hand safety; these topics included firework safety, gardening safety, lawn mower safety, pumpkin carving safety, snow blower safety, table saw safety, turkey carving safety, and how to remove a ring. A total of 88 unique hand surgery patient educational articles in the English language were included in this study.
The readability of each of the 88 articles was assessed using 5 validated and widely accepted readability metrics by one of the authors (D.Z.): Flesch reading ease formula,
The Flesch reading ease score ranges from 0 to 100, where higher scores represent more readable materials. A Flesch reading ease score of 90–100 is suitable for elementary school students, a score of 60–70 is suitable for middle school students, and a score of 0–30 is suitable for university students. The remaining 4 readability metrics (the Flesch-Kincaid grade level, Coleman-Liau index, Gunning-Fog index, and SMOG grade level) generate scores that correspond directly to the academic grade level required to comprehend the material. Therefore, lower scores on these readability metrics represent more readable materials. Twenty of the 88 articles were selected by a random number generator. Assessments of the readability metrics, by copying the plain-text format articles into an open-source readability score software, were performed by a second independent investigator (E.K.). The intraclass correlation coefficient among the scores of these 20 articles was used to assess the interobserver reliability of the task of calculating readability scores from the plain-text articles using the open-source readability score software.
To evaluate the trend in the readability of ASSH online hand surgery patient educational materials, a readability scale was used. The Flesch-Kincaid grade level was chosen because it is the most commonly used readability metric in the field of orthopedic surgery and has been previously used for temporal comparisons.
Descriptive statistics for the readability scores were calculated. Histograms were generated to graphically depict the distribution of readability scores obtained using each readability metric. Concordance between the readability metrics was calculated using the Pearson correlation coefficient and depicted graphically to show pairwise relationships. Multiple linear regression was used to test for differences in the readability of the articles based on anatomic location. The t test was used to test for differences in the readability of articles classified as hand safety-related educational materials because we hypothesized that these articles are written at lower reading grade levels. The t test was used to analyze differences in the readability of ASSH patient educational materials in 2020 compared with that in 2008 and 2015. A standard significance criteria of α of 0.05 was employed.
Results
Readability scores were calculated for all 88 online hand surgery patient educational articles (Table 2). The average Flesch reading ease score of the patient educational materials was 57.6, which is at the high-school reading level (Fig. 1A). The average reading grade level of the patient educational materials ranged from 9.0 to 12.3 depending on the readability metric used (Fig. 1B–E). No article was written below the sixth grade reading level, as determined by any of the readability metrics. The intraclass correlation coefficient was 99.2%, indicating excellent interobserver reliability among the study investigators.
Table 2Readability scores of Online Hand Surgery Patient Educational Materials
Figure 1Histograms demonstrating the readability of online hand surgery patient educational materials assessed using the A Flesch reading ease formula, B Flesch-Kincaid grade level, C Coleman-Liau index, D Gunning-Fog index, and E SMOG grade level.
The readability scores differed based on algorithm. The Coleman-Liau index and Gunning-Fog index consistently resulted in higher reading grade levels than the Flesch-Kincaid grade level test and SMOG grade level test. Concordance was observed between the algorithms for reading grade level used in this study (Fig. 2). Strong positive correlations were seen between the Flesch-Kincaid grade level and Gunning-Fog index, Flesch-Kincaid grade level, and SMOG grade level, between the Gunning-Fog index and SMOG grade level, and between the Coleman-Liau index and SMOG grade level. Moderate positive correlations were seen between the Flesch-Kincaid grade level and Coleman-Liau index and between the Coleman-Liau index and Gunning-Fog index.
Figure 2Scatter plots with regression lines demonstrating correlations between the A Flesch-Kincaid grade level and Coleman-Liau index, B Flesch-Kincaid grade level and Gunning-Fog index, C Flesch-Kincaid grade level and SMOG grade level, D Gunning-Fog index and Coleman-Liau index, E Gunning-Fog index and SMOG formula, and F Coleman-Liau index and SMOG grade level.
Sixty-seven of the 88 articles were categorized based on anatomic location, including 41 articles on hand and digit conditions, 11 articles on wrist and forearm conditions, 9 articles on elbow and arm conditions, and 6 articles on shoulder and brachial plexus conditions. The average (± SD) Flesch-Kincaid grade level of the articles pertaining to these anatomic locations was 9.5 ± 1.7, 9.6 ± 1.2, 9.7 ± 1.5, and 10.7 ± 1.2, respectively. The multiple linear regression showed no significant difference in the Flesch-Kincaid grade level based on anatomic location (P = .3).
The average (± SD) Flesch-Kincaid grade level of the 8 articles on hand safety was 8.2 ± 1.4. The average Flesch-Kincaid grade level of the remaining patient educational articles was 10.0 ± 1.6. This difference was statistically significant (P < .05).
The average (± SD) Flesch-Kincaid grade level of all ASSH patient educational materials was 9.8 ± 1.7 in 2020. This was significantly better than a reading grade level of 10.4 ± 1.6 in 2008 (P < .05)
From 2015 to 2020, 15 new articles were added to HandCare.org, and 4 existing articles were removed. Among the 73 articles on identical topics, paired t test showed a significant difference in the Flesch-Kincaid grade level (P < .05), and only 4 articles had the same Flesch-Kincaid grade level, suggesting that the content had been revised over time. For the 73 topics carried over from 2015 to 2020, the reading grade level was significantly higher in 2020 (9.6 ± 1.5) compared with that in 2015 (8.5 ± 1.5, P < .05). The reading grade level of the newly added educational materials was 10.7 ± 2.2, which was significantly higher than 9.6 ± 1.5 for the revised existing materials (P < .05).
Figure 3Violin plot demonstrating the reading grade level of online hand surgery patient educational materials in 2008 (n = 49), 2015 (n = 77), and 2020 (n = 88). The width of each violin represents the density of articles written at a particular reading grade level. The red line represents the NIH-recommended sixth grade reading level for health information. NIH, National Institutes of Health.
The internet has become an important source of health information for patients; however, patients with varying levels of health literacy approach online resources. Since the average adult reading grade level in the United States is at the eighth grade level, the National Institutes of Health and numerous other agencies have recommended that health information be written at the sixth grade or lower reading level to ensure appropriate readability.
U.S. Department for Health and Human Services. Centers for Disease Control and Prevention Simply Put: A Guide for Creating Easy-To-Understand Materials.
but no option has been shown to consistently provide readable materials on hand surgery conditions. The problem of poor readability of patient educational materials is pervasive and comparable with other surgical subspecialties, such as spine surgery,
Digital era of mobile communications and smartphones: a novel analysis of patient comprehension of cancer-related information available through mobile applications.
In this study of ASSH online hand surgery patient educational materials, we showed that information is written at a ninth to 12th grade reading level on an average. Of the 88 articles on HandCare.org, no article was written at the recommended sixth grade reading level, as assessed by any of the 5 readability metrics used.
Although there has been a significant improvement in the readability of online hand surgery patient educational materials since 2008, the progress has been modest and liable to fluctuation. Wang et al
demonstrated that ASSH online hand surgery patient educational materials had an average Flesch-Kincaid grade level of 10.4 in 2008. In a follow-up study, Hadden et al
demonstrated an improvement in the average reading grade level to 8.5 in 2015. In our study conducted in 2020, we demonstrated an average reading grade level of 9.8. Although this represents a modest improvement from the level in 2008, it is worse than that in 2015. Because the volume of materials available to patients has increased, it is possible that newer materials are being produced at higher-than-recommended reading levels. It is unclear whether these minor fluctuations in readability have an important effect on the comprehension of online patient educational materials by the general public. We did not find a substantial improvement in online patient educational materials in this replication study, and the materials continue to be written at a higher-than-suggested reading grade level.
A number of strategies exist for improving the accessibility of online patient health information. One strategy is the establishment of a committee to oversee online content and ensure appropriate readability. Using shorter words and sentences, avoiding unnecessary jargon, and minimizing compound sentences can help improve readability. Information on even the most complicated medical diagnoses can be effectively conveyed at the recommended reading grade level (Table 3). Another strategy is the incorporation of multimedia supplements, such as pictures or videos, which can aid in the understanding of hand conditions, regardless of reading proficiency. The use of interactive web interfaces can leverage motion, animation, and user participation to provide the end user with a customized educational experience, which may translate to better content comprehension compared with that by reading text alone.
The brachial plexus is a group of nerves that come from the spinal cord in the neck and travel down the arm. These nerves control the muscles of the shoulder, elbow, wrist and hand, as well as provide feeling in the arm. Some brachial plexus injuries are minor and will completely recover in several weeks. Other injuries are severe enough and could cause some permanent disability in the arm.
The brachial plexus is a group of nerves that travel from the neck down the arm. These nerves allow the shoulder, elbow, wrist, and hand to move. They provide a sense of feeling in the hand and arm. When injuries happen, some are minor and get better in weeks. Others are severe and can cause permanent disability of the arm.
These nerves can be damaged by stretching, pressure or cutting. Stretching can occur when the head and neck are forced away from the shoulder, such as during a motorcycle fall or car accident. If severe enough, the nerves can tear out of the spinal cord in the neck. Pressure could occur from the crushing of the brachial plexus between the collarbone and first rib, which can happen during a fracture or dislocation. Swelling in this area from excessive bleeding or injured soft tissues can also cause an injury.
These nerves can be hurt by a stretch, a squeeze, or a cut. Bad stretches can occur when the head and neck are pulled away from the shoulder. This can happen in motorcycle accidents, car accidents, or falls. If bad enough, the nerves can tear away from the neck. These nerves can also be hurt by a squeeze, such as from a broken collar bone or first rib. Bleeding or swelling in this area can also causes these nerves to get squeezed.
Nerve injuries can stop signals to and from the brain, preventing the muscles of the arm and hand from working properly, and causing loss of feeling in the area.
Nerve injuries can stop signals to and from the brain. They can cause the muscles of the arm and hand to not work. They can also cause loss of feeling in the area.
Many adult injuries will not recover on their own, and early evaluation by physicians who have experience treating these problems is essential. Some injuries can recover with time and therapy. The time for recovery can be weeks or months. When an injury is unlikely to improve, several surgical techniques can be used to improve the recovery.
Many adult injuries do not get better on their own. It is important to be seen early on by a doctor who deals with these problems. Some injuries can get better with time and exercise. This can take weeks or months. There are surgeries that can help treat bad injuries not likely to recover on their own.
To help decide which injuries are likely to recover, your physician will rely upon multiple examinations of the arm and hand to check the strength of muscles and presence of feeling in different areas. Additional testing, such as an MRI scan or CT scan/myelography, may be used. A Nerve Conduction Study/Electromyogram (NCS/EMG), a test that measures the electrical activity transmitted by nerves and muscles, may also be performed.
To judge which injuries are likely to get better, your doctor will examine muscle strength and feeling in different areas of the arm time and again. Additional tests, such as special MRI scans, CT scans, and nerve tests, that measure the electrical activity of injured nerves, may also be used.
There are several limitations to this study. First, we assessed readability using 5 validated and widely used readability algorithms. Although comprehensive, these algorithms are not exhaustive. The other metrics of readability include the Fry readability graph and New Dale-Chall score.
Second, because we analyzed the plain-text version of online patient educational materials, we were unable to account for multimedia illustrations, which may improve comprehension. To the authors’ knowledge, no tool exists that measures the overall comprehensibility of a website that accounts for illustrations and videos, although there may be a role for further studies of the patient-perceived utility of educational materials.
Third, we assessed readability at 1 time point, and we were unable to comment on whether or how frequently online materials changed from 2008 to 2020. Finally, any metric of readability is algorithmic, and scores are generally calculated based on the number of letters, syllables, words, complex words, or sentences, but these metrics are unable to account for the nuances of medical jargon. For example, the words “gout” and “pseudogout” describe similar conditions, with comparable medical sophistication; however, many readability algorithms would consider “gout” a simple word but “pseudogout” a complex word because it has 3 or more syllables. Although these algorithms are validated measures of reading grade level, their correlation with patients’ perception of the utility of educational materials has not been demonstrated.
Our findings suggest that online hand surgery patient educational materials, even those published by our national society, are written for the general public at too high a reading grade level. No substantial improvements in the readability of online hand surgery patient educational materials have been made since 2015. Our results support a need for continual readability assessments of patient educational materials from official organizations to ensure that patients with all levels of health literacy levels are able to comprehend and benefit from online information. Future research is needed to study the effects of multimedia content, interactive modules, and social-networking features on the effective delivery of health information.
Acknowledgments
The authors acknowledge Dr Jia Zhu for her help in creating the violin plot.
References
Rao A.J.
Dy C.J.
Goldfarb C.A.
Cohen M.S.
Wysocki R.W.
Patient preferences and utilization of online resources for patients treated in hand surgery practices.
Digital era of mobile communications and smartphones: a novel analysis of patient comprehension of cancer-related information available through mobile applications.
00202-1&id=gr1.jpg){kind=link}
00202-1&id=gr2.jpg){kind=link}
00202-1&id=gr3.jpg){kind=link}