If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Minimal Clinically Important Difference for PROMIS Physical Function and Pain Interference in Patients Following Surgical Treatment of Distal Radius Fracture
Corresponding author: Warren C. Hammert, MD, Professor of Orthopaedic Surgery and Plastic Surgery, Department of Orthopaedics and Rehabilitation, University of Rochester Medical Center, 601 Elmwood Ave. Box 665, Rochester, NY 14642.
We estimated the minimal clinically important difference (MCID) for the Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function (PF) and Pain Interference (PI) computer adaptive tests (CATs) following surgical treatment of distal radius fracture (DRF).
Methods
Adult patients surgically treated between November 2017 and November 2020 for isolated DRF were identified. Demographic and patient-reported outcome data were extracted from the electronic health record. Outcomes of interest were the PROMIS PF and PI CATs. Inclusion criteria were met if: (1) PROMIS PF and PI scores were available at preoperative and postoperative visits; and (2) a postoperative clinical anchor question asking about overall response to treatment was answered. An anchor-based MCID estimate was determined by calculating the average absolute score change in PROMIS PF and PI for patients who indicated a mild change to the anchor question. A distribution-based MCID estimate was also calculated using the standard error of measurement and effect sizes of change.
Results
The changes in PROMIS PF and PI scores were significantly different between patients who gave responses of much change (n = 73), mild change (n = 51), and no change (n = 19) to the clinical anchor question. The average score changes in the mild change group for PROMIS PF and PI were 5.2 (SD, 3.7) and 6.8 (SD, 4.3) points, respectively, representing the anchor-based MCID estimates. The PROMIS PI anchor-based estimate was moderately correlated with the preoperative score (r = −0.41), time between visits (r = −0.39), and age (r = 0.30). The distribution-based MCID estimates were 3.8 (SD, 1.3) and 3.7 (SD, 1.3) points for the PROMIS PF and PI, respectively.
Conclusions
The MCIDs were estimated as 5.2 and 6.8 for the PROMIS PF and PI CATs, respectively, following surgery for DRF.
Clinical relevance
As reports continue to publish a consistent range of MCID values, researchers can be confident in these values and begin using them across a broader spectrum of conditions treated by hand surgeons.
Patient-reported outcome (PRO) measures are being adopted by practitioners to capture the impact of disease on patient health status and determine the effectiveness of treatment. By providing a patient-centered view of current health status, PRO measures can promote shared clinical decision-making and guide expectations following treatment.
Commonly used region- and condition-specific PRO measures in hand surgery include the Disabilities of the Arm, Shoulder, and Hand questionnaire (DASH),
Development of an upper extremity outcome measure: the DASH (disabilities of the arm, shoulder and hand) [corrected]. The Upper Extremity Collaborative Group (UECG).
The shortened disabilities of the arm, shoulder and hand questionnaire (QuickDASH): validity and reliability based on responses within the full-length DASH.
These questionnaires are validated and reliable measures of hand function. However, barriers exist preventing their full adoption and standardization across hand surgery. Several authors have reported concern regarding their narrow scopes, administrative burdens, and notable ceiling and floor effects.
In 2004, the PROMIS was developed by the National Institutes of Health to improve the overall reporting of physical, mental, and social health in a maximally precise and efficient manner.
The PROMIS is a general PRO measure that uses item response theory (IRT) and computerized adaptive testing (CAT), which have the benefits of decreased respondent and administrative burdens while maintaining accuracy and reproducibility.
When compared to legacy instruments, the PROMIS has performed favorably by demonstrating excellent reliability (ie, reproducibility), responsiveness (ie, ability to detect score changes), one-dimensionality (ie, low unexplained variance), and coverage (ie, minimal ceiling and floor effects).
The PROMIS is increasingly being used to evaluate outcomes following hand/wrist procedures. To understand the clinical implications of postoperative score changes over time, it is important to know the minimal clinically important difference (MCID), which represents the smallest change in an outcome score that a patient may perceive as clinically meaningful (either beneficial or harmful).
The MCID is an intrinsic property of a PRO measure, and variation in estimates of the MCID reflects the responsiveness of the PRO measure to the different conditions studied. MCID calculations are most often performed with anchor- or distribution-based methods. Anchor-based methods compare changes in PRO scores to an external, independent assessment of clinical change,
whereas distribution-based methods use statistical calculations based on the variability in scores from the sampled population to determine the level of change that is beyond random occurrence.
The anchor-based method is generally regarded as the preferred approach because it is patient-centered; however, both methods have their limitations, and it has therefore been recommended that MCID estimates be based on multiple approaches, ideally resulting in values that converge on a single value or narrow range of values.
Minimal clinically important differences for PROMIS physical function, upper extremity, and pain interference in carpal tunnel release using region- and condition-specific PROM tools.
While there is no consensus regarding the amount of data necessary prior to establishing the true MCID for a PRO measure, there is no doubt that additional data extend confidence in the previously reported values. Therefore, the objective of this study was to estimate the MCID for PROMIS PF and PI CATs in patients surgically treated for DRF using an anchor- and distribution-based approach.
Materials and Methods
Institutional review board approval was obtained from the University of Rochester Medical Center to review our longitudinally maintained PROMIS database. Patients who presented to our hand clinic between November 2017 and November 2020 for elective outpatient surgical treatment of isolated DRF were identified using Current Procedural Terminology (CPT) billing codes 25607, 25608, and 25609. Demographic and PRO data were automatically extracted from the electronic health record. Inclusion criteria were patients 18 years of age and older with DRF who were treated with surgery using open reduction and internal fixation. In addition, PROMIS data had to be available at both preoperative (<21 days before surgery) and postoperative (<365 days after surgery) visits. Patients who failed nonoperative management and initially presented beyond 21 days prior to surgery were excluded. The postoperative visit closest to the 6-week follow-up date was chosen for analysis. A clinical anchor question also had to be completed at the same postoperative visit as the PROMIS score selected to be included in the analysis. A flowchart diagram detailing our inclusion and exclusion criteria is illustrated in Figure 1.
Figure 1Flowchart illustrating inclusion and exclusion criteria.
The PROMIS CATs of interest were PF (versions 1.2 and 2.0) and PI (version 1.1). As part of routine clinical care, these CATs were administered to patients at office visits using an Apple iPad. During the study period, we transitioned from using PROMIS PF version 1.2 to version 2.0; however, scores across these versions can be compared to each other.
The PROMIS PF CAT measures self-reported physical capability, whereas the PROMIS PI CAT measures the degree to which pain interferes with physical, mental, and social activities.
Higher PROMIS PF scores indicate better physical capability, whereas lower PROMIS PI scores indicate better pain coping. PROMIS scores are reported as a t score, which is a standardized score referenced against the general US population.
To independently assess patient-perceived clinical changes, the following anchor question was asked to each patient at their postoperative visits: “since your first visit to the provider you are seeing today, has your condition treated by this provider changed?” Responses to the anchor question were limited to a scale from 0 to 5, with 0 meaning “I am seeing the physician for a new problem,” 1 indicating the condition was “much better,” 2 that the condition was “mildly better,” 3 that there was “no change,” 4 that the condition was “mildly worse,” and 5 that the condition was “much worse.”
Statistical analysis
Descriptive statistics were calculated. Continuous variables are reported as means ± SDs and categorical variables are presented as frequencies and percentages. Statistical significance was set at a P value <.05.
Analysis of variance with Tukey’s post hoc testing was done for significant pairwise comparisons to determine whether the average absolute changes in PROMIS PF and PI t scores were significantly different between visits when patients reported much change (eg, “much better” or “much worse”), mild change (eg, “mildly better” or “mildly worse”), and “no change.” This allowed us to determine whether numerical changes in PROMIS scores reflected a true change as perceived by the patient. The average change in score among patients reporting mild change was chosen as the anchor-based MCID estimate for the respective PROMIS domain.
Bivariate statistical testing was performed to identify patient variables affecting the anchor-based MCID estimate. Among patients who reported mild change since their preoperative visit, Spearman’s correlation coefficients were calculated to determine the association between the change in PROMIS PF and PI scores and the preoperative PROMIS score for the respective PROMIS domain, number of days between visits, age, and body mass index (BMI).
A distribution-based estimate was also calculated using methods originally described by Yost et al.
Minimally important differences were estimated for six Patient-Reported Outcomes Measurement Information System–cancer scales in advanced-stage cancer patients.
Minimally important differences were estimated for the Functional Assessment of Cancer Therapy–Colorectal (FACT-C) instrument using a combination of distribution- and anchor-based approaches.
These methods rely on the statistical distribution of PRO data, including standard error of measurement (SEM) and effect size measures. The SEM, which represents the minimal detectable change (MDC) in a scale, reflects the smallest score change likely to be a true change rather than measurement error. In IRT, each t score is associated with a standard error. Therefore, using the previous 3,900 patient encounters at our hand clinic, the MDC was calculated as the average standard error across the PROMIS PF and PI CATs. The MDC values for the PROMIS PF and PI CATs were estimated to be 2.4 and 2.2, respectively. To be clinically relevant, the MCID estimate must exceed the MDC. Next, an effect size filter was applied to the anchor-based MCID t scores by taking the change in PROMIS score and dividing by the baseline SD of the sample.
Only score differences corresponding to effect sizes between 0.2 and 0.8 were considered in the final distribution-based MCID estimate. Score differences corresponding to effect sizes less than 0.2 (Cohen’s small effect size) were deemed unlikely to be clinically important, whereas scores differences corresponding to effect sizes greater than 0.8 (Cohen’s large effect size) were deemed unlikely to be minimal.
Results
Baseline patient characteristics are summarized in Table 1. Of the 143 patients that met the inclusion criteria, most were female (n = 112; 78.3%), White (n = 132; 90.2%), and non-Hispanic (n = 119; 83.2%). The average patient age was 56.6 years old (SD, 16.3 years), and most patients were nonsmokers (n = 91; 63.6%) and not obese (n = 92; 64.3%).
Patients presented to our clinic at a median of 5 days (interquartile range [IQR], 3–6 days) prior to surgery. Preoperative PROMIS PF scores averaged 33.8 (SD, 8.1), with a range of 22.8 to 66.6, and preoperative PROMIS PI scores averaged 65.1 (SD, 7.1), with a range of 38.7 to 80.1.
Patients had a postoperative visit with clinical anchor data recorded at a median of 34 days (IQR, 11–49 days) after surgery. The average postoperative PROMIS PF score was 39.4 (SD, 8.9; range, 23.2–66.0), while the average postoperative PROMIS PI score was 57.5 (SD, 8.1; range, 38.7–77.8).
The changes in PROMIS PF and PI scores between visits were significantly different between patients who reported much change (n = 73), mild change (n = 51), and no change (n = 19) to the clinical anchor question (P <.05). Using an anchor-based approach for estimating the MCID, the average changes in PROMIS PF and PI scores for the mild change group were +5.2 (SD, 3.7) and −6.8 (SD, 4.3), respectively (Tables 2 and 3); thus, both MCID anchor-based estimates exceed the MDC level of instrument error (PROMIS PF MDC, +2.4; PROMIS PI MDC, −2.2).
Table 2Average Change in PROMIS Physical Function According to the Clinical Anchor Response
Among patients who reported a mild change since their preoperative visit, using Spearman’s correlation coefficient, the degree of score change for the PROMIS PF CAT was not correlated with the preoperative score (r = −0.19; P =.25), age (r = −0.23; P =.46), BMI (r = 0.04; P =.79), nor time between visits (r = −0.07; P =.68). For the PROMIS PI CAT, no correlation was observed between the degree of score change and BMI (r = 0.17; P =.24); however, weak to moderate correlations were observed between the degree of score change and the preoperative score (r = −0.41; P =.003), time between visits (r = −0.39; P =.005), and age (r = 0.30; P =.04).
A distribution-based MCID estimate was also calculated for the PROMIS PF and PI CATs. After applying effect size parameters of 0.2 to 0.8 to data from patients who reported a mild change since their preoperative visit, distribution-based MCID values were estimated to be +3.8 (SD, 1.3) and −3.7 (SD, 1.3) for the PROMIS PF and PI, respectively (Tables 4 and 5). These distribution-based MCID estimates also exceed the MDC values of +2.4 for the PROMIS PF and −2.2 for the PROMIS PI.
Table 4Average Change in PROMIS Physical Function After Distribution Correction
This study estimates, using an anchor- and distribution-based approach, the MCID values for the PROMIS PF and PI CATs in patients surgically treated for DRF. The range of estimated MCID values for the PROMIS PF CAT was +3.8 to +5.2 points, while the range of estimated MCID values for the PROMIS PI CAT was −3.7 to −6.8 points. These estimates were greater than the MDC, and therefore likely reflect a true change rather than measurement error. An important point in looking at MCID and MDC is the direction of change. While the common thought is that the MCID shows improvement, the change can indicate improvement or worsening. Similarly, the direction of the MDC can be up or down and represent an improvement or worsening change (depending on the specific PROMIS domain) that was noticeable. We believe this should be taken into consideration when looking at values, planning prospective studies, and determining power or sample size.
The estimates reported in our study compare favorably to previously reported MCID values for the PROMIS PF and PI CATs in the hand literature. Sandvall et al
reported a range of MCID values for PROMIS PF between 3.6 and 4.6 points in patients treated nonsurgically for DRF. Similarly, in a cohort of patients treated both surgically and nonsurgically for thumb carpometacarpal arthritis, Lee and Calfee
reported a range of MCID values for PROMIS PF between 3.5 to 3.9 points. Based on validated region-specific (MHQ) and condition-specific (BCTQ) PRO measures, Bernstein et al
Minimal clinically important differences for PROMIS physical function, upper extremity, and pain interference in carpal tunnel release using region- and condition-specific PROM tools.
provided MCID estimates for the PROMIS PF (1.8 to 2.8 points) and PI (−4.1 to −9.7 points) for patients undergoing carpal tunnel release. Alternatively, using a half-SD distribution-based method, Kazmers et al
calculated MCID values of 4.6 and −3.4 for the PROMIS PF and PI CATs in patients with hand and elbow conditions. Our findings seem comparable, particularly given that we looked at a traumatic injury that likely resulted in an acute decline in function and greater pain interference at the initial evaluation and thus would be expected to show greater improvement following treatment when compared to a condition that the patient had potentially adapted to over time. The fact that our estimates fall within a narrow range of previously reported values strongly suggests that a true MCID value exists for the PROMIS PF and PI scales. This supports the notion that the MCID is not a fluid value, but rather an intrinsic value of a PRO measure. The extent to which estimates of the MCID vary purely reflects varying degrees of responsiveness of the PRO measure to the different hand conditions investigated.
Prior studies in the literature have suggested that MCID estimates are not influenced by the severity of patients’ baseline scores nor the length of time since treatment.
Similarly, our study found that higher preoperative PROMIS PI scores were moderately associated with the MCID estimate. These findings reflect the ability of the PROMIS scale to detect greater changes among patients with worse baseline scores. In addition, our MCID estimate for PROMIS PI seemed to be influenced by the length of time between the preoperative and postoperative visits. Consequently, future work may be needed to determine whether our PROMIS PI estimate can be applied to patients outside of the 6-week time frame.
Estimating the MCID for the PROMIS CATs is valuable in understanding the instruments’ ability to detect clinically meaningful changes in outcome scores over time. Furthermore, MCID estimates can be used to evaluate treatment effectiveness between 2 cohorts, as well as to sufficiently power a clinical study.
However, summarized by the concept of responsiveness, MCID estimates for a PRO measure cannot be used blindly across all clinical applications and patient samples. Studies have shown that MCID estimates for PRO measures are dependent on the clinical context in which they were initially calculated.
This includes patient characteristics of the sample population (eg, socioeconomic status, patient expectations) and the clinical picture (eg, disease type, intervention). For example, a large estimated MCID value for a certain hand condition may indicate that the PRO measure is not sufficiently sensitive to clinical change for that hand condition; therefore, a different, more responsive measure may be required to enhance the efficiency of a clinical study. Alternatively, in the case of a less responsive PRO measure, a greater number of subjects may be required in order to detect significant differences between groups of patients. Consideration for these factors should be made before applying an MCID estimate in clinical research. Future research can look at the responsiveness of PROMIS for a given condition as the MCID ranges seem to be established for a variety of traumatic and degenerative conditions in the hand and wrist.
Finally, some authors have proposed that MCID values be used to evaluate clinical changes of individual patients.
However, from our experience as well as that of others, we believe that the best application of MCID values is in the evaluation of PRO scores at the group level.
At the individual level, we observed a significant variation in scores among patients reporting their condition as “mildly better” or “mildly worse,” and therefore the MCID estimate would likely not be a reliable threshold for individual clinical change following surgical treatment of DRF. For individual patients, following the direction of change seems to be more meaningful.
This study has several limitations. First, our MCID estimates are derived from a patient cohort that presented to a single hand clinic for elective outpatient surgery for DRF. Patients were predominately female, White, and non-Hispanic. In addition, our anchor-based estimate was based on PROMIS data and clinical anchor responses from just 51 patients reporting mild changes. These factors should be considered before applying our MCID estimates to a larger, more diverse patient population. Furthermore, while our MCID estimates fall within the range of previously reported estimates for the PROMIS PF and PI CATs, they may not correlate to all patients with operatively treated DRF, particularly those with open injuries or other associated injuries in the same extremity. Second, anchor-based MCID estimates are prone to recall bias because patients are asked to compare their current, posttreatment health status to their pretreatment state. We strived to use postoperative data from the 6-week follow-up visit; however, for patients without these data available, data from earlier or later postoperative time points were chosen. This was to select a relatively uniform time point when most patients are still in treatment but have recovered enough to detect change. Third, our clinical anchor question asked about overall health status when compared to the pretreatment period. Because our anchor question did not explicitly identify the target PRO measure (ie, PF or PI), it is possible that responses to our anchor did not truly reflect the clinical outcome that the PRO instrument intended to measure. However, we used an anchor question that has been previously reported in the hand literature.
Lastly, while we determined the MCID for the PROMIS PF and PI CATs, the PROMIS Upper Extremity (UE) CAT is also relevant to hand surgery. At the time the patients in our study were initially treated at our hand clinic, the PROMIS UE CAT was not routinely collected. As hand surgeons work toward a consensus regarding a primary PRO measure to use, future work may need to estimate the MCID as well as responsiveness to the PROMIS UE CAT in patients surgically treated for DRF.
In conclusion, this study adds to the growing hand literature defining an approximate 3- to 6-point score change on the PROMIS PF and PI CATs as the MCID. As studies continue to report a consistent range of MCID values for patients seeking hand care, researchers can gain confidence in these estimates and begin using them across a broader spectrum of conditions treated by hand surgeons.
References
Karl J.W.
Olson P.R.
Rosenwasser M.P.
The epidemiology of upper extremity fractures in the United States, 2009.
Development of an upper extremity outcome measure: the DASH (disabilities of the arm, shoulder and hand) [corrected]. The Upper Extremity Collaborative Group (UECG).
The shortened disabilities of the arm, shoulder and hand questionnaire (QuickDASH): validity and reliability based on responses within the full-length DASH.
Minimal clinically important differences for PROMIS physical function, upper extremity, and pain interference in carpal tunnel release using region- and condition-specific PROM tools.
Minimally important differences were estimated for six Patient-Reported Outcomes Measurement Information System–cancer scales in advanced-stage cancer patients.
Minimally important differences were estimated for the Functional Assessment of Cancer Therapy–Colorectal (FACT-C) instrument using a combination of distribution- and anchor-based approaches.
00600-6&id=gr1.jpg){kind=link}