Approaches to Using CAHPS Items and Composites

Guidance on analytic approaches when modeling items and composites in analyses using SEER-CAHPS data

This document is intended to provide information and guidance on analytic approaches for analysis of the Medicare CAHPS data within the SEER-CAHPS resource. This guidance is based on approaches used in various CAHPS surveys, including Hospital CAHPS (HCAHPS), Surgical CAHPS (S-CAHPS), and Medicare CAHPS. Investigators are encouraged to consider this information when conducting SEER-CAHPS studies.

Linear Mean Scoring

Linear mean scores assign sequential values from 1 to k for ordered response with k response categories. For example, never, sometimes, usually, and always are assigned initial values of 1, 2, 3, and 4. Some responses, such as those for 0-10 global ratings, are already on a numerical scale. Next, number responses are then transformed to a 0-100 scale, where 0 represents the lowest possible mean score, and 100 represents the highest possible mean score. The formula is:

Y = 100 * (X-a)/(b-a),

where Y is the transformed score, and X is on the original numerical scale that has a minimum possible value of a and a maximum possible value of b. This approach maximizes statistical power as it minimizes degrees of freedom. However, using the linear mean scoring approach also requires the assumption that the intervals between scores are equivalent (e.g., 0 to 1 and 9 to 10).

Linear mean scoring is the preferred Medicare CAHPS scoring method, as it produces the most reliable estimates and the greatest statistical power.1 Given that meaningful effect sizes can be small with Medicare CAHPS measures, adequate statistical power is a significant concern for analyses that are limited to subgroups of patients, as is the case with SEER-CAHPS data. Linear mean scoring is the approach used for Medicare CAHPS public reporting and comparing patient ratings across health plans.

References using linear mean scoring:

Elliott MN, Cohea CW, Lehrman WG, Goldstein EH, Cleary PD, Giordano LA, Beckett MK, Zaslavsky AM. Accelerating Improvement and Narrowing Gaps: Trends in Patients' Experiences with Hospital Care Reflected in HCAHPS Public Reporting. Health Serv Res 2015 Dec;50(6):1850-67. [View Abstract]External Web Site Policy

Orr N, Elliott MN, Burkhart Q, Haviland A, Weinick RM. Racial/Ethnic differences in Medicare experiences and immunization: the role of disease burden. Med Care 2013 Sep;51(9):823-31. doi: 10.1097/MLR.0b013e31829c8d77. [View Abstract]External Web Site Policy

Paddison CA, Elliott MN, Haviland AM, Farley DO, Lyratzopoulos G, Hambarsoomian K, Dembosky JW, Roland MO. Experiences of care among Medicare beneficiaries with ESRD: Medicare Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey results. Am J Kidney Dis 2013 Mar;61(3):440-9. [View Abstract]External Web Site Policy

Top-Box Scoring

Top-box scoring examines proportions of responses in the top category (top-box) responses for CAHPS composites and global ratings, with responses of 9 or 10 often used as the top category for 0-10 global ratings.2, 3 This method has been used for public reporting of HCAHPS and Home Health CAHPS (HHCAHPS). Top-box scoring has been shown to demonstrate favorable hospital-level reliability in composite and single-item measures,4 and may be easier for patients to understand.5 However, analyses of Medicare CAHPS data (which is included in the SEER-CAHPS data) using top-box scoring compared to linear mean scoring have shown reduced reliability. In general, the top-box approach reduces statistical power and reliability.

References using top-box scoring:

Indovina K, Keniston A, Reid M, Sachs K, Zheng C, Tong A, Hernandez D, Bui K, Ali Z, Nguyen T, Guirguis H, Albert RK, Burden M. Real-time patient experience surveys of hospitalized medical patients. J Hosp Med 2016 Apr;11(4):251-6. [View Abstract]External Web Site Policy

Toomey SL, Zaslavsky AM, Elliott MN, Gallagher PM, Fowler FJ Jr, Klein DJ, Shulman S, Ratner J, McGovern C, LeBlanc JL, Schuster MA. The Development of a Pediatric Inpatient Experience of Care Measure: Child HCAHPS. Pediatrics 2015 Aug;136(2):360-9. [View Abstract]External Web Site Policy

Dichotomous Scoring

Because responses for single items and composite scores in Medicare CAHPS are often at the upper end of the scales, an alternative approach involves dichotomizing global ratings and CAHPS composites at a particular cut-point.6, 7 The cut-point may be informed by the distribution of patient-level CAHPS composite scores. Previous research has used 100 vs <100 or 90-100 vs <90. There may be situations where it is more clinically relevant to use a dichotomous approach in SEER-CAHPS analyses, particularly when the CAHPS items or composites are used as predictor variables rather than outcome variables. Disadvantages are similar to using top-box scoring in increasing the risk of misclassification and limiting statistical power.

Reference using dichotomous scoring:

Halpern MT, Urato MP, Kent EE. The health care experience of patients with cancer during the last year of life: Analysis of the SEER-CAHPS data set. Cancer 2017 Jan 1;123(2):336-344. [View Abstract]External Web Site Policy

Recommendations

We suggest that researchers consider using linear mean scoring first when conducting SEER-CAHPS analyses. Other approaches, however, such as top-box scoring and dichotomizing scores, may be useful under limited circumstances. Under such circumstances, investigators are encouraged to consider their research aims to determine the most appropriate analytic approach, as well as conduct sensitivity analyses to compare methods. We suggest providing rationale to justify the specific analytic approach chosen.

References

  1. Summary of comparison of linear mean scoring with top-box scoring for MA CAHPS measures. https://ma-pdpcahps.org/globalassets/ma-pdp/technical-specifications/comparison_of_linear_mean_and_top_box_scoring_for_mcahps_final.pdf. (PDF)External Web Site Policy Accessed January 10, 2018.
  2. Hospital Consumer Assessment of Healthcare Providers and Systems: CAHPS Hospital Survey. 2017; http://www.hcahpsonline.org/External Web Site Policy. Accessed August 30, 2017.
  3. Centers for Medicare & Medicaid Services: Patient Survey (HHCAHPS) Star Ratings: Frequently asked questions 2015; 2017. https://homehealthcahps.org/Portals/0/HHCAHPS_Stars_FAQs_5_7_15.pdf (PDF)External Web Site Policy
  4. Toomey S, Zaslavsky A, Elliott M, et al.: The development of a pediatric inpatient experience of care measure: Child HCAHPS. Pediatrics. 2015;136:360-369. https://www.ncbi.nlm.nih.gov/pubmed/26195542External Web Site Policy
  5. Giordano LA, Elliott M, Goldstein E, et al.: Development, implementation, and public reporting of the HCAHPS survey. Med. Care Res. Rev. 2010;67:27-37. https://www.ncbi.nlm.nih.gov/pubmed/?term=Development%2C+implementation%2C+and+public+reporting+of+the+HCAHPS+surveyExternal Web Site Policy
  6. Halpern M, Urato M, Kent E: The health care experience of patients with cancer during the last year of life: Analysis of the SEER-CAHPS data set. Cancer. 2016;123(2):336-344. https://www.ncbi.nlm.nih.gov/pubmed/?term=The+health+care+experience+of+patients+with+cancer+during+the+last+year+of+life%3A+Analysis+of+the+SEER-CAHPS+data+setExternal Web Site Policy
  7. Jha A, Orav J, Zheng J, et al.: Patients' perception of hospital care in the United States. The New England Journal of Medicine. 2008;359:1921-1931. https://www.ncbi.nlm.nih.gov/pubmed/18971493External Web Site Policy
Last Updated: 09 Dec, 2020