Development, Testing, and Implementation

Development and Testing of the PRO-CTCAE Measurement System

PRO-CTCAE was initially developed and tested under contracts awarded to Memorial-Sloan Kettering Cancer Center (HHSN261200800043C and HHSN261201000063C), and partial support from a contract awarded to the NCI NCCCP program (HHSN261200800001E).

The National Cancer Institute’s overall goal is to support the continued development of a PRO-based measurement system to capture symptomatic adverse events by self-report in cancer clinical trials that:

  • Offers a psychometrically robust library of PRO items to support patient self-reporting;
  • Integrates smoothly into clinical trials workflow;
  • Is well-accepted by patients, clinicians and regulators;
  • Accommodates diverse linguistic preferences;
  • Support self-reporting by respondents across the developmental spectrum;
  • Supplies meaningful data to improve understanding of symptomatic AEs

Initial and continued development of the PRO-CTCAE Measurement System employs both qualitative and quantitative methodologies, and follows international guidelines and best practices, including recommendations from the US Food and Drug Administration, the PROMIS initiative, EORTC, and international outcomes research organizations, including ISPOR and ISOQOL.

Timeline of PRO-CTCAE Measurement System Development and Testing

PRO-CTCAE Measurement System: Item Development and Measurement Properties

  1. Concept Elicitation and Item Development: Trialists, clinical experts, PRO methodologists, patient advocates, and representatives from the US Food and Drug Administration identified 78 symptomatic AEs that can be meaningfully self-reported by patients. Items were developed to evaluate these symptomatic AEsExternal Web Site Policy with respect to frequency, severity, interference with daily activities, amount, presence/absence.

    PRO-CTCAE Attributes and Item Structures
    PRO-CTCAE Attributes and Item Structures
    View PRO-CTCAE Attributes and Item Structures

    Cognitive interviews were conducted to evaluate comprehension, clarity and ease of judgement in adults who were undergoing or had recently completed cancer treatment.

  2. Validity, Reliability and Responsiveness: PRO-CTCAE has demonstrated favorable validity, reliability, and responsiveness in a sample of 975 patients who had received cancer-directed therapy in the prior two weeks. PRO-CTCAE also exhibited acceptable test-retest reliability across the subset of items tested (Median ICC 0.77). There is also evidence that the 1-week recall corresponds most closely to daily reporting, supporting the validity of the 7-day recall period.

The PRO-CTCAE Item Library is comprised of 78 symptomatic adverse events, assessed by 124 items. It has been linguistically validated in more than 30 languages.

Preview the item library using the Quick Guide (PDF, 136 KB) .

More information about the development and testing of PRO-CTCAE is available in the PRO-CTCAE Resources.

Ped-PRO-CTCAE Module: Item Development and Measurement Properties

  1. Concept Elicitation and Item Development: Experienced pediatric oncology clinicians prioritized CTCAE terms with high potential salience for children and adolescents in pediatric oncology trials and amenable to child self-reporting. Trialists, clinical experts, PRO methodologists, and patient advocates employed best practices for the design of pediatric PRO measures to develop 130 Ped-PRO-CTCAE items to evaluate various attributes (including frequency, severity, interference, and presence/absence of these 62 Ped-PRO-CTCAE symptom terms.

    Ped-PRO-CTCAE* Attributes and Item Structures
    Ped-PRO-CTCAE Attributes and Item Structures

    *Note: Ped-PRO-CTCAE [Caregiver] employs comparable attributes; phrasing of items for caregiver-reporting replaces "you" with "your child"

    View Ped-PRO-CTCAE* Attributes and Item Structures

    *Note: Ped-PRO-CTCAE [Caregiver] employs comparable attributes; phrasing of items for caregiver-reporting replaces "you" with "your child"

    Subsequently, cognitive interviews were conducted with children and their caregivers in the United States to evaluate comprehension, clarity and ease of judgement.

  2. Validity, Reliability and Responsiveness: Construct validity, responsiveness, and test-retest reliability of Ped-PRO-CTCAE items has been evaluated in 482 triads (children and adolescents ages 7 to 18 years of age, their caregiver, and one of their treating clinicians) undergoing cancer treatment pediatric oncology hospitals in the United States.

    Ped-PRO-CTCAE items demonstrated strong convergent and known-groups validity, test-retest reliability, and responsiveness over time. Across age groups, Ped-PRO-CTCAE correlated with other conceptually relevant patient-reported outcome measures such as PROMIS and MSAS.

Ped-PRO-CTCAE and Ped-PRO-CTCAE [Caregiver] Item Libraries are each comprised of 62 symptomatic adverse events, assessed by 130 items. Translation and linguistic validation of Ped-PRO-CTCAE in Italian and Traditional Chinese is in progress.

Preview the item libraries using the Quick Guide (PDF, 135 KB) .

More information about the development and testing of Ped-PRO-CTCAE is available in the PRO-CTCAE Resources.

Continued Development and Implementation of the PRO-CTCAE Measurement System

PRO-CTCAE is strategically directed by a multi-disciplinary team of scientific leaders in patient-reported outcomes measurement in cancer clinical trials, drawn from across four NCI divisions. This team continues to guide the scientific development of the PRO-CTCAE Measurement System and directs efforts to scale the system for use across a wide variety of cancer clinical trials contexts, including early phase trials and precision medicine trials

The current evidence base supports the use of PRO-CTCAE to provide descriptive reporting of symptomatic toxicities, alongside CTCAE grading by clinicians. Additional work is needed to understand the utility of PRO-CTCAE in various trial designs, to transform PRO-CTCAE scores into a CTCAE grade, and to interpret PRO-CTCAE scores in making decisions at the level of the patient, or drawing conclusions at the level of the trial, with respect to dose-limiting toxicity, dosing or schedule adjustments, and overall tolerability, including comparative tolerability.

There are more than 150 early adopters of the PRO-CTCAE Measurement System, including investigators in academic settings and in industry-sponsored trials. PRO-CTCAE is being used in a variety of study designs, including treatment trials and observational studies.

Activities in the next phase of the development of the PRO-CTCAE Measurement System are designed to ensure that PRO-CTCAE fulfills the requirements of a mature patient-reported outcome (PRO) measure (see Figure 1 below), and that data derived from PRO-CTCAE are interpretable and meaningful to trialists/clinicians, patients, and regulatory agencies.

Figure 1: PRO Maturity Model

PRO-CTCAE™

This graphic shows several boxes, each representing (from top to bottom) one of the three steps in the Patient-Reported Outcome Maturity Model.

This graphic shows several boxes, each representing (from left to right) one of the three steps in the Patient-Reported Outcome Maturity Model.

I. Basic Methods / Tool Development

  • Item development
  • Conduct qualitative studies of content validity
  • Evaluate measurement properties including measurement invariance and responsiveness to change using modern psychometric techniques
  • Test in broader population, and clinical samples and subpopulations
  • Analyze and interpret the above results towards instrument refinement

II. Dissemination

  • Validate in clinical samples
  • Develop evidence to support meaningful interpretation at the group and individual level
  • Develop and validate language translations
  • Articulate standards for use (e.g. new items, retiring items, cut-points)
  • Develop outside partnerships
  • Encourage use in observational studies and clinical trials

III. Implementation & Adoption

  • Support widespread use in observational studies and clinical trials
  • Encourage use in real-world settings, comparative effectiveness research and pragmatic trials
  • Develop the business model to ensure sustainability
  • Strengthen interpretability and actionability for decision-making by clinicians, investigators, and regulators
  • Incorporate information about the use of the measure in training and educational curricula

Source: Smith AW, Mitchell SA, Deaguiar C, Moy C, Riley WT, Wagster M, Werner E. Person-Centered Outcomes Measurement: NIH-Supported Measurement Systems to Evaluate Self-Assessed Health, Function, and Symptomatic Toxicity. Translational Behavioral Medicine, 1-5. doi:10.1007/s13142-015-0345-9.

Last Updated: 13 May, 2020