Development, Testing, and Implementation

Development and Testing of the PRO-CTCAE Measurement System

PRO-CTCAE was initially developed and tested under contracts awarded to Memorial-Sloan Kettering Cancer Center (HHSN261200800043C and HHSN261201000063C), and partial support from a contract awarded to the NCI NCCCP program (HHSN261200800001E).

The National Cancer Institute’s overall goal is to support the continued development of a PRO-based measurement system to capture symptomatic adverse events by self-report in cancer clinical trials that:

• Offers a psychometrically robust library of PRO items to support patient self-reporting;
• Integrates smoothly into clinical trials workflow;
• Is well-accepted by patients, clinicians and regulators;
• Accommodates diverse linguistic preferences;
• Support self-reporting by respondents across the developmental spectrum;
• Supplies meaningful data to improve understanding of symptomatic AEs

Continued Development and Implementation of the PRO-CTCAE Measurement System

Initial and continued development of the PRO-CTCAE Measurement System employs both qualitative and quantitative methodologies, and follows international guidelines and best practices, including recommendations from the U.S. Food and Drug Administration, the PROMIS initiative, EORTC, and international outcomes research organizations, including ISPOR and ISOQOL.

PRO-CTCAE Measurement System: Item Development and Measurement Properties

1. Concept Elicitation and Item Development: Trialists, clinical experts, PRO methodologists, patient advocates, and representatives from the U.S. Food and Drug Administration identified 78 symptomatic AEs that can be meaningfully self-reported by patients. Items were developed to evaluate these symptomatic AEs with respect to frequency, severity, interference with daily activities, amount, presence/absence. Cognitive interviews were conducted to evaluate comprehension, clarity and ease of judgement in adults who were undergoing or had recently completed cancer treatment.

   PRO-CTCAE Attributes and Item Structures

   Frequency

   In the last 7 days, how often did you have ____?

   • Never
   • Rarely
   • Occasionally
   • Frequently
   • Almost constantly

   Severity

   In the last 7 days, what was the severity of your ____ at its worst?

   • None
   • Mild
   • Moderate
   • Severe
   • Very severe

   Interference

   In the last 7 days, how much did ____ interfere with your usual or daily activities?

   • Not at all
   • A little bit
   • Somewhat
   • Quite a bit
   • Very much

   Amount

   In the last 7 days, did you have any ____?

   • Not at all
   • A little bit
   • Somewhat
   • Quite a bit
   • Very much

   Presence / Absence

   In the last 7 days, did you have any ____?

   • No
   • Yes

2. Validity, Reliability and Responsiveness: PRO-CTCAE has demonstrated favorable validity, reliability, and responsiveness in a sample of 975 patients who had received cancer-directed therapy in the prior two weeks. PRO-CTCAE also exhibited acceptable test-retest reliability across the subset of items tested (Median ICC 0.77). There is also evidence that the 1-week recall corresponds most closely to daily reporting, supporting the validity of the 7-day recall period.

The PRO-CTCAE Item Library is comprised of 78 symptomatic adverse events, assessed by 124 items. It has been linguistically validated in more than 30 languages.

Preview the item library using the Quick Guide (PDF, 124 KB).

More information about the development and testing of PRO-CTCAE is available in the PRO-CTCAE Resources.

Ped-PRO-CTCAE Module: Item Development and Measurement Properties

1. Concept Elicitation and Item Development: Experienced pediatric oncology clinicians prioritized CTCAE terms with high potential salience for children and adolescents in pediatric oncology trials and amenable to child self-reporting. Trialists, clinical experts, PRO methodologists, and patient advocates employed best practices for the design of pediatric PRO measures to develop 130 Ped-PRO-CTCAE items to evaluate various attributes (including frequency, severity, interference, and presence/absence of these 62 Ped-PRO-CTCAE symptom terms. Subsequently, cognitive interviews were conducted with children and their caregivers in the United States to evaluate comprehension, clarity and ease of judgement.

   Ped-PRO-CTCAE* Attributes and Item Structures

   Frequency

   How often did you have ____?

   • Never
   • Sometimes
   • Most of the time
   • Almost all the time

   Severity

   How bad was your ____?

   • Did not have any
   • A little bad
   • Bad
   • Very bad

   Interference

   How much did ____ keep you from doing things you usually do?

   • Not at all
   • Some
   • A lot
   • A whole lot

   Presence / Absence

   Did you have ____?

   • No
   • Yes
   • I don't know

   *Note: Ped-PRO-CTCAE [Caregiver] employs comparable attributes; phrasing of items for caregiver-reporting replaces "you" with "your child"

2. Validity, Reliability and Responsiveness: Construct validity, responsiveness, and test-retest reliability of Ped-PRO-CTCAE items has been evaluated in 482 triads (children and adolescents ages 7 to 17 years of age, their caregiver, and one of their treating clinicians) undergoing cancer treatment pediatric oncology hospitals in the United States.

   Ped-PRO-CTCAE items demonstrated strong convergent and known-groups validity, test-retest reliability, and responsiveness over time. Across age groups, Ped-PRO-CTCAE correlated with other conceptually relevant patient-reported outcome measures such as PROMIS and MSAS.

Ped-PRO-CTCAE and Ped-PRO-CTCAE [Caregiver] Item Libraries are each comprised of 62 symptomatic adverse events, assessed by 130 items. Ped-PRO-CTCAE and Ped-PRO-CTCAE [Caregiver] have been linguistically validated in Italian and Simplified Chinese, and validation of additional language translations is in progress.

Preview the item libraries using the Quick Guide (PDF, 174 KB).

More information about the development and testing of the Ped-PRO-CTCAE module is available in the PRO-CTCAE Resources.

Continued Development and Implementation of the PRO-CTCAE Measurement System

PRO-CTCAE is strategically directed by a multi-disciplinary team of scientific leaders in patient-reported outcomes measurement in cancer clinical trials, drawn from across four NCI divisions. This team continues to guide the scientific development of the PRO-CTCAE Measurement System and directs efforts to scale the system for use across a wide variety of cancer clinical trials contexts, including early phase trials and precision medicine trials.

The current evidence base supports the use of PRO-CTCAE to provide descriptive reporting of symptomatic toxicities, alongside CTCAE grading by clinicians. Additional work is needed to understand the utility of PRO-CTCAE in various trial designs, to transform PRO-CTCAE scores into a CTCAE grade, and to interpret PRO-CTCAE scores in making decisions at the level of the patient, or drawing conclusions at the level of the trial, with respect to dose-limiting toxicity, dosing or schedule adjustments, and overall tolerability, including comparative tolerability.

There are more than 150 early adopters of the PRO-CTCAE Measurement System, including investigators in academic settings and in industry-sponsored trials. PRO-CTCAE is being used in a variety of study designs, including treatment trials and observational studies.

Activities in the next phase of the development of the PRO-CTCAE Measurement System are designed to ensure that PRO-CTCAE fulfills the requirements of a mature patient-reported outcome (PRO) measure (see Figure 1 below), and that data derived from PRO-CTCAE are interpretable and meaningful to trialists/clinicians, patients, and regulatory agencies.

PRO Maturity Model

Source: Smith AW, Mitchell SA, Deaguiar C, Moy C, Riley WT, Wagster M, Werner E. Person-Centered Outcomes Measurement: NIH-Supported Measurement Systems to Evaluate Self-Assessed Health, Function, and Symptomatic Toxicity. Translational Behavioral Medicine, 1-5. doi:10.1007/s13142-015-0345-9.