Frequently Asked Questions

Visit the Development, Testing, and Implementation page for more information.

Selecting which symptomatic adverse events to assess and determining the time points for measurement are critical trial design decisions. In trials developed to date, items from the PRO-CTCAE or from Ped-PRO-CTCAE and the time points of measurement have generally been specified using an approach similar to that used to define the adverse event surveillance plan for the trial more broadly. That is, the study team reviews published data, as well as data from earlier phase trials or animal models, if available, and incorporates information about the known or anticipated on- and off-target effects of agents in a similar mechanistic class, to identify those symptomatic adverse events likely to be associated with the regimens in the trial.

After establishing a pre-treatment baseline, more frequent PRO-CTCAE or Ped-PRO-CTCAE administration is generally warranted during the first few cycles of therapy (e.g., weekly reporting during the initial several months of therapy). Thereafter, the assessment intervals may be extended (e.g., monthly or quarterly, depending on the regimen under study), particularly in trial contexts where the duration of investigational treatment is prolonged or where toxicities are anticipated to have stabilized. In summary, the time points of measurement should reflect the anticipated pattern of toxicity, and the scientific objectives of the trial.

At this time, we cannot recommend use of longer or shorter PRO-CTCAE or Ped-PRO-CTCAE recall periods. The standard PRO-CTCAE recall period is “the past 7 days”. A recent study suggests that longer recall periods (2-, 3- or 4- week recall) are associated with small but successively increasing measurement error. This must be considered if recall periods longer than 1 week are going to be used in a trial for logistical reasons.

Based on our findings, PRO-CTCAE items are completed rapidly by adults. We estimate that a 20-item PRO-CTCAE survey would take on average 3.4 minutes by paper, 3.7 minutes by web, and 5.4 minutes by interactive voice response system (IVRS).

Research in adults also indicates that there is no evidence of clinically meaningful variation in completion times by participant characteristics, including impairments in physical or cognitive functioning (Mode equivalence and acceptability of tablet computer-, interactive voice response system-, and paper-based administration of the U.S. National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).). These findings suggest that completion of PRO-CTCAE items by adults is generally not laborious, even for those respondents who may have some degree of functional limitation.

Estimates of the total time for adults to complete a survey by each mode depend upon the number of items presented to the respondent. As an additional consideration, because paper surveys do not incorporate conditional branching patterns, as do web-based and IVRS surveys, a participant completing a paper survey would likely have to complete more PRO-CTCAE items, compared to the same participant completing PRO-CTCAE using an electronic version of the questions.

As an example of an estimate of respondent burden in adults for human subjects research applications, 75% of respondents would be able to complete twenty items in 3.8 minutes by web, in 6.1 minutes by IVRS, and in 4.3 minutes by paper.

There is no clear guidance from the literature or from regulatory agencies that when establishing an electronic system to collect PRO-CTCAE responses, the system should require the respondent to provide a response before they are able to progress within the survey.

In deciding whether to constrain an electronic PRO collection system in a manner that allows respondents to decline to answer a question (i.e., to skip a question or a whole set of questions but continue with the survey), it is important to consider three aspects. These are: (a) the possible effects on the integrity of the PRO-CTCAE conditional branching logic; (b) the impact on data quality; and (c) human subjects considerations.

Allowing respondents to choose not to provide an answer to a question and continue with the survey could have adverse effects on the integrity of the PRO-CTCAE conditional branching logic. Allowing respondents to skip an entire set of questions for a specific symptomatic adverse event could also result in considerable missing data. This is because data will not only be missing for the specific item that was skipped, but data will also be missing for the other PRO-CTCAE items that branch from that item. At the same time, requiring that the respondent provide a response before progressing within the survey could serve to produce missing data because of participant withdrawal. It is also possible that if you require respondents to complete each item, that they will provide a random response or just abandon your survey entirely.

There may also be human subjects considerations in requiring respondents to provide a response. Consultation with your Institutional Review Board is important in weighing this study design decision. For some PRO-CTCAE questions that address sensitive topics (e.g., PRO-CTCAE items that address sexual function), there are response options that allow the respondent to indicate if they would prefer not to respond or if they deem a question as not applicable (for example, because they are not sexually active). These response choices should be explicitly represented in the dataset, and not be coded as missing.

Perhaps the overall best approach that serves to balance both respondent burden and data quality may be to constrain the electronic data capture system to require responses to all questions, while simultaneously implementing conditional branching to conserve respondent burden. Keep in mind, that if one chooses to allow respondents to skip individual items or to skip whole PRO-CTCAE item sets, that lack of response should be explicitly represented in the dataset as ‘missing, participant did not answer’.

In summary, if having missing data will cause significant difficulties with data quality or in your analyses, it may be necessary to prohibit respondents from declining to answer a question (i.e., to prevent them from skipping a question or a whole set of questions but continuing with the survey). However, you do want to be sure to carefully consider the implications of this choice. For more information about the conditional branching logic and PRO-CTCAE scoring, please see the FAQ page at our website.

PRO-CTCAE has been developed and tested with a conditional branching logic that helps to reduce respondent burden. Conditional branching should be employed for electronic administration of PRO-CTCAE symptom terms that have two or more attributes (e.g., fatigue severity and interference or pain frequency, severity and interference). The logic branches from frequency, then to severity, and then to interference. For example, if frequency is greater than "Never", you next pose the severity question, and if severity is greater than "None", you pose the interference question.

Conditional branching logic implemented through electronic administration of Ped-PRO-CTCAE and Ped-PRO-CTCAE [Caregiver] can help to manage respondent burden. For both Ped-PRO-CTCAE and Ped-PRO-CTCAE [Caregiver] the logic branches from frequency, then to severity, then to interference. For example, if frequency is greater than "Never", you next pose the severity question, and if severity is greater than "Did not have any", you pose the interference question.

In addition, there are several Ped-PRO-CTCAE/Ped-PRO-CTCAE [Caregiver] symptom terms that branch from a presence/absence item to an interference item. For presence/absence questions where the selected response is "No" or "I do not know", the conditional branching logic should skip the subsequent interference question.

PRO-CTCAE questions were developed and tested using a ‘wide’ layout (see below for an example), and have been shown to be equivalent when administered on tablet computer, interactive voice response system (IVRS), or paper. Equivalence using other layouts (eg. vertical response choices) or other devices (eg. smartphone) has not been studied.

PRO-CTCAE and Ped-PRO-CTCAE questions should be grouped in bundles (eg. frequency, severity and interference) so as to preserve the sequencing of the cognitive task patients are completing when answering (ie. reflecting on the different attributes of a single symptom). For an example, please see below.

Mouth and Throat Sores

In the last 7 days, what was the SEVERITY of your MOUTH OR THROAT SORES at their WORST?
None	Mild	Moderate	Severe	Very severe
In the last 7 days, how much did MOUTH OR THROAT SORES INTERFERE with your usual or daily activities?
Not at all	A little bit	Somewhat	Quite a bit	Very much

In our studies in adults older than age 18, the whole group of questions about a particular symptom has been presented on a single screen, although because of conditional branching, questions only appear on the screen if the answer to the first question is greater than never (frequency) or none (severity).

A PRO-CTCAE mode equivalence study in adults older than age 18 has been performed examining psychometric equivalence of responses gathered using web-enabled tablets (screen size ranging from 10-12 inches), IVRS and paper. Participants were drawn from one of 6 clinical sites, and included older respondents and those with poor performance status. All respondents were currently undergoing cancer treatment.

There is psychometric evidence in adults older than age 18 to justify comparison of results and pooled analyses across studies that employ different PRO-CTCAE modes of administration (Mode equivalence and acceptability of tablet computer-, interactive voice response system-, and paper-based administration of the U.S. National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).). Administration of PRO-CTCAE via different modes including desk top computers, web-enabled tablet computers, IVRS, and paper offers flexibility for patients and for study operations personnel.

PRO-CTCAE responses are scored from 0 to 4 (or 0/1 for absent/present). At present there are no guidelines yet established for how to combine attributes into a single score or how best to analyze PRO-CTCAE data longitudinally. These are topics of ongoing studies.

When PRO-CTCAE is administered using an electronic platform and the conditional branching logic is employed, there are additional scoring considerations. Specifically, if the response provided to the first question in a PRO-CTCAE item set is the lowest response on the scale (e.g., ‘Never’ for frequency, or ‘None’ for severity), scores for the conditionally branched PRO-CTCAE items should be scored/coded as 0 and are not to be coded as missing values.

In addition, the following response choices for selected PRO-CTCAE items should be coded as indicated below:

PRO-CTCAE scores for each attribute (frequency, severity and/or interference) should be presented descriptively (e.g., summary statistics or graphical presentations). CTCAE grades for the corresponding time period should be presented alongside PRO-CTCAE scores. The proportion of missing data should also be summarized to aid interpretation.

Ped-PRO-CTCAE responses are scored from 0 to 3 (or 0/1 for absent/present), and there are no established guidelines for how to combine attributes into a single score or how best to analyze Ped-PRO-CTCAE data longitudinally. These are topics of ongoing studies.

When Ped-PRO-CTCAE or Ped-PRO-CTCAE [Caregiver] is administered using an electronic platform and the conditional branching logic is employed, there are additional scoring considerations. Specifically, if the response provided to the first question in a PRO-CTCAE item set is the lowest response on the scale (e.g., ‘Never’ for frequency, or ‘Did not have any’ for severity), scores for the conditionally branched Ped-PRO-CTCAE or Ped-PRO-CTCAE [Caregiver] items should be scored/coded as 0 and are not to be coded as missing values.

In addition, if the response provided is ‘I do not know’, no numeric coded value should be assigned, and the response should not be coded as missing. Rather, the specific text response chosen should be preserved as a string value.

Ped-PRO-CTCAE scores for each attribute (frequency, severity and/or interference) should be presented descriptively (e.g., summary statistics or graphical presentations). CTCAE grades for the corresponding time period should be presented alongside Ped-PRO-CTCAE and Ped-PRO-CTCAE [Caregiver] scores. The proportion of missing data should also be summarized to aid interpretation.

No there is not, nor should there be. One should expect this for many reasons, including:

• Clinicians and patients providing their reports at differing time points
• Clinicians and patients reporting on slightly different phenomena (eg. sadness vs. depression)
• Frequent management of severe toxicity by clinicians may skew clinician perceptions of symptom severity
• Patients may under-report symptoms to their clinicians due to time constraints, a desire to avoid an appearance that they are complaining or bothering their clinician, as well as concerns about not qualifying for continued treatment, and/or being removed from a study protocol or receiving a lower dose of treatment, if they report toxicities.

Disagreement between patient-reported and clinician-reported symptomatic toxicity has been extensively described. Research has shown a varying relationship between self-report of symptoms and clinician-based CTCAE grades (See: Xiao C, Polomano R, Bruner DW. Comparison between patient-reported and clinician-observed symptoms in oncology. Policy Cancer Nurs 2013 Nov-Dec;36(6):E1-E16. Review).

Several additional key points:

• Criteria for grading on the CTCAE scale vary by toxicity, however by convention, grade 1 typically refers to asymptomatic or mild symptoms not requiring intervention, grade 2 refers to moderate symptoms that interfere somewhat with daily function and where some intervention may be indicated, and grade 3 refers to severe symptoms that interfere with daily activities or require more significant intervention. Grade 4 toxicity (life-threatening, with urgent intervention indicated) and Grade 5 (death) are indicative of significantly higher levels of toxicity, and are rarely applied for symptomatic toxicities such as fatigue, tinnitus, hoarseness or xerostomia. Thus, a patient-reported symptom of 'severe' or 'very severe' may not therefore correspond to CTCAE grade 4 or 5 toxicity. In addition many of the symptomatic toxicities in the CTCAE do not permit assignment of a grade higher than grade 3.
• While there is apparent similarity between grading scales for PRO-CTCAE or Ped-PRO-CTCAE and CTCAE, differences between CTCAE grades and PRO-CTCAE or Ped-PRO-CTCAE derived scores should not be over-interpreted. Disagreement between PRO-CTCAE or Ped-PRO-CTCAE scores and CTCAE grades may not represent 'under-ascertainment' by clinicians or 'over-reporting' by patients. For example, a patient may meet CTCAE criteria for grade 1 vomiting (1-2 episodes separated by 5 minutes in a 24 hour period), but to the patient, two daily episodes of vomiting constitutes a severe symptom.
• There are unique challenges when attempting to directly compare CTCAE grades and PRO-CTCAE or Ped-PRO-CTCAE scores. These toxicity scoring instruments were not developed for direct comparison but rather were designed to be complementary strategies for capturing symptomatic adverse events. The NCI is leading an effort to determine the best approaches to scoring PRO-CTCAE, and ultimately to using the PRO-CTCAE scores to assign a CTCAE grade.

PRO-CTCAE items should not be reprinted in a publication. Instead, please cite the appropriate reference to our PRO-CTCAE web site (https://healthcaredelivery.cancer.gov/pro-ctcae/) in the reference list.

We have several early adopters testing additional or revised items that may one day be included in the PRO-CTCAE Measurement System.

To protect the integrity of the PRO-CTCAE Measurement System, the U.S. National Cancer Institute holds the trademark for PRO-CTCAE, all translations, and all related documents. To assure standardization, users of the PRO-CTCAE Measurement System are not permitted to adapt or otherwise modify the PRO-CTCAE instruments (including symptom terms, item phrasing, response choices and recall period) as specified on the PRO-CTCAE Instrument page without prior written consent of the National Cancer Institute.

If you have interest in developing and testing additional or revised PRO-CTCAE items, please be in touch with us by e-mail to discuss this interest and your planned approach.

Please visit the Language Translation page for more information.

There are a number of resources that exist to support study design, analysis, interpretation and reporting when a PRO is included as a primary, secondary or exploratory endpoint in a clinical trial. One resource is the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials)-PRO Extension which provides guidelines for the inclusion of PROs in clinical trial protocols. Associated resources include an explanation and elaboration of the SPIRIT-PRO Extension, and tools to support the involvement of patient partners in the research. Another resource is the PROTEUS (Patient-Reported Outcomes Tools: Engaging Users & Stakeholders) Consortium (also see https://theproteusconsortium.org/). PROTEUS brings together international guidelines and offers training materials and implementation checklists. Together, these resources can be used by investigators to justify their selection of PRO domains, specify procedures for data collection, and make decisions about the analysis and interpretation of the data.

The pediatric module of the PRO-CTCAE Measurement System permits self- reporting by children and adolescents ages 7-17 years, or caregiver- reporting for children ages 7-17 who are unable to self-report (e.g., due to cognitive impairment, age, or other reasons why the participants are unable to reliably self-report). Ped-PRO-CTCAE has not been validated for use in patients younger than 7 years of age, and Ped-PRO-CTCAE [Caregiver] has not been validated for use as a proxy measure for patients younger than age 7.

Ped-PRO-CTCAE and Ped-PRO-CTCAE [Caregiver] have psychometric evidence to support their reliability and validity for use with children ages 7-17. Considerations in selecting which PRO-CTCAE measure to use include the respondent’s chronological age, educational attainment, and achievement of the cognitive milestones expected at each developmental stage. There may also be cases where a research participant is not considered to be legally an adult until age 21. However, irrespective of this determination of legal adulthood (and this definition may vary by legal rights, country, and psychological development), if a respondent is at least 18 years of age, PRO-CTCAE is generally recommended for use. PRO-CTCAE has been tested in individuals 18 years of age and older and has demonstrated strong measurement properties for this age group.

Study protocol documents should clearly specify that the Ped-PRO-CTCAE [Caregiver] version be used when a child 7 years of age or older is unable to complete the pediatric version. This may be due to cognitive impairment, age, or other reasons (e.g. illness severity) that make self-reporting unreliable or infeasible. Studies have shown that there are varying levels of agreement between participant and proxy ratings. Given these known issues, when child-participants are able to self-report, collecting both child- and proxy-reported data allows for quantification of the size and direction of any bias that may later be statistically adjusted for, if needed.

The trial protocol should indicate clearly who is eligible to provide the proxy report, and the protocol document should include explicit administration guidelines for completion of proxy measures. These administration guidelines should address how the proxy report is to be captured, whether that same individual must be the consistent rater across all timepoints of assessment (this is preferable, for consistency) or whether varying proxy reports will be permissible. Further data may be gathered about the proxy (e.g., age, relationship to the patient, gender, proxy literacy, and exposure to the patient), as these variables may guide interpretation of results and any subgroup/sensitivity analyses. Whether caregiver proxy-reported data will be analyzed separately or pooled with child self-reported data should also be detailed in both the protocol and the statistical analysis plan.

More information about the use of proxy more generally for any PRO measure can also be found in two recently published papers that extend and elaborate on the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidance with respect to study design and analysis of PRO endpoints.