Frequently Asked Questions

Visit the Development, Testing, and Implementation page for more information.

Selecting which symptomatic adverse events to assess and determining the time points for measurement are critical trial design decisions. In trials developed to date, items from the PRO-CTCAE or from Ped-PRO-CTCAE and the time points of measurement have generally been specified using an approach similar to that used to define the adverse event surveillance plan for the trial more broadly. That is, the study team reviews published data, as well as data from earlier phase trials or animal models, if available, and incorporates information about the known or anticipated on- and off-target effects of agents in a similar mechanistic class, to identify those symptomatic adverse events likely to be associated with the regimens in the trial.

After establishing a pre-treatment baseline, more frequent PRO-CTCAE or Ped-PRO-CTCAE administration is generally warranted during the first few cycles of therapy (e.g., weekly reporting during the initial several months of therapy). Thereafter, the assessment intervals may be extended (e.g., monthly or quarterly, depending on the regimen under study), particularly in trial contexts where the duration of investigational treatment is prolonged or where toxicities are anticipated to have stabilized. In summary, the time points of measurement should reflect the anticipated pattern of toxicity, and the scientific objectives of the trial.

At this time, we cannot recommend use of longer or shorter PRO-CTCAE or Ped-PRO-CTCAE recall periods. The standard PRO-CTCAE recall period is “the last 7 days”. Prior research suggests that longer recall periods (2-, 3- or 4- week recall) are associated with small but successively increasing measurement error (Evaluation of different recall periods for the US National Cancer Institute's PRO-CTCAE). While use of longer recall periods may seem to be an attractive approach to address burden on patient and study personnel, trialists must consider the fact that longer recall periods and less frequent assessments may introduce both measurement unreliability and under-reporting of important within-cycle treatment experiences. At the same time, concerns about temporal breaks in reporting may be less pertinent in study contexts where symptomatic AEs are anticipated to be stable or to be changing only subtly, as, for example, with chronically administered oral therapies or during post-treatment follow-up. The anticipated patterns of change in symptomatic adverse events and the study aims and design must both be considered if recall periods longer than 1 week are going to be used in a trial for logistical reasons. The measurement properties and study design considerations for use of a 24-hour recall period are addressed in this FAQ.

While the standard recall period for PRO-CTCAE is the ‘last 7 days’, in some specific trial contexts, investigators may be interested to consider the use of a 24-hour recall period.

It is important to note that administration of PRO-CTCAE with a 24-hour recall period has had fairly limited psychometric testing, whereas the 7-day recall has been shown in comparative analysis to have the strongest measurement properties and thus is the standard recall period for PRO-CTCAE (Evaluation of different recall periods for the US National Cancer Institute's PRO-CTCAE). However, when paired with daily assessment, the 24-hour recall period has also demonstrated acceptable test-retest reliability and construct validity (Reliability and validity of PRO-CTCAE® daily reporting with a 24-hour recall period).

Importantly, research also indicates that many PRO-CTCAE symptomatic adverse events demonstrate considerable intra-individual variability day-to-day, across a week of daily assessments. Specifically, symptoms such as sad or unhappy feelings, constipation, diarrhea, vomiting, and insomnia all demonstrated prominent within-subject variability across daily PRO-CTCAE assessments. This suggests that unless a shorter recall period is also paired with daily reporting, use of a 24-hour recall period could significantly bias the interpretation of PRO-CTCAE data (Reliability and validity of PRO-CTCAE® daily reporting with a 24-hour recall period).

A recent report underscores the importance of this interplay between the PRO-CTCAE recall period and the symptom reporting interval (e.g. daily reporting, weekly reporting or some other reporting interval). Using an interrupted time-series design, Paudel et al. evaluated the effects of shortening the recall from ‘last 7 days’ to ‘past 24 hours’ in a large sample of patients completing weekly PRO-CTCAE assessments. Paudel et al. observed that the shortened recall period was associated with a 17%-35% (p=.02) reduction in the proportion of patients reporting moderate or severe symptoms in a large cohort of patients undergoing chemotherapy treatment or a cancer-directed surgical procedure (Effects of a change in recall period on reporting severe symptoms: an analysis of a pragmatic multisite trial).

Taken together, the results of these three studies indicate that although 24-hour recall may demonstrate acceptable measurement properties when paired with daily assessment, simply shortening the recall period results in significant under-detection of treatment side effects. As such, use of a 24-hour recall period necessitates more frequent PRO-CTCAE assessment intervals to avoid having time periods of information loss with respect to the frequency, severity and/or interference of a particular symptomatic adverse event.

In some specific trial contexts, investigators may be interested to precisely characterize the onset or offset of specific symptomatic adverse events within a 24-hour period. These trial contexts could include capturing acute symptomatic adverse events such as infusional reactions, post-surgical complications, and other phenomena that have a rapid onset and offset or are self-limiting.

However, while acute symptomatic toxicities such as nausea, diarrhea, chills, itching, or pain surrounding sentinel treatment events may be validly and reliably captured using a 24-hour recall, there are several important study design considerations. These include the intervals of assessment, the administrative and analytic burden of daily reporting, and the potential for there to be more missing data. Moreover, given the significant intra-individual variability observed with daily reporting of domains such as sleep quality, mood, or sexual concerns (Reliability and validity of PRO-CTCAE® daily reporting with a 24-hour recall period), meaningful interpretation of the daily reports of these symptomatic adverse events could also prove analytically challenging. Nevertheless, in contexts where it is important to interpret acute symptomatic adverse events in relation to a treatment event (e.g. a sentinel treatment event such as surgery, infusion of a cellular product), use of a 24-hour recall period and daily reporting, at least during the period surrounding the event of interest, could be valuable.

For a specific study, if a 24-hour recall period will be employed instead of the standard 7-day recall, the study protocol, presentations, abstracts, manuscripts and all similar materials should: (i) explicitly indicate that the standard PRO-CTCAE 7-day recall period was modified to a 24-hour recall period, (ii) specify the PRO-CTCAE assessment intervals (e.g. daily assessment), and (iii) provide a scientific justification for these design decisions.

Lastly, for registration trials and/or trials where a labelling indication is being sought based on PRO-CTCAE data, consultation with the relevant regulatory agency (e.g. FDA, EMA) regarding use of a shortened recall period should also be considered.

Based on our findings, PRO-CTCAE items are completed rapidly by adults. We estimate that a 20-item PRO-CTCAE survey would take on average 3.4 minutes by paper, 3.7 minutes by web, and 5.4 minutes by interactive voice response system (IVRS).

Research in adults also indicates that there is no evidence of clinically meaningful variation in completion times by participant characteristics, including impairments in physical or cognitive functioning (Mode equivalence and acceptability of tablet computer-, interactive voice response system-, and paper-based administration of the U.S. National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).). These findings suggest that completion of PRO-CTCAE items by adults is generally not laborious, even for those respondents who may have some degree of functional limitation.

Estimates of the total time for adults to complete a survey by each mode depend upon the number of items presented to the respondent. As an additional consideration, because paper surveys do not incorporate conditional branching patterns, as do web-based and IVRS surveys, a participant completing a paper survey would likely have to complete more PRO-CTCAE items, compared to the same participant completing PRO-CTCAE using an electronic version of the questions.

As an example of an estimate of respondent burden in adults for human subjects research applications, 75% of respondents would be able to complete twenty items in 3.8 minutes by web, in 6.1 minutes by IVRS, and in 4.3 minutes by paper.

There is no clear guidance from the literature or from regulatory agencies that when establishing an electronic system to collect PRO-CTCAE responses, the system should require the respondent to provide a response before they are able to progress within the survey.

In deciding whether to constrain an electronic PRO collection system in a manner that allows respondents to decline to answer a question (i.e., to skip a question or a whole set of questions but continue with the survey), it is important to consider three aspects. These are: (a) the possible effects on the integrity of the PRO-CTCAE conditional branching logic; (b) the impact on data quality; and (c) human subjects considerations.

Allowing respondents to choose not to provide an answer to a question and continue with the survey could have adverse effects on the integrity of the PRO-CTCAE conditional branching logic. Allowing respondents to skip an entire set of questions for a specific symptomatic adverse event could also result in considerable missing data. This is because data will not only be missing for the specific item that was skipped, but data will also be missing for the other PRO-CTCAE items that branch from that item. At the same time, requiring that the respondent provide a response before progressing within the survey could serve to produce missing data because of participant withdrawal. It is also possible that if you require respondents to complete each item, that they will provide a random response or just abandon your survey entirely.

There may also be human subjects considerations in requiring respondents to provide a response. Consultation with your Institutional Review Board is important in weighing this study design decision. For some PRO-CTCAE questions that address sensitive topics (e.g., PRO-CTCAE items that address sexual function), there are response options that allow the respondent to indicate if they would prefer not to respond or if they deem a question as not applicable (for example, because they are not sexually active). These response choices should be explicitly represented in the dataset, and not be coded as missing.

Perhaps the overall best approach that serves to balance both respondent burden and data quality may be to constrain the electronic data capture system to require responses to all questions, while simultaneously implementing conditional branching to conserve respondent burden. Keep in mind, that if one chooses to allow respondents to skip individual items or to skip whole PRO-CTCAE item sets, that lack of response should be explicitly represented in the dataset as ‘missing, participant did not answer’.

In summary, if having missing data will cause significant difficulties with data quality or in your analyses, it may be necessary to prohibit respondents from declining to answer a question (i.e., to prevent them from skipping a question or a whole set of questions but continuing with the survey). However, you do want to be sure to carefully consider the implications of this choice. For more information about the conditional branching logic and PRO-CTCAE scoring, please see the FAQ page at our website.

PRO-CTCAE has been developed and tested with a conditional branching logic that helps to reduce respondent burden. Conditional branching should be employed for electronic administration of PRO-CTCAE symptom terms that have two or more attributes (e.g., fatigue severity and interference or pain frequency, severity and interference). The logic branches from frequency, then to severity, and then to interference. For example, if frequency is greater than "Never", you next pose the severity question, and if severity is greater than "None", you pose the interference question.

Conditional branching logic implemented through electronic administration of Ped-PRO-CTCAE and Ped-PRO-CTCAE [Caregiver] can help to manage respondent burden. For both Ped-PRO-CTCAE and Ped-PRO-CTCAE [Caregiver] the logic branches from frequency, then to severity, then to interference. For example, if frequency is greater than "Never", you next pose the severity question, and if severity is greater than "Did not have any", you pose the interference question.

In addition, there are several Ped-PRO-CTCAE/Ped-PRO-CTCAE [Caregiver] symptom terms that branch from a presence/absence item to an interference item. For presence/absence questions where the selected response is "No" or "I do not know", the conditional branching logic should skip the subsequent interference question.

PRO-CTCAE questions were developed and tested using a ‘wide’ layout (see below for an example), and have been shown to be equivalent when administered on tablet computer, interactive voice response system (IVRS), or paper. Equivalence using other layouts (eg. vertical response choices) or other devices (eg. smartphone) has not been studied.

PRO-CTCAE and Ped-PRO-CTCAE questions should be grouped in bundles (eg. frequency, severity and interference) so as to preserve the sequencing of the cognitive task patients are completing when answering (ie. reflecting on the different attributes of a single symptom). For an example, please see below.

Mouth and Throat Sores

In the last 7 days, what was the SEVERITY of your MOUTH OR THROAT SORES at their WORST?
None	Mild	Moderate	Severe	Very severe
In the last 7 days, how much did MOUTH OR THROAT SORES INTERFERE with your usual or daily activities?
Not at all	A little bit	Somewhat	Quite a bit	Very much

In our studies in adults older than age 18, the whole group of questions about a particular symptom has been presented on a single screen, although because of conditional branching, questions only appear on the screen if the answer to the first question is greater than never (frequency) or none (severity).

A PRO-CTCAE mode equivalence study in adults older than age 18 has been performed examining psychometric equivalence of responses gathered using web-enabled tablets (screen size ranging from 10-12 inches), IVRS and paper. Participants were drawn from one of 6 clinical sites, and included older respondents and those with poor performance status. All respondents were currently undergoing cancer treatment.

There is psychometric evidence in adults older than age 18 to justify comparison of results and pooled analyses across studies that employ different PRO-CTCAE modes of administration (Mode equivalence and acceptability of tablet computer-, interactive voice response system-, and paper-based administration of the U.S. National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).). Administration of PRO-CTCAE via different modes including desk top computers, web-enabled tablet computers, IVRS, and paper offers flexibility for patients and for study operations personnel.

PRO-CTCAE responses are scored from 0 to 4 (or 0/1 for absent/present). At present there are no guidelines yet established for how to combine attributes into a single score or how best to analyze PRO-CTCAE data longitudinally. These are topics of ongoing studies.

When PRO-CTCAE is administered using an electronic platform and the conditional branching logic is employed, there are additional scoring considerations. Specifically, if the response provided to the first question in a PRO-CTCAE item set is the lowest response on the scale (e.g., ‘Never’ for frequency, or ‘None’ for severity), scores for the conditionally branched PRO-CTCAE items should be scored/coded as 0 and are not to be coded as missing values.

In addition, the following response choices for selected PRO-CTCAE items should be coded as indicated below:

PRO-CTCAE scores for each attribute (frequency, severity and/or interference) should be presented descriptively (e.g., summary statistics or graphical presentations). CTCAE grades for the corresponding time period should be presented alongside PRO-CTCAE scores. The proportion of missing data should also be summarized to aid interpretation.

Ped-PRO-CTCAE responses are scored from 0 to 3 (or 0/1 for absent/present), and there are no established guidelines for how to combine attributes into a single score or how best to analyze Ped-PRO-CTCAE data longitudinally. These are topics of ongoing studies.

When Ped-PRO-CTCAE or Ped-PRO-CTCAE [Caregiver] is administered using an electronic platform and the conditional branching logic is employed, there are additional scoring considerations. Specifically, if the response provided to the first question in a PRO-CTCAE item set is the lowest response on the scale (e.g., ‘Never’ for frequency, or ‘Did not have any’ for severity), scores for the conditionally branched Ped-PRO-CTCAE or Ped-PRO-CTCAE [Caregiver] items should be scored/coded as 0 and are not to be coded as missing values.

In addition, if the response provided is ‘I do not know’, no numeric coded value should be assigned, and the response should not be coded as missing. Rather, the specific text response chosen should be preserved as a string value.

Ped-PRO-CTCAE scores for each attribute (frequency, severity and/or interference) should be presented descriptively (e.g., summary statistics or graphical presentations). CTCAE grades for the corresponding time period should be presented alongside Ped-PRO-CTCAE and Ped-PRO-CTCAE [Caregiver] scores. The proportion of missing data should also be summarized to aid interpretation.

No there is not, nor should there be. One should expect this for many reasons, including:

• Clinicians and patients providing their reports at differing time points
• Clinicians and patients reporting on slightly different phenomena (eg. sadness vs. depression)
• Frequent management of severe toxicity by clinicians may skew clinician perceptions of symptom severity
• Patients may under-report symptoms to their clinicians due to time constraints, a desire to avoid an appearance that they are complaining or bothering their clinician, as well as concerns about not qualifying for continued treatment, and/or being removed from a study protocol or receiving a lower dose of treatment, if they report toxicities.

Disagreement between patient-reported and clinician-reported symptomatic toxicity has been extensively described. Research has shown a varying relationship between self-report of symptoms and clinician-based CTCAE grades (See: Xiao C, Polomano R, Bruner DW. Comparison between patient-reported and clinician-observed symptoms in oncology. Policy Cancer Nurs 2013 Nov-Dec;36(6):E1-E16. Review).

Several additional key points:

• Criteria for grading on the CTCAE scale vary by toxicity, however by convention, grade 1 typically refers to asymptomatic or mild symptoms not requiring intervention, grade 2 refers to moderate symptoms that interfere somewhat with daily function and where some intervention may be indicated, and grade 3 refers to severe symptoms that interfere with daily activities or require more significant intervention. Grade 4 toxicity (life-threatening, with urgent intervention indicated) and Grade 5 (death) are indicative of significantly higher levels of toxicity, and are rarely applied for symptomatic toxicities such as fatigue, tinnitus, hoarseness or xerostomia. Thus, a patient-reported symptom of 'severe' or 'very severe' may not therefore correspond to CTCAE grade 4 or 5 toxicity. In addition many of the symptomatic toxicities in the CTCAE do not permit assignment of a grade higher than grade 3.
• While there is apparent similarity between grading scales for PRO-CTCAE or Ped-PRO-CTCAE and CTCAE, differences between CTCAE grades and PRO-CTCAE or Ped-PRO-CTCAE derived scores should not be over-interpreted. Disagreement between PRO-CTCAE or Ped-PRO-CTCAE scores and CTCAE grades may not represent 'under-ascertainment' by clinicians or 'over-reporting' by patients. For example, a patient may meet CTCAE criteria for grade 1 vomiting (1-2 episodes separated by 5 minutes in a 24 hour period), but to the patient, two daily episodes of vomiting constitutes a severe symptom.
• There are unique challenges when attempting to directly compare CTCAE grades and PRO-CTCAE or Ped-PRO-CTCAE scores. These toxicity scoring instruments were not developed for direct comparison but rather were designed to be complementary strategies for capturing symptomatic adverse events. The NCI is leading an effort to determine the best approaches to scoring PRO-CTCAE, and ultimately to using the PRO-CTCAE scores to assign a CTCAE grade.

PRO-CTCAE items should not be reprinted in a publication. Instead, please cite the appropriate reference to our PRO-CTCAE web site (https://healthcaredelivery.cancer.gov/pro-ctcae/) in the reference list.

We have several early adopters testing additional or revised items that may one day be included in the PRO-CTCAE Measurement System.

To protect the integrity of the PRO-CTCAE Measurement System, the U.S. National Cancer Institute holds the trademark for PRO-CTCAE, all translations, and all related documents. To assure standardization, users of the PRO-CTCAE Measurement System are not permitted to adapt or otherwise modify the PRO-CTCAE instruments (including symptom terms, item phrasing, response choices and recall period) as specified on the PRO-CTCAE Instrument page without prior written consent of the National Cancer Institute.

If you have interest in developing and testing additional or revised PRO-CTCAE items, please be in touch with us by e-mail to discuss this interest and your planned approach.

Please visit the Language Translation page for more information.

There are a number of resources that exist to support study design, analysis, interpretation and reporting when a PRO is included as a primary, secondary or exploratory endpoint in a clinical trial. One resource is the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials)-PRO Extension which provides guidelines for the inclusion of PROs in clinical trial protocols. Associated resources include an explanation and elaboration of the SPIRIT-PRO Extension, and tools to support the involvement of patient partners in the research. Another resource is the PROTEUS (Patient-Reported Outcomes Tools: Engaging Users & Stakeholders) Consortium (also see https://theproteusconsortium.org/). PROTEUS brings together international guidelines and offers training materials and implementation checklists. Together, these resources can be used by investigators to justify their selection of PRO domains, specify procedures for data collection, and make decisions about the analysis and interpretation of the data.

The pediatric module of the PRO-CTCAE Measurement System permits self- reporting by children and adolescents ages 7-17 years, or caregiver- reporting for children ages 7-17 who are unable to self-report (e.g., due to cognitive impairment, age, or other reasons why the participants are unable to reliably self-report). Ped-PRO-CTCAE has not been validated for use in patients younger than 7 years of age, and Ped-PRO-CTCAE [Caregiver] has not been validated for use as a proxy measure for patients younger than age 7.

Ped-PRO-CTCAE and Ped-PRO-CTCAE [Caregiver] have psychometric evidence to support their reliability and validity for use with children ages 7-17. Considerations in selecting which PRO-CTCAE measure to use include the respondent’s chronological age, educational attainment, and achievement of the cognitive milestones expected at each developmental stage. There may also be cases where a research participant is not considered to be legally an adult until age 21. However, irrespective of this determination of legal adulthood (and this definition may vary by legal rights, country, and psychological development), if a respondent is at least 18 years of age, PRO-CTCAE is generally recommended for use. PRO-CTCAE has been tested in individuals 18 years of age and older and has demonstrated strong measurement properties for this age group.

Study protocol documents should clearly specify that the Ped-PRO-CTCAE [Caregiver] version be used when a child 7 years of age or older is unable to complete the pediatric version. This may be due to cognitive impairment, age, or other reasons (e.g. illness severity) that make self-reporting unreliable or infeasible. Studies have shown that there are varying levels of agreement between participant and proxy ratings. Given these known issues, when child-participants are able to self-report, collecting both child- and proxy-reported data allows for quantification of the size and direction of any bias that may later be statistically adjusted for, if needed.

The trial protocol should indicate clearly who is eligible to provide the proxy report, and the protocol document should include explicit administration guidelines for completion of proxy measures. These administration guidelines should address how the proxy report is to be captured, whether that same individual must be the consistent rater across all timepoints of assessment (this is preferable, for consistency) or whether varying proxy reports will be permissible. Further data may be gathered about the proxy (e.g., age, relationship to the patient, gender, proxy literacy, and exposure to the patient), as these variables may guide interpretation of results and any subgroup/sensitivity analyses. Whether caregiver proxy-reported data will be analyzed separately or pooled with child self-reported data should also be detailed in both the protocol and the statistical analysis plan.

More information about the use of proxy more generally for any PRO measure can also be found in two recently published papers that extend and elaborate on the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidance with respect to study design and analysis of PRO endpoints.

• Check our Release Notes page, which covers the technical documentation we develop and distribute alongside the launch of an updated, corrected, or revised PRO-CTCAE Item Library. Our PRO-CTCAE Release Notes document any change made to a PRO-CTCAE Item Library, including correction of typographical errors and adjustments to phrasing warranted to create a single harmonized universal language version after validation testing in a wider range of countries. Release notes can be filtered by date, language, or library type (PRO-CTCAE or Ped-PRO-CTCAE).
• You may wish to monitor the Release Notes page using a website change monitoring software service. There are several services that are free of charge and can be configured to your specific needs and interests.
• We recommend that you check the Release Notes page as part of the planning and/or pre-implementation stages of your research study, to determine if there have been updates made to PRO-CTCAE content that should be reflected in your implementation materials, including your protocol documents, case report forms, and/or your electronic patient-reported outcome (ePRO) data capture system build.