Measures that are Limited or not Available in the Data

Historically, the SEER data identified patients with metastatic disease at diagnosis but did not identify the specific site of metastasis. In 2004 the SEER data started to record the presence of bone metastases among men with advanced prostate cancer and then in 2010 started to document metastases to the bone, brain, liver and lung among all patients but these variables have not been fully vetted. These measures are listed below, divided into those that are problematic and those that are not feasible. A more detailed explanation is provided for each item listed in the Summary chart. The list includes topics that are of frequent interest. There may be other less commonly identified measures that have limitations that are not included with this document.

In general, the challenge of using Medicare data to identify specific clinical indications is due to the lack of precision of diagnosis and procedure codes and CMS's coding requirements. Over-interpretation of how frequently specific vs. general codes are used can lead to situations where measures are being used that have poor sensitivity (do a poor job of identifying cases) or poor positive predictive values. Medical conditions are frequently reported inconsistently or rarely because they do not impact reimbursement from Medicare (e.g., metastases). A good rule of thumb is that measures that impact payment will be more precisely coded than measures that do not. Of course, not all services have the same payment rules.

Despite being problematic, some of the measures listed below have been included in peer-reviewed publications that have used SEER-Medicare data. Because these measures appear in a peer-reviewed journal does not mean that they have been validated. The researchers who have published using these measures and the journal reviewers may not have understood the limitations of these measures. The topics listed below should be used selectively or avoided completely.

Summary Table

Measures that are problematic	Measures that cannot be determined from the SEER-Medicare data
Adverse events. Behavioral and other patient factors. Curative vs. palliative chemotherapy and radiation therapy. Date of diagnosis for chronic conditions. Digital rectal exam and fecal occult blood tests (FOBT). Patient out of pocket payments. Prostate cancer- watchful waiting, expectant management, active surveillance. Recurrence. Screening colonoscopy. Site of metastasis.	Activities of daily living (ADL) status, performance status, functional status; patient's quality of life. Care paid by entities other than Medicare. Disease-free survival. Metastasis occurring after diagnosis. Physician treatment recommendations and patient preferences related to treatment. Services Medicare does not cover. Test results - lab, imaging, surgical margins. Treatments or conditions occurring before Medicare eligibility.

Detailed Explanations Related to Each Measure

Measures that are Problematic

Measure	Comment	Resources and References (if relevant)
Adverse events (AEs)	Claims may include information about AEs if the patient sought treatment for the adverse event and the event was reported on the hospital or physician claim. Conditions for which the patient does not receive additional treatment will be significantly underreported in the data. For example, a validation study compared self-reported outcomes for men with prostate cancer with information on their Medicare claims. The Medicare claims had a diagnosis of incontinence for 29% of men who reported having incontinence. However, Medicare claims included procedure codes for 83% of men who reported having surgery for urethral strictures.	Potosky AL et al. Measuring complications of cancer treatment using the SEER-Medicare data. Med Care. 2002 Aug;40(8 Suppl):IV-62-8. PubMed [View Abstract].
Behavioral and other patient factors	Although researchers frequently want to include behavioral and other patient risk factors (e.g., smoking/alcohol use, obesity, body mass index (BMI), and personal or family history) in analyses, these measures are incompletely reported in the SEER-Medicare data. The SEER registries do not collect information about risk factors. Medicare data include ICD-9 diagnoses codes related to some risk factors, such as V15.82 (personal history of smoking), 305.1 (tobacco use disorder), V85.X (BMI categories), V10.X (personal history of cancer) and V16.X (family history of cancer) However, the sensitivity of these codes is low. Additionally, ICD9 diagnoses codes for other conditions have been used to impute the presence of these characteristics (e.g., COPD diagnosis for smoking history or alcoholic cirrhosis diagnosis for alcohol history) but the sensitivity of this methodology can also be low and those identified may not represent a general population having the risk factor of interest.	Free ICD-9 codes Free ICD-10 codes
Curative vs. palliative chemotherapy and radiation therapy (RT)	Medicare claims have no information about whether chemotherapy or radiation therapy is for curative or palliative purposes. Some investigators have attempted to overcome lack of information in the claims by assuming that all treatments for patients with distant disease at diagnosis are intended to be palliative. However, this approach assumes that all providers treat patients according to guideline recommendations. Equally problematic is using dose/duration of treatment to differentiate curative vs. palliative care (ex. palliative RT will be no more than x doses at y level). There are a variety of reasons a patient may get fewer doses and it is not possible to differentiate between a patient ending treatment early and a patient who received all care initially planned.
Date of diagnosis for chronic conditions	Investigators often want to identify when a patient first developed a chronic condition, such as COPD or diabetes. If the condition developed prior to attaining Medicare eligibility, there is no way to determine this from Medicare claims. In order to conclude the condition developed after Medicare enrollment, one needs evidence that there was a period during which the patient did not have the condition. This generally requires an observable "clean" period of one or two years prior to the first evidence of presence of the chronic condition. If a patient enrolled in Medicare less than one or two years prior to the first appearance of the diagnosis in the claims, it will be unclear whether the condition existed prior to Medicare enrollment. Those patients will have to be excluded from analyses requiring date of onset. A general approach is to exclude all Medicare enrollees younger than 67 from these types of analyses.
Digital rectal examination and fecal occult blood tests (FOBT)	Some procedures are covered by Medicare but may not appear in the regularly in the claims. This is true for digital rectal examination for prostate cancer and fecal occult blood tests (FOBT) for colorectal cancer. Digital Rectal Examination (DRE) Medicare began covering annual DRE in 2000 when it began covering prostate-specific antigen (PSA) test for all male beneficiaries over 50 years old. Medicare will only pay for DRE if it is the only service provided during a physician visit or is part of a visit that is not covered by Medicare. For example, if a patient is seeing the doctor for a routine office visit, the DRE would not be paid separately. Procedures that are not paid separately are usually underreported in the Medicare claims. FOBT Annual FOBT has been covered by Medicare since 1998. This includes FOBT performed in a physician's office and at a patient's home. There have been concerns about how completely claims for FOBT have been submitted to Medicare. An analysis of FOBT claims billed to Medicare in 2001-2002 compared with data from the medical record reported the claims had 58% Sensitivity 88% specificity and 53% positive predictive value for FOBT. In January 1, 2007, CMS implemented a new CPT code to help distinguish tests based on a single card obtained in the physician's office and tests based on three cards completed at home. This new code may improve the future ability of Medicare claims to capture CRC screening with FOBT. There has been no validation of the additional FOBT codes. Pending any validations, researchers should use Medicare claims to assess FOBT cautiously.	Schenck AP, Klabunde CN, Warren JL, Peacock S, Davis WW, Hawley ST, Pignone M, Ransohoff DF. Evaluation of claims, medical records, and self-report for measuring fecal occult blood testing among medicare enrollees in fee for service. Cancer Epidemiol Biomarkers Prev 2008 Apr;17(4):799-804. [View Abstract]
Patient out of pocket payments	The Medicare claims include variables such as coinsurance and deductible amounts that can be used to identify a patient's responsibility for a bill. These values represent the amount the provider is allowed to bill the beneficiary. While the beneficiary may be responsible for this amount, they may have supplemental coverage, such as Medigap, that paid the provider. There is no information on the claim as to who paid the patient's portion of the bill. This means true patient burden for healthcare costs is not known.
Prostate cancer: Watchful waiting/ Active surveillance/ Expectant management	Researchers who use SEER-Medicare data to study prostate cancer treatment may want to identify men who are managed with watchful waiting. Some investigators assume that a patient is being managed through watchful waiting if there is no evidence of treatment in Medicare claims or SEER data. However, it is impossible to determine if the lack of treatment is an intentional plan or if there is no follow up care due patient or physician decision making.
Recurrence (see related comment below about metastasis)	Neither SEER nor Medicare collect information about recurrence. Medicare claims can be used to identify recurrences indirectly only if the patient receives treatment for the recurrence. It is not possible to identify recurrence through diagnosis codes only. Investigators who have used claims to identify treated recurrence have used an approach that involves reviewing claims longitudinally for cancer related treatment (cancer-related surgery, chemotherapy, RT) after the initial care period. The later surgeries should be selected carefully (ex. hepatic resection for a colon cancer patient likely shows recurrence while a hemicolectomy may be for disease recurrence or adhesions). Using a treatment-based approach is dependent on the patient receiving additional treatment in the event of a recurrence. However, many elderly patients are not offered/decline additional treatment if their cancer recurs. One study (Warren et al) showed that 40% of elderly patients treated for incident Stage II/III cancer did not receive additional cancer treatment prior to a cancer death. Those who were not treated were more likely to be age 70 and older and female. A second study (Hassett et al) using later chemotherapy as evidence of recurrence found that health claims had low sensitivity and specificity to identify recurrence. CONCLUSION: While it is possible to use SEER-Medicare to identify patients with "treated recurrence", this approach will miss a large number of cases and the cases identified are a biased sample of the elderly.	Warren JL, Mariotto A, Melbert D, Schrag D, Doria-Rose P, Penson D, Yabroff KR. Sensitivity of Medicare Claims to Identify Cancer Recurrence in Elderly Colorectal and Breast Cancer Patients. Med Care 2013 Dec 26. [Epub ahead of print] [View Abstract] Hassett MJ, Ritzwoller DP, Taback N, Carroll N, Cronin AM, Ting GV, Schrag D, Warren JL, Hornbrook MC, Weeks JC. Validating Billing/Encounter Codes as Indicators of Lung, Colorectal, Breast, and Prostate Cancer Recurrence Using 2 Large Contemporary Cohorts. Med Care 2012 Dec 6. [Epub ahead of print] [View Abstract]
Screening colonoscopy	Identifying screening colonoscopy from the SEER-Medicare data is challenging for several reasons: Coding policy The SEER data include no information about cancer screening. Colonoscopies are well captured in Medicare claims, although the reason the colonoscopy was performed is problematic. For patients who undergo colonoscopy for screening reasons, Medicare policy is that if during the course of the procedure a polyp or lesion is found, the colonoscopy becomes diagnostic and should coded to report colonoscopy with removal of polyp. This policy makes it impossible to determine which colonoscopies were initiated as diagnostic procedures and which were initiated as screening but were coded differently based on findings during the procedure. In 2011, Medicare introduced a modifier code, "PT" that indicates the colonoscopy began as a screening procedure. It is not known if this modified code is being used regularly. Guidelines about colorectal (CRC) cancer screening. The U.S. Preventive Services Task Force recommends that screening colonoscopies be performed no more frequently than every 10 years for person age 50-74. Medicare claims include no information about colonoscopy prior to Medicare enrollment, which occurs at age 65 for most beneficiaries. The 10-year recommended interval for screening colonoscopy makes it difficult to determine if Medicare patients ages 65-74 have not had a screening colonoscopy or if they had a screening colonoscopy prior to becoming Medicare eligible. In addition, the U.S. Preventive Services Task Force recommends against CRC screening in most patients age 75 or older. As a result of this recommendations, the time period available to identify whether patients are compliant with screening colonoscopies is limited and with incomplete information.	Schenck AP, Klabunde CN, Warren JL, Peacock S, Davis WW, Hawley ST, Pignone M, Ransohoff DF. Data sources for measuring colorectal endoscopy use among Medicare enrollees. Cancer Epidemiol Biomarkers Prev 2007 Oct;16(10):2118-27. [View Abstract] Coding FAQ - Screening Colonoscopy
Site of Metastasis (see metastasis)	Historically, the SEER data identified patients with metastatic disease at diagnosis but did not identify the specific site of metastasis. In 2004 the SEER data started to record the presence of bone metastases among men with advanced prostate cancer and then in 2010 started to document metastases to the bone, brain, liver and lung among all patients but these variables have not been fully vetted. The Medicare data includes codes for surgeries performed. If a surgery is only performed for cancer treatment and is limited to a specific organ, it may be possible to infer metastatic site, ex hepatic resection for patients diagnosed only with colon cancer. Medicare claims also include ICD-9 diagnosis codes that indicate specific anatomic sites for secondary neoplasms. A population-based assessment has compared bone metastasis diagnosis codes in the Medicare claims with bone metastasis collected by the SEER registries at the time of a prostate cancer diagnosis. The investigators found that the Medicare claims had 58.8% sensitivity, 54.1% specificity, and 68.4% positive predictive value for identifying bone metastasis at diagnosis. These findings raise concerns about the accuracy of the metastasis coding for specific sites.	SEER coding requirements for prostate cancer and bone metastasis. Bone metastasis based on SEER registry versus Medicare claims among metastatic prostate cancer patients in SEER-Medicare.
Incidence of Mental Illness	The claims data are most useful for identifying conditions for which one would commonly seek treatment. Or, in the case of chronic conditions, they are most useful for identifying conditions for which one would regularly seek treatment or monitoring. Using claims data to identify other types of conditions may yield low sensitivity and a biased sample because those who are identified may not represent a general population with that condition. In these cases, it may not be possible to produce valid, generalizable conclusions using these data. Using Medicare data to identify a mental health population is particularly problematic. Mental health disorders are under-identified in this population and those that do get treated may not be representative of the general mental disorder population. Historically, co-pays for mental health treatment have been much higher than other healthcare (50% through 2009 and 45% in 2010 vs 20% for other services). This may have lowered the probability that a patient will seek treatment. Those that do seek treatment may not be representative of a general population with mental health conditions of interest.	Noyes K, Liu H, Lyness JM, Friedman B. Medicare beneficiaries with depression: comparing diagnoses in claims data with the results of screening. Psychiatr Serv. 2011;62(10):1159-1166. [View Abstract] Hwang S et al. Concordance Between Clinical Diagnosis and Medicare Claims of Depression Among Older Primary Care Patients. Am J Geriatr Psychiatry. 2015;23(7):726-734. [View Abstract] Fung V, et al. Coverage Parity And Racial And Ethnic Disparities In Mental Health And Substance Use Care Among Medicare Beneficiaries. Health Aff (Millwood). 2023;42(1):83-93. [View Abstract]

Measures that cannot be Determined from the SEER - Medicare Data

Measure	Comment	Resources and References (if relevant)
Activities of daily living (ADL) status, performance status, functional status; patient's quality of life (QoL)	The claims data contain information necessary for a provider to receive payment. There is no patient self-reported or survey (e.g. SF-36) information in the SEER data or Medicare claims. This means ADLs, performance status, functional status and QoL cannot be directly measured. Researchers have attempted to infer performance status or functional status using claims-based evidence such as receipt of home health services or claims for oxygen and wheelchairs. While presence of these types of claims may be good indicators of poor performance, absence of these types of claims does not necessarily imply that a patient is in good health.	If interested in these and other patient-reported outcomes, researchers are advised to consider the SEER-MHOS linked data, which contains SEER cancer registry data linked to the Medicare Health Outcomes Study, an annual CMS survey of Medicare Advantage beneficiaries that contains information on ADLs, QoL, and other patient-reported outcomes.
Care paid by entities other than Medicare	If a service was not submitted to Medicare for payment, it will not appear in the claims data. Examples: claims for HMO enrollees; care provided in other settings, such as the Veterans Administration; care for persons with Medicare as the secondary payer (in some cases); reimbursement for covered services not captured by the Medicare data, e.g. out of pocket expenditures; or coverage provided by Medigap policies.
Disease free survival	Determining disease-free survival requires knowing when a cancer recurred. As noted above under "recurrence", a large portion of elderly cancer patients do not receive additional treatment in the event of recurrence. Also noted below, use of metastasis codes will misclassify those patients who did and did not have metastatic disease. Therefore the only patients for whom it is possible to identify disease free survival from the SEER-Medicare data are those who receive treatment for their recurrence. For these patients, the date of the first claim for additional cancer treatment would be the date that disease free survival ended.
Metastasis occurring after diagnosis (see recurrence and site of metastasis)	The SEER registries do not conduct active follow-up of patients other than to determine vital status. Therefore, the SEER data will not include information about metastasis occurring after initial diagnosis. The Medicare claims include ICD-9 diagnosis codes for secondary neoplasms to specific anatomic sites, ex. ICD-9 198.5 Secondary neoplasm to bone and bone marrow. Including these codes on a Medicare claim does not impact the amount of reimbursement to the provider; therefore metastases are less likely to be completely and accurately coded. Several population-based studies have examined the completeness and accuracy of the ICD-9 diagnosis codes for secondary neoplasm, primarily at the time of diagnosis. These studies have reported that the sensitivity, specificity and positive predictive value of the codes on the Medicare claims have considerable variability. In none of these studies, has the sensitivity, specificity and positive predictive value of the algorithms simultaneously exceeded 80%. Conclusion: Using ICD-9 codes on the SEER-Medicare data to identify patients with later metastatic disease will result in an incompletely and inaccurately classified cohort.	Whyte JL, Engel-Nitz NM, Teitelbaum A, Gomez Rey G, Kallich JD. An Evaluation of Algorithms for Identifying Metastatic Breast, Lung, or Colorectal Cancer in Administrative Claims Data. Med Care 2013 Mar 21. [Epub ahead of print] [View Abstract] Chawla N et al. Accuracy and completeness of diagnosis codes for cancer metastasis on Medicare claims. J Clin Oncol 31, 2013 (suppl; abstr 6521). Hassett MJ, Ritzwoller DP, Taback N, Carroll N, Cronin AM, Ting GV, Schrag D, Warren JL, Hornbrook MC, Weeks JC. Validating Billing/Encounter Codes as Indicators of Lung, Colorectal, Breast, and Prostate Cancer Recurrence Using 2 Large Contemporary Cohorts. Med Care 2012 Dec 6. [Epub ahead of print] [View Abstract] Nordstrom BL, Whyte JL, Stolar M, Mercaldi C, Kallich JD. Identification of metastatic cancer in claims data. Pharmacoepidemiol Drug Saf 2012 May;21 Suppl 2:21-8. doi: 10.1002/pds.3247. [View Abstract] Cooper GS, Yuan Z, Stange KC, Amini SB, Dennis LK, Rimm AA. The utility of Medicare claims data for measuring cancer stage. Med Care 1999 Jul;37(7):706-11. [View Abstract]
Physician treatment recommendations and patient preferences related to treatment	There are no diagnoses or procedures contained in Medicare files that can signal either treatment intent or whether specific treatments were considered and not given. Likewise, there is no information about patients' preferences for aggressive or palliative approaches. The claims contain information about treatment received but not treatment recommended, wanted or needed but not received.
Services Medicare does not cover	Services and treatments will only be found in the claims if they are covered by Medicare. Medicare has expanded its covered services multiple times since 1991. If a service is currently covered, this does not mean it has always been covered. Types of services that have not been continuously covered or have never been covered: Cancer screening. More information on coverage of screening. Preventive services. Drugs prior to FDA approval. For most clinical care, Medicare does not monitor if the drug is provided for the condition approved by the FDA. Long-term nursing home care that is not considered rehabilitative or skilled-nursing care. Routine physicals.	To identify what is covered, annual Medicare & You (PDF) handbooks are a good place to start. The "What's New" section in each handbook highlights some newly-covered services. For example, in 2005, these services were added: Cardiovascular screening blood tests. Diabetes screening tests. "Welcome to Medicare" physical examination. More detail may be found in provider manuals, such as: Medicare Claims Processing Manual and the Internet-Only Manuals (IOMs)
Test results - lab, imaging, surgical margins	With few exceptions, test results from lab tests and imaging are not identifiable in these data. The exceptions are some tumor marker tests (e.g. ER and PR status for breast cancers) that are found in the SEER-Medicare Cancer Data File and hemoglobin and hematocrit results for a subset of Medicare patients in the more recent Medicare claims data. Neither the SEER nor the Medicare data include any information about imaging results or surgical margins. Some researchers have tried to impute results based on subsequent treatment found in the claims. However, the lack of subsequent treatment does not necessarily indicate negative results. Rather lack of subsequent treatment only indicates that following the test there was no additional treatment.
Treatments or conditions occurring before Medicare eligibility	The Medicare data do not contain any information about treatments received or conditions diagnosed prior to Medicare eligibility. It is possible to use the SEER data to identify cancers diagnosed prior to Medicare enrollment if they were reported to a SEER registry. The amount of information in the SEER data is dependent on the year the patient was diagnosed with a prior cancer and the year the registry became part of SEER. Some registries, such as Connecticut, have diagnoses as early as 1973. Others, such as New Jersey have no details on cancers diagnosed prior to 2000. Thus, it is possible to find cancer cases diagnosed prior to age 65, but treatment information would be limited to that information contained in SEER records.	Inception dates for each SEER registry

Last Updated: 05 Sep, 2024