Stage II–III colon cancer: a comparison of survival calculators

Introduction

Colorectal cancer (CRC) is the second leading cause of cancer-related death in the United States and the third most common cancer in both men and women. In 2014, over fifty thousand people died from CRC in the United States (1). An additional 140,250 new cases are expected in 2018 (2). The role and efficacy of adjuvant chemotherapy for CRC have been well established. Modern chemotherapeutic regimens have been demonstrated to improve overall survival (OS), especially for stage II and III patients (3,4).

Accurate prognosis for patients diagnosed with cancer is critical. Stage of the disease has been demonstrated to be strongly associated with OS (5,6). However, predicting prognosis for individual patients remains imprecise. In recent years, several studies have included more clinicopathological features in order to enhance the accuracy of the predicted prognosis. Different prediction models have been developed to assess survival rate for individual patients, who have been treated for stage II and III colon cancer: (I) Roswell Park Comprehensive Cancer Center (RPCCC) calculator (7); (II) Memorial Sloan-Kettering Cancer Center calculator (MSKCC, available at https://www.mskcc.org/nomograms/colorectal) (8); (III) MD Anderson Cancer Center calculator (MDACC, available at http://www3.mdanderson.org/app/medcalc/index.cfm?pagename=coloncancer) (9); (IV) Mayo Clinic ACCENT calculator (available at https://www.mayoclinic.org/medical-professionals/cancerpredictiontools/colon-cancer) (10); and (V) Adjuvant! Online (available at http://www.adjvantonline.com/) (11). All of these offers post-operative survival calculation, with the exception of the MDACC calculator, which generates an estimate of 5-year conditional survival (CS).

While these models have been validated and assistance with clinical decision making (12,13), concordance in predicting survival rates can often be variable among each of these calculators. The purpose of this study was to compare these selected calculators and determine which, if any, was the most accurate in predicting the 5-year OS rate for individual patients with stage II/III colon cancer. We hypothesized that the more extensive inclusion of patient, tumor, and outcome-specific variables would optimize prediction of OS.

Methods

Patients

This was a retrospective review of a multi-site institutional experience for all patients diagnosed with clinical stage II or III colon cancer. Mayo Clinic Institutional Review Board (IRB) approval was obtained for this study; informed consent was waived by our IRB as this retrospective study was deemed minimal risk to patients. Patient selection was limited to the diagnosis made from January 1, 2012 to March 31, 2013. This criterion was influenced by the RPCCC OS calculator. The RPCCC calculator includes patients from National Cancer Data Base (NCDB) up to 2011. Due to the participation of Mayo Clinic in the NCDB, a cohort of patients outside the years used in the calculator (specifically 2004–2011) was required in order to perform appropriate external validation of this specific calculator. This criterion was not applicable to the other calculators, as these were developed using patients prior to 2011. Specifically, the Mayo Clinic calculator included patients accrued to 12 phase III clinical trials from 1989 to 2002 (10). The MSKCC and MDACC are based on the Surveillance, Epidemiology and End Results (SEER) Program and included patients from 1996 to 2005, and from 1988 to 2000, respectively (8,9). The Adjuvant! Online tool is also based on SEER prior to 2011 (11), although it is currently being updated.

As the Adjuvant! Online calculator is currently being revised, not every calculator could be included in this study. Adjuvant! Online was neither available nor accessible because of website maintenance. The Mayo Clinic ACCENT calculator could not be used as well, because of insufficient or incompatible data within our cohort. Specifically, 65 patients did not receive adjuvant chemotherapy, which is a mandatory variable for estimating survival rate using Mayo Clinic ACCENT calculator. Therefore, the Mayo Clinic and Adjuvant! Online calculators were excluded from the analysis. Figure 1 shows the selection criteria for the calculators.

Figure 1 Selection criteria for colon cancer survival rate calculators. RPCCC, Roswell Park Comprehensive Cancer Center; MSKCC, Memorial Sloan Kettering Cancer Center; MDACC, MD Anderson Cancer Center.

Surgical resection with confirmation of pathological stage II or III colon cancer was included among the primary inclusion criteria. Patients were identified using the International Statistical Classification of Disease (ICD CM-10) codes C18.0 to C18.9. Clinical or pathological stage I or IV patients were excluded. Patients with histology different from adenocarcinoma, carcinoma in situ, more than 1 recorded malignancy were excluded as well. Figure 2 summarizes the inclusion and exclusion criteria adopted for this study.

Figure 2 Flow chart showing the inclusion and exclusion criteria used to select patient for the comparison of the calculators.

The following patient-specific characteristics were collected: age, gender, race, and Charlson-Deyo comorbidity score as a measure of comorbidity status. Treatment and outcome-related variables included the surgical procedure (partial colectomy, sub-colectomy/hemicolectomy, total colectomy, total proctocolectomy), margins status, number of nodes examined and positive nodes at pathological examination, post-operative hospital length of stay (days), length of follow-up (months), unplanned readmission, and administration of adjuvant chemotherapy and/or adjuvant radiotherapy. The tumor-related characteristics included preoperative carcinoembryonic antigen (CEA) levels, tumor grade, size, pathological T and N stage (7th edition AJCC staging). Table 1 summarizes the calculator-specific collected variables as they pertain to the selected OS calculators.

Table 1 Variables needed to calculate survival rates for each calculator
Full table

Statistical analysis

Patient characteristics were summarized using the mean, median, and standard deviation for continuous variables and using frequencies and relative frequencies for categorical variables. OS was summarized using standard Kaplan-Meier methods, where estimates of median and 5-year OS were obtained with 95% confidence intervals. The 5-year OS survival scores were summarized for each calculator using the mean, median, and standard deviation. The association between scores from different calculators was examined using a scatter plot matrix, while their agreement was assessed using the intra-class correlation coefficient (ICC). The performance of the calculators was assessed using time-dependent receiver operating characteristic (ROC) curves and corresponding area under the curve (AUC) values. All analyses were conducted in SAS v9.4 (Cary, NC, USA) at a significance level of 0.05.

Results

A total of 504 patients were extracted from our multi-site institutional database. Following the application of the inclusion and exclusion criteria, a total of 97 patients were suitable for the analysis. The overall patient and tumor characteristics used to calculate survival rates are shown in Table 2. Figure 3 shows the Kaplan-Meier curve for OS. The survival data were available for all 97 patients, of which there were 23 events. The median follow-up was 57.6 months. The 5-year OS rate was 0.74 (95% CI: 0.64–0.82).

Table 2 Patients characteristics used to compare survival calculators
Full table

Figure 3 Kaplan-Meier curve showing 5-year OS. OS, overall survival.

The scatter plot matrix used to examine the agreement between all calculators is shown in Figure 4. Results indicate that the different calculators tended to rank patients similarly. Thus, if a patient obtained the highest survival score using the RPCCC calculator, then that patient was likely to obtain a high score for the other two calculators. However, there was variation on what the predicted survival score should be, indicated by the overall ICC of 0.017. When comparing how the MDACC performed with the other calculators, a negative ICC was obtained (−0.294 and −0.541), which is due to the survival scores of MD Anderson being generally more optimistic. Conversely, there was a reasonable level of agreement between the RPCCC and MSKCC calculators (ICC =0.723).

Figure 4 Agreement plot between Prediction Tools. RPCCC and MSKCC calculators show a reasonable level of agreement. ICC, intra-class correlation coefficient; RPCCC, Roswell Park Comprehensive Cancer Center; MSKCC, Memorial Sloan Kettering Cancer Center.

When evaluating individual calculator performance with the observed OS, the RPCCC calculator had the highest AUC (0.913), followed by the MSKCC calculator (AUC =0.803), and the MDACC calculator (AUC =0.644). Therefore, given the AUC >0.8, RPCCC and MSKCC calculators were similarly able to differentiate between good and poor outcome patients. Figure 5 shows the performance evaluation and the AUCs for each calculator.

Figure 5 Evaluating Prediction Tool Performance. The AUC for both RPCCC and MSKCC calculators is greater than 0.8; indicating that these calculators can differentiate between good and poor outcome patients. AUC, area under the curve; RPCCC, Roswell Park Comprehensive Cancer Center; MSKCC, Memorial Sloan Kettering Cancer Center; MDACC, MD Anderson Cancer Center.

Discussion

One of the most significant values of survival prediction models lies in the ability to estimate survival outcomes for individual patients, and then to quantify the potential benefit from adjuvant treatment. In this study, a comparison between existing calculators was performed, hypothesizing that a more comprehensive inclusion of patient, treatment, and tumor specific variables would lead to a more accurate prediction of OS.

The present analysis involved patients from three sites in different areas of the United States, which increased the heterogeneity of the studied population. Moreover, the multi-site institutional database was queried in order to perform an external validation with complete data for all RPCCC, MSKCC, and MDACC calculators. The median length of follow up (57.6 months) was sufficient to compare 5-year survival rates for each calculator.

The RPCCC calculator achieved the best performance (AUC =0.913) in estimating 5-year survival rates. Thus, the inclusion of more variables in the equation likely allows a higher level of input and analysis for each patient, resulting in a more accurate OS prediction. Variables which contribute to OS, including comorbid status (Charlson-Deyo index) and post-operative outcomes, such as length of stay and potential readmissions as surrogates for post-operative complications, increase the completeness of the prediction. These variables have been demonstrated to have a strong correlation with OS. As additional benefit, they can also be used as a measure of quality for surgery outcomes (7,14,15). At present, the RPCCC calculator is the only one incorporating these variables in its model (8-11). The MSKCC and MDACC calculators showed a lower performance (AUC =0.803 and 0.644, respectively), although these still show a good to very good ability for OS prediction. The lower AUC scores may be explained by the relatively limited number of variables included in their models. Furthermore, the MDACC calculator does not generate post-operative outcomes but rather predicts conditional 5-year survival. Thus, its performance is extrapolated to this patient population, which had surgery as part of their treatment course.

In the recent years, the development of calculators and tools estimating outcomes has increased and their use is becoming part of the assessment and decision-making process (16-18). Even though equations and nomograms, which these tools are based on, can be quite complex, they result practicable and easily accessible aids for patients and physicians (12,13). The RPCCC calculator equation is quite complex but was developed into a free, user friendly, mobile application (7) that will allow for more feasible use of the calculator. In addition, all of the other calculators are available online through web-based interactive pages. However, in spite of these promising advantages, the use of these tools is not intended as a replacement of the clinical judgement. Instead, these tools should be used an instrument that can refine the prognosis appraisal and treatment decision making, and potentially act as a bridge between individual patient and large-scale statistics.

Limitations of this comparison include those involved in retrospective studies, which introduces bias in patient selection. Moreover, patients were accrued from an institutional database, and thus this may affect the reproducibility of results on the whole population. Indeed, the OS of the cohort used in this study was higher than that reported in the cohort used to generate the RPCCC calculator. Thus, this may have influenced the accuracy of the calculators tested as a more population with more favorable observed outcomes may be better suited for certain calculators, whereas a different population with poorer observed outcomes may potentially be better suited for different calculators. The available data were also limited to a modest number of patients (n=97), and, as a consequence, this may influence the strength of the comparison. In this analysis, it was not possible to compare all existing calculators that estimate survival rates for colon cancer. A significant number of patients were not eligible to be used in the Mayo Clinic ACCENT calculator because of incompatible data. In addition, the Adjuvant! Online tool was unavailable online due to current updating and maintenance (19). The inclusion of these two calculators in the comparison would have granted a more comprehensive overview on the impact of all calculators in the clinical practice. Lastly, calculators estimate 5-year OS or CS instead of disease-free (DFS) or disease-specific (DSS) survival, which act as a valid surrogate for OS (20) and provide more specific information about mortality rates directly related to cancer.

Nonetheless, this study reports a valuable comparison between three important existing survival rate calculators for colon cancer. Based on the hypothesis that a more comprehensive list of variables, spanning patient, tumor, and treatment related characteristics would result in the most optimal post-operative OS survival calculator, the RPCCC calculator was shown to be the most accurate tool. A study utilizing a larger population, and perhaps in a multi-institutional cohort, would enhance the comparison between the different calculators, and is the focus of further studies.

Acknowledgements

Funding: This work was supported by Roswell Park Cancer Institute and National Cancer Institute (NCI) grant P30CA016056, and also by the Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery. We acknowledge and thank the American College of Surgeons Committee on Cancer for providing access to the Participant User File from the National Cancer Data Base.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Disclaimers: The American College of Surgeons Committee on Cancer provided the Participant User File from the National Cancer Data Base, but has not reviewed or validated the results or conclusions of our study.

References

1. Colorectal Cancer Statistics. Centers for Disease Control and Prevention. Available online: https://www.cdc.gov/cancer/colorectal/statistics/
2. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2018. CA Cancer J Clin 2018;68:7-30. [Crossref] [PubMed]
3. Babaei M, Balavarca Y, Jansen L, et al. Administration of adjuvant chemotherapy for stage II-III colon cancer patients: An European population-based study. Int J Cancer 2018;142:1480-9. [Crossref] [PubMed]
4. Gill S, Loprinzi CL, Sargent DJ, et al. Pooled analysis of fluorouracil-based adjuvant therapy for stage II and III colon cancer: who benefits and by how much? J Clin Oncol 2004;22:1797-806. [Crossref] [PubMed]
5. Lan YT, Yang SH, Chang SC, et al. Analysis of the seventh edition of American Joint Committee on colon cancer staging. Int J Colorectal Dis 2012;27:657-63.
6. Edge SB, Compton CC. The American Joint Committee on Cancer: the 7th edition of the AJCC cancer staging manual and the future of TNM. Ann Surg Oncol 2010;17:1471-4.
7. Gabriel E, Attwood K, Thirunavukarasu P, et al. Predicting Individualized Postoperative Survival for Stage II/III Colon Cancer Using a Mobile Application Derived from the National Cancer Data Base. J Am Coll Surg 2016;222:232-44. [Crossref] [PubMed]
8. Weiser MR, Gonen M, Chou JF, et al. Predicting survival after curative colectomy for cancer: individualizing colon cancer staging. J Clin Oncol 2011;29:4796-802. [Crossref] [PubMed]
9. Chang GJ, Hu CY, Eng C, et al. Practical application of a calculator for conditional survival in colon cancer. J Clin Oncol 2009;27:5938-43. [Crossref] [PubMed]
10. Renfro LA, Grothey A, Xue Y, et al. ACCENT-based web calculators to predict recurrence and overall survival in stage III colon cancer. J Natl Cancer Inst 2014.106. [PubMed]
11. Ravdin PM, Siminoff LA, Davis GJ, et al. Computer program to assist in making decisions about adjuvant therapy for women with early breast cancer. J Clin Oncol 2001;19:980-91. [Crossref] [PubMed]
12. Papamichael D, Renfro LA, Matthaiou C, et al. Validity of Adjuvant! Online in older patients with stage III colon cancer based on 2967 patients from the ACCENT database. J Geriatr Oncol 2016;7:422-9. [Crossref] [PubMed]
13. Gill S, Loprinzi C, Kennecke H, et al. Prognostic web-based models for stage II and III colon cancer: A population and clinical trials-based validation of numeracy and adjuvant! online. Cancer 2011;117:4155-65. [Crossref] [PubMed]
14. Chen VW, Hsieh MC, Charlton ME, et al. Analysis of stage and clinical/prognostic factors for colon and rectal cancer from SEER registries: AJCC and collaborative stage data collection system. Cancer 2014;120 Suppl 23:3793-806. [Crossref] [PubMed]
15. Hari DM, Leung AM, Lee JH, et al. AJCC Cancer Staging Manual 7th edition criteria for colon cancer: do the complex modifications improve prognostic assessment? J Am Coll Surg 2013;217:181-90.
16. Bardia A, Loprinzi C, Grothey A, et al. Adjuvant chemotherapy for resected stage II and III colon cancer: comparison of two widely used prognostic calculators. Semin Oncol 2010;37:39-46. [Crossref] [PubMed]
17. Gabriel E, Attwood K, Shah R, et al. Novel Calculator to Estimate Overall Survival Benefit from Neoadjuvant Chemoradiation in Patients with Esophageal Adenocarcinoma. J Am Coll Surg 2017;224:884-94.e1. [Crossref] [PubMed]
18. Sjoquist KM, Renfro LA, Simes RJ, et al. Personalizing Survival Predictions in Advanced Colorectal Cancer: The ARCAD Nomogram Project. J Natl Cancer Inst 2018;110:638-48. [Crossref] [PubMed]
19. Adjuvant! Online. Adjuvant! Inc. Available online: http://www.adjuvantonline.com/indexjsp
20. Abrams J. Disease-free survival versus overall survival as a primary end point for adjuvant colon cancer studies: a commentary. J Clin Oncol 2005;23:8564-5. [Crossref] [PubMed]