Nomogram predicting early recurrence defined by the minimum P value approach for colorectal liver metastasis patients receiving colorectal cancer resection with simultaneous liver metastasis resection: development and validation
Highlight box
Key findings
• The predictive nomogram for early recurrence of CRLM patients who received simultaneous resection demonstrated high specificity and accuracy in the training cohort and achieved good verification results in the test cohort.
What is known and what is new?
• Early recurrence in CRLM patients was associated with unfavourable prognoses in comparison with late recurrence, but previous studies mainly focused on patients who received hepatectomy alone or hepatectomy followed by neoadjuvant chemotherapy.
• Early recurrence in CRLM patients who received simultaneous resections was also associated with unfavourable clinical outcomes and the interval was defined as <13 months after surgery. Preoperative NLR levels, preoperative BUN levels and TBS scores were first proven to be useful in prognosis predictions for these patients.
What is the implication, and what should change now?
• It should be vigilant about the early recurrence in CRLM patients who received simultaneous resections, and preoperative NLR levels, BUN levels and TBS scores should be included in the risk stratification.
Introduction
Colorectal cancer (CRC) has become a predominant cancer and ranks as the second most common cause of death due to cancer worldwide (1). Liver metastases represent the major cause of CRC-related mortality (2), and synchronous liver metastases occur in up to 25% (3) of CRC patients at diagnosis.
Compared with systemic therapies, complete resection of both the primary tumour and liver metastases was associated with favourable oncological outcomes in colorectal cancer liver metastases (CRLM) patients (4,5). For resectable CRLM, a traditional staged procedure in which CRC resection was performed before or after liver metastasis resection increased the risk of liver metastasis progression and dissemination to other distant sites in the interval between the two operations (6), while simultaneous resections of both lesions could reduce the abovementioned risks, provide the benefit of short hospital stays, reduce hospital costs and improve patient experiences (6,7). As previous studies suggested, simultaneous resections of primary lesions and liver metastases exhibit comparable short-term outcomes and have a tendency to improve long-term outcomes such as progression-free survival and overall survival (OS) in comparison with traditional staged procedures (6-8). Although simultaneous resections have been increasingly (8) performed for resectable CRLM patients, only a handful of studies have aimed to investigate the risk stratification of these patients following the simultaneous operation.
Mounting evidence indicates that compared with late recurrences that occur a long period after surgery, early recurrences usually indicate a significantly worse survival expectancy for multiple cancers, including hepatocellular carcinoma, gastric cancer, and renal cell carcinoma (9-11). Previous studies have identified multiple factors that may predict early recurrence, including demographics characteristics such as male gender (12), clinical/pathological information such as the size of liver metastasis (13), features of the intervention like resection margin (14), and cancer-specific biomarkers like carcinoembryonic antigen (CEA) (15). In addition, early recurrence in CRLM patients (15,16) was also associated with unfavourable prognoses in comparison with late recurrence, but previous studies mainly focused on patients who received hepatectomy alone or hepatectomy followed by neoadjuvant chemotherapy (NAC). Patients who received simultaneous resections were ignored. Identifying risk factors and categorizing patients based on those factors is vital to effective treatment planning for CRLM patients (17). By doing so, customized treatment and the most appropriate care for patients’ individual situation could be achieved to meet the unique needs of each patient. Since simultaneous resections have been performed more often in recent years, investigating the relationship between early recurrence and these patients would provide clinicians with new insights into selecting patients for receiving simultaneous resection decisions, and fill the gap of the prognostic stratification for these patients. A clear definition of early recurrence remains controversial, and recurrence-free survival (RFS) times varying arbitrarily from 6 months to 1 year (15,16) were previously recognized as the cut-off value for early recurrence. In the current study, we performed the minimum P value approach (18,19) to explore the evidence-based cut-off value of early recurrence.
A nomogram is a reliable, convenient tool for predicting risk and is helpful for clinical decision-making in the management of cancer patients. Given the abovementioned evidence, this study aimed to statistically define early recurrence, construct a nomogram for predicting early recurrence in CRLM patients receiving simultaneous resections of the primary tumour and liver metastases, and validate the model in an external test cohort. We present this article in accordance with the TRIPOD reporting checklist (available at https://jgo.amegroups.com/article/view/10.21037/jgo-22-934/rc).
Methods
Study population
We retrospectively conducted the collection and analysis of the medical records of CRLM patients who received simultaneous resection from December 2008 to August 2020. The training cohort enrolled patients from Cancer Hospital, Chinese Academy of Medical Sciences while the test cohort enrolled patients from Sun Yat-sen University Cancer Center, the enrollment were performed complying with same criteria as described below. Patients with all of the following criteria were enrolled: (I) clinical or histological evidence of CRLM; (II) simultaneous resection of primary tumour and liver metastases with curative purpose; and (III) developed recurrence. Patients with either of the following criteria were excluded: (I) loss to follow-up; (II) incomplete clinical information; and (III) synchronous other malignant disease or infectious disease or other related diseases of liver and kidney. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Institutional Review Board of the Cancer Hospital, Chinese Academy of Medical Sciences and Sun Yat-sen University Cancer Center (No. 81972311) and informed consent was taken from all the patients.
Data collection and study variables
Detailed data were retrieved from medical records of each patient, including demographics characteristics, clinicopathologic features [preoperative cholesterol level, lymphocyte counts, neutrophil counts, platelet counts, albumin (ALB), blood urea nitrogen (BUN), creatine, serum CEA], medical treatment and survival results. Abovementioned preoperative markers were included in the current study as they were routinely tested and supposed to be correlated with the prognosis of CRLM patients. Besides, since there were no verified biomarkers associated with the early recurrence in CRLM patients who received simultaneous resection, we included as many predictors as possible. All the included information were determined to be collected before the study began. Peripheral samples of each patient were collected by venous punctures within 1–3 weeks before the surgery.
To better predict the prognosis of early recurrence in CRLM patients, we made some combination of preoperative biomarkers and tumor morphology characteristics. Tumor burden scores (TBS) was calculated from the distance from the origin on a Cartesian plane combining the maximum tumor size (x-axis) and number of lesions (y-axis) [TBS2 = (maximum tumor diameter)2 + (number of liver lesions)2], and the predictive utility of this index in CRLM patients was proved to be of higher specialty and sensitivity than that of tumor size or liver lesion number alone (20,21), so we here included the combined biomarker instead of singular characteristics of tumor morphology, to predict the early recurrence. Neutrophil-to-lymphocyte ratio (NLR) and platelet-to-lymphocyte ratio (PLR) was calculated as neutrophil counts/lymphocyte counts and platelet counts/lymphocyte counts, separately.
All postoperative complications were accessed by the Clavien-Dindo classification system, and minor complications were recognized as Clavien-Dindo I–II while major complications were defined as Clavien-Dindo III–V.
Treatment
For complete treatment procedure, a routinely discussion was performed by a multidisciplinary team (MDT) including surgeons, oncologists and radiologists, and a consensus was reached before determining the appropriate treatment for each CRLM patients. Surgical margin (R0 resection or not), extent of liver resection (major resection, or not), and hepatic portal occlusion were the surgical data taken into analysis. Resections ≥three segments of liver metastases were defined as major resection (22). Preoperative chemotherapy regimens, with or without bevacizumab and cetuximab, mainly including 5-flfluorouracil or capecitabine, and oxaliplatin or irinotecan, the regimens were recommended to CRLM patients with one or more risk factors of recurrence as previously defined (17,23).
Follow-up and outcomes
Postoperative complications consisted of incision dehiscence, postoperative infection, anastomotic leakage, hemorrhage, pleural and peritoneal effusion, and so on, and were previously accessed by clinicians and recorded accordingly.
Regular follow-up examinations were performed for patients after surgery. The first follow-up interval period was one month from the date of surgery. Then, the interval periods were every 3 months for 5 years and thereafter every 1 year. The period of time from the date of simultaneous resection to recurrence or latest follow-up if recurrence did not occur, was defined as RFS. The period of time from the date of simultaneous resection to death or latest follow-up was defined as OS. The period time from the first recurrence to either death or latest follow-up was defined as post-recurrence survival (PRS). The recurrence of each patient was comprehensively evaluated by experienced clinicians via monitoring level of CEA and computed tomography (CT)/magnetic resonance imaging (MRI), then judgement of the results was made complying with the European Society for Medical Oncology (ESMO) guideline (5).
Statistical analysis
CRLM patients were divided into potential early recurrence group and late recurrence group by different cut-off values of RFS which was assessed by minimum P value method to determine its optimal threshold. And the log-rank test was conducted to compare the according PRS of different cut-off value of RFS to find the minimum P value, and cut-off value with the minimum P value was defined as the value of early recurrence in our study. The Kaplan-Meier method was performed to analyse RFS and OS, and the log-rank test was conducted to do the statistically comparison.
For continuous variables, the Mann-Whitney U test was used to do the comparison, and the categorical variables were compared via the Chi-squared test or the Fisher’s exact test. The receiver operating characteristic (ROC) curves for early recurrence were constructed to determine the optimal threshold of TBS, NLR, PLR, creatine and BUN. To determine the optimal cut-off value of above factors, the highest Youden’s index (sensitivity + 1 − specificity) which presented graphically as the distance between the 45° line and the ROC graph was calculated for each factor.
In the multivariate logistic regression model, all predictors with a P value of less than 0.10 in the univariate analysis were retained to determine the relationship between the variables and early recurrence. Only those predictors with a P value less than 0.05 were defined as significant and then included in the final model. The resulting multivariate logistic regression model incorporated all significant risk factors to develop the nomogram in the training set, and the nomogram was validated externally in the test cohort. The discrimination performance of the nomogram was estimated by C-index, and the area under ROC curves (AUROCs). The ROC curve generated the values of sensitivity and specificity for the predictive model. The sensitivity, specificity, C-index or area under the curve (AUC) values range from 0.5 to 1.0, with 0.5 indicating random chance and 1.0 signifying the ideal capability to correctly predict the outcome. We considered a value of 0.6 to be an acceptable threshold and a value of 0.7 to be a good threshold for these measures. The calibration of the models was assessed by calibration plots and the Hosmer-Lemeshow chi-square test. Besides, the decision curve analysis was performed to test the clinical effectiveness of the model.
A two-sided P value ≤0.05 was defined as of statistically significance. All the statistical analysis were performed by SPSS version 25 software (Armonk NY, USA) and R software (http://www.r-project.org).
Results
Clinicopathological characteristics of CRLM patients with recurrence
A total of 323 CRLM patients from the Cancer Hospital, Chinese Academy of Medical Sciences, were enrolled in the training cohort and had a median age of 58.0 [interquartile range (IQR), 51.0–65.0] years. Thirty-eight (11.8%) patients who had American Society of Anesthesiologists (ASA) scores of 3–4, and 153 (47.4%) patients who had body mass index (BMI) ≥24 kg/m2. In the training set, 209 (64.7%) were male, and 114 (35.3%) were female. In addition, 137 (42.4%) patients who had comorbidities, 20.1% (65/323) of primary tumours were located in the right hemicolon, 79.6% (257/323) of positive lymph node metastases were observed at diagnosis, and 145 patients in the training cohort (44.9%) had bilobular distributions of liver lesions. And there were 204 (63.2%) patients who had multiple liver metastases and 39 (12.1%) patients who had highest diameter of liver metastases ≥5 cm. One hundred seventy-two (172/323, 53.3%) patients underwent major liver resection. The median operation time was 345.0 (IQR, 275.0–720.0) min, and the median blood loss volume during the operations was 200.0 (IQR, 150.0–500.0) mL. Extrahepatic metastases occurred in 39 (12.1%) patients in this cohort. In total, 56.0% (181/323) and 65.9% (213/323) of patients received preoperative chemotherapy and postoperative chemotherapy, respectively.
For the preoperative laboratory indicators, the median level of preoperative cholesterol was 178.65 (IQR, 155.45–207.66) mg/dL. The median lymphocyte counts, neutrophil counts, and platelet counts were 1.67 (IQR, 1.34–2.08) ×109/L, 3.10 (IQR, 2.41–4.35) ×109/L, and 200.0 (IQR, 157.0–248.0) ×109/L, respectively. The median levels of BUN and creatine were 4.70 (IQR, 4.20–5.40) µmol/L and 67.0 (IQR, 58.0–77.0) µmol/L, respectively. Seventeen (17/323, 5.3%) patients who had CEAs ≥200 ng/mL. And the median TBS scores was 4.24 (2.79–6.10) in the training cohort. The optimal cut-off values for early recurrence were calculated by using the highest Youden’s index values for TBS, NLR, PLR, creatine and BUN, which were 4.09, 1.44, 133.97, 70.5, and 3.55 µmol/L, respectively.
From December 2008 to August 2020, we included 71 patients from Sun Yat-sen University Cancer Center in this research, as the test cohort followed the same inclusion and exclusion criteria as the training cohort. Detailed information on the demographics and clinicopathological features is described in Table 1.
Table 1
Item | Training cohort | Test cohort | |||||||
---|---|---|---|---|---|---|---|---|---|
All patients (n=323) | Early recurrence (<13 m) (n=241) | Late recurrence (≥13 m) (n=82) | P | All patients (n=71) |
Early recurrence (<13 m) (n=49) | Late recurrence (≥13 m) (n=22) | P | ||
Demographics and clinicopathological features | |||||||||
Age ≥60 years | 146 (45.2) | 109 (45.2) | 37 (45.1) | 0.987 | 26 (36.6) | 20 (40.8) | 6 (27.3) | 0.273 | |
Male | 209 (64.7) | 161 (66.8) | 48 (58.5) | 0.176 | 47 (66.2) | 32 (65.3) | 15 (68.2) | 0.813 | |
BMI ≥24 kg/m2 | 153 (47.4) | 119 (49.4) | 34 (41.5) | 0.215 | 25 (35.2) | 18 (36.7) | 7 (31.8) | 0.688 | |
Comorbidity | 137 (42.4) | 106 (44.0) | 31 (37.8) | 0.328 | 22 (31.0) | 16 (32.7) | 6 (27.3) | 0.650 | |
ASA score 3–4 | 38 (11.8) | 27 (11.2) | 11 (13.4) | 0.591 | 2 (2.8) | 2 (4.1) | 0 (0) | 0.336 | |
Primary site in colon | 181 (56.0) | 133 (55.2) | 48 (58.5) | 0.598 | 42 (59.2) | 29 (59.2) | 13 (59.1) | 0.994 | |
Right hemicolon | 65 (20.1) | 51 (21.2) | 14 (17.1) | 0.425 | 12 (16.9) | 10 (20.4) | 2 (9.1) | 0.239 | |
Diameter of liver metastases ≥5 cm | 39 (12.1) | 33 (13.7) | 6 (7.3) | 0.126 | 15 (21.1) | 14 (28.6) | 1 (4.5) | 0.022 | |
Multiple liver metastases | 204 (63.2) | 161 (66.8) | 43 (52.4) | 0.02 | 39 (54.9) | 31 (63.3) | 8 (36.4) | 0.035 | |
Bilobar liver distribution | 145 (44.9) | 115 (47.7) | 30 (36.6) | 0.08 | 25 (35.2) | 20 (40.8) | 5 (22.7) | 0.140 | |
Preoperative TBS levels | 4.24 (2.79–6.10) | 4.47 (3.16–6.40) | 3.31 (2.24–5.07) | <0.001 | 3.20 (2.06–5.83) | 4.39 (2.24–7.18) | 2.50 (1.72–3.17) | 0.004 | |
Poor differentiation | 104 (32.2) | 84 (34.9) | 20 (24.4) | 0.08 | 13 (18.3) | 10 (20.4) | 3 (13.6) | 0.495 | |
T3–T4 stage | 303 (93.8) | 227 (94.2) | 76 (92.7) | 0.625 | 66 (93.0) | 46 (93.9) | 20 (90.9) | 0.651 | |
Positive lymph node metastasis | 257 (79.6) | 200 (83.0) | 57 (69.5) | 0.009 | 49 (69.0) | 34 (69.4) | 15 (68.2) | 0.919 | |
Extrahepatic metastases | 39 (12.1) | 34 (14.1) | 5 (6.1) | 0.054 | 6 (8.5) | 5 (10.2) | 1 (4.5) | 0.428 | |
Preoperative laboratory testing levels | |||||||||
CEA ≥200, ng/mL | 17 (5.3) | 12 (5.0) | 5 (6.1) | 0.695 | 4 (5.6) | 4 (8.2) | 0 (0.0) | 0.168 | |
Cholesterol, mg/dL | 178.65 (155.45–207.66) |
178.27 (154.10–206.69) |
182.13 (164.06–211.04) |
0.223 | 190.67 (173.58–215.72) |
192.14 (177.26–217.07) |
175.71 (169.72–210.31) |
0.271 | |
Lymphocyte counts, ×109/L | 1.67 (1.34–2.08) | 1.64 (1.30–2.05) | 1.76 (1.44–2.27) | 0.053 | 1.60 (1.20–2.10) | 1.60 (1.20–2.05) | 1.80 (1.20–2.20) | 0.558 | |
Neutrophil counts, ×109/L | 3.10 (2.41–4.35) | 3.10 (2.43–4.36) | 3.14 (2.32–4.30) | 0.737 | 3.60 (2.60–4.80) | 3.60 (2.78–4.85) | 3.39 (2.53–4.43) | 0.422 | |
Platelet counts, ×109/L | 200.0 (157.0–248.0) |
196.0 (156.5–246.5) |
206.0 (158.3–266.5) |
0.415 | 227.0 (185.0–284.0) |
221.0 (174.9–278.0) |
242.5 (203.1–314.3) |
0.052 | |
Blood urea nitrogen, μmol/L | 4.70 (4.20–5.40) | 4.70 (4.25–5.40) | 4.70 (4.10–5.70) | 0.706 | 4.41 (3.63–5.80) | 4.65 (3.70–5.94) | 4.40 (3.20–5.18) | 0.323 | |
Creatine, μmol/L | 67.0 (58.0–77.0) | 68.0 (58.5–78.5) | 65.5 (58.0–74.0) | 0.078 | 67.0 (54.0–80.7) | 67.1 (55.3–80.1) | 64.9 (51.1–81.4) | 0.663 | |
Treatment details | |||||||||
R0 resection | 229 (70.9) | 162 (67.2) | 67 (81.7) | 0.013 | 60 (84.5) | 41 (83.7) | 19 (86.4) | 0.772 | |
Major liver resection | 172 (53.3) | 139 (57.7) | 33 (40.2) | 0.006 | 22 (31.0) | 14 (28.6) | 8 (36.4) | 0.511 | |
Pretreatment chemotherapy | 181 (56.0) | 135 (56.0) | 46 (56.1) | 0.99 | 34 (47.9) | 27 (55.1) | 7 (31.8) | 0.069 | |
Postoperative chemotherapy | 213 (65.9) | 152 (63.1) | 61 (74.4) | 0.062 | 60 (84.5) | 43 (87.8) | 17 (77.3) | 0.259 | |
Hepatic portal occlusion | 222 (68.7) | 177 (73.4) | 45 (54.9) | 0.002 | 27 (38.0) | 16 (32.7) | 11 (50.0) | 0.164 | |
All laparoscopic operation | 56 (17.3) | 38 (15.8) | 18 (22.0) | 0.201 | 1 (1.4) | 1 (2.0) | 0 (0.0) | 0.500 | |
Operation time, min | 345.0 (275.0–720.0) |
350.0 (281.0–435.0) |
302.0 (238.5–401.3) |
0.019 | 330.0 (240.0–450.0) |
360.0 (270.0–465.0) |
300.0 (240.0–397.5) |
0.141 | |
Blood loss, mL | 200.0 (150.0–500.0) |
200.0 (150.0–500.0) |
200.0 (100.0–400.0) |
0.181 | 100.0 (50.0–100.0) |
100.0 (50.0–100.0) |
100.0 (50.0–162.5) |
0.452 | |
Blood transfusion | 81 (25.1) | 56 (23.2) | 25 (30.5) | 0.191 | 8 (11.3) | 5 (10.2) | 3 (13.6) | 0.672 | |
Postoperative complications | 162 (50.2) | 130 (53.9) | 32 (39.0) | 0.02 | 18 (25.4) | 13 (26.5) | 5 (22.7) | 0.733 | |
ICU | 20 (6.2) | 15 (6.2) | 5 (6.1) | 0.967 | 17 (23.9) | 12 (24.5) | 5 (22.7) | 0.872 | |
Postoperative hospital stay, days | 10.0 (9.0–14.0) | 11.0 (9.0–14.0) | 10.0 (9.0–12.3) | 0.2 | 10.0 (8.0–12.0) | 10.0 (8.5–13.0) | 10.0 (8.0–11.0) | 0.221 |
Data are presented as n (%) or median (IQR). CRLM, colorectal liver metastases; m, months; BMI, body mass index; ASA, American Society of Anesthesiology; IQR, interquartile range; TBS, tumour burden scores; CEA, carcinoembryonic antigen; ICU, intensive care unit.
Defining early and late recurrence
The various cut-off values that were examined to define early recurrence and the corresponding survival outcomes are shown in Table 2. Among the 323 CRLM patients in the training cohort with recurrence, the optimal RFS value, which could yield the optimal discrimination efficiency for early and late recurrence based on subsequent PRS, was 13 months (P=0.00083). For patients with early recurrence (<13 months, n=241, 74.6%), the median RFS was 4.7 [95% confidence interval (CI): 3.7–5.7] months, and the median PRS was 27.0 (95% CI: 24.6–29.5) months. Patients with late (≥13 months) recurrence (n=82, 25.4%) had a median PRS of 19.0 (95% CI: 17.4–20.6) months and median PRS of 52.8 (95% CI: 38.8–66.8) months. The median OS of patients with early recurrence was 33.8 (95% CI: 30.7–36.9) months, and the median OS of patients with late recurrence was 70.9 (95% CI: 64.6–77.2) months. CRLM patients with early recurrence had significantly unfavourable survival outcomes, including RFS (P<0.0001), PRS (P=0.00083), and OS (P<0.0001) (Figure 1). Patients with early recurrence were more likely to have longer operation times [350.0 (IQR, 281.0–435.0) vs. 302.0 (IQR, 238.5–401.3) min, P=0.019], multiple liver metastases (P=0.02), positive lymph node metastases (P=0.009), postoperative complications (P=0.02), major resection (P=0.006) and hepatic portal occlusion (P=0.002) than patients with late recurrence. Patients with late recurrence were likely to have R0 resection (P=0.013) and had a tendency to receive postoperative chemotherapy (P=0.062), while they had a tendency to not have bilobar liver distributions (P=0.08), poor differentiation (P=0.08) or extrahepatic metastases (P=0.054) in comparison with patients with early recurrence (Table 1). Nevertheless, the validation of the early recurrence value was conducted in the test cohort, and this cut-off value exhibited the ability to remarkably distinguish the RFS probability as well as the OS probability among these CRC patients (Figure S1). To further validate this cut-off value, we retrieved relevant clinical data from two colorectal cancer Gene Expression Omnibus (GEO) cohorts including GSE103479 and GSE106584. In both cohorts, this cut-off value could also significantly differentiate the RFS probability as well as the OS probability among these CRC patients (Figures S2,S3).
Table 2
Evaluated cut-off | P | Potential early recurrence cohort | Potential late recurrence cohort | |||||||
---|---|---|---|---|---|---|---|---|---|---|
N | RFS (m) | PRS (m) | OS (m) | N | RFS (m) | PRS (m) | OS (m) | |||
2 | 0.4337436 | 79 | 1.1 | 28.0 | 30.0 | 244 | 10.0 | 31.8 | 44.1 | |
3 | 0.1717507 | 95 | 1.3 | 26.9 | 28.0 | 228 | 10.2 | 32.4 | 46.0 | |
4 | 0.1086597 | 112 | 1.43 | 27.0 | 28.0 | 211 | 11.0 | 32.8 | 46.9 | |
5 | 0.1326299 | 125 | 1.6 | 28.0 | 30.0 | 198 | 11.2 | 32.4 | 52.7 | |
6 | 0.02438825 | 144 | 1.9 | 27.0 | 30.1 | 179 | 12.5 | 35.7 | 57.7 | |
7 | 0.007911529 | 158 | 2.0 | 27.0 | 30.0 | 165 | 13.0 | 35.7 | 58.3 | |
8 | 0.04603317 | 178 | 2.8 | 28.3 | 32.0 | 145 | 14.0 | 35.7 | 58.3 | |
9 | 0.02406499 | 187 | 3.0 | 28.0 | 31.1 | 136 | 14.5 | 37.5 | 58.8 | |
10 | 0.004676895 | 207 | 3.5 | 28.0 | 32.0 | 116 | 15.7 | 41.8 | 70.2 | |
11 | 0.00483053 | 219 | 4.0 | 28.0 | 32.1 | 104 | 17.0 | 42.7 | 70.2 | |
12 | 0.002170195 | 230 | 4.2 | 28.0 | 32.6 | 93 | 17.9 | 47.9 | 70.9 | |
13 | 0.0008316524 | 241 | 4.7 | 27.0 | 33.8 | 82 | 19.0 | 52.8 | 70.9 | |
14 | 0.01058733 | 252 | 5.1 | 28.0 | 34.1 | 71 | 19.9 | 47.9 | 70.9 | |
15 | 0.05074251 | 259 | 5.3 | 28.7 | 34.6 | 64 | 21.1 | 42.7 | 70.9 | |
16 | 0.09058815 | 267 | 5.4 | 28.8 | 35.2 | 56 | 23.0 | 47.9 | 70.9 | |
17 | 0.1104402 | 273 | 5.6 | 29.3 | 35.2 | 50 | 23.3 | 47.9 | 75.9 | |
18 | 0.104548 | 277 | 5.7 | 29.3 | 35.7 | 46 | 23.4 | 47.9 | 75.9 |
m, months; OS, overall survival; PRS, post-recurrence survival; RFS, recurrence-free survival.
Factors associated with early recurrence
In the univariate analysis (Table 3), R0 resection (P=0.014), positive lymph node metastases (P=0.010), postoperative complications (P=0.020), major resection (P=0.007), hepatic portal occlusion (P=0.002), NLR ≥1.44 (P=0.005), PLR ≥133.97 (P=0.041), BUN ≥3.55 µmol/L (P=0.039), creatine ≥70.5 µmol/L (P=0.014), and TBS ≥4.09 (P<0.001) were significantly correlated with early recurrence in the training cohort. In addition, extrahepatic metastases (P=0.062), poor differentiation (P=0.082) and postoperative chemotherapy (P=0.063) exhibited tendencies towards early recurrence. A multivariate logistic regression analysis was performed to identify those factors that were independently associated with early recurrence. The above predictors (P<0.10) were included in the multivariate analysis, and positive lymph node metastases [odds ratio (OR) =2.549, 95% CI: 1.366–4.756, P=0.003); postoperative complications (OR =1.763, 95% CI: 1.021–3.046, P=0.042); NLR ≥1.44 (OR =2.192, 95% CI: 1.254–3.830, P=0.006); BUN ≥3.55 µmol/L (OR =2.593, 95% CI: 1.189–5.652, P=0.017); and TBS ≥4.09 (OR =2.545, 95% CI: 1.473–4.398, P=0.001) were identified as independently correlated with early recurrence (Table 3).
Table 3
Factor | Univariate analysis | Multivariate analysis | |||
---|---|---|---|---|---|
P | OR (95% CI) | P | OR (95% CI) | ||
Demographics and clinicopathological features | |||||
Age ≥60 years | 0.987 | 1.004 (0.607–1.662 | – | – | |
Male | 0.177 | 1.426 (0.852–2.385) | – | – | |
BMI ≥24 kg/m2 | 0.216 | 1.377 (0.830–2.286) | – | – | |
Comorbidity | 0.329 | 1.292 (0.773–2.159) | – | – | |
ASA score 3–4 | 0.592 | 0.814 (0.384–1.725) | – | – | |
CEA ≥200 ng/mL | 0.696 | 0.807 (0.275–2.364) | – | – | |
Primary site in colon | 0.598 | 0.872 (0.525–1.449) | – | – | |
Right hemicolon | 0.426 | 1.304 (0.679–2.505) | – | – | |
T3–T4 stage | 0.625 | 1.280 (0.475–3.448) | – | – | |
Positive lymph node metastasis | 0.010 | 2.139 (1.200–3.813) | 0.003 | 2.549 (1.366–4.756) | |
Bilobar liver distribution | 0.081 | 1.582 (0.945–2.649) | – | – | |
Extrahepatic metastases | 0.062 | 2.529 (0.955–6.703) | – | – | |
Poor differentiation | 0.082 | 1.659 (0.939–2.931) | – | – | |
Preoperative laboratory indicators | |||||
TBS ≥4.09 | <0.001 | 2.760 (1.639–4.648) | 0.001 | 2.545 (1.473–4.398) | |
NLR ≥1.44 | 0.005 | 2.104 (1.247–3.551) | 0.006 | 2.192 (1.254–3.830) | |
PLR ≥133.97 | 0.041 | 1.794 (1.025–3.139) | – | – | |
BUN ≥3.55 μmol/L | 0.039 | 2.157 (1.041–4.471) | 0.017 | 2.593 (1.189–5.652) | |
Creatine ≥70.5 μmol/L | 0.014 | 1.962 (1.144–3.365) | – | – | |
Treatment details | |||||
R0 resection | 0.014 | 0.014 (0.247–0.854) | – | – | |
Major liver resection | 0.007 | 2.023 (1.215–3.370) | – | – | |
Pretreatment chemotherapy | 0.990 | 0.997 (0.602–1.651) | – | – | |
Postoperative chemotherapy | 0.063 | 0.588 (0.336–1.030) | – | – | |
Hepatic portal occlusion | 0.002 | 2.274 (1.351–3.827) | – | – | |
Operation time ≥345 min | 0.351 | 1.270 (0.769–2.098) | – | – | |
Intraoperative blood loss ≥200 mL | 0.483 | 1.221 (0.699–2.132) | – | – | |
Blood transfusion | 0.192 | 0.690 (0.395–1.205) | – | – | |
Postoperative complications | 0.020 | 1.830 (1.098–3.050) | 0.042 | 1.763 (1.021–3.046) |
CRLM, colorectal liver metastases; OR, odds ratio; CI, confidence interval; BMI, body mass index; ASA, American Society of Anesthesiology; CEA, carcinoembryonic antigen; TBS, tumour burden scores; NLR, neutrophil-to-lymphocyte ratio; PLR, platelet-to-lymphocyte ratio; BUN, blood urea nitrogen.
Construction and validation of the nomogram for early recurrence
All five independent risk factors for early recurrence that were identified by the multivariate regression analysis were included to establish the predictive nomogram. The specific scores of the independent factors were as follows: positive lymph node metastases, 99; postoperative complications, 60; NLR ≥1.44, 82; BUN ≥3.55 µmol/L, 100 and TBS ≥4.09, 99. For each patient, the total risk scores were calculated based on the nomogram, and the total number of points ranged from 0 to 440 (Figure 2). According to the ROC curve, the cut-off value of risk scores for early recurrence was set at 289.5 with a sensitivity of 0.618 and specificity of 0.707 in the training cohort. In the validation cohort, the sensitivity of the nomogram was 0.735, and the specificity was 0.682. The nomogram for predicting early recurrence had an AUROC of 0.720 (95% CI: 0.660–0.781) (Figure 3A), indicated acceptable model calibration (P=0.7612) (Figure 3B) and showed good clinical applicability (Figure 3C) in the training cohort. And in the test cohort, the nomogram for predicting early recurrence had an AUROC of 0.740 (95% CI: 0.625–0.854) (Figure 4A), exhibited acceptable consistency between the predicted and observed survival probability (P=0.8671) (Figure 4B) and could generate net benefits (Figure 4C).
Discussion
In this study, we first verified the prognostic role of early recurrence in CRLM patients who received simultaneous resections of the primary lesion and liver metastases. A nomogram was constructed and externally validated for predicting early recurrence in these patients. The main findings were as follows: (I) early recurrence in CRLM patients who received simultaneous resections of the primary lesion and liver metastases was 13 months; (II) positive lymph node metastases, preoperative TBS ≥4.09, preoperative NLR ≥1.44, preoperative BUN ≥3.55 µmol/L and postoperative complications were independently associated with early recurrence; and (III) the predictive nomogram for early recurrence demonstrated high specificity and accuracy in the training cohort and achieved good verification results in the test cohort.
Early recurrence usually exerts a significant negative impact on postoperative survival for cancer patients, including CRLM patients who receive hepatectomy alone or hepatectomy followed by NAC (15,16,24). In our study, we statistically calculated the optimal cut-off value for early recurrence and verified that early recurrence was associated with unfavourable survival outcomes in CRLM patients who received simultaneous resections. For improve postoperative surveillance, it is very important to perform risk stratification of patients and recognize those who are more likely to experience early recurrence. In addition, it was reported that the same or different therapies with curative intent after initial treatment for CRLM patients who suffered from early recurrence, such as repeated local treatments (25), might improve survival outcomes, so the early recognition of these patients would help to guide personalized medical management. Also, the proportion of early recurrence among CRLM patients with recurrence in the current study was consistent with previous studies (26-28). TBS is a prognostic index that combines the impacts of tumour size and number, and many previous studies have explored its utility in discriminating the survival of CRLM patients. A recent study (20) combined DNA ploidy and TBS to stratify patients into different recurrence risk groups. TBS showed high discriminatory value in predicting conversion outcomes in patients with initially unresectable CRLM (21). Nevertheless, determining the relationship between TBS and early recurrence in CRLM patients is elusive. In our study, a higher TBS was first verified as being independently associated with early recurrence, and accumulated evidence that TBS was a reliable tool to comprehensively represent tumour morphology and could accurately predict the survival of CRLM patients.
In this study, we identified the NLR and BUN levels as independent risk factors for early recurrence in CRLM patients who received simultaneous resections. Since these factors could be routinely calculated from laboratory test results before surgery, it was convenient and easy to repeat these measurements to perform risk stratification of patients by considering these indices. A dysfunctional inflammatory response, such as a chronic inflammatory status, plays an essential role in the initiation, development, and progression of cancer cells (29); thus, inflammation-based prognostic indices, such as NLR and PLR, have been developed and explored for their predictive value in many cancers (30,31). Among them, NLR was presumed to be more frequently used to predict prognoses than other indices, and elevated NLR was reported to be associated with poor prognoses in CRLM patients (30). However, the role of NLR in the early recurrence of CRLM remains unknown. In our research, an elevated NLR was proven to be independently associated with early recurrence. Elevated NLR means relative lymphocytopenia in patients, and depletion of lymphocytes usually indicates a deficiency of potent cytotoxins, such as perforin (31), which play an important role in anti-tumour immunity, thus promoting the protection of cancer cells from immune surveillance and attack. In addition, the relatively increased amounts of neutrophil cells could upregulate the vascular endothelial growth factor and aid the development and progression of cancer (31). BUN and creatinine are generally considered to be biomarkers of catabolism, and both have been reported to be correlated with poor diagnoses in many cancers (32-34); however, evidence in CRLM patients is lacking. A previous study (17) demonstrated that higher creatine levels were associated with early recurrence in CRLM patients. However, in our study, BUN rather than creatine was recognized as an independent predictive factor for early recurrence, which could be ascribed to the discrepancies in the different treatments used for patients between the two studies. The CRLM patients in Chen’s research all received NAC, while only a portion of patients in our study did, and the surgical procedures performed for patients in the two studies were different. Cancer cachexia is a devastating syndrome that might be responsible for 20% of cancer-related deaths and is likely to be orchestrated by inflammation-related hypoanabolism or hypercatabolism (35). Therefore, it was reasonable to postulate that elevated BUN levels indicated potential cancer-related cachexia, thus leading to poor survival in CRLM patients. In addition, BUN is an easy-to-obtain, frequently retested, fast and simple laboratory index, and using BUN levels to predict the conditions and prognoses of CRLM patients could yield significant clinical value.
Besides, in the current study, the TBS score for individuals was calculated during preoperative evaluation, and NLR level, BUN level was collected by venous punctures within 1 to 3 weeks before the surgery. Therefore, during the period before operation in our study, the indexes were less likely to variate a lot. Together with the abovementioned studies in which TBS score, NLR level or BUN level was widely considered as independent risk factors for cancer patients, we could conclude that the stability of these indexes was guaranteed.
As simultaneous resections have been performed more frequently, the increased incidence rate of postoperative complications (36,37) compared with traditional staged resection has raised concern among clinicians. In the current study, the presence of postoperative complications was independently associated with early recurrence. To decrease the risk of postoperative complications, perioperative care should be improved. A recent study (38) reported that multimodal prehabilitation in the preoperative period could decrease the incidence of postoperative complications by improving the functional capacity of CRC patients. Regular postoperative intensive care units (39) were reported to be cost-effective in reducing the risk of postoperative complications in elderly CRC patients. A nomogram is a convenient tool for guiding the management of patients, and many studies have constructed models to predict recurrence and survival in CRLM patients (40,41), but these studies neglected CRLM patients who received simultaneous resections of the primary tumour and liver metastases. As simultaneous resections were performed more frequently, we filled the gap in this field and added more evidence for the personalized management of CRLM patients. CRLM patients with higher nomogram scores should receive more frequent monitoring and better perioperative care than those with low scores, which would achieve a more efficient distribution of medical resources.
There are several limitations in our study. First, the retrospective nature and relatively small number of patients in our design limited our efforts, and further prospective, large-scale research needs to be conducted to verify our findings. Second, although we collected as many preoperative laboratory indicators as we could, there was still the risk of ignoring some factors related to early recurrence, so in future studies, we would include more clinical indicators to more accurately predict early recurrence. Third, information on gene signatures, such as KRAS mutations, was lacking due to the retrospective design, and we also included this information in our further prospective cohort.
Conclusions
To the best of our knowledge, this is the first study to explore the relationship between early recurrence and CRLM patients who received simultaneous resections of the primary tumour and liver metastases. We statistically determined the optimal cut-off value for early recurrence and constructed and externally validated a predictive nomogram for early recurrence in these patients. The models exhibited high specificity and accuracy in the training set and achieved good verification results in the test set. Preoperative NLR levels, preoperative BUN levels and TBS scores were first proven to be useful in prognosis predictions for CRLM patients who received simultaneous resections. Our findings will be helpful for accurate risk stratification and improving the management of CRLM patients.
Acknowledgments
Funding: This study was supported by the National Natural Science Foundation of China (Nos. 81972311, and 82141127), the CAMS Innovation Fund for Medical Sciences (CIFMS) (No. 2021-I2M-1-066), the Non-profit Central Research Institution Fund of Chinese Academy of Medical Sciences (No. 2019PT310026) and Sanming Project of Medicine in Shenzhen (No. SZSM202011010).
Footnote
Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://jgo.amegroups.com/article/view/10.21037/jgo-22-934/rc
Data Sharing Statement: Available at https://jgo.amegroups.com/article/view/10.21037/jgo-22-934/dss
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jgo.amegroups.com/article/view/10.21037/jgo-22-934/coif). The authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work and in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Institutional Review Board of the Cancer Hospital, Chinese Academy of Medical Sciences and Sun Yat-sen University Cancer Center (No. 81972311) and informed consent was taken from all the patients.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
References
- Sung H, Ferlay J, Siegel RL, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin 2021;71:209-49. [Crossref] [PubMed]
- Zhao G, Yuan H, Li Q, et al. DDX39B drives colorectal cancer progression by promoting the stability and nuclear translocation of PKM2. Signal Transduct Target Ther 2022;7:275. [Crossref] [PubMed]
- Wang SH, Song L, Tang JY, et al. Safety and long-term prognosis of simultaneous versus staged resection in synchronous colorectal cancer with liver metastasis: a systematic review and meta-analysis. Eur J Med Res 2022;27:297. [Crossref] [PubMed]
- Adam R, de Gramont A, Figueras J, et al. Managing synchronous liver metastases from colorectal cancer: a multidisciplinary international consensus. Cancer Treat Rev 2015;41:729-41. [Crossref] [PubMed]
- Van Cutsem E, Cervantes A, Adam R, et al. ESMO consensus guidelines for the management of patients with metastatic colorectal cancer. Ann Oncol 2016;27:1386-422. [Crossref] [PubMed]
- Boudjema K, Locher C, Sabbagh C, et al. Simultaneous Versus Delayed Resection for Initially Resectable Synchronous Colorectal Cancer Liver Metastases: A Prospective, Open-label, Randomized, Controlled Trial. Ann Surg 2021;273:49-56. [Crossref] [PubMed]
- Dulundu E, Attaallah W, Tilki M, et al. Simultaneous resection for colorectal cancer with synchronous liver metastases is a safe procedure: Outcomes at a single center in Turkey. Biosci Trends 2017;11:235-42. [Crossref] [PubMed]
- Wang LJ, Wang HW, Jin KM, et al. Comparison of sequential, delayed and simultaneous resection strategies for synchronous colorectal liver metastases. BMC Surg 2020;20:16. [Crossref] [PubMed]
- Zeng J, Zeng J, Lin K, et al. Development of a machine learning model to predict early recurrence for hepatocellular carcinoma after curative resection. Hepatobiliary Surg Nutr 2022;11:176-87. [Crossref] [PubMed]
- Zeng CDD, Jin CC, Gao C, et al. Preoperative Folate Receptor-Positive Circulating Tumor Cells Are Associated With Occult Peritoneal Metastasis and Early Recurrence in Gastric Cancer Patients: A Prospective Cohort Study. Front Oncol 2022;12:769203. [Crossref] [PubMed]
- Liu W, Yang Y, Liu Y, et al. p21-Activated kinase 4 predicts early recurrence and poor survival in patients with nonmetastatic clear cell renal cell carcinoma. Urol Oncol 2015;33:205.e13-21. [Crossref] [PubMed]
- Bhogal RH, Hodson J, Bramhall SR, et al. Predictors of early recurrence after resection of colorectal liver metastases. World J Surg Oncol 2015;13:135. [Crossref] [PubMed]
- Jung SW, Kim DS, Yu YD, et al. Risk factors for cancer recurrence or death within 6 months after liver resection in patients with colorectal cancer liver metastasis. Ann Surg Treat Res 2016;90:257-64. [Crossref] [PubMed]
- Viganò L, Capussotti L, Lapointe R, et al. Early recurrence after liver resection for colorectal metastases: risk factors, prognosis, and treatment. A LiverMetSurvey-based study of 6,025 patients. Ann Surg Oncol 2014;21:1276-86. [Crossref] [PubMed]
- Dai S, Ye Y, Kong X, et al. A predictive model for early recurrence of colorectal-cancer liver metastases based on clinical parameters. Gastroenterol Rep (Oxf) 2021;9:241-51. [Crossref] [PubMed]
- Imai K, Allard MA, Benitez CC, et al. Early Recurrence After Hepatectomy for Colorectal Liver Metastases: What Optimal Definition and What Predictive Factors? Oncologist 2016;21:887-94. [Crossref] [PubMed]
- Zhang D, Hu D. Exploring better stratification methods for individualized treatment of patients with colorectal cancer liver metastases. Ann Palliat Med 2022;11:3369-70. [Crossref] [PubMed]
- Groot VP, Gemenetzis G, Blair AB, et al. Defining and Predicting Early Recurrence in 957 Patients With Resected Pancreatic Ductal Adenocarcinoma. Ann Surg 2019;269:1154-62. [Crossref] [PubMed]
- Sahara K, Tsilimigras DI, Kikuchi Y, et al. Defining and Predicting Early Recurrence after Resection for Gallbladder Cancer. Ann Surg Oncol 2021;28:417-25. [Crossref] [PubMed]
- Peng J, Li W, Fan W, et al. Prognostic value of a novel biomarker combining DNA ploidy and tumor burden score for initially resectable liver metastases from patients with colorectal cancer. Cancer Cell Int 2021;21:554. [Crossref] [PubMed]
- Peng J, Liu Y, Li W, et al. Application of Tumor Burden Score for predicting conversion outcome in patients with initially unresectable colorectal liver metastases after first-line systemic therapy. Therap Adv Gastroenterol 2021;14:17562848211066206. [Crossref] [PubMed]
- Wu Y, Guo T, Xu Z, et al. Risk scoring system for recurrence after simultaneous resection of colorectal cancer liver metastasis. Ann Transl Med 2021;9:966. [Crossref] [PubMed]
- Chen Q, Mao R, Zhao J, et al. Upgraded nomograms for the prediction of complications and survival in patients with colorectal liver metastases treated with neoadjuvant chemotherapy followed by hepatic resection. Ann Transl Med 2021;9:265. [Crossref] [PubMed]
- Yang L, Li A, Wang Y, et al. Intratumoral microbiota: roles in cancer initiation, development and therapeutic efficacy. Signal Transduct Target Ther 2023;8:35. [Crossref] [PubMed]
- Hellingman T, Kuiper BI, Buffart LM, et al. Survival Benefit of Repeat Local Treatment in Patients Suffering From Early Recurrence of Colorectal Cancer Liver Metastases. Clin Colorectal Cancer 2021;20:e263-72. [Crossref] [PubMed]
- Chen Q, Zhang Y, Li X, et al. Nomogram Incorporating Preoperative Testing Markers for the prediction of Early Recurrence for Colorectal Liver Metastases with Neoadjuvant Chemotherapy followed by Hepatectomy. J Cancer 2022;13:1758-67. [Crossref] [PubMed]
- Zheng Z, Wang X, Huang Y, et al. Defining and predicting early recurrence in patients with locally advanced rectal cancer treated with neoadjuvant chemoradiotherapy. Eur J Surg Oncol 2020;46:2057-63. [Crossref] [PubMed]
- Yamashita Y, Adachi E, Toh Y, et al. Risk factors for early recurrence after curative hepatectomy for colorectal liver metastases. Surg Today 2011;41:526-32. [Crossref] [PubMed]
- Fang T, Wang Y, Yin X, et al. Diagnostic Sensitivity of NLR and PLR in Early Diagnosis of Gastric Cancer. J Immunol Res 2020;2020:9146042. [Crossref] [PubMed]
- Verter E, Berger Y, Perl G, et al. Neutrophil-to-Lymphocyte Ratio Predicts Recurrence Pattern in Patients with Resectable Colorectal Liver Metastases. Ann Surg Oncol 2021;28:4320-9. [Crossref] [PubMed]
- Ishii K, Pouzolles M, Chien CD, et al. Perforin-deficient CAR T cells recapitulate late-onset inflammatory toxicities observed in patients. J Clin Invest 2020;130:5425-43. [Crossref] [PubMed]
- Lafleur J, Hefler-Frischmuth K, Grimm C, et al. Prognostic Value of Serum Creatinine Levels in Patients with Epithelial Ovarian Cancer. Anticancer Res 2018;38:5127-30. [Crossref] [PubMed]
- Peng X, Huang Y, Fu H, et al. Prognostic Value of Blood Urea Nitrogen to Serum Albumin Ratio in Intensive Care Unit Patients with Lung Cancer. Int J Gen Med 2021;14:7349-59. [Crossref] [PubMed]
- Lin HL, Chen CW, Lu CY, et al. High preoperative ratio of blood urea nitrogen to creatinine increased mortality in gastrointestinal cancer patients who developed postoperative enteric fistulas. Kaohsiung J Med Sci 2012;28:418-22. [Crossref] [PubMed]
- Argilés JM, Busquets S, Stemmler B, et al. Cancer cachexia: understanding the molecular basis. Nat Rev Cancer 2014;14:754-62. [Crossref] [PubMed]
- Snyder RA, Hao S, Irish W, et al. Thirty-Day Morbidity after Simultaneous Resection of Colorectal Cancer and Colorectal Liver Metastasis: American College of Surgeons NSQIP Analysis. J Am Coll Surg 2020;230:617-627.e9. [Crossref] [PubMed]
- Ono Y, Saiura A, Arita J, et al. Short-Term Outcomes after Simultaneous Colorectal and Major Hepatic Resection for Synchronous Colorectal Liver Metastases. Dig Surg 2017;34:447-54. [Crossref] [PubMed]
- van Rooijen S, Carli F, Dalton S, et al. Multimodal prehabilitation in colorectal cancer patients to improve functional capacity and reduce postoperative complications: the first international randomized controlled trial for multimodal prehabilitation. BMC Cancer 2019;19:98. [Crossref] [PubMed]
- Fahim M, Visser RA, Dijksman LM, et al. Routine postoperative intensive care unit admission after colorectal cancer surgery for the elderly patient reduces postoperative morbidity and mortality. Colorectal Dis 2020;22:408-15. [Crossref] [PubMed]
- Imai K, Allard MA, Castro Benitez C, et al. Nomogram for prediction of prognosis in patients with initially unresectable colorectal liver metastases. Br J Surg 2016;103:590-9. [Crossref] [PubMed]
- Liu W, Wang K, Han Y, et al. Nomogram predicted disease free survival for colorectal liver metastasis patients with preoperative chemotherapy followed by hepatic resection. Eur J Surg Oncol 2019;45:2070-7. [Crossref] [PubMed]