Predictive models of lymph node metastasis in patients with gastrointestinal stromal tumors based on machine learning algorithms: a SEER-based retrospective study

Introduction

Gastrointestinal stromal tumors (GISTs) are among the most prevalent mesenchymal tumors originating from the gastrointestinal tract, arising from interstitial cell of Cajal (1). They are predominantly found in the stomach (60–65%) and small intestine (20–35%). Only a minority of patients with GISTs (<10%) experience lymph node metastasis (2-4).

Previous studies have suggested that the presence or absence of lymph node metastasis does not significantly affect patient prognosis, and in some cases, patients with lymph node metastasis may even have a better prognosis (5,6). Therefore, lymph node dissection is not currently considered a standard treatment approach (7-9). However, as more studies have been conducted, some scholars also believe that lymph node metastasis can lead to a poor prognosis (10-12). Currently, in clinical practice, the National Institutes of Health (NIH) risk category recommended by the National Comprehensive Cancer Network (NCCN) for grading the malignancy of GISTs is often used to predict the risk of lymph node metastasis. However, cases have also emerged where patients assessed as low-risk by the NIH risk category experienced lymph node metastasis (13). Due to a lack of relevant standards, many clinicians face significant challenges in predicting the risk of lymph node metastasis in patients with GISTs. With the continuous advancement of personalized and precision medicine, enhancing the success rate of predicting lymph node metastasis in patients with GISTs is a topic worthy of further exploration.

Computed tomography (CT) is currently recommended as the initial screening modality for GISTs. In cases where lymph node or distant metastasis is suspected, enhanced CT or positron emission tomography (PET)/CT is recommended for a more comprehensive assessment of the metastatic landscape and associated risk evaluation (9,14). However, the cost of enhanced CT and PET/CT is considerable, making them impractical as routine diagnostic approaches for most patients worldwide. Therefore, there is a need for more cost-effective and precise methods for routine screening. Given the rarity of lymph node metastasis in GISTs, there is a lack of research focused on predicting lymph node metastasis in this field. Most studies have primarily focused on predicting the risk of distant organ metastasis and the therapeutic efficacy of GISTs (15). While some studies have highlighted the primary tumor site, mitotic index, tumor size, and number of lymph nodes as important predictive indicators of lymph node metastasis (16-19), many of these studies did not account for multifaceted factors, resulting in models that exhibit shortcomings in predictive efficacy and fitting adequacy.

With the advancement of machine learning technologies, the emergence of various algorithms has opened new avenues for research on disease diagnosis and prognostic prediction. Recently, researchers had increasingly applied machine learning algorithms to develop models for predicting lymph node metastasis and prognostic outcomes in gastric cancer. Compared to conventional algorithms, machine learning algorithms have achieved superior accuracy and enhanced stability, highlighting their increased utility in prognostic predictions for digestive system tumors (20-22). In conclusion, machine learning approaches hold significant promise for the accurate prediction of the risk of lymph node metastasis in patients with GISTs. However, the current lack of research has hampered the application of machine learning for forecasting the risk of lymph node metastasis in GISTs.

In this study, we used patient data from the Surveillance, Epidemiology and End Results (SEER) database of GISTs to investigate the performance of machine learning algorithms in predicting lymph node metastasis. We included patients with GISTs who underwent surgical treatment, trained the corresponding models, and developed an ensemble model with three best-performing machine learning algorithms. Furthermore, we utilized a logistic regression model as baseline. Finally, we compared the best-performing machine learning model with the baseline model. We present this article in accordance with the TRIPOD reporting checklist (available at https://jgo.amegroups.com/article/view/10.21037/jgo-24-777/rc).

Methods

Study population

The clinical information of patients with GISTs was obtained from the SEER database, which contains data from 2010 to 2017. We included 6,621 patients with GISTs based on the third edition of histological coding (ICD-O-3 code 8936). The exclusion criteria were as follows: (I) no surgical therapy; (II) diagnosis without positive histological confirmation; and (III) no lymph node dissection during surgery. The patient selection process is shown in Figure 1. We gathered demographic and clinicopathological data from the patients with GISTs, including age, sex, race, marital status, neoadjuvant therapy, tumor size, mitotic index, primary tumor location, number of dissected lymph nodes, the number of positive lymph nodes, M stage, Union for International Cancer Control (UICC) Tumor Node Metastasis (TNM) stage and overall survival (OS). The lymph node status of the patients, in relation to the existence of metastasis, was ascertained based on the category of “Regional nodes positive”. The parameter “RX Summ--Systemic/Sur Seq” was used to evaluate the neoadjuvant therapy, and “CS site-specific factor 6” was utilized to evaluate the mitotic index of the patients.

Figure 1 Flowchart demonstrating patient selection. SEER, Surveillance, Epidemiology and End Results.

Due to the lack of a universal NIH risk score in the SEER database, patients’ NIH scores were calculated based on tumor size, mitotic index, and primary tumor location included in the database. The standard definitions are as follows (23): very-low risk: tumor size <2 cm, mitotic index <5/50 high power fields (HPF), primary tumor location unrestricted; low risk: tumor size ≥2 and <5 cm, mitotic index <5/50 HPF, primary tumor location unrestricted; intermediate risk: tumor size ≥2 and <5 cm, mitotic index >5/50 HPF, primary tumor location gastric or tumor size <5 cm, mitotic index 6–10/50 HPF, primary tumor location unrestricted or tumor size ≥5 and ≤10 cm, mitotic index <5/50 HPF, primary tumor location gastric; high risk: tumor size >10 cm, mitotic index unrestricted, primary tumor location unrestricted or tumor size unrestricted, mitotic index >10/50 HPF, primary tumor location unrestricted or tumor size >5 cm, mitotic index >5/50 HPF, primary tumor location unrestricted or tumor size ≥2 and <5 cm, mitotic index >5/50 HPF, primary tumor location not in the stomach or tumor size ≥5 and ≤10 cm, mitotic index <5/50 HPF, primary tumor location not in the stomach.

Data preprocessing

During the data cleaning process, stringent screening of patient data was conducted in accordance with the stipulated exclusion criteria. The data were subsequently segregated into distinct groups, contingent on the outcome variable of lymph node metastasis, thereby differentiating between positive and negative groups.

During the machine-learning procedure, it was imperative to transform all categorical data into numerical data. Consequently, we employed the one-hot encoding technique to process all categorical data, thereby facilitating a more comprehensive interpretation of the model and examining the efficacy of various data preprocessing methodologies. Furthermore, we used unprocessed data to construct the model. All samples were partitioned into training and validation cohorts in a ratio of 6:4. To mitigate the influence of data dimensionality on the outcomes and avert data leakage, data within the training cohort were centered and scaled. Given the comparatively lower number of patient samples with lymph node metastasis compared to those without, we used Adaptive Synthetic Sampling (ADASYN), the synthetic minority over-sampling technique (SMOTE), and the synthetic minority over-sampling technique for regression with Gaussian noise (SMOGN) to amplify the data, with the aim of optimizing the imbalanced distribution between positive and negative samples. The specific parameters for the data balancing algorithm are detailed in Table S1. Based on the procedure described above, we obtained six training cohorts. We analyzed the results and investigated approaches to compare the outcomes.

Model building and cross-validation

This study involved the instruction of five distinct machine learning classifiers: Elastic Net (ELR) (24), Random Forest (RF) (25), Extreme Gradient Boosting (XGBoost) (26), Support Vector Machine (SVM) (27), and artificial neural networks (ANN). Table S2 provides a concise description of these algorithms. We selected the optimal hyperparameters using a grid search with ten repeats of 10-fold cross-validation, prioritizing cross-entropy loss as the primary evaluation metric. Table S3 provides details of the final hyperparameters. Considering that lymph node metastasis might lead to a worse prognosis for patients with GISTs, so our model should be equipped with not only high accuracy but high sensitivity. As a result, achieving higher sensitivity was also one of our objectives. Therefore, the classification performances of the algorithms were assessed using sensitivity metrics [true positive/(true positive + false negative)] and the area under the receiver operating characteristic curve (AUC). To determine the final sensitivity and AUC values, 500 bootstrap iterations were conducted on the internal validation cohort. Ensemble models were built by selecting the three algorithms with the highest sensitivity from each training cohort. The best model among the single or ensemble models was selected as the final model. We assessed the robustness of the models by evaluating them using calibration curve (28). Acknowledging the scarcity of positive patients relative to negative patients in this study, we plotted a precision-recall curve (PRC) and calculated the area under the precision-recall curve (PRAUC) to evaluate the model’s ability to identify cases (29,30).

Our baseline models for the six training cohorts were built using logistic regression to substantiate the advantages of machine learning algorithms. The establishment and evaluation methods for the baseline models were aligned with those for the machine learning models. Decision curve analysis (DCA) was employed to assess the clinical utility of the method, a DCA was employed (31).

Risk score

To examine the performance of the machine-learning models we built for predicting patient prognosis, we used the Survminer package in R software to calculate and determine the optimal classification thresholds for predictive value in the training cohorts. This threshold was used to separate the patients into high-risk (predictive value above the threshold) and low-risk groups (predictive value below the threshold). Survival analysis was performed according to this grouping in the validation and entire cohort.

Variable visualization

Compared to the commonly used logistic regression and Cox regression models, more complex machine learning models resemble a black box to a greater extent. Therefore, elucidating the importance of individual variables in the model construction process has been a long-standing concern for machine learning researchers. We analyzed the importance of each variable in the final model using the VarImp function within the caret package in R (32).

Statistical analysis

In the present investigation, Chi-squared tests were employed to analyze categorical data, whereas t-tests or Fisher’s exact tests were used to examine continuous data, with the aim of detecting the heterogeneity of intervention effects between the two cohorts. Subsequently, 1:3 propensity score matching (PSM) was conducted between the two cohorts using nearest-neighbor matching with a caliper of 0.3. Survival outcomes were subsequently assessed by survival analysis, using Kaplan-Meier survival curves and log-rank tests. All P values are 2-sided, with P values less than 0.05 indicating statistical significance. R version 4.2.3 was used for all statistical analyses (R-project, Institute for Statistics and Mathematics, Vienna, Austria).

Results

Baseline characteristics of patients

We obtained the clinical information of 5,041 patients with GISTs from the SEER database. A total of 1,404 patients were included in this study after data screening (Table 1). Among them, 94 patients were lymph node positive, accounting for 6.70% of the cohort. Age (P=0.003), race (P=0.03), primary site of tumor (P=0.03), tumor size (P<0.001), UICC TNM stage (P=0.001), mitotic index (P=0.005), summary stage (P<0.001), NIH risk category (P=0.003) and M stage (P<0.001) were significantly associated with the presence of lymph node metastasis based on univariate analysis. But there were no statistically significant differences observed between the lymph node positive group and negative group in terms of sex (P=0.51), neoadjuvant therapy (P=0.76), or marital status (P=0.13). Although there was no statistically significant difference in adjuvant therapy between the two groups, given its pivotal role in the treatment of GIST, we still incorporate information regarding adjuvant therapy into our model construction. Finally, sex and marital status were excluded. Following the processing of the cohort, we constructed training and validation cohorts in a ratio of 6:4. All features were well balanced between the training and validating cohorts (Table 2). After undergoing data balancing algorithms and one-hot encoding, six training cohorts were individually constructed for model building.

Prognostic impact of lymph node metastasis

The median survival time of the positive group was 60.29±38.97 months and the negative group was 73.74±37.35 months. Survival analysis revealed that patients in the positive group (60.29±38.97 months) exhibited a significantly lower OS compared to those in the negative group (73.74±37.35 months) (P<0.001) (Figure 2A). We used PSM to balance confounders between two groups. There were no significant differences in the various factors between the two groups after matching (all P>0.05, Table S4). The findings continued to demonstrate that patients with positive test results exhibited a significantly lower OS (58.10±38.92 months) in comparison to those with negative test results (68.14±38.98 months) (P=0.03) (Figure 2B).

Figure 2 Kaplan-Meier curve of overall survival in GISTs patients with and without lymph node metastasis. (A) Untreated cohort; (B) the matched cohort. CI, confidence interval; GISTs, gastrointestinal stromal tumors; HR, hazard ratio; LNM, lymph node metastasis; PSM, propensity score matching.

Optimal model selection

All the models in the training cohorts displayed excellent performance, with high AUC and sensitivity (Figure S1). The specific parameters for the oversampling algorithm are detailed in Table S1, and the hyper-parameters for all the models are listed in Table S3. Figure 3 shows the AUC and sensitivity of each model on the validation cohort. Based on the transverse contrast, most of the models trained on one-hot encoding processed training sets performed better than those trained on non-processed training sets (Figure 3). Although the AUC remained good for all models in the validation cohort, the sensitivities of the individual models varied significantly. As a result, we chose the ANN [AUC: 0.951 95% confidence interval (CI): 0.901–0.992, sensitivity: 0.752] (Figure 3) trained on the training cohort processed using one-hot encoding and the ADASYN algorithm, which demonstrated the highest sensitivity, as the final model.

Figure 3 Heatmap of model performance for predicting lymph node metastasis in GISTs across various machine learning algorithms in the validation group. ada, Adaptive Synthetic Sampling; ada-hot, Adaptive Synthetic Sampling and one-hot encoding; AUC, area under the ROC curve; GISTs, gastrointestinal stromal tumors; ROC, receiver operator characteristic; SMOGN, synthetic minority over-sampling technique for regression with Gaussian noise; SMOGN-hot, synthetic minority over-sampling technique for regression with Gaussian noise and one-hot encoding; SMOTE, synthetic minority over-sampling technique; SMOTE-hot, synthetic minority over-sampling technique and one-hot encoding.

Assessment of final model

The calibration curve of the final model was presented in Figure 4A and shows a favorable goodness-of-fit. This indicates that the probabilities predicted by the final model were consistent with the actual observations.

Figure 4 Final model performance. (A,B) Calibration curve and DCA for Artificial Neural Networks algorithm lymph node metastasis prediction; (C,D) ROC and precision-recall curve for the final and baseline models. AUC, area under the ROC curve; DCA, decision curve analysis; PRAUC, area under the precision-recall curve; ROC, receiver operator characteristic; CI, confidence interval.

We evaluated the clinical effectiveness of the final model by plotting a DCA curve. We selected none as the reference line. The results indicated that both the final and the baseline models exhibited a favorable net benefit, with the final model showing a higher net benefit than the baseline (Figure 4B).

We selected the logistic regression model, trained on the training cohort processed using one-hot encoding and the ADASYN algorithm, to serve as our baseline model (AUC: 0.932, 95% CI: 0.896–0.968, sensitivity: 0.528) (Figure 3). Although there was no significant difference in the AUC between the final and the baseline models (Figure 4C), the final model demonstrated a higher PRAUC than the baseline model in identifying positive cases. Furthermore, the balance point of the PRC for the final model lies to the upper right of the balance point of the PRC of the baseline model, once again demonstrating the superior ability of the final model to identify positive cases compared to the baseline model (Figure 4D).

Survival analysis

As further testing of the predictive value of the final model, we used survival analysis with the ’surv_cutpoint’ function in the Caret package to determine the best classification threshold. The best cut-off value for predicting OS in the validation set using the final model was 0.87, while the best cut-off value for predicting OS in the entire cohort was 0.86. The OS of patients in the high-risk group was significantly lower than that of patients in the low-risk group (all P<0.001, Figure 5).

Figure 5 Kaplan-Meier survival curves for low and high-risk groups in the validation group and entire cohort of the final model. The efficacy of the Artificial Neural Networks predictive value in distinguishing the prognosis of patients in the validating group (A) and the entire group (B). CI, confidence interval; HR, hazard ratio.

Importance of variables

The final model’s variable importance in the original dataset is presented in Table 3 to better explain the importance of each variable in building the model. As shown in Table 3, the most significant feature is M stage, followed by age, summary stage, tumor size, primary site, NIH risk category, mitotic index, total number of regional lymph nodes, UICC TNM stage and race, in descending order of importance. As shown in Table 3, the most significant feature was M stage, followed by age, summary stage, and tumor size, in descending order of importance. The importance of each variable in the dataset after one-hot encoding of the final model is presented in Table S5.

Discussion

This retrospective study included 1,404 patients with GISTs who underwent lymph node dissection from the SEER database. We preliminarily confirmed through univariate analysis that lymph node metastasis leads to a poorer prognosis in patients with GISTs, reaffirming the necessity of research investigating lymph node metastasis in GIST (2,6). We chose routinely available clinical data to build both machine learning and logistic regression models. The results indicated that machine learning models demonstrated excellent performance in predicting lymph node metastasis and forecasting the prognosis of patients with GISTs. In addition, the machine learning model exhibited superior efficacy compared with the logistic regression model. It is essential to ensure that the models are interpretable because clinicians must consider multiple variables when treating patients to make the most informed decisions. In our study, the final model highlighted that M stage was the most important predictor of lymph node metastasis in GISTs. Previous research has indicated that when GISTs undergo lymph node metastasis, they are often in an advanced stage or have already developed distant metastasis, which should be considered malignant (10,33,34). Consequently, the 7th edition of the American Joint Committee on Cancer (AJCC) guidelines universally classifies GISTs with lymph node metastases as stage IV (35). GISTs generally metastasize to distant organs through the circulatory system and rarely spread to distant sites through the lymphatic system, which is one of the factors differentiating GISTs from other gastrointestinal malignancies. Therefore, lymphatic metastasis in GISTs primarily involves regional spread to the lymph nodes rather than distant spread. Although distant organ and lymph node metastases are not directly related, the occurrence of distant metastasis in GISTs often signifies a higher malignancy, which is associated with a greater likelihood of lymph node metastasis.

Based on the machine learning algorithms used in our study, we found that lymph node metastases of GISTs was significantly influenced by age. There is no age difference in the onset of GISTs since the condition can affect anyone. However, the relationship between lymph node metastasis and age remains controversial issue. Prakash et al. (36) reported that the rate of lymph node metastasis among underage patients with GISTs was 26%. We believe that this is primarily associated with a higher incidence of wild-type GISTs in younger patients indicating a higher risk of lymph node metastasis due to the lack of succinate dehydrogenase (37). However, Miettinen et al. (38) found no occurrence of lymph node metastasis in younger GISTs patients. Gong et al. (2) found that older patients with GISTs are more susceptible to lymph node metastasis. We consider the observed disparities in the aforementioned results to be influenced, to some extent, by the fortuity of the studies. Through a review of retrospective studies on GIST lymph node metastasis, we found that the average age of patients with lymph node metastasis ranged from 43 to 55 years (18,19,39,40). In our study, the average age of the positive patients was 58.15 years, whereas that of the negative patients was 62.60 years. The age of positive patients is significantly lower than that of negative patients, which implies a lower likelihood of lymph node metastasis in patients with GISTs with a relatively advanced onset age. Machine learning algorithms can struggle to effectively explore the relationship between variables and outcomes in certain contexts, particularly when data are limited or the relationships are complex. Therefore, additional samples are required to explore the impact of age on lymph node metastasis in patients with GISTs.

Our study also showed that tumor size, primary site of GISTs, UICC TNM stage, NIH risk category, and mitotic index are crucial variables influencing lymph node metastasis, which has already been demonstrated to be significant in previous research (16-19,41). It is worth noting that in examining the importance of factors in the dataset after one-hot encoding, we found that ‘M stage I’, ‘primary site is the stomach’, ‘NIH risk category High’ and ‘Mitotic index >10/50’ also played significant roles in model construction (Table S5).

Although several methods have been employed to prevent overfitting, our model exhibited overfitting for some training data. We considered this as the primary reason for the scarcity of positive cases within the dataset. However, this overfitting is deemed acceptable. First, several techniques were used to control overfitting when the models were built. Our findings revealed that the model performance did not exhibit significant enhancement after controlling for overfitting and, in some cases, even declined. Second, our model demonstrated exceptional performance on the validation set, achieving outstanding results in terms of both AUC and sensitivity. Before building the models, we assumed that the dataset after one-hot encoding would offer more variables and features to the machine learning algorithms, which could lead to better performance of the models. However, the outcomes revealed that one-hot encoding had an unstable effect, predominantly enhancing model performance in most cases, while also reducing the performance in some models. One-hot encoding is not perfect and may pose risks such as high-dimensional sparsity and reduced interpretability of clinical data. Thus, further scrutiny and testing is necessary in research and clinical practice before applying one-hot encoding.

This study has some limitations. First, this was a retrospective study and the study population was limited to the patients who have undergone lymphadenectomy, which may carry the risk of selection bias. Second, further testing of the model’s accuracy and generalizability is necessary because the training and test data both originate from the same database, which requires a substantial amount of external patient data for validation. Third, given that our study relied on the SEER database, some data were inevitably missing. For example, imaging and biological data encompassing a wealth of additional information were not included in this study. In our opinion, machine learning models built using a multimodal dataset would utilize clinical information comprehensively and provide much better performance.

Conclusions

This study established multiple prediction models for GIST lymph node metastasis by using machine learning algorithms. Due to its superior predictive capabilities, the ANN model has been chosen as the final model. The application of machine learning algorithms in predicting lymph node metastasis in GISTs has been lacking. Thus, given its strong predictive capabilities, machine learning models may assist clinicians in delivering more efficient, precise, and individualized treatment for patients with GISTs.

Acknowledgments

All authors appreciate the SEER Program tumor registries for their contribution in building the SEER Database.

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://jgo.amegroups.com/article/view/10.21037/jgo-24-777/rc

Peer Review File: Available at https://jgo.amegroups.com/article/view/10.21037/jgo-24-777/prf

Funding: None.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jgo.amegroups.com/article/view/10.21037/jgo-24-777/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Klug LR, Khosroyani HM, Kent JD, et al. New treatment strategies for advanced-stage gastrointestinal stromal tumours. Nat Rev Clin Oncol 2022;19:328-41. [Crossref] [PubMed]
2. Gong N, Wong CS, Chu YC. Is lymph node metastasis a common feature of gastrointestinal stromal tumor? PET/CT correlation. Clin Nucl Med 2011;36:678-82. [Crossref] [PubMed]
3. Cavnar MJ, Seier K, Curtin C, et al. Outcome of 1000 Patients With Gastrointestinal Stromal Tumor (GIST) Treated by Surgery in the Pre- and Post-imatinib Eras. Ann Surg 2021;273:128-38. [Crossref] [PubMed]
4. Joensuu H. Risk stratification of patients diagnosed with gastrointestinal stromal tumor. Hum Pathol 2008;39:1411-9. [Crossref] [PubMed]
5. Qiao Z, Zhang Z, Chen J, et al. Influence of lymph node removal on the prognosis of high malignancy potential gastric gastrointestinal stromal tumors: Insights from population-based study. PLoS One 2024;19:e0314504. [Crossref] [PubMed]
6. Gaitanidis A, El Lakis M, Alevizakos M, et al. Predictors of lymph node metastasis in patients with gastrointestinal stromal tumors (GISTs). Langenbecks Arch Surg 2018;403:599-606. [Crossref] [PubMed]
7. Gronchi A, Bonvalot S, Poveda Velasco A, et al. Quality of Surgery and Outcome in Localized Gastrointestinal Stromal Tumors Treated Within an International Intergroup Randomized Clinical Trial of Adjuvant Imatinib. JAMA Surg 2020;155:e200397. [Crossref] [PubMed]
8. Demetri GD, von Mehren M, Antonescu CR, et al. NCCN Task Force report: update on the management of patients with gastrointestinal stromal tumors. J Natl Compr Canc Netw 2010;8:S1-41; quiz S42-4. [Crossref] [PubMed]
9. von Mehren M, Randall RL, Benjamin RS, et al. Soft Tissue Sarcoma, Version 2.2018, NCCN Clinical Practice Guidelines in Oncology. J Natl Compr Canc Netw 2018;16:536-63. [Crossref] [PubMed]
10. Agaimy A, Wünsch PH. Lymph node metastasis in gastrointestinal stromal tumours (GIST) occurs preferentially in young patients < or = 40 years: an overview based on our case material and the literature. Langenbecks Arch Surg 2009;394:375-81. [Crossref] [PubMed]
11. Serex CA, Dulguerov N. Rapidly progressive stridor in a case of GIST. BMJ Case Rep 2021;14:e237360. [Crossref] [PubMed]
12. Woodall CE 3rd, Brock GN, Fan J, et al. An evaluation of 2537 gastrointestinal stromal tumors for a proposed clinical staging system. Arch Surg 2009;144:670-8. [Crossref] [PubMed]
13. Wang M, Cao H, Jin X, et al. Gastrointestinal stromal tumor of the lymph node metastasis of clinical and pathological characteristics analysis. Journal of Surgery Concepts & Practice 2011;16:298-300.
14. Wong CS, Chu YC, Khong PL. Unusual features of gastrointestinal stromal tumor on PET/CT and CT imaging. Clin Nucl Med 2011;36:e1-7. [Crossref] [PubMed]
15. Liu L, Zhang R, Shi Y, et al. Automated machine learning for predicting liver metastasis in patients with gastrointestinal stromal tumor: a SEER-based analysis. Sci Rep 2024;14:12415. [Crossref] [PubMed]
16. Miettinen M, Fetsch JF, Sobin LH, et al. Gastrointestinal stromal tumors in patients with neurofibromatosis 1: a clinicopathologic and molecular genetic study of 45 cases. Am J Surg Pathol 2006;30:90-6. [Crossref] [PubMed]
17. Fletcher CDM, Bridge JA, Hogendoorn PCW, et al. WHO classification of tumours of soft tissue and bone [M]. 4th edition. Lyon: LARC Press; 2013:164-7.
18. Kubo N, Takeuchi N. Gastrointestinal stromal tumor of the stomach with axillary lymph node metastasis: A case report. World J Gastroenterol 2017;23:1720-4. [Crossref] [PubMed]
19. Petrelli F, Spagnoli L, Giusti A, et al. Jejunal gastrointestinal stromal tumor with rectal lymph nodes metastases synchronous to intestinal adenocarcinoma: a possible common origin. Dig Liver Dis 2021;53:796-8. [Crossref] [PubMed]
20. Meng L, Dong D, Chen X, et al. 2D and 3D CT Radiomic Features Performance Comparison in Characterization of Gastric Cancer: A Multi-Center Study. IEEE J Biomed Health Inform 2021;25:755-63. [Crossref] [PubMed]
21. Wu J, Yang Y, Cheng L, et al. GCdiscrimination: identification of gastric cancer based on a milliliter of blood. Brief Bioinform 2021;22:536-44. [Crossref] [PubMed]
22. Jiang S, Gao H, He J, et al. Machine learning: A non-invasive prediction method for gastric cancer based on a survey of lifestyle behaviors. Front Artif Intell 2022;5:956385. [Crossref] [PubMed]
23. Agaimy A. Gastrointestinal stromal tumors (GIST) from risk stratification systems to the new TNM proposal: more questions than answers? A review emphasizing the need for a standardized GIST reporting. Int J Clin Exp Pathol 2010;3:461-71. [PubMed]
24. Zou H, Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc Series B Stat Methodol 2005;67:301-20. [Crossref]
25. Chen T, Guestrin C. XGBoost: A Scalable Tree Boosting System. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM; 2016:785-94.
26. Breiman L. Random Forests. Machine Learning 2001;45:5-32. [Crossref]
27. Decoste D, Schölkopf B. Training Invariant Support Vector Machines. Machine Learning 2002;46:161-90. [Crossref]
28. Redelmeier DA, Bloch DA, Hickam DH. Assessing predictive accuracy: how to compare Brier scores. J Clin Epidemiol 1991;44:1141-6. [Crossref] [PubMed]
29. Boyd K, Eng KH, David Page C. Area under the precision-recall curve: point estimates and confidence intervals. Berlin, Heidelberg: Springer-Verlag; 2013;8190:451-66. doi: 10.1007/978-3-642-40994-3_29.10.1007/978-3-642-40994-3_29
30. Saito T, Rehmsmeier M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One 2015;10:e0118432. [Crossref] [PubMed]
31. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 2006;26:565-74. [Crossref] [PubMed]
32. Kuhn M. caret: Classification and Regression Training. Astrophys Source Code Libr. 2015 May 1;ascl:1505.003.
33. Bucher P, Egger JF, Gervaz P, et al. An audit of surgical management of gastrointestinal stromal tumours (GIST). Eur J Surg Oncol 2006;32:310-4. [Crossref] [PubMed]
34. Blay JY, Shen L, Kang YK, et al. Nilotinib versus imatinib as first-line therapy for patients with unresectable or metastatic gastrointestinal stromal tumours (ENESTg1): a randomised phase 3 trial. Lancet Oncol 2015;16:550-60. [Crossref] [PubMed]
35. Edge SB, Compton CC. The American Joint Committee on Cancer: the 7th edition of the AJCC cancer staging manual and the future of TNM. Ann Surg Oncol 2010;17:1471-4.
36. Prakash S, Sarran L, Socci N, et al. Gastrointestinal stromal tumors in children and young adults: a clinicopathologic, molecular, and genomic study of 15 cases and review of the literature. J Pediatr Hematol Oncol 2005;27:179-87. [Crossref] [PubMed]
37. Shen L, Cao H, Qin S, et al. Gastrointestinal Stromal Tumor Diagnosis and Treatment Consensus in China (2017 Edition). Journal of Multidisciplinary Cancer Management (Electronic Version) 2018;31-43.
38. Miettinen M, Lasota J, Sobin LH. Gastrointestinal stromal tumors of the stomach in children and young adults: a clinicopathologic, immunohistochemical, and molecular genetic study of 44 cases with long-term follow-up and review of the literature. Am J Surg Pathol 2005;29:1373-81. [Crossref] [PubMed]
39. Takada N, Watanabe M, Koya Y, et al. A case of gastrointestinal stromal tumor of the stomach with lymph node metastasis. Nihon Gekakei Rengo Gakkaishi 2020;45:14-20. [Crossref]
40. DeMatteo RP, Lewis JJ, Leung D, et al. Two hundred gastrointestinal stromal tumors: recurrence patterns and prognostic factors for survival. Ann Surg 2000;231:51-8. [Crossref] [PubMed]
41. Liang SQ, Cui YT, Hu GB, et al. Development and validation of a machine-learning model for preoperative risk of gastric gastrointestinal stromal tumors. J Gastrointest Surg 2025;29:101864. [Crossref] [PubMed]