ORIGINAL ARTICLE
Check for updates
A nomogram for individualized estimation of survival among adult patients with adrenocortical carcinoma after surgery: a retrospective analysis and multicenter validation study
Jianqiu Kong1,2+, Junjiong Zheng1,21, Jinhua Cai3+, Shaoxu Wu1,2, Xiayao Diao1,2, Weibin Xie1,2, Xiong Chen1,2, Chenyi Liao4, Hao Yu1,2, Xinxiang Fan1,2, Chaowen Huang5, Zhuowei Liu5, Wei Chen6, Qiang Lv7, Haide Qin1,2,8, Jian Huang1,2 and Tianxin Lin 1,2,8*
Abstract
Background: Clinical outcome of adrenocortical carcinoma (ACC) varies because of its heterogeneous nature and reliable prognostic prediction model for adult ACC patients is limited. The objective of this study was to develop and externally validate a nomogram for overall survival (OS) prediction in adult patients with ACC after surgery.
Methods: Based on the data from the Surveillance Epidemiology, and End Results (SEER) database, adults patients diagnosed with ACC between January 1988 and December 2015 were identified and classified into a training set, comprised of 404 patients diagnosed between January 2007 and December 2015, and an internal validation set, com- prised of 318 patients diagnosed between January 1988 and December 2006. The endpoint of this study was OS. The nomogram was developed using a multivariate Cox proportional hazards regression algorithm in the training set and its performance was evaluated in terms of its discriminative ability, calibration, and clinical usefulness. The nomogram was then validated using the internal SEER validation, also externally validated using the Cancer Genome Atlas set (TCGA, 82 patients diagnosed between 1998 and 2012) and a Chinese multicenter cohort dataset (82 patients diag- nosed between December 2002 and May 2018), respectively.
Results: Age at diagnosis, T stage, N stage, and M stage were identified as independent predictors for OS. A nomo- gram incorporating these four predictors was constructed using the training set and demonstrated good calibration and discrimination (C-index 95% confidence interval [CI], 0.715 [0.679-0.751]), which was validated in the internal validation set (C-index [95% CI], 0.672 [0.637-0.707]), the TCGA set (C-index [95% CI], 0.810 [0.732-0.888]) and the Chi- nese multicenter set (C-index [95% CI], 0.726 [0.633-0.819]), respectively. Encouragingly, the nomogram was able to successfully distinguished patients with a high-risk of mortality in all enrolled patients and in the subgroup analyses. Decision curve analysis indicated that the nomogram was clinically useful and applicable.
Conclusions: The study presents a nomogram that incorporates clinicopathological predictors, which can accurately predict the OS of adult ACC patients after surgery. This model and the corresponding risk classification system have the potential to guide therapy decisions after surgery.
*Correspondence: lintx@mail.sysu.edu.cn
+Jianqiu Kong, Junjiong Zheng, and Jinhua Cai are co-first authors.
1 Department of Urology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen
University, 107 Yan Jiang West Road, Guangzhou 510120, Guangdong, P. R. China
☒ BMC
Keywords: Adrenocortical carcinoma, Adult patients, Overall survival, Nomogram, Validation, Decision curve analysis, Surveillance Epidemiology, and End Results (SEER), The Cancer Genome Atlas (TCGA), Multicenter
Background
Adrenocortical carcinoma (ACC) is a rare disease in both pediatric and adult patients with an overall incidence of 0.5-2.0 cases per million people per year [1]. Complete surgical resection is considered as the main curative form of treatment for localized ACC [2]. However, ACC is a vicious tumor with a high degree of malignancy and recurrence rate [3-6]. Its 5-year overall survival (OS) rate is estimated to range between 16 and 60% [7-9].
Adjuvant therapy including mitotane has demonstrated the potential to improve the prognosis of ACC patients [10-12], although confirmatory randomized, prospective trials on adjuvant therapy are yet to be published. If the prognosis of ACC could be accurately predicted, com- prehensive treatment would be timely given to high-risk patients to improve their survival outcome. The Ameri- can Joint Committee on Cancer (AJCC) TNM staging system is globally recognized and implemented to esti- mate the survival of ACC patients [13, 14], but it is largely constrained by its inability to consider other determin- ing clinicopathological factors, such as age, gender, and tumor size, which may also have considerable impact on the patients’ survival [15, 16].
Only a few studies have established prediction mod- els for clinicians and researchers to access the prognosis of ACC patients because of its low incidence [17-20]. However, these studies were partly limited for clinical applicability as in some, the patient’s age was used as a categorical variable rather than continuous variable [17, 18], in others, the cases with insufficient data (data of radiation therapy, chemotherapy, and histologic grade et al.) were not excluded for analysis or even lacked external validation [17, 20]. In addition, another impor- tant determining limitation was that these proposed models were developed using the data of ACC patients of all ages and thereby neglected the differences in prog- nosis predictors between pediatric and adult patients [17-20]. In fact, adult and pediatric ACC patients are different not only in incidence and clinical presentation but also in some aspects of biological behaviors. ACC in adult patients is more aggressive and is associated with poorer clinical outcomes despite undergoing complete surgical resection as compared to pediatrics ACC. The 5-year survival rate in adult patients was reported of being 37%-39% and in pediatric patients 53%-56% [21, 22]. One study analyzing the data from the Surveillance, Epidemiology and End Results (SEER) database found that the overall 5-year survival of ACC patients in adults
was mediocrely between 30 and 40% while that of pedi- atrics was 57% [23]. The genomic characteristics of ACC are also different between pediatric and adult patients. For instance, germline TP53 mutations are less common in adults with ACC, and IGF2 overexpression is a marker of poor prognosis in adult ACC patients, but not in pedi- atric patients [24, 25]. Meanwhile, adult patients seem to have less obvious symptoms of hormonal overproduc- tion, i.e. virilization and precocious puberty, and have clear-cut pathological criteria for malignancy [26] mean- ing that tumors among adult patients can be adequately classified based on the Weiss or Van Slooten scores. As such, an easy-to-implement model for prognostication of the postoperative survival tailored for adult ACC patients is greatly needed to provide more personalized treat- ment, especially for high-risk patients.
In the present study, we aimed to develop a nomo- gram for predicting the survival of post-operative adult ACC patients using the SEER database and to validate it using external validation using the Cancer Genome Atlas (TCGA) database and a multicenter Chinese cohort for wider clinical application.
Methods
Patients and data collection
In this multicenter retrospective study, three inde- pendent datasets of adult ACC patients were retrieved. The cases recruitment methodology is illustrated in Fig. 1. The inclusion criteria for data extraction were (1) pathology-confirmed ACC diagnosis; (2) patients aged ≥20 years who underwent surgery at the primary tumor site; and (3) availability of complete clinicopatho- logical and follow-up data. The exclusion criteria were (1) patients with other synchronous cancers or prior diag- nosis with other tumors; (2) patients with bilateral ACC. Ultimately, eligible ACC patients from the SEER database (January 1988- to December 2015, ICD-O-3) were iden- tified and classified as the SEER training set (diagnosed between January 2007 and December 2015) and the SEER internal validation set diagnosed between January 1988 and December 2006). In addition, two other inde- pendent datasets comprising of the TCGA validation set (TCGA-ACC project, diagnosed between 1998 and 2012) and a Chinese multicenter validation set (diagnosed between December 2002 and May 2018 from four hospi- tals, namely the Sun Yat-sen Memorial Hospital, the First Affiliated Hospital of Sun Yat-sen University, Sun Yat-sen University Cancer Center and Jiangsu Province Hospital)
Adrenocortical carcinoma cases from the SEER database (ICD-O-3) N = 2109
Adrenocortical carcinoma cases from the TCGA database N= 92
Adrenocortical carcinoma cases from the Chinese multicenter database N = 131
Patients excluded due to:
1. Other cancer (n = 270)
2. Age < 20 years old (n = 134)
3. Not had surgery (n = 471)
Patients excluded due to:
Patients excluded due to:
4. With bilateral adrenocortical carcinoma (n = 20)
1. Other cancer (n = 6)
1. Other cancer (n = 42)
2. Age < 20 years old (n = 3)
2. Missing information on
5. Missing information on
3. Missing information on TNM stage (n = 1)
TNM stage (n = 4)
TNM stage (n = 364)
3. With bilateral adrenocortical carcinoma (n = 3)
6. Missing information on tumor size (n = 128)
Training set (Diagnosed between 2007-2015) N = 404
Internal validation set (Diagnosed between 1988-2006) N = 318
External validation set (Diagnosed between 1998-2012) N = 82
External validation set (Diagnosed between 2002-2018) N = 82
Fig. 1 Flowchart illustrating patient selection for this study
were used for external validation. For the Chinese cohort, the retrospective analysis of anonymous patient data was approved by the institutional review board at each partic- ipating institution. Due to the retrospective nature of this study, informed consent was not required and patients’ data were used anonymously.
Demographic and clinicopathological data includ- ing age at diagnosis, gender, tumor laterality, tumor size, TNM stage, tumor stage group, survival status, and survival time were retrieved. TNM stage was defined according to the UICC/AJCC TNM Classification. The tumor stage group was defined based on the 7th AJCC staging system and the European Network for the Study of Adrenal Tumors (ENSAT) staging system consistent with the 8th AJCC staging system. The main outcome was OS, defined as the time from the date of diagnosis to the date of death or last follow-up.
Of note, the SEER data were accessed using the SEER*Stat version 8.3.5 software on January 3, 2019, and data from the TCGA set were downloaded from the TCGA-ACC project on January 23, 2019 (https://porta l.gdc.cancer.gov/). For the Chinese cohort, the data were censored on December 31, 2018.
Development of the nomogram
In the training set, clinicopathological predictors were tested using the univariable Cox proportional hazards regression analyses. Three models for OS prediction using multivariable Cox proportional hazards regres- sion analyses were developed. Model 1 incorporated the TNM stage, while models 2 and 3 incorporated the 7th AJCC stage group and ENSAT stage group, respectively.
Backward stepwise selection was applied by using the Akaike’s Information Criterion (AIC) as the stopping rule [27] and age at diagnosis was included in all three mod- els. The discrimination accuracy of the models was quan- tified using the Harrell’s concordance index (C-index) [28]. The optimal model was selected by comparing their C-indices and based on which the nomogram was developed.
Performance assessment of the nomogram in the training set
C-index was obtained to quantitatively evaluate the dis- criminative ability of the nomogram. Calibration curves were plotted to assess the calibration of the nomogram. Bootstrapping using 1000 resampling procedures was applied to calculate the C-index that was corrected for potential overfitting.
Validation of the nomogram
The performance of the nomogram was validated using the SEER internal validation dataset and externally vali- dated using the TCGA and Chinese dataset. The multi- variate Cox proportional hazards regression formula of the nomogram formed in the training set was applied to the patients in the validation sets, with risk scores calcu- lated for each patient to reflect the risk of cancer mor- tality. Cox proportional hazards regression analyses were performed using the risk scores in the validation sets. The discrimination and calibration of the nomogram were then assessed based on the regression analyses to validate its performance.
Survival risk classification based on the nomogram
In the training dataset, the optimal risk score for ACC mortality cutoff value was identified using the X-tile plots [29]. Based on the value obtained, all patients were clas- sified into a high- and low-risk group. The Kaplan-Meier method and log-rank test were used to assess and com- pare the OS of adult ACC patients after surgery in the different risk groups. Stratified analyses were also per- formed within the various subgroups according to sex and tumor location.
Clinical usefulness of the nomogram
Decision curve analysis (DCA) was performed by calcu- lating the net benefits for a range of threshold probabili- ties to estimate the clinical usefulness of the nomogram. The DCA algorithm, a validated approach, was utilized for evaluating alternative diagnostic and prognostic strat- egies [30].
Statistical analysis
The X-tile software version 3.6.1 (Yale University, New Haven, CT, USA) was used to determine the optimal risk score cutoff value. All other computations were con- ducted using the R software, version 3.5.2 (The R Foun- dation for Statistical Computing, https://www.r-proje ct.org/). The Cox proportional hazards regression analy- ses were performed by the R software “survival” and “MASS” packages. The nomogram and calibration plots were produced using the “rms” package. The DCA was performed using the function “stdca.R”. Statistical sig- nificance was set at P values less than 0.05 in a two-tailed test.
Results
Patient characteristics
In total, 722 eligible ACC patients from the SEER data- base were identified and classified as the SEER train- ing set (n=404) and the SEER internal validation set (n=318). There were also two external validation sets, namely the TCGA validation set (n=82) and the Chi- nese multicenter validation set (n=82). The patients’ characteristics of the training and three validation data- sets are shown in Table 1. The median follow-up of the entire dataset was 51 months (interquartile ranges [IQR], 45-57 months) for the training dataset; 167 months (IQR, 156-178 months) for the internal validation data- set; 61 months (IQR, 42-80 months) for the TCGA vali- dation set; and 22 months (IQR, 15-29 months) for the Chinese multicenter validation set. Furthermore, the generalized 5-year OS of these datasets was also calcu- lated. In the SEER training dataset, the 5-year OS was 40.4% (95% confidence interval [CI], 34.6%-46.1%). For the validation datasets, the 5-year OS was 41.1% (95%
CI 35.7%-46.6%), 60.4% (95% CI 47.9%-72.9%) and 63.6% (95% CI 48.7%-78.5%) for the SEER internal vali- dation, TCGA and Chinese multicenter validation set, respectively.
Development of the nomogram and performance assessment
Table 2 shows the findings of univariate and multivari- ate analyses in the training set. Age at diagnosis, ENSAT stage group, and 7th AJCC T, N, M and TNM stage were found to be significantly associated with OS. As for the multivariate analyses, age at diagnosis was included in all three models. Model 1 incorporated T stage, N stage, and M stage, while models 2 and 3 incorporated the 7th AJCC TNM and ENSAT stage group, respectively. The Cox regression coefficients of each included factors in the three models are displayed in Table 3. The C-indices of the models are listed in Table 4. Model 1 demonstrated the superior discrimination power in predicting OS (C-index [95% CI], 0.715 [0.679-0.751]) compared with model 2 and 3. Thence, model 1 was chosen as the opti- mal model, and a nomogram was developed on the basis of its regression result (Fig. 2a). The calibration curves for the 1-, 3- and 5-year OS showed favorable calibration of the nomogram in the training set (Fig. 2b).
Validation of the nomogram
The favorable discrimination ability of the nomogram was validated in the SEER internal validation dataset (C-index [95% CI], 0.672 [0.637-0.707]). In addition, the performance was also confirmed in the TCGA and Chi- nese multicenter external validation set, with C-indices of 0.810 (95% CI 0.732-0.888) and 0.726 (95% CI 0.633- 0.819), respectively. Good consistency was also observed between actual survival data and the nomogram predic- tion in the three validation datasets (Fig. 2c-e). There- fore, the presented nomogram performed well in both the training and validation sets.
Survival risk classification based on the nomogram
The X-tile plots showed that the optimal mortality risk score cutoff value was 2.96 (Fig. 3), and was used to classify the patients into a high- (risk score ≥ 2.96) and low-risk group. Kaplan-Meier curves for sur- vival outcomes of the different risk subgroups showed significant distinction in survival probability in the training set (Fig. 4a, P<0.001); which was further confirmed in the three validation datasets (Fig. 4b-d, SEER internal validation set, P<0.001; TCGA valida- tion set, P<0.001; Chinese multicenter validation set, P=0.010). Further, the nomogram demonstrated great potential in distinguishing patients with high-risk of all-cause mortality in all the 886 investigated patients
| Characteristics | Training set (n = 404) | High risk (%) | Internal validation set (n = 318) | TCGA validation set (n = 82) | Chinese multicenter validation set (n=82) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Number of patients | Low risk (%) | Number of patients | Low risk (%) | High risk (%) | Number of patients | Low risk (%) | High risk (%) | Number of patients | Low risk (%) | High risk (%) | ||
| Age, years | ||||||||||||
| Median (IQR) | 54 (20-89) | 55 (20-89) | 61 (27-86) | 52 (20-85) | 55 (20-85) | 59 (34-74) | 50 (20-83) | 48 (20-83) | 59 (23-71) | 49 (19-79) | 48 (19-77) | 61 (49-79) |
| Sex | ||||||||||||
| Male | 146 | 128 (87.7%) | 18 (12.3%) | 133 | 125 (94.0%) | 8 (6.0%) | 28 | 26 (92.9%) | 2 (7.1%) | 38 | 34 (89.5%) | 4 (10.5%) |
| Female | 258 | 232 (89.9%) | 26 (10.1%) | 185 | 169 (91.4%) | 16 (8.6%) | 54 | 47 (87.0%) | 7 (13.0%) | 44 | 42 (95.5%) | 2 (4.5%) |
| Tumor location | ||||||||||||
| Left | 224 | 204 (91.1%) | 20 (8.9%) | 163 | 150 (92.0%) | 13 (8.0%) | 43 | 37 (86.0%) | 6 (14.0%) | 56 | 53 (94.6%) | 3 (5.4%) |
| Right | 180 | 156 (86.7%) | 24 (13.3%) | 155 | 144 (92.9%) | 11 (7.1%) | 39 | 36 (92.3%) | 3 (7.7%) | 26 | 23 (88.5%) | 3 (11.5%) |
| Tumor size, cm | ||||||||||||
| Median (IQR) | 11.0 (1.2-80.0) | 10.7 (1.2-80.0) | 11.8 (2.6-21.0) | 11.0 (1.2-34.0) | 11.0 (1.2-34.0) | 12.3 (1.7-22.5) | – | – | 9.1 (0.8-17.8) | 9.5 (0.8-17.8) | 6.5 (4.5-11.1) | |
| 7th AJCC T stage | ||||||||||||
| T1 | 26 | 26 (100.0%) | 0 (0.0%) | 16 | 16 (100.0%) | 0 (0.0%) | 7 | 7 (100.0%) | 0 (0.0%) | 6 | 6 (100.0%) | 0 (0.0%) |
| T2 | 194 | 192 (99.0%) | 2 (1.0%) | 190 | 189 (99.5%) | 1 (0.5%) | 45 | 43 (95.6%) | 2 (4.4%) | 38 | 38 (100.0%) | 0 (0.0%) |
| T3 | 109 | 90 (82.6%) | 19 (17.4%) | 58 | 49 (84.5%) | 9 (15.5%) | 11 | 10 (90.9%) | 1 (9.1%) | 20 | 19 (95.0%) | 1 (5.0%) |
| T4 | 75 | 52 (69.3%) | 23 (30.7%) | 54 | 40 (74.1%) | 14 (25.9%) | 19 | 13 (68.4%) | 6 (31.6%) | 18 | 13 (72.2%) | 5 (17.8%) |
| 7th AJCC N stage | ||||||||||||
| N0 | 371 | 351 (94.6%) | 20 (5.4%) | 290 | 286 (98.6%) | 4 (1.4%) | 73 | 71 (97.3%) | 2 (2.7%) | 74 | 72 (97.3%) | 2 (2.7%) |
| N1 | 33 | 9 (27.3%) | 24 (72.7%) | 28 | 8 (28.6%) | 20 (71.4%) | 9 | 2 (22.2%) | 7 (77.8%) | 8 | 4 (50.0%) | 4 (50.0%) |
| 7th AJCC M stage | ||||||||||||
| M0 | 328 | 319 (97.3%) | 9 (2.7%) | 287 | 277 (96.5%) | 10 (3.6%) | 66 | 66 (100.0%) | 0 (0.0%) | 68 | 68 (100.0%) | 0 (0.0%) |
| M1 | 76 | 41 (53.9%) | 35 (46.1%) | 31 | 17 (54.8%) | 14 (45.2%) | 16 | 7 (43.8%) | 9 (56.2%) | 14 | 8 (57.1%) | 6 (42.9%) |
| 7th AJCC stage group | ||||||||||||
| I | 24 | 24 (100.0%) | 0 (0.0%) | 15 | 15 (100.0%) | 0 (0.0%) | 7 | 7 (100.0%) | 0 (0.0%) | 5 | 5 (100.0%) | 0 (0.0%) |
| II | 173 | 173 (100.0%) | 0 (0.0%) | 176 | 176 (100.0%) | 0 (0.0%) | 41 | 41 (100.0%) | 0 (0.0%) | 31 | 31 (100.0%) | 0 (0.0%) |
| III | 83 | 83 (100.0%) | 0 (0.0%) | 52 | 52 (100.0%) | 0 (0.0%) | 11 | 11 (100.0%) | 0 (0.0%) | 22 | 22 (100.0%) | 0 (0.0%) |
| IV | 124 | 80 (64.5%) | 44 (35.5%) | 75 | 51 (68.0%) | 24 (32.0%) | 23 | 14 (60.9%) | 9 (39.1%) | 24 | 18 (75.0%) | 6 (25.0%) |
| ENSAT stage group | ||||||||||||
| I | 24 | 24 (100.0%) | 0 (0.0%) | 15 | 15 (100.0%) | 0 (0.0%) | 7 | 7 (100.0%) | 0 (0.0%) | 5 | 5 (100.0%) | 0 (0.0%) |
| II | 173 | 173 (100.0%) | 0 (0.0%) | 176 | 176 (100.0%) | 0 (0.0%) | 41 | 41 (100.0%) | 0 (0.0%) | 31 | 31 (100.0%) | 0 (0.0%) |
| III | 131 | 122 (93.1%) | 9 (6.9%) | 96 | 86 (89.6%) | 10 (10.4%) | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) |
| IV | 76 | 41 (53.9%) | 35 (46.1%) | 31 | 15 (48.4%) | 16 (51.6%) | 34 | 25 (73.5%) | 9 (26.5%) | 46 | 40 (87.0%) | 6 (23.0%) |
Data are n or n (%) unless indicated otherwise. The ENSAT staging system was consistent with the 8th AJCC staging system IQR interquartile range, TCGA the Cancer Genome Atlas, AJCC the American Joint Committee on Cancer, ENSAT European Network for the Study of Adrenal Tumors
| Characteristics | Univariable analyses | Model 1 Multivariable analyses | Model 2 Multivariable analyses | Model 3 Multivariable analyses | ||||
|---|---|---|---|---|---|---|---|---|
| HR (95%CI) | P | HR (95% CI) | P | HR (95% CI) | P | HR (95% CI) | P | |
| Age (continuous) | 1.017 (1.008-1.027) | <0.001* | 1.021 (1.012-1.031) | <0.001* | 1.018 (1.008-1.027) | <0.001* | 1.019 (1.009-1.028) | <0.001* |
| Sex (male vs. female) | 0.822 (0.617-1.096) | 0.182 | – | – | – | – | – | – |
| Tumor location (left vs. right) | 0.981 (0.742-1.297) | 0.892 | – | – | – | – | – | – |
| Tumor size (continu- ous) | 1.012 (0.993-1.032) | 0.227 | – | – | – | – | – | – |
| 7th AJCC T stage | <0.001* | <0.001* | ||||||
| T1 | Reference | Reference | – | – | – | – | – | |
| T2 | 1.521 (0.734-3.154) | 0.260 | 1.544(0.743-3.208) | 0.244 | – | – | – | – |
| T3 | 3.412 (1.636-7.116) | 0.001* | 2.981 (1.428-6.225) | 0.004* | – | – | – | – |
| T4 | 4.364 (2.063-9.233) | <0.001* | 3.105 (1.454-6.633) | 0.003* | – | – | – | – |
| 7th AJCC N stage (N0 vs. N1) | 3.448 (2.370-5.341) | <0.001* | 2.789 (1.801-4.319) | < 0.001* | – | – | – | |
| 7th AJCC M stage (M0 vs. M1) | 2.773 (2.019-3.808) | <0.001* | 1.970 (1.391-2.791) | < 0.001* | – | – | – | |
| 7th AJCC TNM stage | <0.001* | <0.001* | ||||||
| I | Reference | – | – | Reference | – | – | ||
| II | 1.297 (0.622-2.705) | 0.489 | – | – | 1.196 (0.572-2.498) | 0.634 | – | – |
| III | 2.599 (1.225-5.515) | 0.013* | – | – | 0.025* | – | – | |
| IV | 4.097 (1.976-8.498) | <0.001* | – | – | 3.897 (1.878-8.087) | <0.001* | – | |
| ENSAT stage group | <0.001* | |||||||
| I | Reference | – | – | – | – | Reference | <0.001* | |
| II | 1.297 (0.622-2.706) | 0.488 | – | – | – | – | 1.191 (0.570-2.489) | 0.642 |
| III | 2.782 (1.339-5.779) | 0.006* | – | – | – | - | 2.545 (1.223-5.299) | 0.013* |
| IV | 4.891 (2.317-10.322) | <0.001* | – | – | – | 4.752 (2.251-10.034) | <0.001* | |
TCGA the Cancer Genome Atlas, AJCC the American Joint Committee on Cancer, ENSAT European Network for the Study of Adrenal Tumors, HR Hazard Ratio, CI confidence interval *P<0.05
(Fig. 4e, P<0.001) and the stratified analyses (Fig. 5). For the entire cohort, the median OS of patients in the low- and high- risk groups was 55.0 months (95% CI 43.1-67.1) and 8.0 months (95% CI 5.6-10.4), respectively.
Clinical usefulness of the nomogram
DCA analysis was performed to illustrate the net bene- fit at 5 years in each cohort. When the threshold prob- abilities exceeded 21% in the SEER training set, ranged between 34% and 95% in the SEER internal validation set, exceeded 6% in the TCGA validation set and 12% in the Chinese multicenter validation set, the use of the nomogram to predict the prognosis of adult ACC patients provided greater net benefit than the “treat all” or “treat none” strategies, indicating the favora- ble potential clinical applicability of the nomogram (Fig. 6).
Discussion
In the present study, a nomogram incorporating age at diagnosis, T stage, N stage, and M stage was developed to predict the OS probability for adult ACC patients after surgery and was externally validated using multiethnicity and multicenter datasets. The nomogram showed good discrimination and calibration in both the training and validation datasets. Also, the DCA revealed it had prom- ising clinical applicability. Thus, the constructed nomo- gram can provide an easy-to-use and individualized tool to help physicians to make more informed treatment decisions for treating adult ACC patients.
Concerning the development of ACC, the only cura- tive approach to ACC is complete tumor resection. How- ever, the 5-year survival after surgery for ACC range between 16 and 60% [22, 31, 32], showing the prognostic heterogeneity associated with this disease. Some stud- ies reported that adjuvant therapy in localized disease may provide survival benefits [10, 33-35]. However, the
| Model and variable | Cox regression coefficient |
|---|---|
| Model 1 | |
| Age | 0.0210 |
| 7th AJCC T stage | |
| T1 | Reference |
| T2 | 0.4343 |
| T3 | 1.0923 |
| T4 | 1.1331 |
| 7th AJCC N stage | 1.0257 |
| 7th AJCC M stage | 0.6781 |
| Model 2 | |
| Age | 0.0176 |
| 7th AJCC stage group | |
| I | Reference |
| II | 0.1788 |
| III | 0.8649 |
| IV | 1.3601 |
| Model 3 | |
| Age | 0.0185 |
| ENSAT stage group | |
| I | Reference |
| II | 0.1750 |
| III | 0.9343 |
| IV | 1.5587 |
SEER the Surveillance Epidemiology, and End Results database, AJCC the American Joint Committee on Cancer, ENSAT European Network for the Study of Adrenal Tumor
| Models | C-index (95% CI) | P* |
|---|---|---|
| Model 1 | 0.715 (0.679-0.751) | – |
| Model 2 | 0.697 (0.660-0.734) | <0.001 |
| Model 3 | 0.698 (0.662-0.734) | <0.001 |
*P values were obtained by comparing model 1 with model 2 and model 3, respectively
necessity of adjuvant therapy remains elusive. Therefore, accurate prognostic predication after surgery for adult ACC patients is significant not only for the adjuvant treatment selection but also to inform patients about
their long-term prognoses. However, there lacked a clear optimal method in present literature to predict the out- come of ACC patients and stratify them into different risk subgroups.
Several, but debatable, factors related to ACC prognosis were identified in previous studies. Some have reported that there were no correlations between age, sex, tumor size to the outcome of ACC [10, 36, 37], while others showed that age, sex, high tumor grade, and tumor size were significantly associated with prognosis [18, 31, 38]. Indeed, all of them focused on ACC patients of all ages (adults and pediatrics). The number of pediatric ACC patients was roughly about 12% to 20% of the total inves- tigated cohort from these studies. Therefore, different biological behavior and clinical presentations between adult and pediatric ACC patients might have accounted for these inconsistent results. In contrast, in this pre- sent study, only adult patients were investigated and we found that age at diagnosis, T stage, N stage and M stage were independent predictors of OS after surgery. Simi- lar to our findings, there have been other studies report- ing that old age was a poorer prognostic factor for OS in adults as compared to the young patients [14, 36, 39]. OS might be affected by age not only related to the clinical course of the disease, but also for age-related complica- tions [40]. Notably, a recent report has proposed a novel staging system incorporating patients’ age and was not based on the patient’s tumor size [14], because age at diagnosis may better inform clinicians about proper indi- vidualized treatment and prognostication. Also, in this present study, the TNM stage contributed as a main part of the final risk score and demonstrated better prognostic performance when combined age. Our nomogram had superior prognostic ability compared to the AJCC and ENSAT stage group models (Table 4). Also, its discrimi- nation and calibration displayed good performance and was validated using internal and external validation data- sets. Thus, it has the potential to be implemented in real- world clinical practice.
Further, the formulated nomogram can be comprehen- sively used for individualized treatment planification due to its potential to accurately stratify adult ACC patients based on their mortality risk [5, 8, 41] into two distinct prognostic groups, namely high- and low-risk groups. To the best of our knowledge, this is the first nomogram for
(See figure on next page.)
Fig. 2 The formulated nomogram and its calibration plots. a This nomogram enables the prognostication of the 1-, 3- and 5-year estimates of the OS of ACC patients after surgery. Calibration plots of the nomogram performed in the b SEER training, c SEER internal validation, d the TCGA validation and e the Chinese multicenter validation set, respectively. Nomogram-predicted OS is plotted on the x-axis; actual OS is plotted on the y-axis. Dots represent nomogram-predicted probabilities. An ideal prediction would correspond to the diagonal 45° gray line slope of b-e. The score range of the nomogram is 0 to 29.3. OS overall survival, ACC adrenocortical carcinoma, SEER the Surveillance Epidemiology, and End Results database, TCGA the Cancer Genome Atlas set
a
0
1
2
3
4
5
6
7
8
9
10
Points
Age (years)
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
T2
T4
T stage
T1
T3
N1
N stage
NO
M1
M stage
MO
Total points
0
5
10
15
20
25
30
1-year OS probability
0.96
0.95
0.9
0.85
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1 0.07
3-year OS probability
0.9
0.85
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.05
0.001
5-year OS probability
0.85
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.05
0.001
b
1.0
SEER training set
C
1.0
SEER validation set
0.8
0.8
Actual OS probability
Actual OS probability
0.6
0.6
0.4
0.4
1-year survival
1-year survival
3-year survival
3-year survival
0.2
5-year survival
0.2
5-year survival
0.0
C-index (95% CI), 0.715 (0.679-0.751)
0.0
C-index (95% CI), 0.672 (0.637-0.707)
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Nomogram-predicted OS probability
Nomogram-predicted OS probability
1.0
TCGA validation set
Q
e
1.0
Chinese validation set
0.8
0.8
Actual OS probability
0.6
Actual OS probability
0.6
0.4
0.4
1-year survival
1-year survival
3-year survival
3-year survival
0.2
5-year survival
0.2
5-year survival
0.0
C-index (95%), 0.810 (0.732-0.888)
0.0
C-index (95% CI), 0.726 (0.633-0.819)
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Nomogram-predicted OS probability
Nomogram-predicted OS probability
a
Risk score, low cutoff value
b
33
C
100
Low risk
High risk
Risk score, high cutoff value
Patients number
Overall survival (%)
10
50
0
V
0
0
0.5
2.96
4.2
0.0
4.5
8.9
Risk score
Time since diagnosis (years)
a
SEER training set
b
SEER validation set
C
TCGA validation set
1.00
Low risk
1.00
1.00
High risk
Low risk
High risk
Low risk
High risk
Survival probability
0.75
Survival probability
0.75
Survival probability
0.75
0.50
0.50
0.50
0.25
0.25
0.25
P < 0.001
P < 0.001
P < 0.001
0.00
0.00
0
2
4
6
8
0.00
0
10
20
30
0
2.5
5
7.5
10
Time since diagnosis (years)
Time since diagnosis (years)
Time since diagnosis (years)
Number at risk
Number at risk
Number at risk
Low risk
360
184
95
48
15
Low risk 294
89
14
0
Low risk 73
46
26
9
3
High risk 44
5
0
0
0
High risk 24
1
0
0
High risk
9
2
0
0
0
d
Chinese validation set
Entire study cohort
e
1.00
Low risk
1.00
High risk
Low risk
High risk
Survival probability
0.75
Survival probability
0.75
0.50
0.50
0.25
0.25
P = 0.010
P < 0.001
0.00
0
2.5
5
7.5
10
0.00
0
10
20
30
Time since diagnosis (years)
Time since diagnosis (years)
Number at risk
Number at risk
Low risk
76
22
15
7
0
High risk
Low risk 803 High risk 83
92
14
0 0
6
1
1
1
0
1
0
a
Sex (male)
b
Sex (female)
1.00
Low risk
1.00
Low risk
High risk
High risk
Survival probability
0.75
Survival probability
0.75
0.50
0.50
0.25
0.25
P < 0.001
P < 0.001
0.00
0.00
0
5
10
15
20
25
0
10
20
30
Time since diagnosis (years)
Time since diagnosis (years)
Number at risk
Number at risk
Low risk
313
92
35
15
8
2
Low risk 490
57
6
0
High risk
32
0
0
0
0
0
High risk
51
1
0
0
C
Tumor location (left-sided ACC)
d
Tumor location (right-sided ACC)
1.00
1.00
Low risk
Low risk
High risk
High risk
Survival probability
0.75
Survival probability
0.75
0.50
0.50
0.25
0.25
P < 0.001
P < 0.001
0.00
0
5
10
15
20
25
0.00
0
10
20
30
Time since diagnosis (years)
Time since diagnosis (years)
Number at risk
Number at risk
Low risk
444
123
45
18
6
0
Low risk
359
47
8
0
High risk
42
1
1
1
0
0
High risk 41
0
0
0
Fig. 5 Kaplan-Meier survival curves categorized into low-risk and high-risk groups in stratified analyses for the entire study cohort. Significance between the OS of the high-risk and low-risk patients was observed in both sex a male and b female, and tumor location, c left-sided ACC, d right-sided ACC. OS overall survival, ACC adrenocortical carcinoma
predicting the OS of adult ACC patients after surgery. Compared with other prognostic models, our model was validated in three independent validation cohorts with promising results. The favorable discriminating ability of the nomogram in all validation sets supports its general- izability for routine clinical use.
Some limitations of the present study were as follows. First, this study may be potentially limited due to its ret- rospective nature and associated with inherent biases. We excluded patients with missing data during data col- lection as their inclusion would have simultaneously
affected the credibility of the results. Second, the mul- tivariable model did not include some potential prog- nostic predictors, such as the hormone status, Ki-67 index, Weiss score, SF-1, calretinin, and SRC1, because these informations were not uniformly available in the retrieved datasets. A more comprehensive model consid- ering all potential risk factors might be expected to have better prognostic performance. Third, the follow-up time was shorter in the Chinese multicenter validation data- set, and close monitoring and five-year follow-up data are still required for these patients.
a
SEER training set
SEER validation set
0.6
b
None
None
0.5
All
0.5
All
Nomogram
Nomogram
0.4
0.4
Net benefit
0.3
Net benefit
0.3
0.2
0.2
0.1
0.1
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Threshold probability
Threshold probability
TCGA validation set
C
d
Chinese validation set
None
None
0.3
All
All
Nomogram
0.25
Nomogram
Net benefit
0.2
Net benefit
0.15
0.1
0.05
0.0
-0.05
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Threshold probability
Threshold probability
Conclusions
In conclusion, we have developed a nomogram able to predict the postoperative OS tailored for adult ACC patients. The nomogram demonstrated favorable pre- dictive accuracy and clinical usefulness after validation in datasets comprised of different populations and eth- nicity. The proposed nomogram is an easy-to-use tool with promising clinical applicability to provide indi- vidualized patient counseling, timely surveillance, and clinical assessments.
Abbreviations
ACC: adrenocortical carcinoma; AIC: Akaike’s Information Criterion; AJCC: the American Joint Committee on Cancer; C-index: concordance index; DCA: decision curve analysis; ENSAT: the European Network for the Study of Adrenal
Tumors; IQR: interquartile range; OS: overall survival; SEER: the Surveillance Epidemiology, and End Results; TCGA: the Cancer Genome Atlas.
Acknowledgements
We thank the contributors and handlers of the Surveillance Epidemiology, and End Results (SEER) database and the Cancer Genome Atlas (TCGA) database for the making these datasets publicly available to promote continuous research.
Authors’ contributions
TXL and JH were responsible for the study design and participated in evalu- ation of results. JOK, JJZ, JHC, SXW, XYD, WBX, XC, CYL, HY, XXF and CWH participated in collection of study materials or patients. JOK, JJZ, JHC, SXW, XYD, WBX, ZWL, QL, WC and HDQ participated in collection and assembly of data. JQK, JJZ, JHC, XYD and HDQ performed the data analysis and interpreta- tion. JOK, JJZ, JHC, XYD and WBX drafted the manuscript. TXL, JH, HDQ, ZWL, QL and WC proofread the manuscript for important intellectual content. All authors contributed to manuscript preparation. All authors read and approved the final manuscript.
Funding
This work was supported by the Natural Science Foundation of China (81572514, U1301221, 81402106, 81272808, 81825016), the Natural Science Foundation of Guangdong, China (2016A030313244), Grant [2013]163 from Key Laboratory of Malignant Tumor Molecular Mechanism and Translational Medicine of Guangzhou Bureau of Science and Information Technology, Grant KLB09001 from the Key Laboratory of Malignant Tumor Gene Regulation and Target Therapy of Guangdong Higher Education Institutes, and grants from the Guangdong Science and Technology Department (2015B050501004, 2017B020227007). The funders had no involvement in study design; in the collection, analysis, or interpretation of data; in the writing of the report; or in the decision to submit the paper for publication.
Availability of data and materials
All data generated or analyzed during this study are included in this published article and its additional information files.
Ethics approval and consent to participate
For the Chinese multicenter dataset, the retrospective analysis of anonymous patient data was approved by the institutional review board at each partici- pating institution.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Author details
1 Department of Urology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen Univer- sity, 107 Yan Jiang West Road, Guangzhou 510120, Guangdong, P. R. China. 2 Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou 510120, Guangdong, P. R. China. 3 Department of Neurology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou 510120, Guangdong, P. R. China. 4 Department of Obstetrics and Gynecology, Third Affiliated Hospital of Sun Yat-sen University, Guangzhou 510630, Guangdong, P. R. China. 5 Department of Urology, Sun Yat-sen University Cancer Center, Guangzhou 510060, Guangdong, P. R. China. 6 Department of Urology, First Affiliated Hospital of Sun Yat-sen University, Guangzhou 510080, Guangdong, P.R. China. 7 Department of Urology, The First Affiliated Hospital of Nanjing Medical University, Nanjing 210029, Jiangsu, P. R. China. 8 State Key Laboratory of Oncology in South China, Guangzhou 510120, Guangdong, P. R. China.
Received: 27 June 2019 Accepted: 14 November 2019 Published online: 27 November 2019
References
1. Sabolch A, Else T, Griffith KA, Ben-Josef E, Williams A, Miller BS, et al. Adju- vant radiation therapy improves local control after surgical resection in patients with localized adrenocortical carcinoma. Int J Radiat Oncol Biol Phys. 2015;92(2):252-9. https://doi.org/10.1016/j.ijrobp.2015.01.007.
2. Datta J, Roses RE. Surgical management of adrenocortical carcinoma: an evidence-based approach. Surg Oncol Clin N Am. 2016;25(1):153-70. https://doi.org/10.1016/j.soc.2015.08.011.
3. Terzolo M, Baudin AE, Ardito A, Kroiss M, Leboulleux S, Daffara F, et al. Mitotane levels predict the outcome of patients with adrenocortical car- cinoma treated adjuvantly following radical resection. Eur J Endocrinol. 2013;169(3):263-70. https://doi.org/10.1530/EJE-13-0242.
4. Gaujoux S, Weinandt M, Bonnet S, Reslinger V, Bertherat J, Dousset B. Surgical treatment of adrenal carcinoma. J Visc Surg. 2017;154(5):335-43. https://doi.org/10.1016/j.jviscsurg.2017.06.010.
5. Mallin K, Phillips JL, Winchester DP. Treatment patterns and outcomes for patients with adrenocortical carcinoma associated with hospital case volume in the United States. Ann Surg Oncol. 2017;24(Suppl 3):617. https ://doi.org/10.1245/s10434-017-6249-9.
6. Dickson PV, Kim L, Yen TWF, Yang A, Grubbs EG, Patel D, et al. Evaluation, staging, and surgical management for adrenocortical carcinoma: an
update from the SSO endocrine and head and neck disease site working group. Ann Surg Oncol. 2018;25(12):3460-8. https://doi.org/10.1245/ s10434-018-6749-2.
7. Else T, Kim AC, Sabolch A, Raymond VM, Kandathil A, Caoili EM, et al. Adrenocortical carcinoma. Endocr Rev. 2014;35(2):282-326. https://doi. org/10.1210/er.2013-1029.
8. Carnaille B. Adrenocortical carcinoma: which surgical approach? Lan- genbecks Arch Surg. 2012;397(2):195-9. https://doi.org/10.1007/s0042 3-011-0852-1.
9. Erickson LA, Rivera M, Zhang J. Adrenocortical carcinoma: review and update. Adv Anat Pathol. 2014;21(3):151-9. https://doi.org/10.1097/ PAP.0000000000000019.
10. Terzolo M, Angeli A, Fassnacht M, Daffara F, Tauchmanova L, Conton PA, et al. Adjuvant mitotane treatment for adrenocortical carcinoma. N Engl J Med. 2007;356(23):2372-80. https://doi.org/10.1056/NEJMoa063360.
11. Dralle H. Adjuvant mitotane treatment after adrenalectomy is not ben- eficial for adrenocortical carcinoma. Chirurg. 2016;87(11):980. https://doi. org/10.1007/s00104-016-0288-9.
12. Terzolo M, Zaggia B, Allasino B, De Francia S. Practical treatment using mitotane for adrenocortical carcinoma. Curr Opin Endocrinol Diabetes Obes. 2014;21 (3):159-65. https://doi.org/10.1097/MED.000000000000005 6.
13. Fassnacht M, Dekkers OM, Else T, Baudin E, Berruti A, de Krijger R, et al. European Society of Endocrinology Clinical Practice Guidelines on the management of adrenocortical carcinoma in adults, in collaboration with the European Network for the Study of Adrenal Tumors. Eur J Endocrinol. 2018;179(4):G1-46. https://doi.org/10.1530/EJE-18-0608.
14. Asare EA, Wang TS, Winchester DP, Mallin K, Kebebew E, Sturgeon C. A novel staging system for adrenocortical carcinoma better predicts survival in patients with stage I/II disease. Surgery. 2014;156(6):1378-85. https://doi.org/10.1016/j.surg.2014.08.018.
15. Fang C, Wang W, Deng JY, Sun Z, Seeruttun SR, Wang ZN, et al. Proposal and validation of a modified staging system to improve the prognosis predictive performance of the 8th AJCC/UICC pTNM staging system for gastric adenocarcinoma: a multicenter study with external validation. Cancer Commun (Lond). 2018;38(1):67. https://doi.org/10.1186/s4088 0-018-0337-5.
16. Yao JJ, Zhou GQ, Wang YQ, Wang SY, Zhang WJ, Jin YN, et al. Prognostic values of the integrated model incorporating the volume of metastatic regional cervical lymph node and pretreatment serum Epstein-Barr virus DNA copy number in predicting distant metastasis in patients with N1 nasopharyngeal carcinoma. Chin J Cancer. 2017;36(1):98. https://doi. org/10.1186/s40880-017-0264-x.
17. Li Y, Bian X, Ouyang J, Wei S, He M, Luo Z. Nomograms to predict overall survival and cancer-specific survival in patients with adrenocortical carcinoma. Cancer Manag Res. 2018;10:6949-59. https://doi.org/10.2147/ CMAR.S187169.
18. Kim Y, Margonis GA, Prescott JD, Tran TB, Postlewait LM, Maithel SK, et al. Nomograms to predict recurrence-free and overall survival after curative resection of adrenocortical carcinoma. JAMA Surg. 2016;151(4):365-73. https://doi.org/10.1001/jamasurg.2015.4516.
19. Zini L, Capitanio U, Jeldres C, Lughezzani G, Sun M, Shariat SF, et al. External validation of a nomogram predicting mortality in patients with adrenocortical carcinoma. BJU Int. 2009;104(11):1661-7. https://doi. org/10.1111/j.1464-410X.2009.08660.x.
20. Wang S, Chen SS, Gao WC, Bai L, Luo L, Zheng XG, et al. Prognostic factors of adrenocortical carcinoma: an analysis of the surveillance epidemiology and end results (SEER) database. Asian Pac J Cancer Prev. 2017;18(10):2817-23. https://doi.org/10.22034/APJCP.2017.18.10.2817.
21. Ayala-Ramirez M, Jasim S, Feng L, Ejaz S, Deniz F, Busaidy N, et al. Adreno- cortical carcinoma: clinical outcomes and prognosis of 330 patients at a tertiary care center. Eur J Endocrinol. 2013;169(6):891-9. https://doi. org/10.1530/EJE-13-0519.
22. Michalkiewicz E, Sandrini R, Figueiredo B, Miranda EC, Caran E, Oliveira- Filho AG, et al. Clinical and outcome characteristics of children with adrenocortical tumors: a report from the International Pediatric Adreno- cortical Tumor Registry. J Clin Oncol. 2004;22(5):838-45. https://doi. org/10.1200/JCO.2004.08.085.
23. McAteer JP, Huaco JA, Gow KW. Predictors of survival in pediatric adrenocortical carcinoma: a Surveillance, Epidemiology, and End Results
(SEER) program study. J Pediatr Surg. 2013;48(5):1025-31. https://doi. org/10.1016/j.jpedsurg.2013.02.017.
24. Raymond VM, Else T, Everett JN, Long JM, Gruber SB, Hammer GD. Prevalence of germline TP53 mutations in a prospective series of unse- lected patients with adrenocortical carcinoma. J Clin Endocrinol Metab. 2013;98(1):E119-25. https://doi.org/10.1210/jc.2012-2198.
25. Giordano TJ, Thomas DG, Kuick R, Lizyness M, Misek DE, Smith AL, et al. Distinct transcriptional profiles of adrenocortical tumors uncovered by DNA microarray analysis. Am J Pathol. 2003;162(2):521-31. https://doi. org/10.1016/S0002-9440(10)63846-1.
26. Kerkhofs TM, Ettaieb MH, Verhoeven RH, Kaspers GJ, Tissing WJ, Loeffen J, et al. Adrenocortical carcinoma in children: first population-based clinicopathological study with long-term follow-up. Oncol Rep. 2014;32(6):2836-44. https://doi.org/10.3892/or.2014.3506.
27. Akaike H. Information theory and an extension of the maximum likeli- hood principle. In: 2nd international symposium on information theory. Abstracts of papers. 1971. p. 276 -.
28. Wang W, Sun Z, Deng JY, Qi XL, Feng XY, Fang C, et al. A novel nomogram individually predicting disease-specific survival after D2 gastrectomy for advanced gastric cancer. Cancer Commun (Lond). 2018;38(1):23. https:// doi.org/10.1186/s40880-018-0293-0.
29. Camp RL, Dolled-Filhart M, Rimm DL. X-tile: a new bio-informatics tool for biomarker assessment and outcome-based cut-point optimization. Clin Cancer Res. 2004;10(21):7252-9. https://doi.org/10.1158/1078-0432. ccr-04-0713.
30. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluat- ing prediction models. Med Decis Making. 2006;26(6):565-74. https://doi. org/10.1177/0272989X06295361.
31. Bilimoria KY, Shen WT, Elaraj D, Bentrem DJ, Winchester DJ, Kebebew E, et al. Adrenocortical carcinoma in the United States: treatment utiliza- tion and prognostic factors. Cancer. 2008;113(11):3130-6. https://doi. org/10.1002/cncr.23886.
32. Vassilopoulou-Sellin R, Schultz PN. Adrenocortical carcinoma Clinical outcome at the end of the 20th century. Cancer. 2001;92(5):1113-21.
33. Luton JP, Cerdas S, Billaud L, Thomas G, Guilhaume B, Bertagna X, et al. Clinical features of adrenocortical carcinoma, prognostic factors, and the
effect of mitotane therapy. N Engl J Med. 1990;322(17):1195-201. https:// doi.org/10.1056/NEJM199004263221705.
34. Wangberg B, Khorram-Manesh A, Jansson S, Nilsson B, Nilsson O, Jakobs- son CE, et al. The long-term survival in adrenocortical carcinoma with active surgical management and use of monitored mitotane. Endocr Relat Cancer. 2010;17(1):265-72. https://doi.org/10.1677/ERC-09-0190.
35. Fassnacht M, Terzolo M, Allolio B, Baudin E, Haak H, Berruti A, et al. Com- bination chemotherapy in advanced adrenocortical carcinoma. N Engl J Med. 2012;366(23):2189-97. https://doi.org/10.1056/NEJMoa1200966.
36. Canter DJ, Mallin K, Uzzo RG, Egleston BL, Simhan J, Walton J, et al. Associ- ation of tumor size with metastatic potential and survival in patients with adrenocortical carcinoma: an analysis of the National Cancer Database. Can J Urol. 2013;20(5):6915-21.
37. Stojadinovic A, Ghossein RA, Hoos A, Nissan A, Marshall D, Dudas M, et al. Adrenocortical carcinoma: clinical, morphologic, and molecular characterization. J Clin Oncol. 2002;20(4):941-50. https://doi.org/10.1200/ JCO.2002.20.4.941.
38. Rossfeld KK, Maithel SK, Prescott J, Wang TS, Fields RC, Weber SM, et al. The prognostic significance of adrenocortical carcinomas identified incidentally. J Surg Oncol. 2018;118(7):1155-62. https://doi.org/10.1002/ jso.25274.
39. Scollo C, Russo M, Trovato MA, Sambataro D, Giuffrida D, Manusia M, et al. Prognostic factors for adrenocortical carcinoma outcomes. Front Endo- crinol (Lausanne). 2016;7:99. https://doi.org/10.3389/fendo.2016.00099.
40. Fang C, Wang W, Zhang Y, Feng X, Sun J, Zeng Y, et al. Clinicopathologic characteristics and prognosis of gastroenteropancreatic neuroendo- crine neoplasms: a multicenter study in South China. Chin J Cancer. 2017;36(1):51. https://doi.org/10.1186/s40880-017-0218-3.
41. Kerkhofs TM, Verhoeven RH, Van der Zwan JM, Dieleman J, Kerstens MN, Links TP, et al. Adrenocortical carcinoma: a population-based study on incidence and survival in the Netherlands since 1993. Eur J Cancer. 2013;49(11):2579-86. https://doi.org/10.1016/j.ejca.2013.02.034.