A
CENTURY
AJR O F
1906 2006
MEDICAL
IMAGING
CT Histogram Analysis in Pathologically Proven Adrenal Masses
Erick M. Remer1 Gaspar A. Motta-Ramirez1 Laura B. Shepardson1 Amir H. Hamrahian2 Brian R. Herts1
Keywords: abdominal imaging, adrenal adenoma, adrenal carcinoma, adrenal gland, adrenal metastasis, CT imaging, pheochromocytoma
DOI:10.2214/AJR.05.0179
Received February 2, 2005; accepted after revision April 15, 2005.
1Division of Radiology, Cleveland Clinic Foundation, 9500 Euclid Avenue, Desk A21, Cleveland, Ohio 44195. Address correspondence to E. M. Remer (remere1@ccf.org).
2Department of Endocrinology and Metabolism, Cleveland Clinic Foundation, Cleveland, Ohio.
AJR 2006; 187:191-196
0361-803X/06/1871-191 @ American Roentgen Ray Society
OBJECTIVE. The purpose of this article is to evaluate a histogram analysis method for dis- tinguishing adrenal adenomas from metastases, pheochromocytomas, and adrenocortical car- cinomas on CT.
MATERIALS AND METHODS. A pathology database was searched, and 335 adrenalec- tomies from 1995 to 2002 were identified. CT images were available for retrospective review in 187 patients (93 males, 94 females; age range, 15-84 years; mean age, 55.2 years) with 208 ad- renal masses. This included 112 adenomas in 104 patients, 48 metastases in 39 patients, 40 pheo- chromocytomas in 36 patients, and eight adrenocortical carcinomas in eight patients. Histogram analysis was performed using a circular region of interest for mean attenuation, number of pixels, number of negative pixels (<0 H), and percentage of negative pixels by two interpreters. Areas of necrosis were excluded from measurements. Observer agreement was calculated.
RESULTS. In 72 of 76 (94.7%) and 63 of 72 (87.5%) adenomas, respectively, interpreters found attenuation values greater than 10 H contained negative pixels on unenhanced CT scans. None of the enhanced adenomas had mean attenuation less than or equal to 10 H, but 24 (38.7%) and 28 (45.2%), respectively, had negative pixels. Negative pixels were present in un- enhanced and enhanced metastases, pheochromocytomas, and carcinomas. Using a 5% or 10% negative pixel threshold value to diagnose adenoma improved specificity but diminished sen- sitivity. Specificity for a 10% negative pixel threshold was approximately 88% for unenhanced CT scans and 99% for enhanced CT scans, with sensitivities of 71% and 12%, respectively.
CONCLUSION. Although specificity for the diagnosis of adenomas on enhanced CT scans with histogram analysis was high when a 10% negative pixel threshold was used, low sen- sitivity likely limits clinical usefulness.
C ommonly, an adrenal mass is dis- covered incidentally on a contrast- enhanced abdominal CT scan. In this situation, CT densitometry and enhanced attenuation washout methods cannot be used to assess if the mass is an ad- enoma because during routine CT examina- tions neither unenhanced nor delayed en- hanced images are obtained. Bae et al. [1] reported a method to diagnose adrenal ade- nomas on routine enhanced abdominal CT examinations based on the distribution of tis- sue attenuation in the mass, termed “histo- gram analysis.” With this technique, a region of interest (ROI) within an adrenal mass is measured-plotting the pixel attenuation ver- sus pixel frequency on a standard CT work- station-and the number of pixels that measure fat attenuation (those < 0 H) are de- termined. Histogram analysis, like unen- hanced CT densitometry, relies on the fact
that adenomas contain large amounts of intra- cytoplasmic lipid [2]. Although promising, further confirmation of this technique in other patient populations is necessary before it can be widely accepted. This report applies the CT histogram analysis method in a group of masses that also include pheochromocytomas and primary adrenocortical carcinomas in ad- dition to adenomas and metastases. Unlike the prior series, all diagnoses were confirmed by histopathology. Two different observers performed a histogram analysis of each mass in this series and their observer agreement was assessed.
Materials and Methods
Our institutional review board approved this study, and informed consent was waived. A pathology data- base was searched and 335 adrenalectomies from 1995 to 2002 were identified. CT images were avail- able for retrospective review in 187 patients (93
32
Pixel Count
5
Pixel Count
4
24
Pixels 1: 957
Pixels 1: 13
Mean 1: 88.70
Mean 1: - $.69
SD 1: 16.72
SD 1: 4.80
Limits 1: - 1000/1000
3
Limits 1: - 1000/-1
16
2
3
1
0
HU
0
HU
-999
500
0
500
1000
-999
750
500
-250
0
A
B
24
Pixel Count
Pixel Count
18
Pixels 1: 749
Pixels 1: 0
Mean 1: 85.76
Mean 1: - 1024.00
SD 1: 19.56
SD 1: 0.00
Limits 1: - 1000/1000
Limits 1: - 1000/-1
12
0
HU
HU
-999
500
0
500
1000
-999
750
500
-250
0
C
D
C, Enhanced image shows mean attenuation of 86 H.
D, Enhanced histogram analysis shows zero negative pixels.
males, 94 females; age range, 15-84 years, mean age, 55.2 years) with 208 adrenal masses. CT scans were available for retrospective review. This included 112 adenomas in 104 patients, 48 metastases in 39 pa- tients, 40 pheochromocytomas in 36 patients, and eight adrenocortical carcinomas in eight patients.
The adenoma group included 50 masses with unenhanced CT scans only, six with enhanced CT scans only, and 56 with both unenhanced and en- hanced CT scans. The metastasis group included four lesions with unenhanced CT scans only, 14 with enhanced CT scans only, and 30 with both un-
enhanced and enhanced CT scans. The pheochro- mocytoma group included five masses with unen- hanced CT scans only, five with enhanced CT scans only, and 30 with both unenhanced and enhanced CT scans. The carcinoma group included one en- hanced CT scan only and seven with unenhanced
CT Histogram Analysis in Adrenal Masses
and enhanced CT scans. All diagnoses were con- firmed at histopathology after surgical excision, ex- cept for two who had needle biopsies of metastases.
CT Technique
CT scans were performed using single or MDCT units (Somatom Plus 4, Volume Zoom, or Sensa- tion 16, Siemens Medical Solutions). Imaging pro- tocols varied considerably over the review period because of changes in CT technology. Contrast-en- hanced CT scans were obtained 60-70 seconds af- ter IV administration of 150 mL of 300 mg I/mL nonionic contrast medium (most commonly iopro- mide; Ultravist 300, Berlex) injected at a rate of 2-4 mL/s using a power injector. Reconstruction thick- ness was 8 mm at 4-mm intervals from 1995-1999 and 5 mm at 2.5-mm intervals for 1999-2002. De- tector collimation was 5 or 8 mm for single-row scanners and 2.5 or 5 mm for multirow scanners (120 kVp, 200-250 mAs). Images were recon- structed with a standard soft-tissue algorithm.
Data
All CT studies were retrieved from the institu- tional image archive to a clinical PACS workstation (MagicView 1000, Siemens Medical Solutions). Measurements were obtained as previously de- scribed [1]. For each examination, the image that contained the maximal cross-sectional diameter of the adrenal mass was chosen. Maximal diameter was measured. A circular ROI was placed on the adrenal mass, taking care to avoid the periphery of the mass to eliminate partial volume effects or any areas of necrosis (Fig. 1). Necrosis was defined as a focal area of hypoattenuation compared with the remainder of the adrenal mass. If possible, approx- imately two thirds of the mass was measured.
The ROI measures were displayed to include a histogram, mean attenuation, number of pixels, and range of pixel attenuation. The measurement tool was initially set to include a range of pixels from -1000 to 1000 H. The measurement was then re- peated with the pixels’ limits set at -1 to -1000 H, so that the number of pixels less than zero (negative pixels) could be counted. A subset of the mean at- tenuation values of unenhanced masses from 1995-2002 [3] and 1997-2002 [4] were published previously as parts of other series.
Data Analysis
The percentage of negative pixels was calculated by dividing the number of negative pixels by the to- tal number of pixels for each adrenal mass. The per- centage of adrenal masses with a mean attenuation of 10 H or less was calculated for unenhanced and enhanced studies of adenomas, metastases, pheo- chromocytomas, and adrenocortical carcinomas. The percentages of each type of adrenal mass that
contained any, more than 5% negative pixels, and more than 10% negative pixels were calculated. Observer concordance was assessed by determin- ing the percent agreement for each observation and then calculating kappa statistics [5].
Results
Adrenal Adenomas on Unenhanced CT
The mean attenuation of the 105 unen- hanced adenomas was 18.3 H ± 14.3 [SD] for interpreter 1 and 15.5 H ± 14.4 for interpreter 2 (Table 1). The range of attenuation values for interpreters 1 and 2 was -27 to 51 and -17 to 45.2, respectively. Only 29 of 105 (27.6%) and 33 of 105 (31.4%) unenhanced ade- nomas had attenuation of 10 H or less, but all contained negative pixels. Interpreter 1 found 72 of 76 (94.7%) and interpreter 2 found 63 of 72 (87.5%) adenomas with attenuation values greater than 10 H that contained neg- ative pixels. The number of unenhanced ade- nomas with the percentage of negative pixels greater than 5% was 85 of 105 (81%) for in- terpreter 1 and 82 of 105 (78.1%) for inter- preter 2. The number of adenomas with a per- centage of negative pixels greater than 10% was 73 (69.5%) for interpreter 1 and 76 (72.4%) for interpreter 2. The overall range for percentage of negative pixels for both in-
terpreters was 0-90 with a mean of 24 ± 0.2. The numbers of masses with negative pixels within various mean attenuation ranges are listed in Table 1. Sensitivities for interpreters 1 and 2 using 1 negative pixel, 5% negative pixels, and 10% negative pixel threshold val- ues to diagnose an adenoma on unenhanced scans are found in Table 2.
Adrenal Adenomas on Enhanced CT
The mean attenuation of the 62 enhanced adenomas was 64.9 ± 26.1 for interpreter 1 and 63.1 ± 30.8 for interpreter 2. The range of attenuation values was 26 to 199 and 19 to 176, respectively. None had mean attenuation less than or equal to 10 H, but 24 (38.7%) and 28 (45.2%) had negative pixels. The number of adenomas with more than 5% negative pix- els was 14 (22.6%) for interpreter 1 and 16 (25.8%) for interpreter 2. The number with more than 10% negative pixels was 7 (11.3%) for each interpreter. The range of the percent- age of negative pixels was zero to 30 and zero to 29 with a mean of 4 ± 0.07 for each inter- preter. Overall, 24 of 62 (38.7%) and 29 of 62 (46.8%) adenomas with attenuation measure- ments greater than 10 H had negative pixels. The sensitivities for interpreters 1 and 2 using one negative-pixel, 5% negative-pixel, and
| Histogram Analysis | Adrenal Adenomas | |||
|---|---|---|---|---|
| Unenhanced CT (n= 105) | Enhanced CT (n=62) | |||
| Interpreter 1 | Interpreter 2 | Interpreter 1 | Interpreter 2 | |
| Attenuation (H) | ||||
| Mean ± SD | 18.3 ± 14.3 | 15.5 ±14.4 | 64.9 ± 26.1 | 63.1 ± 30.8 |
| Range | -27 to 51 | -17 to 45.2 | 26 to 199 | 19 to 176 |
| Percent negative pixels | ||||
| Mean ± SD | 24 ± 0.2 | 25 ±0.2 | 4 ± 0.07 | 4 ± 0.07 |
| Range | 0 to 90 | 0 to 83 | 0 to 30 | 0 to 29 |
| No. with any negative pixels | 102 (97.1%) | 99 (94.3%) | 24 (38.7%) | 28 (45.2%) |
| With > 5% | 85 (81.0%) | 82 (78.1%) | 14 (22.6%) | 16 (25.8%) |
| With > 10% | 73 (69.5%) | 76 (72.4%) | 7 (11.3%) | 7 (11.3%) |
| No. with mean ≤ 10 H | 29 | 33 | 0 | 0 |
| No. with negative pixels | 29 | 33 | 24 | 29 |
| No. with negative pixels; >10 H | 72/76 (94.7%) | 63/72 (87.5%) | 24/62 (38.7%) | 29/62 (46.8%) |
| No. with H = 10-20 | 23 | 23 | 2 | 4 |
| No. with negative pixels | 23 | 23 | 2 | 4 |
| No. with H = 20-30 | 29 | 32 | 3 | 4 |
| No. with negative pixels | 26 | 30 | 3 | 4 |
| No. with H = 30-40 | 17 | 8 | 2 | 3 |
| No. with negative pixels | 17 | 6 | 1 | 2 |
| Performance Measure | 1 Negative Pixel | 5% Negative Pixels | 10% Negative Pixels | |||
|---|---|---|---|---|---|---|
| Interpreter | Interpreter | Interpreter | Interpreter | Interpreter | Interpreter | |
| 1 | 2 | 1 | 2 | 1 | 2 | |
| Sensitivity unenhanced | 97.1 | 94.3 | 81.0 | 78.1 | 69.5 | 72.4 |
| Specificity unenhanced | 38.2 | 28.9 | 76.3 | 67.1 | 89.5 | 85.5 |
| Specificity unenhanced, met only | 38.2 | 41.2 | 79.4 | 73.5 | 97.1 | 91.2 |
| Sensitivity enhanced | 38.7 | 47.8 | 22.6 | 27.4 | 11.3 | 12.9 |
| Specificity enhanced | 88.5 | 64.4 | 96.6 | 90.8 | 98.9 | 98.9 |
| Specificity enhanced, met only | 93.2 | 95.5 | 97.7 | 95.5 | 97.7 | 100.0 |
Note-met only = only metastases and adenomas.
10% negative-pixel threshold values to diag- nose an adenoma on enhanced CT scans are found in Table 2.
Nonadenomas on Unenhanced CT
Negative pixels were present in 21 (inter- preter 1) and 20 (interpreter 2) of 34 unen- hanced metastases, three and 12 of 44 en- hanced metastases, 21 and 28 of 35 unenhanced pheochromocytomas, four and 14 of 34 enhanced pheochromocytomas, five and six of seven unenhanced carcinomas and three and five of nine enhanced carcinomas, respectively. The mean attenuation values can be found in Tables 3-5. None of the unen- hanced and enhanced metastases, pheochro- mocytomas, or carcinomas had attenuation values less than or equal to 10 H. The speci- ficities for each interpreter associated with choosing a threshold for a positive study of one negative pixel, 5% negative pixels, or 10% negative pixels for unenhanced and en- hanced studies are found in Table 2.
Observer Agreement
Interobserver agreement was evaluated by calculating the percentage of agreement be- tween the two interpreters for each observa- tion, that is, identifying adenomas (with and without IV contrast) and nonadenomas (with and without IV contrast) at thresholds of one negative pixel and 5% negative pixels. Kappa statistics and percentages of agreement are found in Table 6.
Discussion
Although unenhanced CT densitometry and delayed contrast-enhanced CT are accu- rate methods to diagnose adenoma, most ad- renal masses are identified incidentally after a standard contrast-enhanced abdominal CT
has been performed. Without unenhanced or delayed-enhanced images available, a repeat study is necessary to attempt to diagnose an adenoma. Bae et al. [1] described a histogram analysis method with the goal of improving sensitivity in distinguishing between ade- nomas and metastases on standard contrast- enhanced CT studies. The histogram analysis method uses an ROI within the adrenal mass and determines if pixels measuring less than 0 H are present. Bae et al. [1] found that only 10.9% (20/184) of adenomas had a mean at- tenuation of 10 H or less on contrast-en- hanced examinations but that 52.7% (97/184) of them could be diagnosed by the presence of negative pixels using the histogram method. In our series, none of the 62 enhanced ade- nomas had a mean attenuation of 10 H or less, but between 38.7% and 45.2% (24/28) had negative pixels.
Unlike Bae et al. [1], however, our series did have a number of nonadenomas that contained negative pixels. We found that there were me- tastases, pheochromocytomas, and adrenocor- tical carcinomas that contained negative pix- els. In our series, this diminished the specificity of the histogram technique on en- hanced scans to between 64.4% and 88.5%. This drop in specificity was even greater on un- enhanced CT scans where using one negative pixel as a diagnostic threshold led to specifici- ties between 28.9% and 38.2%. Comparing ad- enomas and metastases only improved speci- ficity to 93.2-95.5% on enhanced scans and 38.2-41.2% on unenhanced scans.
Despite having no metastases with nega- tive pixels in their series, Bae et al. [1] recom- mended using a threshold of 10% negative pixels to diagnose adenomas because the low- est percentage of negative pixels in their ade- noma group that measured less than 10 H on
enhanced examinations was 9.8%. None of our histopathologically diagnosed adenomas measured less than 10 H on enhanced studies. Adenomas had negative pixel values that ranged from 0-30%. In our series, using a 10% negative pixel threshold maintained high specificity of 98.9%, but sensitivity dropped to unacceptably low levels of between 11.3-12.9% on enhanced studies. These lev- els are similar to what the Bae et al. [1] series found when assessing CT densitometry on enhanced scans.
The mean attenuation value for unenhanced adenomas in this series is higher than many published previously. An analysis of prior studies reported a sensitivity of 71% for a 10- H threshold value [6]. The percentage of ade- nomas with attenuation of 10 H or less in this series was only approximately 30%. This is likely related to a high percentage of lipid-poor adenomas in our study population. An alterna- tive explanation is that some unknown differ- ence exists in our study population of patho- logically proven adenomas from those in other series that were mainly proved by lack of inter- val growth or characteristic appearance. Many of the adenomas in this series were resected si- multaneously with renal cell carcinomas.
Lesions with both unenhanced and en- hanced CT images had an increase in mean at- tenuation and a decrease in the percentage of negative pixels on enhanced images com- pared with unenhanced. This was seen in the Bae et al. [1] series and was attributed to the pseudoenhancement effect that has been de- scribed for renal cysts [7-9].
Bae et al. [1] enumerated factors that may affect image quality and noise that were, sim- ilarly, not standardized in this study. These in- clude patient body habitus, breathing motion artifact, size and location of ROI, kilovolt peak and milliampere second values, slice collimation, section thickness, reconstruction kernel, IV contrast media injection rate, and scan delay. Other factors that could impact image noise include differences in CT scan- ner technology-both single-detector and MDCT scanners were used in this series. For MDCT systems, more patient length is scanned per rotation; thus, for extended- length studies, the X-ray tube current can be higher than for single-section units. The higher current reduces image noise and im- proves image quality, which is critical for thin-section extended-length studies, espe- cially of large patients [10]. This could reduce the number of spuriously negative pixels. However, thinner slice thickness may lead to
CT Histogram Analysis in Adrenal Masses
| Histogram Analysis | Adrenal Metastases | |||
|---|---|---|---|---|
| Unenhanced CT (n = 34) | Enhanced CT (n = 44) | |||
| Interpreter 1 | Interpreter 2 | Interpreter 1 | Interpreter 2 | |
| Attenuation (H) | ||||
| Mean ± SD | 38.4 ± 9.7 | 38.5 ± 14.4 | 76.0 ± 23.4 | 77.7 ± 27 |
| Range | 14-77.8 | 19.3-56.7 | 32.2-139.2 | 42.1-120.5 |
| Percent negative pixels | ||||
| Mean ± SD | 2.4 ± 2.9 | 3.7 ± 4.8 | 0.66 ± 4 | 0.78 ± 1.5 |
| Range | 0-10 | 0-14 | 0-27 | 0-6 |
| No. with any negative pixels | 21 | 20 | 3 | 12 |
| With > 5% | 7 | 9 | 1 | 2 |
| With > 10% | 1 | 5 | 1 | 0 |
| No. with mean ≤ 10 H | 0 | 0 | 0 | 0 |
| No. with negative pixels | 0 | 0 | 0 | 0 |
| Histogram Analysis | Pheochromocytomas | |||
|---|---|---|---|---|
| Unenhanced CT (n = 35) | Enhanced CT (n = 35) | |||
| Interpreter 1 | Interpreter 2 | Interpreter 1 | Interpreter 2 | |
| Attenuation (H) | ||||
| Mean ± SD | 36.6 ± 9.8 | 34.7 ± 9.4 | 98.8 ± 10.5 | 77.84 ± 25 |
| Range | 12-49 | 19-49 | 62-156 | 18-119 |
| Percent negative pixels | ||||
| Mean ± SD | 5.3 ± 0.07 | 5.3 ± 0.06 | 0 ± 0.02 | 1.9 ± 0.04 |
| Range | 0-23 | 0-17 | 0-9 | 0-21 |
| No. with any negative pixels | 21 | 28 | 4 | 14 |
| With > 5% negative pixels | 10 | 14 | 1 | 4 |
| With > 10% negative pixels | 7 | 6 | 0 | 1 |
| No. with mean ≤ 10 H | 0 | 0 | 0 | 0 |
| No. with negative pixels | 0 | 0 | 0 | 0 |
| Histogram Analysis | Adrenocortical Carcinomas | |||
|---|---|---|---|---|
| Unenhanced CT (n= 7) | Enhanced CT (n= 9) | |||
| Interpreter 1 | Interpreter 2 | Interpreter 1 | Interpreter 2 | |
| Attenuation (H) | ||||
| Mean ± SD | 39.3 ± 6.5 | 38.7 ± 3.3 | 63.6 ± 22.5 | 59 ± 17 |
| Range | 37-49 | 34-44 | 35-100 | 35-89 |
| Percent negative pixels | ||||
| Mean ± SD | 2.7 ± 0.03 | 3.9 ± 0.05 | 2.4 ± 0.05 | 2.2 ± 0.08 |
| Range | 0-9 | 0-13 | 0-15 | 0-7 |
| No. with any negative pixels | 5 | 6 | 3 | 5 |
| With > 5% negative pixels | 1 | 2 | 1 | 2 |
| With > 10% negative pixels | 0 | 0 | 1 | 0 |
| No. with mean ≤ 10 H | 0 | 0 | 0 | 0 |
| No. with negative pixels | 0 | 0 | 0 | 0 |
nosier images. A recent phantom study (Tongdee R, et al., Comparison of CT histo- gram analysis and mean attenuation methods in characterization of adrenal masses: a phan- tom study, presented at the 2004 annual meet- ing of the Radiological Society of North America) by Bae’s group evaluated the effects of changes in image thickness and the image reconstruction kernel on mean attenuation and the percentage of negative pixels. They found that the percentage of negative pixels was highly correlated to an increase in the SD of the mean attenuation values. However, the percentage of negative pixels and SDs in- creased together only as the reconstruction kernel changed from smooth to sharp and the slice thickness decreased. This suggests that thin slices or sharp kernels may be inappro- priate for histogram analysis because of ex- cessive image noise.
We chose to include adrenal abnormalities other than adenomas and metastases in our se- ries. Although the clinical goal in patients with a primary malignancy and an adrenal mass is to distinguish between a metastasis and an inci- dentaloma, most typically an adenoma, other abnormalities such as pheochromocytomas [3] and adrenocortical carcinomas can also be dis- covered incidentally. A comprehensive analy- sis of histogram performance should include these other adrenal lesions. Although no my- elolipomas were included in this series, they often have specific imaging features of macro- scopic fat [2]. Because of the large amount of mature fat, most myelolipomas are easily rec- ognized on CT and would either be excluded from this type of analysis or cause them to be mistaken for adenomas. Either situation is clin- ically acceptable. Similarly, the inclusion of adrenocortical carcinoma and pheochromocy- tomas in the series includes lesions that can be centrally necrotic. Although the presence of necrosis was not specifically recorded, areas of necrosis or cystic change were intentionally
| Histogram Analysis | 1 Negative Pixel (%) | 5% Negative Pixels (%) |
|---|---|---|
| Adenomas | ||
| Without contrast | 92.2 (0.16) | 87.4 (0.63) |
| With contrast | 81.6 (0.63) | 81.6 (0.49) |
| Nonadenomas | ||
| Without contrast | 70.1 (0.33) | 76.1 (0.45) |
| With contrast | 76.7 (0.37) | 89.0 (0.15) |
Note-Kappa values are in parentheses.
avoided in the ROI measurement in this series. Much like the performance of mean attenua- tion measurements to diagnose adenomas or adrenal enhancement washout [2], atypical im- aging features should clinically supersede his- togram analysis in arriving at a diagnosis. Findings such as large size, central necrosis, or calcification should raise suspicion for carci- noma [2] or large degenerated adenoma [11] based on mass morphology alone and obviate the need for histogram analysis.
One strength of our study is that we relied on histopathologic proof to determine the diagno- sis of each of the adrenal masses. This dimin- ishes uncertainty and potential misdiagnoses that can occur when criteria such as lack of in- terval change on imaging studies or another imaging test such as MRI are used as indicat- ing the true diagnoses of the adrenal masses.
Two interpreters independently measured ROIs on all patients in each group. Interob- server agreement was measured at one nega- tive pixel and 5% negative pixel thresholds (Table 6). Percent agreement ranged from 70-92%, with three values in the 70-79% range, three values in the 80-89% range, and one value greater than 90%. One might have predicted higher agreement than the interpret- ers achieved. Factors that may have influ- enced interpreter agreement include ROI placement, ROI size, and image choice for ROI measurement. The interpreters were in- structed to choose approximately two thirds of the mass and to exclude borders with retro- peritoneal fat and areas of heterogeneously low attenuation or calcifications from their measurements. However, they may have had different placement of the ROI despite the same instructions. Thus, retroperitoneal fat or areas of necrosis may have been unintention- ally included. Although Bae et al. [1] ex- cluded masses with areas of cystic change, calcification, or necrosis, other studies [12, 13] using ROI measurements to differentiate different adrenal masses excluded these areas from the ROI. Excluding these masses from the analysis could potentially have dimin- ished interpreter errors, improving agree- ment, but would have diminished applicabil- ity of the test. Second, although instructions specified that the largest axial size of the mass should be selected from which to measure the ROI, the interpreters could have chosen dif- ferent axial images, adding to variability.
Lastly, interpreters may have chosen different sized ROIs that could have added variability, especially with larger masses.
In several circumstances in this study, the overall agreement was high (such as 89% for nonadenomas 5% negative pixels with con- trast), but the kappa value was slight (0.15). The kappa value is a measure of agreement corrected for chance. This paradoxical result is caused by the high prevalence of negative cases. Prevalence effects can lead to situa- tions in which the values of k do not corre- spond with intuition [14].
The issue then becomes the actual percent- agreement values themselves. Exactly what constitutes an acceptable level of agreement between interpreters for any test is subjective and may vary depending on the setting, tech- nique, or clinical application. We could only find one study using ROIs to differentiate adre- nal masses that evaluated interpreter concor- dance [15]. Three interpreters’ ROI measure- ments were used to create receiver operating characteristic curves. Although these were de- termined to be “virtually identical,” kappa val- ues were not reported. Therefore, it is hard to judge what level of agreement one should ex- pect for this technique. Should the level of agreement between our two interpreters call into question the reproducibility of the test? Further studies are needed.
There are several limitations of our study. There is the potential for verification bias in our study. This occurs when the study results are only reported for subjects with verified disease status. However, the most typical scenario in which the results of the diagnos- tic test under study affect whether the gold standard procedure is used to verify the test result is not the case. Second, the study was retrospectively performed and, therefore, imaging parameters were not controlled within the population.
In conclusion, our study shows that using a 10% negative pixel threshold to discriminate adenomas from nonadenomas on enhanced scans has a high specificity of 98.9% but suf- fers from poor sensitivity. Further modifica- tions of this technique may be necessary be- fore it is clinically useful.
References
1. Bae KT, Fuangtharnthip P, Prasad SR. Joe BN,
Heiken JP. Adrenal masses: CT characterization with histogram analysis method. Radiology 2003; 228:735-742
2. Dunnick NR, Korobkin M. Imaging of adrenal inci- dentalomas: current status. AJR 2002; 179:559-568
3. Motta-Ramirez GA, Remer EM, Herts BR, Gill IS, Hamrahian AH. Comparison on CT findings in symptomatic and incidentally discovered pheo- chromocytomas. AJR 2005; 185:684-688
4. Hamrahian AH, Ioachimescu AG, Remer EM, et al. Clinical utility of noncontrast CT attenuation value (HU) to differentiate adrenal adenomas/hyperpla- sias from nonadenomas: Cleveland Clinic experi- ence. J Clin Endocrinol Metab 2005; 90:871-877
5. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33:159-174
6. Boland GW, Lee MJ, Gazelle GS, Halpern EF, Mc- Nicholas MM, Mueller PR. Characterization of ad- renal masses using unenhanced CT: an analysis of the CT literature. AJR 1998; 171:201-204
7. Maki DD, Birnbaum BA, Chakraborty DP, Jacobs JE, Carvalho BM, Herman GT. Renal cyst pseu- doenhancement: beam-hardening effects on CT numbers. Radiology 1999; 213:468-472
8. Coulam CH, Sheafor DH, Leder RA, Paulson EK, DeLong DM, Nelson RC. Evaluation of pseudoen- hancement of renal cysts during contrast-enhanced CT. AJR 2000; 174:493-498
9. Bae KT, Heiken JP, Siegel CL, Bennett HF. Renal cysts: is attenuation artifactually increased on con- trast-enhanced CT images? Radiology 2000; 216:792-796
10. Rydberg J, Buckwalter KA, Caldemeyer KS, et al. Multisection CT: scanning techniques and clinical applications. RadioGraphics 2000; 20:1787-1806
11. Newhouse JH, Heffess CS, Wagner BJ, Imray TJ, Adair CF, Davidson AJ. Large degenerated adrenal adenomas: radiologic-pathologic correlation. Ra- diology 1999; 210:385-391
12. Szolar DH, Kammerhuber FH. Adrenal adenomas and nonadenomas: assessment of washout at de- layed contrast-enhanced CT. Radiology 1998; 207:369-375
13. Caoili EM, Korobkin M, Francis IR, et al. Adrenal masses: characterization with combined unen- hanced and delayed enhanced CT. Radiology 2002; 222:629-633
14. Kundel HL, Polansky M. Measurement of observer agreement. Radiology 2003; 228:303-308
15. Lee MJ, Hahn PF, Papanicolau N, et al. Benign and malignant adrenal masses: CT distinction with at- tenuation coefficients, size, and observer analysis. Radiology 1991; 179:415-418