Original Investigation

An Exploratory Multi-reader,
Multi-case Study Comparing
Transmission Ultrasound to
Mammography on Recall Rates and
Detection Rates for Breast Cancer
Lesions
Bilal Malik, PhD, Elaine Iuanow, MD, John Klock, MD

Background: Three-dimensional Quantitative Transmission (QT) ultrasound imaging is an emerging modality for improving the detection
and diagnosis of breast cancer. QT ultrasound has high resolution and high contrast to noise ratio, making it effective in evaluating breast
tissue. This study compares radiologists’ performance of noncancer recall rates and lesion detection rates using QT Ultrasound versus
full-field digital mammography (FFDM) in a cross section of female subjects.
Materials and Methods: In this multi-reader multi-case (MRMC) study, we examined retrospective data from two clinical trials conducted
at five sites. All subjects received FFDM and QT scans within 90 days. Data were analyzed in a reader study with full factorial design
involving 22 radiologists and 108 breast cases (42 normal, 39 pathology-confirmed benign, and 27 pathology-confirmed cancer cases).
The main results used a random-reader random-case analysis adjusted for location bias performed after a primary predefined random-
reader fixed-case analysis.
Results: The readers’ mean rate of detecting lesions of any type was 4% higher (p-value > 0.05) with QT imaging. The mean non-cancer
recall rate improved significantly, showing a decrease of 16% with QT (p-value = 0.03), at the expense of a 2% decrease in the mean can-
cer recall rate (p-value >0.05) in comparison to FFDM. Combining performance on cancer and noncancer recall rates, the mean area
under the receiver operator curve of confidence scores improved significantly by 10% with QT (p-value = 0.01).
Conclusion: This MRMC study indicates that QT improves non-cancer recall rates without substantially affecting cancer recall rates. The
main limitation is the small number of cases from retrospective data. A larger prospective MRMC study is warranted for further assess-
ment.

Key Words: Transmission ultrasound; medical imaging; breast cancer; reader study; cancer detection.
© 2020 The Association of University Radiologists. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND
license (http://creativecommons.org/licenses/by-nc-nd/4.0/)

INTRODUCTION superimposition effects inherent to projection imaging.

Overlying fibroglandular tissue can mask a cancer, decreasing

A
lthough two-dimensional (2D) full-field digital
the sensitivity for breast cancer detection particularly for
mammography (FFDM) is used to screen for breast
women with dense breasts (1,2). It can also cause summation
cancer, the diagnostic accuracy is limited by
artifacts, resulting in false-positive findings and more recalls,
leading to additional studies and/or procedures (3).
Acad Radiol 2022; 29:S10–S18 Digital breast tomosynthesis (DBT), a form of limited-angle
From the QT Ultrasound LLC, Novato, California. Received August 31, 2020; tomography, was developed to reduce the issues caused by
revised November 5, 2020; accepted November 12, 2020. Address corre-
spondence to B.M. e-mail: [email protected]
superimposed breast tissue observed in conventional mammog-
© 2020 The Association of University Radiologists. Published by Elsevier Inc.
raphy. The ability of DBT to view the breast in slices reduces
This is an open access article under the CC BY-NC-ND license breast tissue overlap, thus potentially revealing lesions that would
(http://creativecommons.org/licenses/by-nc-nd/4.0/) have otherwise been missed. Previous studies have demonstrated
https://doi.org/10.1016/j.acra.2020.11.011

S10
Academic Radiology, Vol 29, No S1, January 2022 COMPARING TRANSMISSION ULTRASOUND TO MAMMOGRAPHY

an increase in cancer detection rates and reduction in recall rates often need to be recalled to evaluate findings seen on the
using DBT as an adjunct to FFDM compared to FFDM alone, views performed after the patient has left the clinic.
for both screening and diagnostic purposes, even in women While FFDM, HHUS, and MRI constitute a majority of the
with dense breasts (4 7). The limitations of DBT include rela- imaging performed in breast screening and diagnostic workup,
tively longer interpretation times, higher costs for patients and other imaging modalities have also made their way into research
for medical facilities needing larger imaging storage capacities, and limited clinical use. Dedicated breast CT has been studied
and increased radiation dose compared to FFDM (5-8). Many over the past few years (19). Additionally, other CT based multi-
of these issues have been improved upon over the years so that modality based systems (SPECT-CT and PET-CT) have also
DBT is now common practice, however recent studies are find- been developed, however, the use of these systems in the clinic
ing that DBT does not improve the sensitivity for detecting can- remains limited, given the potential radiation safety issues for
cers in women with dense breasts (9). screening in younger women (20,21).
To offset the limitations of mammography, other modalities As a result, there is a clinical need for a breast imaging modal-
such as Magnetic Resonance Imaging (MRI) and Hand-Held ity that has high sensitivity and specificity, is safe and affordable,
Ultrasound (HHUS) have been used as secondary or adjunctive and can be used in young, high risk women who currently do
screening modalities. MRI is recommended for screening not receive mammography due to radiation effects. To address
women with a 20% or greater lifetime risk of breast cancer due this need, an automated, fully tomographic ultrasound modality,
to known risk factors, such as a family history of breast cancer, quantitative transmission (QT) ultrasound has been developed.
certain inherited genetic alterations, or high breast density (10). QT is a novel imaging modality shown to have high resolution
Studies have shown MRI to be a very sensitive method for and high contrast to noise ratio, providing anatomic details as
detecting invasive breast cancer and comparable to FFDM for well as information about tissue characteristics using the speed of
detecting ductal carcinoma in situ, but MRI has also demon- sound data, thus improving diagnostic accuracy (22,23). A pho-
strated lower specificity with relatively higher false positive and tograph of the QT scanner is shown in Figure 1 and representa-
biopsy rates than screening mammography (11). In addition, tive QT images and comparison with other modalities are
MRI requires administration of intravenous contrast agents, shown in Figure 2. Additionally, QT can use imaging bio-
which can be associated with the deposition of heavy metal ele- markers to differentiate between benign and malignant masses
ments in the brain and elsewhere in the body, is expensive, is (24), which help improve the accuracy and performance. In
not suitable for patients with implanted devices or claustropho- comparison to HHUS which is based on reflection B mode
bia and is not available to all women, particularly those in low- data only and assumes a constant speed of sound throughout the
income areas. Thus, it is not considered appropriate for screen- tissue, QT imaging uses transmitted ultrasound waves through
ing a general population (11,12). the breast alongside reflected ultrasound output rendering
In addition to MRI, HHUS, and Automated Breast Ultra- improved accuracy of breast imaging. In addition, the scan is
sound (ABUS) systems have been used to improve screening automated, resulting in consistent and reproducible imaging.
for breast cancer. HHUS in breast imaging has shown to The projection data acquired by the scanner is used in image
improve cancer detection rates, especially in women with reconstruction algorithms which are based on inverse scattering
dense breast tissue (13-16). Studies have shown additional and fully account for the three-dimensional (3D) nature of
cancers detected in adjunctive screening settings, however, acoustic wave propagation. The result of image reconstruction is
the improved cancer detection has happened at the expense the generation of coregistered 3D image volumes of reflection,
of an increase in false positives (17). Moreover, the operation and speed-of-sound maps that together can be used to quantita-
of HHUS is highly operator dependent resulting in a lack of tively identify breast tissue types (25-27). While there has been
standardization and integration of HHUS in the breast considerable work performed in describing the theory behind
screening paradigm (18). Automated ultrasound systems like imaging reconstruction for transmission ultrasound, and charac-
ABUS can be difficult to integrate into practices as patients terizing QT imaging performance, there have been no reader

Figure 1. (A) Photograph of the QT Scanner www.qtultra

sound.com.

S11
MALIK ET AL Academic Radiology, Vol 29, No S1, January 2022

Figure 2. (A) Representative speed-of-sound image (right) and the corresponding whole breast H&E section (left) showing correlation of glan-
dular and ductal histology with the speed-of-sound image; (B) Representative image of a mature apocrine cyst showing the cyst membrane
and the internal cyst contents; (C) representative images of fibroadenomas showing internal cellular detail; (D) QT image of an invasive ductal
carcinoma showing internal detail; (E) MRI image of the same mass as in D.

studies reported that compare radiologists’ performance of non- case collection clinical trials conducted between 2006 and
cancer recall rates and lesion detection rates for transmission 2018 at the following five sites: UC San Diego Health, Mayo
ultrasound and FFDM. In this study, we compared the perfor- Clinic, Long Beach Medical Center, Marin Breast Health Trial
mance of QT and FFDM for breast cancer detection in a select Center, and Universitatsklinikum Freiburg University Hospi-
population of women enrolled for mammography and QT tal. Within a 90-day period, participating adult female subjects
ultrasound. received both a standard FFDM and a transmission ultrasound
scan (QT Ultrasound Breast Scanner, QT Ultrasound, Novato,
California) in the case collection study where cases were col-
MATERIALS AND METHODS lected when a mammographic abnormality was seen on at least
one view of the mammogram and subsequent QT imaging
Study Subjects
was performed. A team of three board certified breast radiolog-
In this study, we directly compared detection rates for all ists adjudicated all mammograms and QT images for image
lesions and recall rates for transmission ultrasound with those quality and colocation of breast masses. Normal mammograms
for FFDM. We retrospectively examined data collected from were also enrolled with their corresponding QT ultrasound
two HIPAA-compliant, institutional review board-approved scans. The transmission ultrasound scan was performed by

S12
Academic Radiology, Vol 29, No S1, January 2022 COMPARING TRANSMISSION ULTRASOUND TO MAMMOGRAPHY

placing the subject’s breasts in the scan tank of the scanner, one The 22 participating readers were all board-certified diag-
breast at a time, where the breast was positioned in a water nostic radiologists from academic and nonacademic institu-
bath. A 360-degree data acquisition occurred around the breast tions with a variety of experience, ranging from breast-
with a curvilinear array with the array moving up 2 mm fol- fellowship trained imagers to general radiologists. Out of 22
lowing a 360-degree rotation around the breast where the readers, 12 readers were breast-fellowship trained radiologists;
entire breast was imaged from the nipple to the pectoralis mus- 10 readers had 1 10 years’ experience, 4 readers had 11 20
cle of the chest wall. The images were reconstructed and years’ experience, and 8 readers had greater than 20 years’
viewed in a proprietary software, QTviewerÒ . experience. Sixteen of the 22 readers worked in private prac-
tice, four in academia, and two in community practices. Prior
to the study, all readers successfully completed a standard QT
Image Interpretation and Analyses and FFDM reader training program, which included instruc-
tion on the operation and functionality of the standardized
The FFDM and QT breast imaging data were analyzed in an QT and FFDM workstations used to review all QT and
exploratory multi-reader multi-case (MRMC) study with full FFDM images, respectively.
factorial design involving 22 radiologists and 108 breast cases, The readers were blinded to all diagnostic reports, prior
including 42 normal, 39 pathology-confirmed benign (18 imaging, and contralateral breast imaging. To avoid memory
cyst and 21 benign solid space-occupying lesions), and 27 effects, they interpreted all of the QT breast cases in a ran-
pathology-confirmed cancer cases. The cases were collected domized fashion on a single day, and similarly interpreted all
when a mammographic abnormality was seen on mammog- of the FFDM cases in a randomized order the following day.
raphy and subsequent QT imaging was performed in the For each case and imaging modality, the readers were asked
diagnostic setting; however, since mammography misses a to mark up to three findings. Instances of marking multiple
substantial number of QT-identified masses, cases with nega- findings was relatively rare and only seven readers com-
tive mammograms were also added to control for bias against mented on multiple findings: five readers marked two find-
QT resulting from “only positive mammography” entry cri- ings and two readers marked three findings. An adjudicator
teria. Breast cases were excluded if the image sets were not assessed whether any of these potential findings correctly
properly processed due to image reconstruction algorithm located a ground truth lesion to assess the detection rate for
failure, were incomplete, or showed a clip or marker. Cases lesions. In addition, the readers were asked to recommend
were also excluded if the one-year follow-up mammograms recall or no-recall, as well as provide a confidence score
(normal cases) or pathology reports (benign and cancer cases) between 0 and 100 on their decision. For this study, a no-
were not available to establish the ground truth; the ground- recall was recommended for QT for normal cases and benign
truth lesions were outside the QT field of view; or the loca- cysts, whereas the presence of any solid space-occupying
tion of the biopsy results were not concordant with both QT lesions required a recall (28). For FFDM, a no-recall was
and FFDM. A patient flow chart is provided in Figure 3.

Figure 3. Patient flow chart depicting the number of patients included and excluded and the distribution between normal cases and cases
with benign and cancer findings.

S13
MALIK ET AL Academic Radiology, Vol 29, No S1, January 2022

recommended for normal cases only; the presence of any The ground truth was established by one-year follow-up
space-occupying lesion required a recall in agreement with mammogram results for the normal cases and pathology
standard of care (29). These recall criteria were guided by the results for the benign and cancer cases. All RRFC and
desire to compare the performance of FFDM and QT with- RRRC results were adjusted post-hoc for location bias,
out having priors to compare. considering recalls as correct only when the decisions were
based on the correct ground-truth lesions. This adjustment
is indicated because the severity of location bias is dissimilar
Statistical Analyses
for the two imaging modalities (34). Therefore, we
The data were analyzed for the entire cohort of 108 breast adjusted for location bias to avoid favoring the modality
cases (42 normal, 39 pathology-confirmed benign, and 27 with higher false-positive rates.
pathology-confirmed cancer cases) using two general
approaches: a random-reader fixed-cases (RRFC) analysis
and a random-readers random cases (RRRC) analysis.
RESULTS
RRFC analysis generalizes to the population of readers, but is
specific to the particular case set and is termed random-reader From the reader study comparing QT imaging and FFDM,
fixed-cases analysis. In comparison, RRRC analysis general- overall improvement was noted with QT imaging in all the
izes both the case set and the population of readers. The defined metrics including noncancer recall rate, ROC-AUC,
RRRC analysis was expected to provide results more gener- and rate of detecting lesions of any type. QT imaging showed
alizable to new readers reading new cases, but with wider a slight, but statistically insignificant decrease in cancer recall
confidence levels compared to the RRFC analysis. rate in comparison to FFDM discussed below.
For both approaches, performance comparisons between After appropriately adjusting for location bias, the 22 read-
QT and FFDM were summarized in terms of mean differen- ers’ mean non-cancer recall rate was 16% lower with QT
ces between readers and 95% confidence intervals (CI) for than FFDM with a 95% confidence interval of
these differences with p-values determining the degree of sta- ( 0.20, 0.12), p-value <0.01 in the RRFC analysis and
tistical significance. The performance metrics included non- ( 0.24, 0.08), p-value = 0.03 in the RRRC analysis for all
cancer and cancer recall rates and detection rates for all 108 breast cases, as shown in Table 1 and Figure 4. The read-
lesions. In addition, we analyzed the mean area under the ers’ mean cancer recall rate was 2% lower with QT than
receiver operator curve (ROC-AUC) based on the readers’ FFDM with a 95% confidence interval of ( 0.07, 0.04), p-
confidence scores as a statistically efficient approach to evalu- value >0.05 in the RRFC analysis and ( 0.16, 0.13), p
ating the cancer and noncancer performance metrics com- >0.05 in the RRRC analysis, as shown in Table 1.
bined into a single measurement. The overall accuracy of these recall decisions was also ana-
These analyses were performed according to the method of lyzed using the combined metric of ROC-AUC based on
Obuchowski & Rockette with Hillis adjustment to the degrees the readers’ confidence scores. The readers’ mean ROC-
of freedom (30,31). The RRRC analysis of ROC-AUC was AUC was 10% higher with QT than FFDM with a 95% con-
performed with the software package ORDBM MRMC 2.5, fidence interval of (0.07, 0.13), p-value <0.01 in the RRFC
written by Stephen L, Kevin M. Schwartz, and Kevin S. Ber- analysis and (0.02, 0.18), p-value = 0.01 in the RRRC analy-
baum (32). The trapezoidal/Wilcoxon method for curve fitting sis for all 108 breast cases, as shown in Table 2 and Figure 5.
and jackknifing for the covariance estimation were used in the Finally, the readers’ mean rate of detecting lesions of any
analysis. All other statistical analyses were performed in the statis- type was 4% higher with QT than FFDM with a 95% confi-
tical computing environment R version 3.4.0 or higher (33). dence interval of (0.01, 0.08), p-value = 0.02 in the RRFC
No statistical adjustments were made for multiple analyses. analysis and ( 0.06, 0.15), p-value > 0.05 in the RRRC

TABLE 1. Comparison of Recall Rates With QT and FFDM for Different Subgroups of Pathologies

Subgroup Count QT recall FFDM recall Difference in recall rates p-value Relative Difference:
rate rate with [95% CI] QT-FFDM

Non-cancer 81 0.31 0.47 -0.16 [-0.20, -0.12] < 0.01* -34.0%

Benign mass 21 0.62 0.68 -0.06 [-0.10, -0.02] <0.01* -8.8%
Cyst 18 0.21 0.66 -0.44 [-0.51, -0.38] <0.01* -68.2%
Benign lesions 39 0.43 0.67 -0.24 [-0.28, -0.20] <0.01* -35.8%
Normal 42 0.19 0.27 -0.08 [-0.15, -0.01] 0.02* -29.6%
Cancer 27 0.6 0.61 -0.02 [-0.07, 0.04] >0.05 -1.6%

The count represents the sample size for that given subgroup. QT: transmission ultrasound; FFDM: full-field digital mammography. The rela-
tive difference is calculated as percentage relative change between the QT and FFDM recall rates. Negative value of the relative difference indi-
cates reduction in recall rates with QT. * denotes statistically significant difference. Non-cancer includes benign mass, benign lesions and
normal cases. Benign lesions group includes benign mass and cyst.

S14
Academic Radiology, Vol 29, No S1, January 2022 COMPARING TRANSMISSION ULTRASOUND TO MAMMOGRAPHY

Figure 4. Overall performance of the radiologists as measured

in terms of recall rates for QT and FFDM. The horizontal axis
marks the difference between the recall rates of QT and FFDM.
A lower than zero value indicates QT shows improvement over
FFDM; a greater than zero value indicates FFDM shows
improvement over QT. Square (&) markers correspond to
RRRC results; circle () markers correspond to RRFC results.

TABLE 2. Comparison of AUC of QT and FFDM for different Subgroups Of Pathologies

Subgroup comparison Count QT FFDM Difference in AUC values p-value Relative Difference:
AUC AUC with [95% CI] QT-FFDM

All vs Cancer 108 0.69 0.59 0.10 [0.07, 0.13] <0.01* 16.9%
Benign Mass vs Cancer 48 0.5 0.45 0.04 [0.02, 0.07] <0.01* 11.1%
Cyst vs Cancer 45 0.74 0.48 0.26 [0.22, 0.31] <0.01* 54.2%
Normal vs Cancer 69 0.76 0.7 0.06 [0.02, 0.10] <0.01* 8.6%

The count represents the sample size for that given subgroup. QT: transmission ultrasound; FFDM: full-field digital mammography; AUC:
area under the curve. The relative difference is calculated as percentage relative change between the QT and FFDM AUC values. Positive value
of the relative difference indicates increase in the AUC with QT. * denotes statistically significant difference. Non-cancer includes benign mass,
benign lesions and normal cases. Benign lesions group includes benign mass and cyst.

Figure 5. Overall performance of the radiologists as measured

in terms of ROC-AUC for QT and FFDM. The horizontal axis
marks the difference between the AUC of QT and FFDM. A
greater than zero value indicates QT shows improvement over
FFDM; a lower than zero value indicates FFDM shows improve-
ment over QT. Square (&) markers correspond to RRRC results;
circle () markers correspond to RRFC results.

analysis for the entire cohort of 108 breast cases, as shown in the ability to image dense breasts, the technology is being
Table 3 in Figure 6. pursued under the FDA’s Breakthrough Device Designation
of the 21st Century Cures Act as potentially useful for screen-
ing younger women who do not qualify for x-ray mammog-
DISCUSSION
raphy (35). This work is the first study assessing performance
Because the scope and cost of conventional prospective of transmission ultrasound for primary breast cancer screening
screening studies in breast cancer are substantial, and because when compared with mammography. The goal for this
the incidence of breast cancer in young women is extremely research is to gather evidence for its effectiveness for this
low, making population-based screening impossible, we have novel indication for use and to assess the detection rates for
discussed with the FDA how to design smaller trials with cre- lesions, benign and cancer, and the recall rates for the same.
ative design that could overcome the obstacles imposed by In the current study, the majority of the metrics used to
large-scale population screening. In support of this approach, quantify readers’ performance with QT imaging compared to
due to its unique safety profile and high image quality, and FFDM, showed overall improvement with QT imaging. The

S15
MALIK ET AL Academic Radiology, Vol 29, No S1, January 2022

TABLE 3. Comparison of Detection Rates For Any Type Of Lesions With QT and FFDM for Different Subgroups Of Pathologies

Subgroup Count QT detection FFDM Difference in detection p-value Relative Difference:

rate detection rate rates with [95% CI] QT-FFDM

All lesions 69 0.71 0.66 0.04 [0.01, 0.06] <0.01* 7.6%

Benign mass 21 0.74 0.71 0.03 [-0.00, 0.07] >0.05 4.2%
Cyst 20 0.77 0.66 0.10 [0.07, 0.14] <0.01* 16.7%
Benign lesions 41 0.75 0.69 0.07 [0.04, 0.09] <0.01* 8.7%
Cancer 28 0.64 0.63 0.01 [-0.04, 0.06] >0.05 1.6%

The count represents the sample size for that given subgroup. QT: transmission ultrasound; FFDM: full-field digital mammography. The rela-
tive difference is calculated as percentage relative change between the QT and FFDM detection rate values. Positive value of the relative differ-
ence indicates increase in the detection rate with QT. * denotes statistically significant difference. Non-cancer includes benign mass, benign
lesions and normal cases. Benign lesions group includes benign mass and cyst.

Figure 6. Overall performance of the radiologists as measured

in terms of detection rates for any type of lesions with QT and
FFDM. The horizontal axis marks the difference between the
detection rates with QT and FFDM. A greater than zero value
indicates QT shows improvement over FFDM; a lower than zero
value indicates FFDM shows improvement over QT. Square (&)
markers correspond to RRRC results; circle () markers corre-
spond to RRFC results.

non-cancer recall rate decreased by 16%, ROC-AUC comparisons of QT imaging and FFDM are first-in-class as
improved by 10%, rate of detecting lesions of any type well as first-in-human results demonstrating the capabilities
improved by 4%. QT imaging showed a slight, but statisti- of transmission ultrasound in breast screening with no direct
cally nonsignificant decrease of 2% in the cancer recall rate and relevant published literature with which to compare.
compared to FFDM. These results suggest that QT imaging In an outside meta-analysis of 20 studies, ROC-AUC was
shows improved accuracy of recall decisions and lesion detec- shown to be an efficient way to simultaneously capture the
tion as compared to FFDM albeit in a small subset of women performance of a device on cancer and non-cancer cases in
imaged. MRMC studies across laboratories and a better method for
For the detection of lesions overall, QT was slightly (+4%), predicting performance in clinical studies than cancer and
but not significantly better (95% CI: [ 0.06, 0.15]) than noncancer recall rates (36). In line with this, it is the preferred
FFDM. A clear improvement was seen for mean noncancer accuracy statistical metric for comparisons of the U.S. Food
recall rate: 16% with QT (95% CI: [ 0.24, 0.08]). As the and Drug Administration (37). Taken together, these results
95% confidence interval RRRC shows, the improvement is indicate that QT improves non-cancer recall rates without
positive also when accounting for case and reader variability. substantially affecting cancer recall rates.
On the other side, the difference in mean cancer recall rate The limitations in the detection of breast masses in FFDM
was 2% (CI [ 0.16,0.13]), which is not statistically signifi- are mainly due to the breast structure since the image is inher-
cant and is considerably smaller than the variability from cases ently a 2D view making it difficult to discern findings due to
and readers as seen in the range between the upper and lower superposition of fibroglandular tissue and summation artifacts
limit of the 95% CI. This poses an interesting question of (38,39). In comparison, the breast structure and the volumet-
what is seen when cancer and noncancer rates are meaning- ric nature of the fibroglandular tissue is fully resolved in QT
fully combined, which we investigated using the mean imaging and the interpretation is similar in nature to other
ROC-AUC. In our study, the ROC-AUC of QT was 10% volumetric imaging modalities such as breast MRI and dedi-
higher with confidence intervals with a positive lower limit cated breast CT (39,40) which is an important advantage. In
for both analyses, indicating a 95% confidence that QT had a addition, both the QT image acquisition and the image
higher recall accuracy than FFDM even when accounting for reconstruction are inherently 3D, so instead of the generation
case variability. Similar trends for the ROC-AUC were seen of concatenated 2D slices in FFDM, QT image synthesis
for both dense and nondense subpopulations. These reported directly results in image volumes and is able to account and

S16
Academic Radiology, Vol 29, No S1, January 2022 COMPARING TRANSMISSION ULTRASOUND TO MAMMOGRAPHY

compensate for full 3D wave physics, resulting in high con- wall radiation or for whom X-ray exposure is contraindi-
trast and highly reproducible images (27,41). Image acquisi- cated.
tion for QT ultrasound takes from 4 to 10 minutes Limitations in our study were the small sample size in a ret-
depending on the size of the breast and can be decreased to rospective study and case selection criteria. In particular,
half those times with software modifications. While this breast cases were excluded if either of the imaging datasets
acquisition time is longer than that for mammography, it is were improperly processed due to algorithm failures that
considerably less than time required for breast MRI which resulted in nondiagnostic images, which may have resulted in
can take up to an hour for both breasts. QT image interpreta- a bias. Since these cases could not be evaluated using both
tion takes less than 2 minutes for an experienced reader, with modalities, it is difficult to determine whether this potential
increases in reading time for complicated cases comparable to bias would be favorable towards QT or FFDM. Cases were
other volumetric breast imaging modalities. This is compara- also excluded if the FFDM findings were outside the QT’s
ble to time required to interpret mammograms and is signifi- imaging field of view. Visualization of the breast near the
cantly less in comparison to average interpretation time of 6 chest wall and axilla were at times limited with the QT scan-
minutes for breast MRI (42). ner used for this study; refinements in the scanner design to
Another advantage of QT is that the imaging is performed increase this field of view are currently underway. Breast
with minimal operator dependence that can allow for stan- tomosynthesis exams were not included in this comparison
dardization of the procedure, in comparison to conventional since many of the cases were collected at a time where tomo-
ultrasound. In clinical practice, HHUS is performed by a synthesis was not widespread. Future studies will include the
technologist, sometimes requiring the radiologist to interrupt comparison of QT Ultrasound with 2D mammography and
their work to re-scan the patient to confirm the technologist’s 3D tomosynthesis in a prospective study design.
findings. QT imaging and ABUS both have automated image Another limitation is that when planning the study, the
acquisition and hence the image review and interpretation information on reader and case variability did not allow a
can be performed by the radiologist on a complete image set. sample size calculation for the MRMC RRRC analyses, so
QT differs from ABUS in that the images are inherently vol- the fixed case analysis was used as predefined primary analysis,
umetric and cross-sectional, eliminating the need for re-scan- and the need and appropriateness of the location bias correc-
ning once proper patient positioning is achieved during tion was also based on seeing the strength of the bias when
scanning. This is critically important in that QT would elimi- analyzing the data.
nate the need for patients to return to the clinic for re-evalua- Studies comparing radiologist performance in a clinical set-
tion of findings seen on ABUS imaging sets when the scan is ting in comparison to that in a reader study show that radiolo-
read after the patient has departed the clinic. We will study gist performance is significantly better and more consistent in
the average time required to interpret the QT images in a clinical setting than in a reader study environment (46).
future studies. These observations can have adverse implications on the
HHUS imaging has proven important in differentiating results assessed on the basis of a reader study. However, such
solid masses from cysts in the breast (43). Transmission ultra- an impact can be reduced when using multiple readers. In the
sound has also been shown to aid in accurately identifying presented study, 22 readers with broad range of radiology
cysts vs solid masses and characterizing cystic components expertise interpreted the same sets of both FFDM and QT
based on speed of sound (44,45). In our results, we also saw images were randomized in each modality in varying order,
that QT imaging showed a higher cyst detection rate (17% potentially minimizing the bias associated with reading
relative improvement over mammography) and reduced FFDM images in a reader study and more effectively simulat-
recall rates for cysts (68% relative improvement over mam- ing the clinical setting.
mography) in comparison to other subgroups, indicating In summary, this preliminary study showed that QT imag-
improved radiologists’ performance when identifying cysts. ing demonstrates improved accuracy of recalls and lesion
In addition, the ROC-AUC relative improvement of 54% detection as compared to FFDM. These results show that
was the highest in all subgroup comparison, further highlight- additional larger prospective studies are needed, but that this
ing the performance improvements with identifying cysts novel ultrasound modality appears to be valuable asset in the
with QT imaging and eliminating the need for recalls in this current breast imaging paradigm and deserves further study.
subgroup of subjects. There may also be a specific niche for QT in young, high-
The clinical implications of the current research are that risk patients with dense breasts, which is currently being eval-
QT is a candidate for being added to the current breast imag- uated with the FDA in a breakthrough designation, or in
ing paradigm with particular application to younger women patients where access to care may be an issue.
that do not qualify for X-ray mammography screening, but
who need to be screened for medical reasons, for women
with dense breasts, for women with breast implants and for
COI STATEMENTS
women who refuse to have X-ray mammograms because of
the radiation risk, for example, women who have had chest JK is employee of QT Ultrasound LLC. BM and EI are con-
sultants for QT Ultrasound LLC.

S17
MALIK ET AL Academic Radiology, Vol 29, No S1, January 2022

TRIAL OVERSIGHT STATEMENT 19. Wienbeck S, Lotz J, Fischer U. Review of clinical studies and first clinical
experiences with a commercially available cone-beam breast CT in
The sponsor, QT Ultrasound LLC (Novato, California), Europe. Clin Imaging 2017; 42:50–59.
designed the trial in consultation with Cytel (Cambridge, 20. Shah JP, Mann SD, McKinley RL, et al. Implementation and CT sampling
characterization of a third-generation SPECT-CT system for dedicated
Massachusetts). All authors participated in the writing of the breast imaging. J Med Imaging (Bellingham) 2017; 4:033502.
manuscript and approved the draft that was submitted for 21. Bowen SL, Wu Y, Chaudhari AJ, et al. Initial characterization of a dedicated breast
publication. The first draft of the manuscript was written by PET/CT scanner during human imaging. J Nucl Med 2009; 50:1401–1408.
22. Klock J, Iuanow E, Smith K, et al. Visual grading assessment of quantitative
the first and last authors. The trial was conducted in accor- transmission ultrasound compared to digital X-ray mammography and
dance with the provisions of the International Conference on hand-held ultrasound in identifying ten breast anatomical structures 2017.
Harmonization Guidelines for Good Clinical Practice and 23. QT Ultrasound website Available from: www.qtultrasound.com.
24. Natesan R LS, Navarro D, Anaje C, et al. Radiomics in Transmission Ultra-
the Declaration of Helsinki. sound Improve Differentiation between Benign and Malignant Breast
Masses. In: Radiological Society of North America 2019 Scientific Assembly
and Annual Meeting; 2019. December 1 - December 6 Chicago IL.
25. Klock JC, Iuanow E, Malik B, et al. Anatomy-correlated breast imaging
and visual grading analysis using quantitative transmission ultrasound.
REFERENCES Int J Biomed Imaging 2016; 2016:7570406.
1. Destounis S, Johnston L, Highnam R, et al. Using volumetric breast den- 26. Malik B, Klock J, Wiskin J, et al. Objective breast tissue image classifica-
sity to quantify the potential masking risk of mammographic density. tion using Quantitative Transmission ultrasound tomography. Sci Rep
AJR Am J Roentgenol 2017; 208:222–227. 2016; 6:38857.
2. Holland K, van Gils CH, Mann RM, et al. Quantification of masking risk in 27. Malik B, Terry R, Wiskin J, et al. Quantitative transmission ultrasound
screening mammography with volumetric breast density maps. Breast tomography: Imaging and performance characteristics. Med Phys 2018;
Cancer Res Treat 2017; 162:541–548. 45:3063–3075.
3. Cohen EO, Tso HH, Phalak KA, et al. Screening mammography findings from 28. Iuanow E, Smith K, Obuchowski NA, et al. Accuracy of cyst versus solid
one standard projection only in the era of full-field digital mammography and diagnosis in the breast using Quantitative Transmission (QT) ultrasound.
digital breast tomosynthesis. AJR Am J Roentgenol 2018; 211:445–451. Acad Radiol 2017; 24:1148–1153.
4. Skaane P. Breast cancer screening with digital breast tomosynthesis. 29. Sickles E, D’Orsi C, Bassett L, et al. ACR BI-RADSÒ Mammograph.
Breast Cancer 2017; 24:32–41. 2013. In: ACR BI-RADSÒ Atlas, Breast Imaging Reporting and Data Sys-
5. Bernardi D, Macaskill P, Pellegrini M, et al. Breast cancer screening with tem [Internet]. Reston, VA: American College of Radiology.
tomosynthesis (3D mammography) with acquired or synthetic 2D mam- 30. Obuchowski NA, Rockette HE. Hypthesis testing of diagnostic accuracy
mography compared with 2D mammography alone (STORM-2): a popu- for multiple readers and multiple tests: an ANOVA approach with depen-
lation-based prospective study. Lancet Oncol 2016; 17:1105–1113. dent observations. Communications in Statistics — Simulation and
6. Skaane P, Bandos AI, Gullien R, et al. Comparison of digital mammogra- Computation 1995; 24:285–308.
phy alone and digital mammography plus tomosynthesis in a popula- 31. Hillis SL, Obuchowski NA, Schartz KM, et al. A comparison of the Dorf-
tion-based screening program. Radiology 2013; 267:47–56. man-Berbaum-Metz and Obuchowski-Rockette methods for receiver
7. Lang K, Andersson I, Rosso A, et al. Performance of one-view breast operating characteristic (ROC) data. Stat Med 2005; 24:1579–1607.
tomosynthesis as a stand-alone breast cancer screening modality: 32. Hillis SL, Schartz KM, Berbaum KS. OR-DBM MRMC 2.5 User Guide.
results from the Malmo Breast Tomosynthesis Screening Trial, a popula- University of Iowa; 2014;4 14.
tion-based study. Eur Radiol 2016; 26:184–190. 33. Core Team R. R: A language and environment for statistical computing.
8. Destounis SV, Morgan R, Arieno A. Screening for dense breasts: digital Vienna, Austria: R Foundation for Statistical Computing, 2018.
breast tomosynthesis. AJR Am J Roentgenol 2015; 204:261–264. 34. McGowan LDA, Bullen JA, Obuchowski NA. Location bias in ROC stud-
9. Lowry KP, Coley RY, Miglioretti DL, et al. Screening performance of digi- ies. Statistics in Biopharmaceutical Research 2016; 8:258–267.
tal breast tomosynthesis vs digital mammography in community practice 35. Available from: https://www.qtultrasound.com/fda-grants-qt-ultrasound-
by patient age, screening round, and breast density. JAMA Network breakthrough-device-designation/.
Open 2020; 3. e2011792-e. 36. Samuelson FW, Abbey CK. The reproducibility of changes in diagnostic
10. Monticciolo DL, Newell MS, Moy L. Breast Cancer screening in women figures of merit across laboratory and clinical imaging reader studies.
at higher-than-average risk: recommendations from the ACR. Journal of Acad Radiol 2017; 24:1436–1446.
the American College of Radiology: JACR 2018; 15(3 Pt A):408–414. 37. Abdolell M, Tsuruda KM, Brown P, et al. Breast density scales: the metric
11. Heywang-Kobrunner SH, Hacker A, Sedlacek S. Magnetic resonance imag- matters. Br J Radiol 2017; 90:20170307.
ing: the evolution of breast imaging. Breast 2013; 22(2):S77–S82. Suppl. 38. Sickles EA. Findings at mammographic screening on only one standard
12. Chetlen A, Mack J, Chan T. Breast cancer screening controversies: who, projection: outcomes analysis. Radiology 1998; 208:471–475.
when, why, and how? Clin Imaging 2016; 40:279–282. 39. Ng KH, 20/20 Lau SVision. Mammographic breast density and its clinical
13. Hooley RJ, Greenberg KL, Stackhouse RM. Screening US in patients applications. Med Phys 2015; 42:7059–7077.
with mammographically dense breasts: initial experience with Connecti- 40. Boone JM, Nelson TR, Lindfors KK, et al. Dedicated breast CT: radiation
cut Public Act 09-41. Radiology 2012; 265:59–69. dose and image quality evaluation. Radiology 2001; 221:657–667.
14. Corsetti V, Ferrari A, Ghirardi M, et al. Role of ultrasonography in detect- 41. Wiskin JW, Borup DT, Iuanow E, et al. 3-D nonlinear acoustic inverse
ing mammographically occult breast carcinoma in women with dense scattering: algorithm and quantitative results. IEEE Trans Ultrason Fer-
breasts. Radiol Med 2006; 111:440–448. roelectr Freq Control 2017; 64:1161–1174.
15. Corsetti V, Houssami N, Ferrari A, et al. Breast screening with ultrasound 42. Ko ES, Morris EA. Abbreviated magnetic resonance imaging for breast
in women with mammography-negative dense breasts: evidence on cancer screening: concept, early results, and considerations. Korean J
incremental cancer detection and false positives, and associated cost. Radiol 2019; 20:533–541.
Eur J Cancer 2008; 44:539–544. 43. Sickles EA, Filly RA, Callen PW. Benign breast lesions: ultrasound detec-
16. Corsetti V, Houssami N, Ghirardi M, et al. Evidence of the effect of adjunct tion and diagnosis. Radiology 1984; 15:467–470.
ultrasound screening in women with mammography-negative dense breasts: 44. Iuanow E, Smith K, Obuchowski NA, et al. Accuracy of cyst versus solid
interval breast cancers at 1 year follow-up. Eur J Cancer 2011; 47:1021–1026. diagnosis in the breast using Quantitative Transmission (QT) ultrasound.
17. Berg WA, Blume JD, Cormack JB, et al. Combined screening with ultra- Acad Radiol 2017; 24:1148–1153.
sound and mammography vs mammography alone in women at elevated 45. Malik BH, Klock JC. Breast cyst fluid analysis correlations with speed of
risk of breast cancer. JAMA 2008; 299:2151–2163. sound using transmission ultrasound. Acad Radiol 2019; 26:76–85.
18. Berg WA, Blume JD, Cormack JB, et al. Training the ACRIN 6666 Investi- 46. Gur D, Bandos AI, Cohen CS, et al. The “Laboratory” Effect: compar-
gators and effects of feedback on breast ultrasound interpretive perfor- ing radiologists' performance and variability during prospective clini-
mance and agreement in BI-RADS ultrasound feature analysis. AJR Am cal and laboratory mammography interpretations. Radiology 2008;
J Roentgenol 2012; 199:224–235. 249:47–53.

S18