Received: 3 May 2020

| Revised: 1 February 2021

| Accepted: 2 March 2021

DOI: 10.1002/mrm.28779

GUIDELINES

A standard system phantom for magnetic resonance imaging

Karl F. Stupic1 | Maureen Ainslie2 | Michael A. Boss3 | Cecil Charles2 |

Andrew M. Dienstfrey1 | Jeffrey L. Evelhoch4 | Paul Finn5 | Zydrunas Gimbutas1 |
Jeffrey L. Gunter6 | Derek L. G. Hill7 | Clifford R. Jack6 | Edward F. Jackson8 |
Todor Karaulanov9 | Kathryn E. Keenan1 | Guoying Liu10 | Michele N. Martin1 |
Pottumarthi V. Prasad11 | Nikki S. Rentz1 | Chun Yuan12 | Stephen E. Russek1
1
Physical Measurement Laboratory, National Institute of Standards and Technology, Boulder, Colorado, USA
2
Department of Radiology, Duke University, Durham, North Carolina, USA
3
American College of Radiology, Philadelphia, Pennsylvania, USA
4
Merck Research Laboratories, West Point, Pennsylvania, USA
5
University of California, Los Angeles, California, USA
6
Department of Radiology, Mayo Clinic, Rochester, Minnesota, USA
7
Centre for Medical Image Computing, University College London, London, United Kingdom
8
Medical Physics, University of Wisconsin, Madison, Wisconsin, USA
9
CaliberMRI, Inc., Boulder, Colorado, USA
10
National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Bethesda, Maryland, USA
11
Radiology/CAI, NorthShore University HealthSystem, Evanston, Illinois, USA
12
Radiology, University of Washington, Seattle, Washington, USA

Correspondence
Karl F. Stupic, Physical Measurement
Purpose: A standard MRI system phantom has been designed and fabricated to as-
Laboratory, National Institute of Standards sess scanner performance, stability, comparability and assess the accuracy of quan-
and Technology, 325 Broadway, Boulder, titative relaxation time imaging. The phantom is unique in having traceability to
CO 80305, USA.
Email: [email protected] the International System of Units, a high level of precision, and monitoring by a
national metrology institute. Here, we describe the phantom design, construction,
imaging protocols, and measurement of geometric distortion, resolution, slice pro-
file, signal-­to-­noise ratio (SNR), proton-­spin relaxation times, image uniformity and
proton density.
Methods: The system phantom, designed by the International Society of Magnetic
Resonance in Medicine ad hoc committee on Standards for Quantitative MR, is a 200
mm spherical structure that contains a 57-­element fiducial array; two relaxation time
arrays; a proton density/SNR array; resolution and slice-­profile insets. Standard im-
aging protocols are presented, which provide rapid assessment of geometric distor-
tion, image uniformity, T1 and T2 mapping, image resolution, slice profile, and SNR.

Contribution of the National Institute of Standards and Technology; not subject to copyright in the United States.
Certain companies, commercial instruments and software are identified to specify the experimental study adequately. This does not imply endorsement by
NIST or that the companies, instruments and software are the best available for the purpose.

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original
work is properly cited.
© 2021 The Authors. Magnetic Resonance in Medicine published by Wiley Periodicals LLC on behalf of International Society for Magnetic Resonance in Medicine.

1194 | 
wileyonlinelibrary.com/journal/mrm Magn Reson Med. 2021;86:1194–1211.
 |

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
STUPIC et al.     1195

Results: Fiducial array analysis gives assessment of intrinsic geometric distortions,

which can vary considerably between scanners and correction techniques. This analy-
sis also measures scanner/coil image uniformity, spatial calibration accuracy, and
local volume distortion. An advanced resolution analysis gives both scanner and pro-
tocol contributions. SNR analysis gives both temporal and spatial contributions.
Conclusions: A standard system phantom is useful for characterization of scanner
performance, monitoring a scanner over time, and to compare different scanners. This
type of calibration structure is useful for quality assurance, benchmarking quantita-
tive MRI protocols, and to transition MRI from a qualitative imaging technique to a
precise metrology with documented accuracy and uncertainty.

KEYWORDS
MRI standards, phantom, quality assurance, quantitative MRI

1 | IN T RO D U C T ION 1. Radio frequency (RF) field, B1, non-­uniformity

2. Static main magnetic field, B0, non-­uniformity
With the rise of quantitative MRI to complement quali- 3. Geometric linearity
tative MRI, it is imperative to assess system stability and 4. Gradient amplitude
understand the comparability of measurements across 5. Slice position and profile
MRI scanners. One solution is to use a standard imaging 6. Image uniformity
phantom, an inanimate object used to characterize or cal- 7. Resolution (high-­contrast detectability)
ibrate an imaging system. A standard phantom would be 8. Signal-­to-­noise ratio (SNR) (low-­contrast detectability)
one that is commonly used and accepted by the imaging 9. Accuracy and precision of measurement of proton spin
community; has stable, precisely defined properties to relaxation times: T1 (longitudinal) and T2 (transverse)
allow monitoring of scanner performance and accuracy and proton density
of image-­based measurements; is fully documented; has 10. System constancy
long-­term monitoring and maintenance; and is traceably
connected to the International System of Measurements. Such a system phantom can be used to: (1) track scanner
The need for a standard MRI system phantom was voiced performance at a particular site over time, as well as compare
at a National Institute of Standards and Technology (NIST) performance with other scanners; (2) determine the accuracy
workshop entitled “Imaging as a Biomarker: Standards of certain quantitative measurements, such as T1 mapping,
for Change Measurements in Therapy,” that was held on and assist in the development of appropriate imaging proto-
September 14-­15, 2006.1 The workshop action items in- cols to obtain desired accuracy; (3) validate scanner and pro-
cluded: (1) Design phantoms that may better characterize tocols for participation in clinical trials; and (4) determine the
the time-­related physical performance of imaging systems, best phantom fabrication techniques for future phantoms, for
and the performance of specific functional and molecular-­ example, application-­specific phantoms.
based measurements; (2) Define the physical performance Previous successful efforts to create standard phantoms in-
of different imaging platforms required to measure change clude the American College of Radiology (ACR) MRI phan-
analysis; and (3) Develop and share open source tools to tom3 and the Alzheimer’s Disease Neuroimaging Initiative
analyze phantom or simulated data. (ADNI) MRI phantom.4 The ACR magnetic resonance ac-
The Ad Hoc Committee for Standards in Quantitative creditation phantom is a water-­ filled cylindrical phantom
Magnetic Resonance (SQMR) was formed, under the aus- with relaxation times and conductivities in the biological
pices of the International Society of Magnetic Resonance range of interest. The phantom contains a resolution inset,
in Medicine (ISMRM), to address these action items in the slice profile wedges, a grid to determine geometric distortion,
areas that pertained to MRI. The committee developed rec- and a contrast array. The phantom was primarily designed as
ommendations for a system phantom2 that could be used to a method to assist site accreditation performed by the ACR
determine the accuracy, stability, and comparability of MRI and as a mechanism to improve quality control by repeated
scanners. The desired measurements include: imaging on a weekly basis. The spherical ADNI phantom
 |   

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1196 STUPIC et al.

is used for measurements of SNR, contrast-­to-­noise ratio 1. SI traceable parameters including geometric dimensions
(CNR), and geometric distortion in scan protocols intended (mm), nuclear magnetic resonance (NMR) parameters
to monitor changes in the brain morphology over time. From such as proton spin relaxation times (s), and material
2004 to 2009, the ADNI study acquired a phantom image compositions that dictate contrast properties.
with each participant study.4 The key results from this phan- 2. Dimensionally stable construction using a rigid precision-­
tom plus human-­participant study were: (1) the drift in scan- joined framework and materials that are both low thermal-­
ner gradient calibration over months to years was less than expansion and low water-­uptake.
the magnitude of step-­wise changes in the gradient calibra- 3. All components including design, 3D models, imaging
tion introduced by field service calibration; (2) using phan- protocols, detailed description of construction materials,
tom derived linear scaling corrections was not better than and imaging analysis routines are in the public domain.
affine registration at correcting within-­subject longitudinal-­ 4. Long-­term maintenance/monitoring of stability and accu-
imaging-­series gradient calibration changes or gradient drift. racy by a national metrology institute.
For ADNI-­2 (2010-­2017), phantom images were acquired for
site certification and after system upgrades. These images
were used to ensure that off-­line gradient non-­linearity correc- 2 | M ETHODS
tions based on scanner model could be performed correctly.5
In ADNI-­3 (2018-­present), all scanners in the study have full This section gives an overview of the design and construc-
three-­dimensional (3D) on-­ board corrections for gradient tion of prototype MRI system phantoms and the process to
non-­linearity, and sites are requested to use ADNI phantoms provide SI-­traceable calibration of phantom properties. Two
for certification if possible. The standard system phantom prototype system phantoms were built for this study. A more
described here builds off the ACR and ADNI phantoms, ad- comprehensive description of the construction, solutions,
dressing the need for a more general and precise calibration and materials can be found in the Supporting Information.
object that has properties traceable to primary standards con- The design and construction specifications are meant as a
nected to the International System of Units (SI).2 common reference for commercial versions, not as specific
This paper describes the design, construction, accuracy requirements. The prototype phantoms are modular so that
requirements, and measurement protocols of prototype sys- all components may be disassembled and reassembled in a
tem phantoms designed by the ISMRM SQMR ad hoc com- variety of configurations for testing alternative designs.
mittee. The system phantom is a precise imaging artifact for
quantitatively measuring image distortions and validating
many types of quantitative mapping. The system phantom 2.1 | System phantom design
has subsequently been commercialized with over 100 units
sold.6 The commercial system phantom has some minor The prototype MRI system phantom, shown in Figure 1, con-
changes in its design to accommodate more efficient man- sists of a water-­filled spherical polycarbonate shell with a 200
ufacture, reduce cost, and incorporate some improvements mm inner diameter (ID). The spherical diameter was chosen
that have been requested by customers. These changes are to mimic a human head and was designed to fit in most head
documented in Supporting Information Sec. 9. Here, we note coil assemblies. A spherical design also allows for easy ro-
that the spin-­relaxation times and geometric tolerances of the tation of the phantom. Inside the spherical shell is a frame-
commercial phantoms vary slightly from the prototypes and work consisting of five 8.0 mm-­thick polyphenylene sulfide
calibration data for the particular phantom, as determined (PPS) plates rigidly connected with PPS rods and kinematic
from the serial number, should be used. The 3D mechani- mounts. PPS was chosen since it is a high-­performance plas-
cal design files, analysis software, analysis instructions, and tic with low water absorption, low thermal expansion, and
imaging data can be found online, as described in the Data good machinability. The PPS plates are annealed for stress
Availability Statement. relief and then machined to an inspected flatness of less than
A single phantom cannot provide all the calibration mea- 0.04 mm. The plates support 57 fiducial spheres, a 14 ele-
surements required for assessing the accuracy of MRI scan- ment NiCl2 array, a 14 element MnCl2 array, a 14 element
ners and quantitative imaging protocols. However, this system proton density array, two resolution insets, and wedges for
phantom can address many of the important measurement slice profiles. The plates, as shown in Figure 1, contain the
and quality control issues. A universally used phantom is es- following structures: Plate 1, 5 fiducial spheres on bottom;
sential to allow sites to measure system stability and readily Plate 2, 13 fiducial spheres on top and slice profile wedges
compare scanner performance with other scanners. There are on bottom; Plate 3, 21 fiducial spheres on top, proton density
several elements of the MRI system phantom described here array on bottom; Plate 4, coarse resolution inset in the plate,
that make it unique: 13 fiducial spheres on top, MnCl2 array on bottom; Plate 5,
 |

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
STUPIC et al.     1197

F I G U R E 1 A, Schematic of system phantom showing fiducial array (gray/brown), NiCl2 array (green), MnCl2 array (red), proton density
array (yellow), and resolution and slice profile insets. Plate 5 contains MR visible engravings with the serial number and phantom type. All plates
have orientational marks and plate 3 contains a grid for manual estimation of geometric distortion. B, Top view of phantom showing eye decals.
C, Sagittal slice in the Y-­Z plane showing the MR parameter arrays, fiducial elements, phantom origin, and the resolution inset.

5 fiducial spheres on top, serial numbers and phantom ID in main MRI vendors, are given in the Supporting Information,
the plate, and NiCl2 array and fine resolution inset on bottom. Tables S4 through S10.
The fine resolution inset can also be mounted on plates 3 or 4. The inner frame of the prototype phantoms was designed
The phantom coordinate system was chosen to match typ- to be very rigid and allow the fiducial-­sphere centers to be
ical scanner/patient coordinates, with +Z along the scanner located within a ±0.1 mm accuracy. Details of the plate locat-
bore axis (superior), +X toward the left, and +Y down (pos- ing mechanism are shown in Supporting Information Figure
terior), when mounting the phantom in the default orienta- S1. A hemispherical machined surface on the post mates
tion. A set of eye decals is included on the phantom shell to with a set of inclined plates to precisely locate the plates and
allow easy recognition and placement of the phantom within eliminate torsional bending. The precision construction of
the scanner, as seen in Figure 1B. The phantom is placed in the prototypes comes at an increased cost due to the need for
the same fashion as a patient laying supine on the imaging computer-­controlled machining. It was deemed necessary for
bed. The origin of the phantom is defined as the center of the the prototypes to have such precision to allow the study of
central fiducial sphere as shown in Figure 1C. The phantom scanner geometric distortion.
is designed so that the plates are in the XZ (coronal) plane
with left (L), right (R), superior (S), and inferior (I) marked
on each plate, along with the plate identifier. Phantom identi- 2.2 | Fiducial array
fication information, along with the serial number, are etched
into plate 5, which are easily readable in the prescribed 3D The system phantom contains a 3D fiducial array that can
scan (Figure 1A). In the default orientation, the MR param- be used to assess geometric distortion, image uniformity
eter arrays (NiCl2, MnCl2, proton density arrays) are imaged and B1 homogeneity. The fiducial array was modeled on the
with a coronal scan. Recommended protocols for measur- ADNI phantom,4 which has successfully been used to correct
ing the arrays and insets on clinical MRI systems, for the geometric distortions and image nonuniformity. The fiducial
 |   

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1198 STUPIC et al.

array spheres all have the same fill solutions with the same spheres are numbered 1 through 57, starting at plate 1, in-
T1 and T2 properties (T1 = 407 ms ± 6 ms, T2 = 347 ms ± 6 creasing from right to left, then inferior to superior. For data
ms at 20.0°C, 3.0 T), which means they can be used to assess presentation purposes, the 57 fiducial array elements are di-
image homogeneity and calculate B1 variation, both transmit, vided into seven groups: 27 internal spheres on central 3 ×
B1+, and receive, B1−, fields. 3 × 3 grid, 5 spheres at each of the six outward faces of the
The fiducial array consists of a set of 57 precision-­ phantom. Each set of fiducials is represented by a different
machined polyvinyl chloride (PVC) 10.0 mm ± 0.1 mm color in the plots in Figure 2.
inner-­diameter spheres on a 3D cubic lattice, with a lattice The spheres are filled with a CuSO4 solution made by
spacing of 40 mm ± 0.1 mm. Figure 2 shows a coronal MRI mixing 0.802 g CuSO4* 5H2O per liter of deionized water
with a subset of the fiducial spheres along with 10 mm di- (Supporting Information Sec. 4). The Cu+2 ion concentra-
ameter circular regions of interest (ROIs) at their prescribed tion was measured, using an inductively coupled plasma-­
positions, before automated sphere location. Figure 2 shows optical emission spectroscopy and a NIST traceable Cu
standard output from the Python-­ based analysis package, standard (such as NIST standard reference material 3114) to
after sphere location, plotting image uniformity and geomet- be 0.18901 ± 0.00055 mg/g or 2.969 ± 0.008 mM. A small
ric distortion along the x, y, z axes, respectively. The fiducial amount of blue dye (erioglaucine disodium salt, 0.12 µg/ml)

F I G U R E 2 A, Coronal 3D gradient echo image of the fiducial array along with 10.0 mm circular ROIs located on prescribed 40 mm grid,
which allows for visual identification of geometric distortion. S, I, L, R refer to superior, inferior, left, right directions, respectively. The scan is
at 3T using a body transmit coil and a head receive coil with 0.97 mm isotropic voxels, TR = 6.3 ms, TE = 1.89 ms. The nonuniform gradient
corrections are turned off. B, Normalized integrated intensity of all 57 spheres obtained after the sphere location procedure. C,D,E, Geometric
distortion along the x, y, z directions, respectively. Geometric distortion, 𝛿 R�⃗ = R�⃗ a − R�⃗ p, is defined as the distance between the apparent position
�⃗ a and the prescribed position R
R �⃗ p, of the sphere centers after an affine transformation has accounted for rotation, translation, and overall scale
correction.
 |

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
STUPIC et al.     1199

was added to assist in the visual recognition of the fiducial el- T2 values (at 20°C, 1.5 T and 3.0 T) for the NiCl2, MnCl2, and
ements. The fiducial spheres are machined in two parts from proton density arrays are listed in the Supporting Information
PVC stock, glued together and attached to the plates using an in Tables S1-­S3, respectively.
M8-­1.25 threaded stub, as shown in Supporting Information The NiCl2 and MnCl2 arrays were chosen to have a wide
Figure S1C. PVC was chosen because it is inexpensive, can range of values to mimic both endogenous tissues and tis-
be easily glued, and has a low permeability to water. The sue containing standard commercial contrast agents. T1 and
spheres are sealed with a tapered polyvinylidene difluoride T2 relaxation times for the arrays, along with typical values
(PVDF) plug. The fiducial solution was chosen to have short for tissue,9 are shown in Figure 3. The MnCl2-­doped array
T1 times to give, for most scans in the protocol, high signal spans a range of relaxation time values much closer to that
and contrast with respect to the surrounding fill composed of of tissue; however, it has considerably more temperature and
long-­T1 deionized water. High contrast is beneficial for auto- field dependence than the NiCl2-­doped array (Supporting
mated location of the fiducial sphere positions. Information Sec. 7). T1 and T2 can be measured on both the
NiCl2-­doped and MnCl2-­doped arrays; although historically,
the NiCl2 array was designed to cover a suitable range of T1
2.3 | MR parameter arrays (NiCl2, values, while the MnCl2 array was chosen to cover a suit-
MnCl2, and proton density arrays) able range of T2 values. Also shown in Figure 3 are model
fits to the data, assuming that the relaxation rates are linearly
The MR parameter array spheres are 20 mm outer diameter proportional to the paramagnetic salt concentration. The de-
(OD), 15 mm internal diameter (ID) polypropylene spheres viation of the measured data from the model and the rela-
that are filled, heat sealed and then, epoxy sealed to an M8-­ tion to the measurement uncertainties are discussed in the
1.25 threaded mounting stud (Supporting Information Figure Supporting Information Sec 2.
S1D), similar to those used for the fiducial spheres. Since the √ The NiCl2 doping concentrations were chosen to give a
spheres are epoxied onto a hemispherical indent, as opposed 2 progression of T1 values from approximately 20 ms to
to being integrally machined with the threaded stud, there 2000 ms at a B0 field of 1.5 T. NiCl2 was chosen because
is more positional uncertainty in the MR parameter arrays the relaxivity of paramagnetic spin-­1 Ni2+ ions on water
than in the fiducial array. Polypropylene spheres were cho- proton spins has been extensively studied.10,11 T1 values are
sen since they are inexpensive, commercially available, and relatively insensitive to the magnetic field strength12 and to
have one of the lowest water/air permeation rates among all temperature over the range of application for this phantom.10
of the plastics tested. However, the OD, ID, and sphericity of However, the paramagnetic relaxation enhancement due to
the polypropylene spheres are not well controlled: see Figure
S2D inset. The polypropylene spheres are made by thermally
welding together two molded hemispheres, which results in a
thicker wall in the weld plane.
The T1 and T2 values were modified by doping high-­
purity water with NiCl2 and MnCl2 in their respective arrays
(Supporting Information Sec. 4). The proton density values
were modified by making solutions of 5 % to 100 % high-­
purity water with the balance D2O. Proton density is defined
as the concentration of MR-­visible protons in a sample/tis-
sue, relative to that in the same volume of water at the same
temperature.7 All solutions use American Chemical Society
(ACS) reagent grade,8 or higher quality, water. A small quan-
tity of NiCl2 (0.26 mg Ni2+/ml) was added to the proton
density solutions to reduce the relaxation times. Green dye
(28 µg/mL erioglaucine disodium salt, 28 µg/mL tartrazine)
was added to the NiCl2-­doped array solutions, red dye (49.6
µg/mL allure red AC) was added to the MnCl2-­doped array
F I G U R E 3 Overview of T1 and T2 properties of MR parameter
solutions, and yellow dye (53 µg/mL tartrazine) was added to
arrays at 20.0°C. 1.5 T data are plotted as circles, 3.0 T data as squares.
the proton density solutions to make the arrays easily iden- Also plotted for reference, are values for various human tissues (in
tifiable by the user. Each array consists of 14 spheres, 10 orange) at 1.5 T and 3.0 T at 37°C, taken from the online IT’IS data
equally spaced on a 50 mm radius, and 4 spheres internal base and values for high purity water at 20°C (in green) The solid lines
to the outside 10 spheres on a 40 mm grid. The sphere posi- are model fits assuming that the relaxivities are linearly dependent on
tions, NiCl2, MnCl2, D2O concentrations, and measured T1, the paramagnetic salt concentrations.
 |   

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1200 STUPIC et al.

Ni2+ has a complex behavior in the region of application with directly into plate 4 with hole diameters of 0.6 mm, 0.7 mm,
non-­monatonic variation of the relaxivities with field and 0.8 mm, 0.9 mm, and 1.0 mm ± 0.1 mm diameters. The fine
temperature.12 T1 and T2 for the higher concentration NiCl2 resolution inset is contained within a rectangular prism filled
solutions, Ni-­5 to Ni-­14, have a minimum near 22°C at 3 T with CuSO4 solution to give high contrast and has hole di-
(Supporting Information Sec 7), close to typical MRI bore ameters of 0.4 mm, 0.5 mm, 0.6 mm, 0.7 mm, and 0.8 mm
temperatures. The presence of a minimum eliminates the ± 0.1 mm diameters. Each hole array is duplicated at a 10°
large linear variation of T1 and T2 with temperature, giving angle, sharing one common hole, to assess resolution along
a maximum variation of 4 % over the range of expected bore non-­principal axes. In total, there are 155 holes in each of the
temperatures (16°C to 26°C). resolution insets (5 hole sizes, 31 per size). The advantage
The MnCl2 concentrations are varied from 0.013 mM to of this resolution inset is that it is easy to interpret visually,
1.704 mM and, as with the √ NiCl2 array, the concentrations compact, easy to manufacture, and there is considerable com-
were spaced by factors of 2. The variation of T1 and T2 is munity experience with it.
a linear function of temperature over the expected range of The slice profile inset consists of two 10° angle wedges
bore temperatures (16°C to 26°C), with a variation at 3 T of contained within a rectangular prism filled with CuSO4
2.7 %/°C and 1.6 %/°C for T1 and T2, respectively. solution. The wedges are oriented so that one has a positive
and one has a negative inclination. The presence of a pair
of wedges allows for the slice profile to be corrected for the
2.4 | Resolution and slice profile insets misalignment of the slice plane relative to the base plane of
the wedges.
The resolution insets are modeled on the ACR guidance,13
and the slice profile wedges are designed in accordance with
National Electrical Manufacturers Association (NEMA) MS 2.5 | Safety
5-­2018 standard.14 Photographs and MR images of resolution
and slice profile insets are shown in Figure 4. For most of the imaging studies, the shell was filled with
The phantom contains two resolution insets consisting 4 L of deionized water. This was done for two reasons: (1)
of 4 × 4 arrays of holes. A coarse resolution inset is drilled safety, if the phantom spilled, the outer fill has no toxicity or

FIGURE 4 A,B, Photo and MRI of resolution inset. C,D, Photo and MRI of slice profile inset.
 |

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
STUPIC et al.     1201

disposal issues; and (2) practicality, the phantom could be Temperature measurement and control are a major
easily refilled at any site. The main toxicity issue is due to source of uncertainty both in the primary calibrations and in
the NiCl2 solutions. A total of 52 mg Ni2+ is included in the phantom-­based MRI scanner assessment. Phantom compo-
phantom. The NiCl2 is contained in the thick-­walled poly- nents can have several percent change in relaxation time per
propylene contrast spheres that are double sealed with both a °C, Supporting Information Sec. 9. For primary calibration in
weld and epoxy. These spheres are very robust and can take the NMR metrology system, sample capillaries are immersed
large impacts without breakage or leakage. in a fluorinated heat transfer solution within a 5 mm diameter
NMR tube. The fluorinated solution also serves as a suscep-
tibility matching solution. A fiber optic thermometer is posi-
2.6 | SI traceability tioned 15 mm away from the sample and is calibrated in the
same NMR tube in a water bath against two NIST-­calibrated
A calibration object requires its properties to be precisely platinum resistance thermometers.
measured and traceable to a common set of definitions and Geometric accuracy is determined at manufacture time
units. The measured values must include a well-­defined uncer- using computer-­ controlled machining techniques. Post-­
tainty. A protocol for traceable spin relaxation measurements assembly geometric accuracy can be checked with computed
for MRI phantoms can be found in NIST SP250-­9715 and tomography techniques using SI traceable geometric calibra-
measurement detail can be found in Supporting Information tions. Here, the geometric distortion is reported relative to the
Sec. 6. Spin relaxation time measurements are SI traceable 3D model values, with uncertainty given by the manufactur-
via a metrology NMR system with a calibrated time base. ing tolerances.
The NMR time base, a 10 MHz oven-­controlled crystal oscil- Composition of the MR parameter arrays is verified using
lator, is calibrated against the NIST hydrogen maser clock. inductively coupled plasma-­mass or optical spectroscopy
Time base errors, which include jitter of transmit and receive techniques calibrated with traceable NIST reference stan-
events, give minor contributions to the overall uncertainty in dards. A reference library of all solutions, flame sealed in
relaxation time measurements. capillaries, is kept to establish stability over time. A test of
More important factors that determine the uncertainty the stability of proton relaxation times of an archived sample
in relaxation time measurements are RF transmit field B1+ of Ni-­12 solution at 3 T and 20.0°C, for example, gave T1 =
calibration, B1+ inhomogeneity, B0 inhomogeneity, and mea- 44.53 ms ± 0.01 ms, T2 = 31.86 ms ± 0.03 ms in May 2015
surement temperature. B1+ is calibrated using a nutation ex- and T1 = 44.67 ms ± 0.003 ms, T2 = 31.97 ms ± 0.026 ms in
periment on the sample to be calibrated, where the spins June 2019, a difference of 0.3 % for T1 and 0.3 % for T2. Here,
are progressively tipped, by varying the RF pulse duration, the reported errors are the SD of three consecutive measure-
at increasing angles through at least 2 cycles (720°). Free ments at each time point and is a measure of system stability
induction decays are recorded, Fourier transformed to ob- and is not the uncertainty in the values, which must incor-
tain a spectral peak, integrated and plotted against RF pulse porate all systematic errors as described in NIST SP250-­97.
duration. The data, which approximate a damped sinusoid, Uncertainties will depend on the material under test, the pre-
are fit to extract the pulse duration required for the desired cise pulse sequence used, the geometry of the sample, and on
tip angle and to extract an RF field inhomogeneity in the the values of the relaxation times. The uncertainty for proton
system. The RF field inhomogeneity is minimized during spin relaxation time calibrations, defined as the range within
calibration by precisely centering a 10 mm long test sample which there is greater than 96 % probability of the true value,
within a 14 mm long RF homogenous zone. This field in- are given in the NIST calibration certificates for each mate-
homogeneity and uncertainty in the B1+ calibration are fed rial and are on the order of 1.5 %.
into Monte Carlo Bloch simulations, along with all other
uncertainties, to calculate the uncertainties in the measured
relaxation times. 3 | RESULTS
B0 inhomogeneity in the NMR metrology system is min-
imized by performing an automated shimming procedure on 3.1 | Geometric distortion, image
the sample to be calibrated. The inhomogeneous line width uniformity, and B1 homogeneity
is required to be no more than 10 Hz larger than the homog-
enous line width. Monte Carlo Bloch simulations15 indicate Geometric distortion is assessed in a manner similar to that
that this level of B0 inhomogeneity gives a negligible con- described in Ref. 4. A 3D gradient echo image, with short
tribution to the overall uncertainty in relaxation times with echo time (TE), of the whole phantom is obtained with iso-
the pulse sequences used (inversion recovery sequence with tropic voxels with typical voxel dimension of 1.0 mm. This
composite 180° pulses for T1 and Carr-­Purcell-­Meiboom-­Gill sequence can be used to assess the intrinsic geometric accu-
(CPMG) sequence with 2.0 ms refocusing times for T2). racy of the scanner, whereas sequences with longer TEs will
 |   

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1202 STUPIC et al.

F I G U R E 5 A, Coronal image of the NiCl2 array at 3 T ~20°C using an IR protocol. B, Average signal in ROIs 1, 5, 9, and 13 in the NiCl2
array, plotted as a function of TI and fits to the inversion recovery model. C, Automated analysis of T1 for each of the 14 NiCl2 array elements,
with the deviation from the NMR reference measurements. The analysis was done both by (1) fitting the average of the ROI signal to the model
(blue squares) and (2) fitting each voxel independently to the model, making a T1 map, and then averaging (red squares). The deviations are shown
in blue and red squares, respectively. The error bars on the deviations are the standard errors in the nonlinear least squares fits and represent the
quality of the model and the fit. D, The inversion efficiency, δ determined from the fits. The line at δ = 1 represents perfect inversion, and the
error bars are the standard errors in the parameter determined from the nonlinear least squares fits. Unobserved error bars are contained within the
symbol.

be more sensitive to distortions due to the patient/phantom. isotropic scale factor), which minimizes the distances be-
The apparent locations of the 10.0 mm fiducial spheres are tween the prescribed and apparent locations is determined
determined by cross-­correlation maximums between a fidu- by a generalized Procrustes algorithm. The resulting rotation
cial sphere mask and the image (discussed in the Supporting and translation define the map from phantom to scanner co-
Information). The sphere center can be determined to <0.1 ordinates. The scale factor provides a measure of the overall
mm, which is comparable to the accuracy in the construc- dimensional accuracy of the scanner. We define geometric
tion. Determination of the center of the fiducial spheres is distortion as the difference between the apparent position
sensitive to defects, such as small air bubbles in the solu- �⃗ a and the prescribed position R
R �⃗ p of the sphere centers after
tion. These defects can be detected manually or numerically, the similarity transformation has been applied to the latter,
and the defective sphere data discarded. After locating the 𝛿R �⃗ p. An example geometric distortion analysis for
�⃗ a − R
�⃗ = R
apparent sphere centers, a similarity transformation (an af- an image without corrections for gradient non-­uniformity,
fine transformation consisting of a translation, rotation, and is shown in Figure 2. A large (~2 mm) x-­component of
 |

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
STUPIC et al.     1203

geometric distortion is seen on the right face, with a negative analysis. Figure 5C shows the results of the T1 analysis along
sign, in Figure 2 since the apparent position is farther along with the deviation relative to the NMR reference values given
the negative x-­axis than the prescribed position. Similarly, in Table S1. The deviation is defined as 100(T1m − T1r)/T1r,
there is a large x-­component of geometric distortion on the where T1m, T1r are the measured and reference values, respec-
left face, with a positive sign, since the apparent position tively. The large deviation for sphere Ni-­14, which has the
is farther along the positive x-­axis. Examples of geometric shortest T1, results from having only three data points where
distortion with non-­uniform gradient corrections applied are the signal is changing before full recovery. This is reflected
given in the Supporting Information and show the need for in the large error bar, which is the standard error in the non-­
the phantom geometric accuracy to be < 0.1 mm. Local volu- linear least squares fit. The analysis was done both by (1) fit-
metric distortion can be assessed by comparing cross correla- ting the average of the ROI signal to the model and (2) fitting
tion volumes. each voxel independently to the model, making a T1 map, and
We assess the uniformity of image intensity as part of this then averaging. Both methods agree within the standard er-
same analyses by calculating the integrated intensity of the 57 rors. A series of T1 measurements made on 3 T scanners over
fiducial spheres. Figure 2 shows image uniformity obtained the course of 7 years is shown in the Figure S13 to illustrate
from a head coil, which shows signal loss in the inferior or phantom stability.
chin region due to low receive coil sensitivity. The image uni- More information is obtained from these fits than simply
formity analysis can also be done as part of other protocols, the relaxation time. The inversion efficiency, δ, is an im-
such as the of T1, T2 protocols. portant indicator of B1 and flip angle uniformity. As seen in
Finally, B1+ mapping can be done with the system phan- Figure 5D, the inversion efficiency is close to 0.85, whereas
tom fiducial array using flip angle mapping techniques.16-­18 the ideal value is 1.0. Typical values obtained from NMR
The fiducial spheres, as well as the surrounding background data on the same solutions, with a more homogenous RF field
fill liquid (DI water), provide regions with identical proper- geometry and using composite inversion pulses, are between
ties to allow identification of RF non-­uniformities. To assess 0.97 and 1.0. A Monte Carlo study of Bloch simulations in-
the effect of fill conductivity and dielectric resonances, the dicate that there is a correlation between the deviation of the
fill solution can readily be swapped out with a conducting inversion efficiency from its ideal value and the error in the
salt solution or a low dielectric constant fluorocarbon fluid. T1 measurements.15
We do not perform this analysis here. IR sequences, while fairly accurate in measuring T1, re-
quire long acquisition times. Variable flip angle (VFA) se-
quences, with shorter TRs and smaller flip angles, are
3.2 | T1 measurement considerably faster with less RF deposition. VFA protocols
typically use 3D gradient echo sequences (see Supporting
Figure 5 shows automated T1 analysis of the NiCl2 array Information) and vary the flip angle 𝛼. Ideally, the signal is
using the Python-­based analysis package. Figure 5A shows − TR
T1
given by S (𝛼) = S90 sin (𝛼) 1−e
, where S90 is the
an image of the NiCl2 array using the inversion-­recovery − TR
1 − e T1 cos(𝛼)
(IR) sequence protocol described in the Table S8 with recom- maximum signal given 𝛼 = 90 ◦ and long TR.20,21 Figure 6A
mended 10.0 mm circular ROIs centered on the NiCl2 array shows a coronal image of the NiCl2 array using the VFA
spheres. The suggested T1-­IR protocol requires 10 inversion protocol described in the Supporting Information, with
times, TI, ranging from 50 ms to 3000 ms and TR = 4500 TR = 5.37 ms and TE = 1.49 ms. Figure 6C shows a series
ms. Note, the actual inversion recovery protocol may vary of sagittal images for a series of flip angles varying from 2°
slightly among the different vendors due to details of the to 30°. The signal in each ROI has a maximum at the Ernst
scanner design and programmable control. Figure 5B shows ( ) − TR
angle, cos 𝛼 E = e T1 , which for ROI 5 is near 10° and
a typical fit to the data using a standard IR model,19-­21 appro-
ROI 2 is near 5°. Figure 6B shows the mean signal for ROIs
priate when TR >> T1, and the inversion and detection flip
2, 5, 8, 11 along with fits to the model listed above and the
angles are precisely 180° and 90°. The average signal in each
| TI | residuals. As seen by the non-­Gaussian distribution of the
ROI is given by S (TI) = A |1 − (1 + 𝛿) e − T1 |, where A is the residuals, there are systematic deviations from the simple
| |
| |
signal amplitude when the magnetization is allowed to fully model listed above due, in large part, to the inability to
relax back to its equilibrium state, and 𝛿 is the inversion effi- achieve the prescribed flip angle at each location in the
ciency,19,21,22 the signal amplitude immediately following in- phantom. Figure 6D shows the measured T1 values and de-
version normalized to A. Here, only magnitude data are used viations from the NMR reference values. The error bars in
since the phase data are often not available and can be hard Figure 6D are the standard error in the fits, which are large
to interpret. This is in contrast with the NMR IR reference for the longer T1 ROIs. The large standard errors on the fits,
protocol, which uses the real part of the complex signal for along with a systematic variation in the residuals, indicate
 |   

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1204 STUPIC et al.

F I G U R E 6 A, Coronal image of the NiCl2 array at 3 T ~20°C using a 3D gradient echo VFA sequence, B, Average signal and fit to VFA
model for ROIs 2, 5, 8, 11. C, Sagittal images showing sphere 2 and sphere 5 for different flip angles. ROI 5 has a maximum signal near 10°, while
ROI 2 has a maximum signal near 5°. The inset shows slice to slice variations that affect T1 values. D, Measured T1 and deviation from the NMR
reference values for each of the 14 ROIs.

that the model, which assumes an accurately prescribed NMR reference values given in Table S2. The error bars in
flip angle that is the same for all regions in the phantom, is Figure 7C are the standard errors derived from the non-­linear
limiting measurement accuracy. More advanced models least squares fitting procedure. For the ROIs with short T2
and protocols exist to improve on VFA T1 measurements23; times, there are not many points used in the fit, and the stand-
however, here we are focusing on using the system phan- ard error is large.
tom and simple protocols to identify and characterize scan-
ner non-­idealities.
3.4 | Proton density and SNR measurement

3.3 | T2 measurement A coronal spin echo image of the PD array is shown in

Figure 8A with the PD ROIs shown in yellow. Substantial
Figure 7 shows T2 measurements of the MnCl2 array using background nonuniformity exists; the signal changes by
the Python-­ based analysis package. Figure 7A shows an a factor of ~6 going from the periphery to the center along
image of the MnCl2 array using the spin echo (SE) sequence the superior/inferior direction. Figure 8B shows the average
listed in the Supporting Information with TE varying from signal in each PD ROI, and the measured MR proton den-
10 ms to 320 ms by 10 ms and TR = 5000 ms. Figure 7B sity plotted versus prescribed proton density. The MR pro-
shows a typical fit to the data using a simple exponential ton density is calculated by normalizing the average signal to
model: S (TE) = S0 e − TE∕T2, where S (TE) is average signal in the local background, as determined by averaging the signal
each ROI and TE is the spin TE. Figure 7C shows the T2 from four points taken 3 mm outside the spheres, and then
obtained from the fits along with the deviation relative to the normalizing to the 100 % water sphere. The average signal
 |

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
STUPIC et al.     1205

F I G U R E 7 A, Coronal image of the MnCl2 array at 3 T ~20°C using a spin-­echo sequence with TR = 5000 ms. Typical fit to a spin echo
protocol (B) and automated analysis of T2 (C) for each of the 14 array elements, with the deviation from the NMR reference measurements. The
TE = 10 ms image was not used due to anomalously low signal, and points below the noise floor, the orange line in B, were not included in the fits.

in the PD ROIs shows a clear nonlinearity due to signal loss temporal drift, which gives rise to the low spatial frequency
towards the periphery of the phantom. The normalized sig- structure in Figure 8C.
nal shows the expected linearity of the signal with respect to
proton density. The maximum error in the measured proton
density is 8.3 %, and the SD of the errors is 2.8 %. 3.5 | Resolution
In addition to measuring proton density, this array is used
to measure SNR using methods similar to those described Resolution is determined either using the manual method sug-
in Ref. 24. The noise is measured by taking the difference gested by ACR13 or an automated computer algorithm that
of two identical scans, ΔS = S2 − S1, taken sequentially compares the MR image to synthesized images with differ-
(Figure 8C), as discussed in Method 1 in Ref. 24. The SNR ent resolution distortions. Figure 9A shows an image of the
is calculated
( for each
) PD ROI by taking the average signal √ resolution inset taken with a voxel size of 0.35 mm and a slice
Sav = 21 S2av + S1av and dividing by the noise N = SD∕ 2 , thickness of 4.0 mm. As per ACR guidelines, visual inspec-
where SD is the SD of the difference signal ΔS within the tion of the image yields a resolution of 0.5 mm. Figure 9B
ROIs. The SNR is plotted versus proton density in Figure 8D. shows a synthetic image generated by analytically comput-
Also shown is the average signal in each ROI normalized to ing the Fourier transform of the resolution-­inset disk array,
the SD within each ROI. This is a measure of the ratio of the calculating a finite k-­space image, and converting into a real-­
average signal to the nonuniformity within each region, only space image. Comparison of the synthetic image to the meas-
part of which comes from noise. ured image shows additional blurring beyond that expected
SNR1 ROI, shown in Figure 8 in blue, is meant to more from the point spread function due to finite k-­space sampling.
closely mimic the geometry suggested in Ref. 24. However, Applying an additional Gaussian point spread function can
the larger ROIs are more susceptible to scanner and phantom improve the match between the synthetic and real image
 |   

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1206 STUPIC et al.

F I G U R E 8 A, Coronal image of proton density array using a spin-­echo sequence with TR = 5000 ms, TE = 10 ms. A background SNR ROI
is shown in blue, in addition to the 14 yellow proton density ROIs. Line scans across the center background fill are shown in yellow and orange
at the bottom to show the image inhomogeneity. B, Average intensity of PD ROIs and measured PD as a function of prescribed proton density.
C, Difference of two identical images and D, plot of the SNR as a function of proton density. Also shown in D, is the ratio of the signal to the SD
within each ROI.

(Figure 9), with the difference between the images minimized differentiating the data with respect to image position, and
using a point spread function with a width 2σ = 0.075 mm, as then multiplying the image position by tan(θ) to determine
seen in Figure 9D. The resolution is then reported as a finite (( 𝛼 = 10) + 𝜃 is the ◦corrected
the slice profile. ◦
( angle,
)) where
k-­space sample limited resolution of 0.35 mm and a machine-­ 𝜃 = 1∕2sin −1
w2 − w1 sin (2 ∗ 10 ) ∕ w2 + w1 meas-
specific Gaussian broadening of 0.075 mm. ures the angle of the image plane with respect to the base of
the wedges (see Figure 10C inset), and w1 , w2 are the pro-
jected thicknesses of the two wedges. The slice thickness, tsl ,
3.6 | Slice profile measurement is then determined by measuring the full width at half maxi-
mum of the corrected slice profile. The suggested prescribed
Slice profile measurements, shown in Figure 10, are done in slice thicknesses are 3 mm and 5 mm, with the maximum
accordance with the NEMA MS-­5-­2018 standard14: the edge being set by the wedge height of 10 mm.
response function is calculated from the imaging data by The slice thickness may also be measured by fitting the edge
placing a 4 mm x 50 mm rectangular ROI over each wedge, response function with a piecewise linear function, extracting
 |

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
STUPIC et al.     1207

of MR fingerprinting,25 assessing changes occurring during

scanner upgrades,26 and assessing variation in multi-­site T1
measurements.27 Many other uses are ongoing, including
monitoring of scanners for quality assurance and homog-
enizing scan protocols across multi-­vendor scanners. The
phantom was designed to be used in several modes, starting
from simple visual inspection for identification of geomet-
ric distortion and resolution, to semi-­automated analysis to
determine relaxation times and SNR, and finally for custom
analysis to validate advanced quantitative protocols. While
application-­specific phantoms may provide better structures
for particular measurements, for example, the ADNI phan-
tom for geometric distortion or a uniform agar phantom for
image homogeneity, the system phantom is meant for char-
acterization of a large number of scanner parameters and to
have sufficient accuracy, generality, and complexity to as-
sess the accuracy of many types of advanced hardware and
protocols. The standard protocols provided with the system
phantom are not intended to be the best protocols for accurate
measurement of geometric distortion, proton spin relaxation
times, resolution, and SNR. Rather, they are meant as a com-
mon method to test the ideality of the scanner and to provide
a comparison for other quantitative protocols that may be
faster or more accurate.
Here we have shown how the phantom can be used for
scanner diagnostics assessing geometric distortion, image
homogeneity, resolution, and slice profile. The resulting data
can provide both parameters to benchmark scanner perfor-
mance and insight on what is determining these parameters.
For example, resolution analysis can separate the pulse-­
sequence component from the intrinsic scanner component
F I G U R E 9 A, MRI of resolution inset taken with a voxel size of
of the point spread function. The SNR protocol can separate
0.35 mm and slice thickness of 4.0 mm. Here the resolution, according
to ACR protocol is 0.5 mm since all the arrays with hole diameters
temporal and spatial noise components, which contribute in
≥0.5 mm can be resolved, while the 0.4 mm hole size arrays cannot. B, different ways to SNR. Finally, the MR-­parameter arrays pro-
Synthetic image of hole array with broadening only from finite k-­space vide a very broad range of properties to allow validation of
sampling. C, Synthetic image with additional blurring from a Gaussian both standard and advanced quantitative protocols.
point spread function of width 2σ = 0.075 mm. D, The magnitude of One specific challenge encountered is that T2 is not a
the difference of the measured and synthetic image, using an L2 norm, rigorously defined quantity.281∕T2 is the component of the
as a function of the width of the point spread function, showing a total transverse spin dephasing rate, 1∕T2∗, that is considered
minimum in the difference at a blurring of 0.075 mm. to be intrinsic to the sample, with the extrinsic component,
1∕T�2, being eliminated using a spin echo: T1∗ = T1� + T1 . For
the projected slice thicknesses, and then( multiplying by the complex materials there is ambiguity on what dephasing
2 2 2

)
tangent of the wedge angle: tsl = 12 w2 + w1 × tan (10 ◦ ). components are considered extrinsic and intrinsic, and the
Averaging both ramp orientations corrects, in lowest order, for components being eliminated by the refocusing pulses will
errors due to the misorientation of the image slice plane relative depend on the details of the refocusing pulses. T2 can be a
to the base of the wedges. The latter method has the advantage function of the refocusing time and the values measured by
that it is easily automated and has a well-­defined uncertainty. CPMG are not the same as measured by SE.29,30 The system
phantom uses simple materials, so this is not a major issue
here. However, to get traceability, a robust operational defini-
4 | D IS C U SSION tion of T2 is required. Here, we define it as the exponential sig-
nal decay measured with a CPMG sequence with a refocusing
The system phantom described here, or modified versions of time of 2 ms. CPMG sequences also effectively null out the
it, have been used for several studies including repeatability longitudinal magnetization, which, if present along with B0
 |   

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1208 STUPIC et al.

F I G U R E 1 0 Slice profile analysis from an MR image with prescribed 3.0 mm slice thickness. A,B, The MR signal and its derivative,
for rectangular ROIs centered on the two wedges (inset in A). C,D, The measurement geometry and the slice profile, corrected for angular
misalignment θ. The measured slice thickness is 3.15 mm using the NEMA MS 5-­2018 protocol and 3.16 mm ± 0.2 mm using the automated
protocol described in the text. The misalignment of the slice profile inset and the scan plane is θ = 0.72°.

and B1 inhomogeneities, can lead to stimulated echoes. Other information may potentially be obtained by random place-
imperfections such as imperfect refocusing pulses, which can ment of the contrast elements, given that their properties
be present in SE sequences, can add long-­time signal and can are highly determined and errors due to location may be
increase the measured T2. The variability in T2 measurements better extracted using computer algorithms. There could be
have been extensively studied in the literature.31-­33 Here, we benefit as well in reducing the number of unique spheres in
note that simple SE sequences, which may appear to be sim- the contrast array to provide replicate spheres. The proto-
ilar, can have very different spin manipulations and signal type phantoms presented here are easily reconfigurable to
production. Using a standard phantom can identify these dif- test these modifications.
ferences, which can be important for both conventional quali- The contrast and fiducial arrays use plastic components
tative interpretation as well as quantitative analysis. that have sharp signal/-­ to-­
no-­signal interfaces that cause
From the feedback of users of the prototype phantoms, Gibb’s ringing artifacts, as can be seen in Figure 6A. These
several limitations and potential improvements have been artifacts can confound SNR and other measurements. While
identified. During the time since this phantom was con- these artifacts can be accounted for in certain measurements,
ceived, head coil geometries are becoming more complex, for instance they can be included in the kernel for the cross-­
and the size is decreasing. Newer head coils have minimum correlations determining geometric distortion, it would be
dimensions of 196 mm or less, which is smaller than the desirable to minimize these artifacts by making the plastic
prototype system phantom design. There is a trade-­off in the shells thinner or by allowing them to have some signal.
system phantom between sampling large volumes to assess Dielectric resonances can give rise to image distor-
uniformity of measurements and the ability to fit into newer, tion, particularly for large water phantoms at high fields.34
smaller coils. Smaller phantom shells have been investigated These effects still need to be studied for the system phantom.
and are available in commercial versions. Another concern However, the outer fill is easily changeable, without affecting
is the cost of the phantom. Tradeoffs can be made, such as the measurement protocols. Fills with different conductivi-
relaxing the 0.1 mm geometric accuracy, to reduce costs to ties and dielectric properties can easily be explored.
make the phantoms commercially viable, as discussed in Temperature control and monitoring remain a critical
Supporting Information Sec. 13. issue, especially when trying to verify the accuracy of re-
The contrast array components are arranged in an or- laxation time measurements to better than 2 %. While the
derly fashion to allow easier interpretation by users. More system phantom is shipped with a thermometer to record
 |

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
STUPIC et al.     1209

temperature, the bore temperatures are often different from Labs for useful discussions on phantom design and imple-
the ambient scanner room temperature or even the control mentation. The authors also thank all the ISMRM SQMR
room, where the temperature is typically measured. There committee members, past and present, for their advice
is no guarantee that the phantom temperature is the same and support. This paper is dedicated to Larry Clarke, NCI
as the recorded temperature. Furthermore, the tempera- Branch Chief, who helped initiate and champion this work
ture information is not easily encoded in the DICOM files. and many other quantitative imaging programs over the past
This establishes the need for in-­situ NMR thermometers, 2 decades.
and an MRI-­readable liquid crystal thermometer has been
developed for MRI phantoms.35,36 Furthermore, the sys- CONFLICT OF INTEREST
tem phantom would greatly benefit from the use of ma- Todor Karaulanov is an employee of CaliberMRI.
terials that are less sensitive to variations in temperature
and magnetic field strength. Most paramagnetic salts such DATA AVAILABILIT Y STATEMENT
as MnCl2 and CuSO4 have considerable temperature10 and The analysis software can be found at https://github.com/
field dependence (see Supporting Information Figures S10 MRISt​andar​
ds/Phant​omViewer. An analysis manual, 3D
and S11) in their relaxivity values r1, r2 (mM−1 s−1). NiCl2, models, and sample datasets can be found at https://github.
while having less temperature and field dependence, has com/MRISt​andar​ds/Syste​mPhantom.
safety/toxicity issues.
The materials used in the system phantom were chosen ORCID
for their stability and simple properties, for example, ex- Karl F. Stupic https://orcid.org/0000-0001-8356-1660
hibiting mono-­exponential spin relaxation. As shown in the Michael A. Boss https://orcid.org/0000-0002-9492-767X
Supporting Information, a reference library sample, sealed in Kathryn E. Keenan https://orcid.
a glass capillary, has 4-­year T1 and T2 stability better than 0.1 org/0000-0001-9070-5255
%. Assessing stability of the phantom components is more Chun Yuan https://orcid.org/0000-0002-6154-5106
challenging, since sampling the solutions for SI-­traceable Stephen E. Russek https://orcid.
NMR measurements would be destructive. The plastic org/0000-0002-8788-2442
spheres in the phantoms have some permeability to water and
may slowly absorb paramagnetic salts, leading to long-­term T WITTER
drift in T1 and T2. A sampling protocol of the MR-­property Karl F. Stupic @KarlStupic
spheres, stored under conditions similar to those contained
within the phantom, is required to assess long term stability R E F E R E NC E S
and is in progress. 1. Clarke LP, Sriram RD, Schilling LB. Imaging as a biomarker: stan-
There is demand for more biomimetic materials that dards for change measurements in therapy workshop summary. J
match multiple tissue properties, such as the apparent water Academic Radiol. 2008;15:501-­530.
2. Keenan KE, Ainslie M, Barker AJ, et al. Quantitative magnetic
diffusion coefficient as well as proton density and proton
resonance imaging phantoms: a review and the need for a system
spin relaxation times. It is challenging to obtain complex
phantom. Magn Reson Med. 2018;79:48-­61.
biomimetic materials that are stable and also have properties 3. Price R, Allison J, Clarke G, et al. 2015 MRI Quality Control
that can be rigorously quantified. However, this work is un- Manual. Virginia: American College of Radiology; 2015.
derway, as illustrated by the stable fat mimic developed for 4. Gunter JL, Bernstein MA, Borowski BJ, et al. Measurement of
the commercial version of the breast phantom developed by MRI scanner performance with the ADNI phantom. Med Phys.
NIST and the University of California San Francisco.37 2009;36:2193-­2205.
While the current implementation of the MRI system 5. ADNI2. Alzheimer’s Disease Neuroimaging Initiative 3T MRI
Technical Procedures Manual. California: Alzheimer's Disease
phantom can be improved and will never be suitable for all
Neuroimaging Initiative; 2011.
calibration needs, it is a first attempt at developing and im-
6. Premium System Phantom, CaliberMRI. Web site. https://qmri.
plementing a rigorous calibration structure with traceable com/qmri-­solut​ions/t1-­t2-­pd-­imagi​ng-­phant​om/. Accessed August
properties, community-­driven open construction details and 1, 2020.
evaluation protocols, and long-­term monitoring and assess- 7. Tofts PS. PD: proton density of tissue water. Quantitative MRI of
ment with state-­of-­the-­art metrology tools. the Brain. 2004;83-­103.
8. ACS. Water, Reagent. ACS Reagent Chemicals. 2017;4A.
ACKNOWLEDGMENTS 9. IT'IS. Tissue Properties Database V4.0. Foundation for Research
on Information Technologies in Society (ITIS). Zurich: IT'IS
The authors thank Michael Snow and William Hollander at
Foundation.
High Precision Devices for the computer aided design and 10. Kraft KA, Fatouros PP, Clarke GD, Kishore PRS. An MRI phan-
manufacturing guidance. The authors also thank Elizabeth tom material for quantitative relaxometry. Magn Reson Med.
Mirowski from Verellium and Joshua Levy from Phantom 1987;5:555-­562.
 |   

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1210 STUPIC et al.

11. Kowalewski J, Maler L. Nuclear Spin Relaxation in Liquids: 30. Ye FQ, Martin WRW, Allen PS. Estimation of the iron concentra-
Theory, Experiments, and Applications. Series in chemical phys- tion in excised gray matter by means of proton relaxation measure-
ics. New York: Taylor & Francis; 2006. ments. Magn Reson Med. 1996;35:285-­289.
12. Kowalewski J, Egorov A, Kruk D, et al. Extensive NMRD stud- 31. Vold RL, Vold RR, Simon HE. Errors in measurements of trans-
ies of Ni(II) salt solutions in water and water-­glycerol mixtures. verse relaxation rates. J Magn Reson (1969). 1973;11:283-­298.
J Magn Reson. 2008;195:103-­111. 32. Majumdar S, Orphanoudakis SC, Gmitro A, O'Donnell M, Gore
13. ACR. Phantom Test Guidance for Use of the Large MRI Phantom JC. Errors in the measurements of T2 using multiple-­echo MRI
for the ACR MRI Accreditation Program. Virginia: American techniques. II. Effects of static field inhomogeneity. Magn Reson
College of Radiology. 2018. Med. 1986;3:562-­574.
14. National Electrical Manufacturers Association. NEMA Standards 33. Majumdar S, Orphanoudakis SC, Gmitro A, O'Donnell M, Gore
Publication MS 5–­2018 Slice Thickness. Determination of Slice JC. Error in the measurements of T2 using multiple-­echo MRI
Thickness in Diagnostic Magnetic Resonance Imaging. New York: techniques. I. Effects of radiofrequency pulse imperfections. Magn
The Association; 2018. Reson Med. 1986;3:397-­417.
15. Boss MA, Dienstfrey AM, Gimbutas Z, et al. Magnetic resonance 34. Collins CM, Liu W, Schreiber W, Yang QX, Smith MB.
imaging biomarker calibration service. Proton Spin Relaxation Central brightening due to constructive interference with, with-
Times. 2018;Special Publication (NIST SP) -­250-­97. out, and despite dielectric resonance. J Magn Reson Imaging.
16. Lutti A, Hutton C, Finsterbusch J, Helms G, Weiskopf N. 2005;21:192-­196.
Optimization and validation of methods for mapping of the radiof- 35. Mirowski E, Snow M, Keenan K. inventors; Qalibre MD, Inc.
requency transmit field at 3T. Magn Reson Med. 2010;64:229-­238. (Boulder, CO, US), National Institute of Standards and Technology.
17. Stollberger RWP, McKinnon G, Justich E, Ebner F. Rf-­ field (Gaithersburg, MD, US), assignee. Liquid Crystal Thermometer
mapping in vivo. In: Proceedings of the 7th Annual Meeting of for MRI. US patent 10,809,331. October 20, 2020.
ISMRM, San Francisco, CA, USA. 1988, p. 106. 36. Keenan KE, Stupic KF, Russek SE, Mirowski E. MRI-­visible
18. van Schie JJ, Lavini C, van Vliet LJ, Vos FM. Feasibility of a fast liquid crystal thermometer. Magn Reson Med. 2020;84:​
method for B1-­inhomogeneity correction for FSPGR sequences. 1552-­1563.
Magn Reson Imaging. 2015;33:312-­318. 37. Keenan KE, Wilmes LJ, Aliu SO, et al. Design of a breast phantom
19. Kingsley PB. Methods of measuring spin-­ lattice (T1) relax- for quantitative MRI. J Magn Reson Imaging. 2016;44:610-­619.
ation times: an annotated bibliography. Concepts Magn Reson.
1999;11:243-­276.
20. Bojorquez JZ, Bricq S, Acquitter C, Brunotte F, Walker PM,
SUPPORTING INFORMATION
Lalande A. What are normal relaxation times of tissues at 3 T?
Magn Reson Imaging. 2017;35:69-­80.
Additional Supporting Information may be found online in
21. Cercignani M, Dowell NG, Tofts P. Quantitative MRI of the Brain: the Supporting Information section.
Principles of Physical Measurement. Boca Raton: CRC Press,
Taylor & Francis Group; 2018. xvii, 342 pages p. FIGURE S1 A, The hemispherically machined surface on
22. Rakow-­Penner R, Daniel B, Yu H, Sawyer-­Glover A, Glover GH. the post matches with the inclined notches, B, to precisely
Relaxation times of breast tissue at 1.5T and 3T measured using locate the plates. Photos of blue fiducial, C, and red MnCl2
IDEAL. J Magn Reson Imaging. 2006;23:87-­91. array, D, spheres showing the sealing and mounting methods.
23. Taylor AJ, Salerno M, Dharmakumar R, Jerosch-­ Herold M.
D, inset shows micro-­computed tomography image of con-
T1 mapping: basic techniques and clinical applications. JACC
Cardiovasc Imaging. 2016;9:67-­81.
trast sphere
24. National Electrical Manufacturers Association. NEMA Standards FIGURE S2 Relaxation rates as a function of paramagnetic
Publication MS 1–­2008 R2014 Determination of Signal-­to-­Noise salt concentration measured by inductively coupled Plasma
Ratio SNR in Diagnostic MRI. New York: The Association; 2014. (ICP) mass spectrometry at 1.5 T and 3.0 T. The solid lines
25. Jiang Y, Ma D, Keenan KE, Stupic KF, Gulani V, Griswold MA. are fits assuming a linear increase in relaxation rate with con-
Repeatability of magnetic resonance fingerprinting T1 and T2 es- centration and a zero-­concentration intercept given by the
timates assessed using the ISMRM/NIST MRI system phantom. measured values for high purity water
Magn Reson Med. 2017;78:1452-­1457.
FIGURE S3 Schematic of the fiducial array analysis show-
26. Keenan KE, Gimbutas Z, Dienstfrey A, Stupic KF. Assessing
effects of scanner upgrades for clinical studies. J Magn Reson
ing input 3D image of the system phantom, cropped image
Imaging. 2019;50:1948-­1954. with just the fiducial spheres, image of fiducial sphere, syn-
27. Bane O, Hectors SJ, Wagner M, et al. Accuracy, repeatability, and thetic k-­space image and real-­space image used as a convo-
interplatform reproducibility of T1 quantification methods used for lution mask, slice of 3D convolution image, and convolution
DCE-­MRI: results from a multicenter phantom study. Magn Reson profiles with fits used to obtain sphere center
Med. 2018;79:2564-­2575. FIGURE S4 Fiducial analysis on a 1.5 T scanner with a
28. Abragam A. The Principles of Nuclear Magnetism. Oxford: Oxford
gradient echo sequence. A, Coronal slice, B, axial slice, C,
Science Publications; 1983:57.
sagittal slice. D, magnified image of fiducial sphere image
29. Bartha R, Michaeli S, Merkle H, et al. In vivo 1H2O T2+ mea-
surement in the human occipital lobe at 4T and 7T by Carr-­Purcell with ROI location after automated location. E, Normalized
MRI: detection of microscopic susceptibility contrast. Magn Reson integrated intensity for all 57 fiducial spheres. F, Difference
Med. 2002;47:742-­750. between center of mass and convolutional sphere centers
 |

15222594, 2021, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mrm.28779 by Jordan Hinari NPL, Wiley Online Library on [23/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
STUPIC et al.     1211

FIGURE S5 A, B, C, Geometric distortion of the 57 fiducial fluoride (PVDF), polyphenylene sulfide (PPS), polypropyl-
sphere centers in x, y and z directions, respectively ene (PP). The samples were 25 mm diameter, 6 mm thick
FIGURE S6 NMR signal for Ni-­12 versus inversion time for disks
the T1-­IR protocol along with fits to the model described in FIGURE S15 Geometric distortion during water soaking of
the text. Data for three consecutive measurements are shown the same samples used in Figure S13. The horizontal line in-
along with the residuals for each measurement (top plot). dicates the threshold for maintaining the specified geometric
The errors listed for T1 and the inversion efficiency are the distortion of the phantom plates
standard deviation of the 3 values obtained for each of the FIGURE S16 Spin relaxation times for NiCl2-­3, NiCl2-­5,
measurements NiCl2-­10 at 3 T, 20°C as a function of pH
FIGURE S7 NMR signal from Ni-­12 using a CPMG se- FIGURE S17 Relaxation times for ACS-­grade and deionized
quence as a function of acquisition time along with exponen- water as a function of temperature
tial fits. Data for three consecutive measurements are shown TABLE S1 NiCl2 Array
along with the residuals for each measurement (top plot). The TABLE S2 MnCl2 Array
errors listed for T2 are the standard deviation of the 3 values TABLE S3 Proton Density Array
obtained for each of the measurements TABLE S4 Water proton spin relaxivities determined from
FIGURE S8 Magnetic field dependence of T1 and T2 for the the slope of the data in Figure S2
MnCl2 and NiCl2 arrays TABLE S5 Isotropic Volume Series
FIGURE S9 Temperature dependence of T1, T2 for the Ni-­12 TABLE S6 Section Thickness Series
solution measured in a metrology NMR at 3.0 T. The plot TABLE S7 Resolution Inset Series
shows data from a flame-­sealed borosilicate-­capillary library TABLE S8 Proton Density and Signal to Noise Series
sample over the course of 4 years TABLE S9 T1 Inversion Recovery Series
FIGURE S10 Variation of normalized relaxation times with TABLE S10 T1 Variable Flip Angle Series
temperature for the NiCl2 array at 3.0 T TABLE S11 T2 Series
FIGURE S11 Temperature coefficient of spin relaxation TABLE S12 Starting Chemicals for Contrast Fluids
times for the MnCl2 array at 3.0 T TABLE S13 NiCl2 Solutions
FIGURE S12 Variation of relaxation times with temperature TABLE S14 MnCl2 Solutions
for the CuSO4 fiducial solution at 3.0 T TABLE S15 Proton Density Solutions
FIGURE S13 Deviation of T1-­IR values from NMR refer-
ence values at 3 T over the course of 7 years. The gray bar
indicated the range of values given a phantom temperature How to cite this article: Stupic KF, Ainslie M, Boss
that can vary between 18°C and 22°C MA, et al. A standard system phantom for magnetic
FIGURE S14 Water mass uptake for various plastics: nylon/ resonance imaging. Magn Reson Med. 2021;86:1194–­
polyamide (PA), poly(methyl methacrylate) (PMMA), poly- 1211. https://doi.org/10.1002/mrm.28779
carbonate (PC), polyvinyl chloride (PVC), polyvinylidene