Home Search Collections Journals About Contact us My IOPscience

A comparison between objective and subjective image quality measurements for a full field

digital mammography system

This content has been downloaded from IOPscience. Please scroll down to see the full text.

2006 Phys. Med. Biol. 51 2441

(http://iopscience.iop.org/0031-9155/51/10/006)

View the table of contents for this issue, or go to the journal homepage for more

Download details:

IP Address: 93.180.53.211
This content was downloaded on 08/01/2014 at 13:10

Please note that terms and conditions apply.

INSTITUTE OF PHYSICS PUBLISHING PHYSICS IN MEDICINE AND BIOLOGY

Phys. Med. Biol. 51 (2006) 2441–2463 doi:10.1088/0031-9155/51/10/006

A comparison between objective and subjective image

quality measurements for a full field digital
mammography system

N W Marshall
Clinical Physics Group, Barts and the London NHS Trust, St Bartholomew’s Hospital,
London EC1A 7BE, UK

Received 6 September 2005, in final form 30 March 2006

Published 27 April 2006
Online at stacks.iop.org/PMB/51/2441

Abstract
This paper presents pre-sampling modulation transfer function (MTF),
normalized noise power spectrum (NNPS) and detective quantum efficiency
(DQE) results for an amorphous selenium (a-Se) full field digital
mammography system. MTF was calculated from the image of an angled
0.5 mm thick Cu edge, acquired without additional beam filtration. NNPS
data were acquired at detector air-kerma levels ranging from 9.1 µGy to
331 µGy, using a standard mammography x-ray spectrum of 28 kV, Mo/Mo
target/filter combination and 4 cm of PMMA additional filtration. Prior to
NNPS estimation, the image statistics were assessed using a variance image.
This method was able to easily identify a detector artefact and should prove
useful in routine quality assurance (QA) measurements. Detector DQE,
calculated from the NNPS and MTF data, dropped to 0.3 for low detector
air-kerma settings but reached an approximately constant value of 0.6 above
50 µGy at the detector. Subjective image quality data were also obtained at
these detector air-kerma settings using the CDMAM contrast-detail (c-d) test
object. The c-d data reflected the trend seen in DQE, with threshold contrast
increasing at low detector air-kerma values. The c-d data were then compared
against predictions made using two established models, the Rose model and a
standard signal detection theory model. Using DQE(0), the Rose model gave
results within approximately 15% on average for all the detector air-kerma
values studied and for detail diameters down to 0.2 mm. Similar agreement
was also found between the measured c-d data and the signal detection theory
results, which were calculated using an ideal human visual response function
and a system magnification of unity. The use of full spatial frequency DQE
improved the agreement between the calculated and observer results for detail
sizes below 0.13 mm.

0031-9155/06/102441+23$30.00 © 2006 IOP Publishing Ltd Printed in the UK 2441

2442 N W Marshall

Introduction

The past few years have seen the introduction to the UK of full field digital mammography
(FFDM) systems, initially for the purpose of breast assessment rather than breast screening.
FFDM technology falls into two distinct categories—the computed radiography (CR) approach
involving the use of photostimulable phosphor plates held in a cassette and the so-called
integrated detector methods. Integrated detector systems use an x-ray converter bonded to
a thin film transistor (TFT) array which performs the image readout. One of the major
advantages of integrated digital detectors is that the image is available directly on the hospital
information network—CR systems currently rely on manual handling of the imaging cassette
by the radiographer between the cassette holder and the CR reader. Hence, integrated digital
systems should allow greater patient throughput and enable the radiographer to stay with the
patient in the x-ray room throughout the duration of the examination.
Assessment of conventional film/screen mammography systems in terms of
commissioning and quality assurance (QA) tests is covered extensively in IPEM Report 89
(2005). One of the challenges of the new full field digital technology lies in extending existing
test protocols to fully assess the capabilities of these new detectors. One such protocol that
has recently been made available is the EUREF Digital Addendum (2003), which gives tests
suitable for the both the commissioning and routine QA assessment of both CR and direct
detection FFDM systems. This protocol suggests that objective measurements such as the
noise power spectrum (NPS) and modulation transfer function (MTF) may also be measured as
part of the commissioning and routine QA measurements performed on these units. Objective
measures are a well-established and powerful means of assessing image quality (Metz et al
1995, ICRU 1996), especially given the relatively easy access to the required image data now
offered by the DICOM format. Within the past 15 years or so, these methods have been applied
to various digital imaging modalities (Hillen et al 1987, Workman and Cowen 1993, Dobbins
et al 1995, Flynn and Samei 1999, Stierstorfer and Spahn 1999, Williams et al 1999, Evans
et al 2002, Samei and Flynn 2003). As demonstrated by Samei and Flynn (2003), accurate
measurement of MTF, NPS and DQE is a complex procedure with many important parameters
that affect the final result.
For imaging modalities such as conventional screen/film (S/F) mammography or
fluoroscopy, it is far harder to apply these objective methods for reasons of image data
access, where images are held by a film or in the form of a TV voltage. Hence, in the field
of conventional screen/film (S/F) mammography, subjective or semi-quantitative test objects
are commonly used to assess image quality. In the UK, this might include the TOR(MAM)
(Cowen et al 1992) test objects, along with the CDMAM 3.4 (Artinis Medical Systems BV,
Netherlands) contrast-detail (c-d) test object. All of these test objects provide a controlled
input signal when imaged under standard conditions and require the observer to perform a
visual assessment of the image. It is likely that some or all of these methods will be carried
over in to the assessment of image quality for FFDM units.
The aim of this work was to assess a Lorad Selenia (Lorad Corp., USA) FFDM imaging
system using both subjective and objective image quality measurements over a range of air-
kerma values at the detector. Subjective image quality was evaluated using c-d data acquired
with CDMAM test object. While there are known problems with the c-d method (ICRU 1996),
current acceptable and achievable image quality standards in the EUREF digital mammography
protocol are couched in terms of a c-d curve. Objective image quality was measured using
NPS and pre-sampling MTF. Results were then compared in the light of two models: the Rose
model (Rose 1948) and a version of the well-known signal detection theory model as applied
to medical imaging systems (Wagner and Brown 1985) supplied by Aufrichtig (1999).
A comparison between objective and subjective image quality measurements for a FFDM system 2443

Materials and methods

Tube and generator

Tube and generator function was assessed under current protocols (IPEM 2005). Accuracy of
the tube potential was within the expected range of ±1 kV of nominal across the range from
24 kV to 34 kV. X-ray output at 50 cm was 232 µGy mAs−1 for a nominal tube potential
of 28 kV. System half value layer (HVL), measured at 30 kV nominal with the compression
plate out of the beam and Mo filter selected was 0.31 mm Al giving an estimated x-ray tube
filtration of 0.54 mm Al equivalent.

Detector
The Lorad Selenia uses a direct conversion flat panel detector in which x-rays are directly
converted to charge by the amorphous selenium (a-Se) (Yorker et al 2002). An in-depth review
of this direct detection technology is given by Rowlands (2000). In this type of detector, a
large potential difference (typically 2000 V) is held across the a-Se layer and this ensures
that charge generated in the photoconductor by x-ray interaction is drawn directly towards
the pixel readout system with very little lateral spread. The a-Se x-ray converter layer in the
Lorad Selenia is 200 µm thick (CEP Report 05084 2006). The holes generated during x-ray
interaction are collected by pixel electrodes and stored on a capacitor located at each pixel.
Readout of the pixel charge is achieved using an amorphous silicon active matrix thin film
transistor (TFT) array to switch out the charge, which is then converted to a voltage by an
amplifier attached to each data line. The 24 × 29 cm detector is mapped on to a 3584 ×
4096 pixel array, giving a 70 µm pixel size and a sampling limit of 7.1 mm−1.

Variance
The NPS describes the spectral decomposition of the noise variance in an image as a function of
spatial frequency. When estimating NPS, various assumptions are made regarding the nature
of the stochastic processes that result in the final image variance. For digital imaging systems,
it is often assumed that the random process generating the variance is wide sense cyclo-
stationary (WSCS) (Cunningham 2000), meaning that the expected value and autocorrelation
of the noise realized in the image are both stationary i.e. they are independent of position in
the image and that these statistics do not change between two positions separated by an integer
multiple of the sample spacing (Cunningham 2003, Maidment et al 2003). The importance
of these assumptions is discussed elsewhere (ICRU 1996, Cunningham 2000, Williams et al
1999).
Most NPS algorithms sample image data from a variety of positions across the image
(Cunningham 2000) and therefore the notion of stationarity is crucial. If we wish to uncover
the spectral decomposition of the image variance using a method that relies on periodicity,
such as Fourier analysis, then it is important the variance remains constant over the region
from which the data are sampled. One of the principal reasons a digital imaging system may
fail this criterion is that of detector artefacts, be it in the form of dead pixels, dead lines or
some other artefact. Before proceeding to full NPS analysis, it has been recommended that an
image of the variance is initially formed (Maidment et al 2003) and assessed for stationarity.
In this study, variance images were produced from a single image using a 10 × 10 pixel
region of interest (corresponding to 0.7 × 0.7 mm). The variance of the pixels within this
ROI was calculated and this value was assigned to all the pixels in the ROI. The ROI was then
moved across the image in steps of 10 pixels, so that the entire image was covered. Finally,
2444 N W Marshall

the variance value for each ROI was assigned a grey scale value; low variance equal to black
and high variance equal to white. This image then allows a quick visual inspection of the
uniformity of the variance in the image and whether there are significant artefacts present.

Signal transfer property (STP)

The first step in objective image quality analysis is measurement of the system signal transfer
property (STP), or detector response as it is defined in the EUREF protocol. This will tell us
whether the system has a linear relationship between the incident x-ray fluence striking the
detector and the output parameter (usually a pixel value). For objective image methods to be
valid the relationship between system input and output must be linear or at least linearizable,
for the case of screen/film and computed radiography systems. The STP was measured
following the EUREF protocol by placing 4 cm PMMA at the x-ray tube head with the
x-ray grid removed and exposing using standard conditions of 28 kV and Mo/Mo target filter
combination. Initially, a calibrated dose meter was placed 8 cm above the detector at the
centre of the x-ray field and x-ray output was measured for a range of mAs product values.
The detector was protected using a lead beam block during this procedure. This enabled the
accurate estimation of air-kerma as a function of mAs at the detector input plane. The lead
block was then removed and mAs product varied from 4 to 140 mAs and the pixel value (PV)
from ‘flat field’ images (i.e. linear images, corrected for detector artefacts) was measured using
a region of interest (ROI) placed at the centre of the image. The STP was derived by plotting
measured PV against air kerma at the detector.

Modulation transfer function (MTF)

Several methods for measuring MTF have been described in the literature (Judy 1976, Sones
and Barnes 1984, Hillen et al 1987, Fujita et al 1992, Samei et al 1998) and these have been
applied to a range of x-ray imaging systems. In this study, a version of the angled edge was
used to measure pre-sampling MTF i.e. the MTF prior to sampling by the pixel matrix (Giger
et al 1984). It is well known that FFDM systems are not shift invariant (Dobbins et al 1995,
Cunningham 2000, Albert and Maidment 2000), as they are based around discrete digital
detector. Hence, the signal (image) produced by the edge used in the MTF measurement
method will vary depending upon its position relative to the pixel matrix. The angled-edge
method is used to produce a very finely sampled edge spread (Samei et al 1998, Albert and
Maidment 2002) that can be used to estimate the pre-sampling MTF i.e. the MTF prior to
sampling by the pixel matrix. The details of the angled edge method are described fully
elsewhere (Samei et al 1998) but some points relevant to the implementation of this method
are given briefly here.
Using this method, the pre-sampling MTF can be estimated in two directions: MTF
parallel to the chest wall edge is defined as the data line direction for this detector and the
direction away from the chest wall towards the nipple edge is the gate line direction. The MTF
was measured at the detector centre; MTF was not studied as a function of position across the
detector. First, a 0.5 mm thick copper sheet was placed on the tabletop at the centre of detector
field and angled slightly with respect to the pixel matrix. An image of the edge was acquired at
28 kV and 20 mAs and saved to CD-R in DICOM format. As will be shown later, the system
has a linear signal transfer property and therefore it was not necessary to linearize the image
before calculating pre-sampling MTF. Next, the angle of the edge was found by differentiating
along the direction of interest. A simple least-squares method was used to fit a first-order
polynomial y = a + bx to the pixel values lying along the edge. The angle of the edge was
A comparison between objective and subjective image quality measurements for a FFDM system 2445

then calculated as tan−1(b) where b is the gradient of the line. Knowing the angle of the edge
allows us to re-sample the image using a finer sample distance than the pixel pitch to create
the over-sample. The pixel values from a square region spanning the edge were then assigned
to the over-sample array, with a position along the array given by the distance from the edge.
This distance was calculated using the equation of the line (Rogge et al 2003).The pixel data
in the over-sample array were then re-binned according to a sub-pixel binning factor β (Samei
et al 1998). In this work we use β = 15, which represents a compromise between resolution
and noise on the MTF result. Finally, the over-sample was normalized for the number of pixels
assigned to each bin.
Before differentiating the over-sampled edge spread function, some filtering can be applied
in order to reduce noise (see the paper by Maidment and Albert (2003) for a discussion on
conditioning of MTF data). Great care should be taken when trying to reduce noise in
the pre-sampling MTF estimate—inappropriate choice of filter type could easily bias the pre-
sampling MTF. In this work, a median filter is applied to the over-sample before differentiating
as this a simple and relatively benign filter. After median filtering, the over-sampled edge
spread function was differentiated, Fourier transformed and normalized to obtain the
pre-sampling MTF.

Noise power spectrum (NPS)

As part of the aim of this paper is to examine the feasibility of measuring objective image
quality parameters as prescribed in the EUREF protocol, a simple NPS method suitable for
routine QA measurement was sought. Some of the important theoretical aspects of noise power
measurements are detailed in the work of Cunningham (2000) and Dobbins (2000). Williams
et al (1999) give a thorough account of the practicalities of noise power estimation for digital
mammography systems. Noise power spectra in digital mammography are estimated using
flat field images, usually produced by imaging a block of PMMA under standard conditions.
In this study, we used a tube potential of 28 kV, a Mo/Mo target filter combination and the
4 cm PMMA blocks supplied with the CDMAM test object. These blocks were suspended at
the x-ray tube port, generating a uniform image with reduced levels of scatter. Images were
acquired with the anti-scatter grid out of the x-ray beam (similar to the geometry employed
by Evans et al (2002)).
In this study we compared objective and subjective image quality parameters across a
range of input air-kerma values at the image receptor. First, the typical operating point for the
automatic exposure control (AEC) was taken to be the mAs delivered under AEC control with
4 cm PMMA and the CDMAM c-d test object placed on the table and with the anti-scatter grid
in place. AEC sensing region was set to position 2 i.e. approximately 3 cm from chest wall
edge of the detector. For 28 kV, Mo/Mo factors, the AEC delivered 105 mAs. The standard
clinical mAs product for 4 cm PMMA and the CDMAM c-d test object was taken to be
110 mAs (i.e. the nearest mAs station to 105 mAs). This gave a mean PV measured at the
centre of the image of 507. Flat field images for 4 cm PMMA at an equivalent detector input
air kerma were acquired by matching the PV for this image; in this case 35 mAs gave a PV
of 497. Flat field NPS images were then acquired at values of 4, 9, 18, 35, 70 and 140 mAs,
corresponding to detector air-kerma values of 9.1, 21.0, 42.3, 82.5 165 and 331 µGy.
The NPS was calculated using a two-dimensional algorithm from a flat field image as
follows. First, a region (e.g. 1024 × 1024 or 2048 × 2048 pixel) was extracted from the
central area of the 3584 × 4096 image. Non-overlapping regions of interest or records of size
128 × 128 pixels were then taken from this area (Evans et al 2002). Williams et al (1999)
acquired their records from a region of approximately 5 cm × 5 cm, however we have found
2446 N W Marshall

that using a central region of 7.2 cm (of 1024 × 1024 pixels) or 14.4 cm (2048 × 2048 pixels)
gave similar results.
The NPS data in this study are calculated from un-subtracted images and therefore contain
contributions from stochastic, varying non-stochastic (i.e. non-stochastic but varying from one
image to another, such as slight ghosting artefacts etc) and fixed pattern noise sources present
in the system (Williams et al 1999). These can be present, despite the flat fielding process that
is routinely applied to this system. Evans et al (2002) excluded fixed pattern noise sources
from their analysis by using subtracted images. In our analysis, each record was corrected for
the presence of background trends by fitting a polynomial and subtracting the fitted surface
S(x, y) from the record. For a 128 × 128 record size, there was some difference between first-
and second-order polynomials at low spatial frequencies (<0.25 cycles mm−1). As second-
order polynomials and above gave similar results, the NPS in this study were calculated using
second-order polynomial detrending.
The 2D digital NPS (in terms of pixel value fluctuation) was then calculated using the
formula (Williams et al 1999, Dobbins 2000):
lim lim 1
NPSP /P (un , vk ) = Nx x Ny y
Nx , Ny → ∞ M → ∞ M


M
× |FTn,k {p(x, y) − S(x, y)}|2 (1)
m=1
where Nx and Ny is the record size in the x and y directions and x and y are the x and y
pixel sizes, M is the number of records used in the ensemble average and FT denotes the fast
Fourier transform operation. The limits in this equation were realized by increasing the size
of Nx and Ny (i.e. the NPS ROI size) until the NPS converged to some value. Values of Nx and
Ny below 128 pixels (approximately 9 mm) resulted in an underestimate of the NPS for spatial
frequencies below approximately 0.7 mm−1.
The value of M governs the number of NPS spectra in the ensemble; increasing the value
of M reduces the uncertainty on the final NPS estimate. Using 128 × 128 pixel records taken
from a 2048 × 2048 region at the centre of one image gave 256 spectra in the ensemble and
a coefficient of variation of approximately 1/(256)1/2 ≈ 6% . For the IDL code used in this
study, FT is given by
1

N−1
F (u) = p(x) e(−2πiux/N) . (2)
N 0

NPSP/P was then divided by the (large area signal)2 to obtain NPSE/E, the normalized NPS
expressed in terms of relative input exposure fluctuation (Dainty and Shaw 1974, Dobbins
2000):
NPSP /P (u, v)
NPSE/E (u, v) = . (3)
(large area signal)2
The term (large area signal) was taken to be the mean PV in the image at the dose/image of
interest and this can be obtained from the STP (after correction for the offset).

Detective quantum efficiency (DQE)

DQE was calculated using the equation:
MTF2 (u, v)
DQE(u, v) = . (4)
q0 NPSE/E (u, v)
A comparison between objective and subjective image quality measurements for a FFDM system 2447

Table 1. Pixel value and air kerma at detector as a function of mAs for CDMAM images and
PMMA images used for NPS analysis.

CDMAM 4 cm PMMA

Pixel Air kerma at Pixel Air kerma at

mAs value detector (µGy) mAs value detector (µGy)
14 246 11.9 4 231 9.1
28 269 18.5 9 278 21.0
55 350 41.0 18 353 42.3
110 507 84.8 35 497 82.5
220 828 174 70 796 165
400 1430 342 140 1388 331

As discussed by several authors (Hillen et al 1987, Stiersorfer and Spahn 1999, Williams et al
1999), most current x-ray detectors integrate the energy of the incident x-ray fluence, rather
than count the individual x-ray photons. This implies that for DQE, we must compare the
detector against an ideal energy integrating device, rather than an ideal photon counter. Hence,
for an input spectral distribution (E ), q0 in equation (4) is given by
 2
(E) E dE
q0 =  . (5)
(E) E 2 dE
We are therefore using the variance of the incident energy fluence to quantify the input quantum
noise (Stierstorpher and Spahn 1999). Using the data given by Cranley et al (1997) gives
a value of q0 = 5300 photons mm–2 µGy–1 for the 28 kV spectrum from a Mo target with
a nominal 30 µm Mo filter used in this study. This is close to the value of 5500 photons
mm–2 µGy–1 adopted by Evans et al (2002). There is a carbon fibre cover plate of unspecified
thickness protecting the detector and grid assembly. The effect of this cover plate on the
number of transmitted photons was not included on the DQE figures presented in this study.

Contrast-detail data
To study the c-d performance of the system as a function of dose/image, two sets of CDMAM
images were acquired at values of 14, 28, 55, 110, 220 and 400 mAs. The detector input air
kerma for these mAs settings were 12, 18, 41, 85, 174 and 342 µGy. These figures were
estimated from the detector STP measured previously for 4 cm PMMA. The detector STP
gradient changes slightly with mean energy of the x-ray beam, however a beam load of 4 cm
PMMA is sufficiently close to 4 cm PMMA with the CDMAM test object not to introduce
significant errors into this estimation. Table 1 compares air kerma at the detector for 4 cm
PMMA (i.e. for the NPS results) against detector air kerma for the CDMAM images.
The contrast for a gold disc of thickness tAu was calculated as follows. First, the number
of photons mm–2 transmitted through the disc at a given energy E (disc (E )) was calculated:
 E max
disc (E) = inc (E) exp(−(µAu (E)tAu + µPMMA (E)tPMMA + µAl (E)tAl )) dE. (6)
0

The number of photons mm–2 in the background spectrum (no gold disc), bkg(E ), is
 E max
bkg (E) = inc (E) exp(−(µPMMA (E)tPMMA + µAl (E)tAl )) dE (7)
0
2448 N W Marshall

where µAu(E ), µPMMA(E ) and µAl(E ) are the linear attenuation coefficients of gold, PMMA
and aluminium at energy E, respectively. inc(E ) is the Mo x-ray spectrum incident on to the
CDMAM test object at 28 kV and includes attenuation by 0.69 mm Be and 0.03 mm Mo tube
filtration. The contrast of the gold disc (Cp) was then calculated in terms of air kerma using
 E max  E max
bkg (E)K(E) dE − 0 disc (E)K(E) dE
Cp = 0
 E max (8)
0 bkg (E)K(E) dE

where K(E ) is the photon to kerma conversion factor (Cranley et al 1997). The contrast is
dimensionless and therefore has no physical units.
As CDMAM is placed between 4 cm of PMMA, the radiation contrasts generated (CP)
will be degraded by scattered radiation:

CP
C= (9)
(1 + S/P )

where S/P is the ratio of scattered to primary radiation (Dance and Day 1984). A typical
factor for 4 cm breast thickness calculated for a linear grid is approximately 0.45. The Selenia
system uses a cellular (HTC) grid (Lorad Corp., USA) however, in the absence of data for this
grid type, a nominal factor of 0.4 was used in this work.
The CDMAM test object is formed of a grid of 1 cm × 1 cm squares, with each square
containing two identical circular discs, one at the centre and the other in a randomly selected
corner (the eccentric disc). Included in the CDMAM manual is a scoring and correction
scheme that utilizes the correct identification of the position of eccentric disc in each square.
This method, requiring at least three observers, produces reproducible results but is time
consuming to implement in a QA programme. CDMAM, as used in the QA programme at
our physics centre, is scored as a conventional c-d test object i.e. the observer counts to the
last disc visible at the centre of the square for each detail diameter. The contrast of this disc
is then taken to be the threshold contrast CT for that detail diameter. This is the method used
when scoring fluoroscopy (Hay et al 1985) and digital subtraction angiography (Cowen et al
1987) c-d test objects. We are therefore comparing subjective and objective measurements
acquired under typical QA conditions.
The c-d images were scored by two experienced observers on the radiologist softcopy
reporting station. This utilizes two Barco (USA) five megapixel cathode ray tube (CRT)
monitors (2500 × 2000 pixels with a display bit depth of 10 bits). The luminance response of
these CRT monitors is checked using a luminance meter as part of the routine quality assurance
checks made on the system. The MediCal Pro (Barco, USA) QA package is used for these
checks with the luminance response calibrated against the DICOM luminance response. The
CRT monitors are situated in the radiologist reporting room and therefore a low level of
ambient light was set during the c-d scoring. Care was also taken to keep reflections on the
CRT screens to a minimum.
When an image is brought up for display on the softcopy station, the whole image is
displayed at reduced resolution (i.e. not at 1 to 1 pixel resolution) so that the 3584 × 4096
image can be displayed on the monitor. The software supplied includes a magnification
function that allows a section of the image to be viewed at full resolution and this was
used during the scoring. Before scoring the images, window width and level were set to a
satisfactory level by the observers. C-d images tend to have a narrow dynamic range and
therefore fairly narrow window widths (high contrast setting) can be used when scoring this
type of image.
A comparison between objective and subjective image quality measurements for a FFDM system 2449

Contrast-detail models
The experimentally derived c-d data were compared against two observer models. The c-d
results were modelled using the version of signal detection given by Aufrichtig (1999) for the
task of detecting a static, circular disc shaped object imaged against a homogeneous noisy
background. The SNR for this detection task can be written as

C |S(u, v) MTF(u, v)O(u, v)|2 du dv
SNR =  1/2 (10)
|S(u, v) MTF2 (u, v)O 2 (u, v)|2 (q0 DQE(u, v))−1 du dv
where S(u, v) is the spectrum of the target object in frequency space, MTF(u, v) and DQE(u, v)
are the imaging system MTF and DQE, respectively and O(u, v) represents the spatial
frequency response function of the observer’s visual system. For detection to occur according
to this model, the SNR will have exceeded some threshold criterion for detection, k say, held by
the observer. At this point, the contrast for the signal detection model will be at the threshold
defined as CTS
  −1
|S(u, v) MTF(u, v)O(u/m, v/m)|2 du dv
CTS = k  1/2 . (11)
|S(u, v) MTF2 (u, v)O 2 (u/m, v/m)|2 (q0 DQE(u, v))−1 du dv
The spatial frequency response of the observer was taken from Kelly (1979). We also studied
the effect of setting O(u, v) = 1 (i.e. with no spatial filtering by the observer).
As the images are presented to the observer with some magnification by the soft copy
workstation (SCW), we could model the effect of display magnification factor using the factor
m in equation (11). A typical viewing distance of 25 cm was set for the calculation of CTS.
The software magnification function was used when scoring the test objects so that sections
of the image were displayed at a 1 to 1 pixel resolution. When this was engaged, 8 cm in the
image was displayed as 18.4 cm on the screen giving a magnification m = 2.3. With the image
displayed at the default size (i.e. without the magnification function), the SCW magnification
was 1.38.
The model of Rose (1948) was used to calculate the threshold contrast CTR for a circular
object of diameter α using the using the relationship:
 1/2
2k 1
CTR = 1/2 (12)
π α q0 DQE(0)
where q0 is the number of photons mm–2 incident on the x-ray detector and DQE(0) is the
detector DQE at zero spatial frequency. Assumptions required for the Rose approach to be
valid include (1) the photons are assumed to be uncorrelated (2) the signal at the detector
is flat-topped and has sharp edges (there is no imaging system blur) (Burgess 1999). The
relationship between the signal detection theory given in equation (10) and the theory of Rose
is discussed at length by Burgess (1999).
In equations (11) and (12), k is the observer’s threshold signal-to-noise ratio (SNR).
Various values of k have been reported in the literature (Rose 1948, Schnitzler 1973, Moran
1982, Ishida et al 1984). k depends on the detection threshold adopted by the observers
during the period of the experiment. Although trained observers can hold this threshold
reasonably constant over a period of time, the actual value adopted by each observer is not
known or controlled in a contrast-detail measurement of this type. In previous studies we have
found that a value of k = 3.8 (Ishida et al 1984) gives calculated results that are close to the
measured results (Marshall et al 2001). A strict observer would demand that the threshold
disc is extremely well visualized (e.g. a clear, continuous edge and well-defined position) and
2450 N W Marshall

(a) (b)

Figure 1. (a) Variance image produced by imaging a 4 cm thick PMMA block at 28 kV, Mo/Mo,
95 mAs using a 10 × 10 pixel ROI. (b) The same grey scale data plotted as an SNR image.

have a correspondingly high value of k (around 4 or 5). A more lenient observer would be
operating nearer 50% detectability and have a lower value of k (closer to 2 or 3) (Burgess
1999). Changing k shifts the calculated or expected threshold contrasts for the system with
respect to the y-axis. This is a shortcoming of the contrast-detail method used in the current
study with regard to the prediction of absolute threshold contrasts (ICRU 1996). However,
this type of experiment can still provide valuable qualitative information on the performance
of the system as various parameters are changed (e.g., detector input dose).

Results and discussion

Variance
All FFDM systems have a software map containing the locations of defective pixels. On
calibration of the detector by the engineer, some form of correction algorithm is applied
whereby a pixel value (PV) interpolated from nearest pixel neighbours is given to the defective
pixel. This is a form of averaging, reducing variance locally and should show up as a region
of reduced variance in the variance image. Defective pixels that have not been mapped by
the engineer, on the other hand, may show up as regions of increased variance in the variance
image.
Figure 1(a) shows a variance image for the Selenia system generated at from one image
acquired at 95 mAs (28 kV, Mo/Mo, 4 cm PMMA). Variance is uniform across the central
and chest wall regions of the image, increasing as we move further away from the chest wall
edge. As a result of the heel effect, the number of x-ray photons mm–2 is greater at the chest
wall than at the nipple edge. X-ray photon noise follows a Poisson distribution and therefore
x-ray variance should increase at the chest wall. This is not seen and is possibly a result of the
flat field and detector gain corrections affecting the noise variance. The variance is lower at
A comparison between objective and subjective image quality measurements for a FFDM system 2451

the image detector edges (top and bottom in this figure) and this may be due to reduced noise
from the lower TFT pixel capacitance at the detector edges.
In addition to the variance image, we can also derive a visual assessment of SNR uniformity
for a single image using this method (figure 1(b)). The SNR in this instance was calculated
from the mean pixel value for the ROI divided by the standard deviation of the pixels within
the ROI. This is of interest as the EUREF Digital Addendum gives a limit of 15% for SNR
uniformity calculated from five regions in the image. The image of the SNR gives a fuller
realization of the SNR variation across a flat field image and shows that SNR increases at the
chest wall. While we can expect excellent pixel value uniformity for a grey scale image as a
result of the flat fielding process, the SNR will tend to be less uniform (especially in the chest
wall to nipple direction) because of the heel effect. This should be taken into account when
devising QA protocols for these systems.
Finally, it should be noted that this system developed a fault that was seen by the
radiographer during the flat fielding QA process (note that this study was performed after
the faulty detector was replaced). A spongy artefact was observed at the chest wall edge of
the detector on the grey scale flat field image. This was investigated by the field engineer
and ascribed to delamination of the a-Se layer and the TFT array. Ultimately, this fault was
traced by the manufacturer to a known set of detectors via the detector serial number—all
detectors with this fault have since been replaced. The detector fault is clearly demonstrated
by the variance image method—see figure 2. In fact, the variance image indicates a second
artefact region that had not been noted by the radiographer. The variance image also uncovers
a line of dead pixels at the bottom left of the image. Extracts from the greyscale image are
presented to show the bubble-like appearance of the greyscale x-ray image in the fault region.
The variance image is now included as part of the routine physics QA measurements for the
FFDM systems at this site.

Signal transfer property

Pixel value was plotted against air kerma at the detector to give the STP shown in figure 3.
Coefficient of variation for the air kerma at the detector in this measurement was approximately
0.2%. While there is a linear response between PV and air kerma at the receptor, there is
clearly an offset to the STP. The offset is added at calibration by the engineers and is used to
avoid negative pixel values in the image.

Modulation transfer function

Pre-sampling MTFs calculated using median filter lengths of 5 and 7 pixels showed a slight
reduction at high spatial frequencies for the 7 pixel filter and therefore a median filter of
just 5 pixels was applied when calculating MTF in this study. Figure 4(a) shows the pre-
sampling MTF in the data line and gate line directions, along with the pixel aperture MTF for
a pixel size of 0.07 mm. Results for both directions are similar, indicating that pre-sampling
MTF is fairly isotropic. All three curves touch the x-axis at the same value of approximately
14 mm−1—this is an indication that the actual pixel size is close to the nominal pixel size of
0.07 mm (a first zero of 14.3 mm−1 is expected for a 0.07 mm pixel). The limit caused by the
aperture response can be higher if the dimension of the sensitive region of the pixel (detector
element) is smaller than the nominal value (Rowlands 2000). The measured pre-sampling
MTF is some way below the limit set by the pixel sampling aperture. Yorker et al (2002) and
Zhao et al (2003) show that re-absorption of K-shell fluorescence x-rays at a position remote
from the initial interaction is a source of blurring, leading to a reduction in MTF compared to
2452 N W Marshall

greyscale extract

greyscale extract

line of dead pixels

Figure 2. Variance image of a 4 cm PMMA block acquired at 28 kV, Mo/Mo and 60 mAs clearly
demonstrating two regions of severe artefact. The extracted regions show greyscale images of the
artefact. The variance image also highlights a line of dead pixels.

the ideal aperture. Zhao et al (2003) also suggest that charge trapping in the blocking layer
just above the pixel electrodes may also be a source of MTF degradation.
Plotted in figure 4(b) are data taken from Yorker et al (2002) for an earlier version of this
detector. The measured pre-sampling MTF in this study is somewhat lower at all frequencies
compared to the result of Yorker et al (2002). Factors that could account for such a difference
include changes in design parameters, such as the voltage across the a-Se layer. This figure
also presents the pre-sampling MTF given by Zhao et al (2003) for a 200 µm thick a-Se
detector with an 85 µm pixel size. For comparison with a typical screen-film detector, the
MTF of a Min-R 2000 mammography screen-film system (Bunch 1999) is plotted. The MTF
A comparison between objective and subjective image quality measurements for a FFDM system 2453

1600

grid out 28 kV, Mo Mo 4 cm

1400 PMMA at tube

1200

1000
Pixel Value

800

600
y = 3.58x + 203
400

200

0
0 50 100 150 200 250 300 350
air kerma at detector (µGy)

Figure 3. Signal transfer property for the a-Se detector.

1.0
1.0
data line (this study)
0.9 data line 0.9
gate line MinR-2000
aperture 0.8 Y orker et al
0.8
Zhao et al data line
0.7 0.7

0.6 0.6
MTF
MTF

0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2

0.1 0.1

0.0 0.0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

spatial frequency (mm-1) spatial f requency (mm-1)

(a) (b)

Figure 4. (a) MTF measured in the data line (parallel to the chest wall edge) and gate line directions.
Also plotted is the response for a sinc function of 0.07 mm (the pixel size for this detector).
(b) MTF result in data line direction compared against a-Se digital mammography detector data
from Yorker et al (2002) (14.3 mm−1 sampling limit) and Zhao et al (2003) (11.8 mm−1 sampling
limit). Also shown is the MTF for a Min-R 2000 mammography film/screen system taken from
Bunch (1999).

of the much thicker a-Se layer (200 µm versus approx 80 µm for Min-R phosphor screen)
is surprisingly close to screen-film MTF and is due to electric field applied to the a-Se layer
stopping lateral charge spread. The screen-film MTF is superior to that of the a-Se layer at
spatial frequencies greater than 12 cycles mm−1. The 70 µm pixel size in the Selenia system
2454 N W Marshall

1.0E-04

1.0E-05
NNPS(u) (mm2)

1.0E-06

9.1 µGy 21.0 µGy 42.3 µGy

82.5 µGy 165.3 µGy 330.9 µGy
gate line

1.0E-07
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0
spatial frequency (mm-1)

Figure 5. Normalized NPS measured in the data line (symbols) and gate line directions (solid
lines) for a range of detector air-kerma settings.

means that only frequencies up to the sampling limit of 7.1 cycles mm−1 can be displayed in
the digital images.

Noise power spectrum

Figure 5 shows NPS in the data line and gate line directions at detector air-kerma values of
9.1, 21.0, 42.3, 82.5 165 and 331 µGy. A detailed exploration of the noise transfer properties
of integrated x-ray detectors (such as that given by Siewardsen et al (1997), Evans et al (2002)
and Zhao et al (2003)) is beyond the scope of the current work—just a few salient points will
be discussed.
The NPS has similar magnitude in both the gate line and data line directions, indicating
good isotropy (as might be expected from the MTF results). In the data line direction, NPS
at the sampling limit has fallen to approximately 60% of the NPS(0) value, while in the gate
line direction, NPS at the sampling limit is roughly 40% of the NPS(0) value. These results
are close to the values given by Zhao et al (2003) for an a-Se detector.
As this is a digital detector, the measured noise power spectrum will suffer from aliasing
because the noise data have been sampled at discrete intervals by the pixel matrix (Giger et al
1984). Williams et al (1999) discuss aliasing of the NPS for digital mammography systems;
detectors that produce images with significant spatial frequency content above the sampling
limit prior to sampling will suffer from aliasing. Figure 4(a) shows that the pre-sampling MTF
of the a-Se layer has fallen to only 50% at 7.1 mm−1 and hence some aliasing of the NPS
for this detector is expected. Ultimately, aliasing leads to an increase in noise power at high
spatial frequencies.
A comparison between objective and subjective image quality measurements for a FFDM system 2455

1.00

0.90 9.1 µGy 21.0 µGy 42.3 µGy

82.5 µGy 165.3 µGy 330.9 µGy
0.80

0.70

0.60
DQE(u)

0.50

0.40

0.30

0.20

0.10

0.00
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0
spatial f requency (mm-1 )

Figure 6. DQE in the data line direction measured for a range of detector air-kerma values. X-ray
spectrum was 28 kV, Mo/Mo target filter and 4 cm PMMA added filtration.

Detective quantum efficiency

Figure 6 present DQE(u) measured for the data line direction at detector air-kerma values of
9.1, 21.0, 42.3, 82.5 165 and 331 µGy. The curves have a similar shape, regardless of the
air kerma at which they were measured, dropping by a factor of approximately 3 as spatial
frequency changes from 0 mm−1 to 7.1 mm−1. It is clear that DQE is dependent on air
kerma to the detector. DQE increases for all spatial frequencies as air kerma at the detector
increases, until reaching a plateau above approximately 50 µGy. At low spatial frequency,
DQE increases from approximately 0.3 at 9.1 µGy to approximately 0.6 for detector air-kerma
values of 50 µGy and above. The reduction in DQE at low detector air kerma is due to the
presence of additive noise from the input stages of charge amplifier read out circuitry (Yorker
et al 2002). Detector air kerma will be low for regions of the breast that have low x-ray
transmission (probably corresponding to areas of highly glandular or dense tissue). In these
regions, it is likely that DQE will be reduced and this may impact on the ability of the system
to detect clinical details in these areas.
Absolute magnitude of the DQE for air-kerma values above 50 µGy is roughly 0.6 at
low spatial frequencies—in good agreement with data published by Yorker et al (2002) and
Zhao et al (2003) for similar detectors. DQE for this a-Se detector is therefore considerably
greater than the figure of 0.35 (at low frequencies) for a well-established mammography
screen/film (S/F) system such as Kodak Min-R 2000 (Bunch 1999).The advantage of the
S/F system is that DQE extends above the 7 mm−1 limit imposed by the sampling matrix
for the Selenia system (albeit at the low value of 0.1). As it is thought that detectability
for a given task is related to the DQE (ICRU 1996), we should expect improved detection
performance compared to S/F detectors (certainly for well-defined signal known exactly,
background known exactly (SKE/BKE) tasks). It should also be noted that DQE is calculated
using the pre-sampling MTF in combination with the digital (sampled) NPS (equation (4)).
As noted above, aliasing results in an increase in the high frequency NPS and hence DQE
calculated in this manner may be an underestimate of the true DQE at high frequencies.
2456 N W Marshall

100.0

10.0

(%) (%)
contrast
10.0
contrast
threshold
threshold

1.0
9.1 µGy
1.0
theory
11.9 µGy
µGy
42.3 theory
µGyµGy
330.941.0

0.1 342 µGy

0.10.01 0.10 1.00 10.00
0.01 0.10 1.00 10.00
detail diameter (mm)
detail diameter (mm)

Figure 7. Observer c-d results at three detector air-kerma settings (symbols) and c-d curves
calculated according to the Rose theory.

Table 2. DQE(0) found by extrapolating DQE(u) back to the y-axis.

Air kerma at detector (µGy) DQE(0)

9.1 0.29
21.0 0.43
42.3 0.54
82.5 0.61
165 0.62
331 0.59

Contrast-detail results and the Rose model

The observer c-d results are shown in figure 7; for reasons of clarity, only data for detector
air-kerma values of 11.9, 41.0 and 342 µGy are plotted out of the six sets of data available.
Note that for the 342 µGy results, all the discs present in the row for diameters of 1.0 mm
and above were seen and these threshold contrast points have also been excluded from the
plot. The errors bars indicate an uncertainty of 15% (Cohen et al 1984), estimated for two
observers reading two images once
Also plotted are the expected c-d curves calculated from Rose theory (equation (12)) at
air-kerma values of 11.9, 41.0 and 342 µGy. Two assumptions were made regarding DQE
for this calculation. The first is a practical point—we are using DQE results obtained at
slightly different detector air-kerma values (9.1, 42.3 and 331 µGy) for the calculation. DQE
is changing quite quickly for the low air-kerma results (11.9 µGy) and this 2 µGy difference
will introduce some error. Above 50 µGy at the detector, DQE reaches a plateau and changes
more slowly with air kerma and virtually no error is introduced by this assumption. Second,
in order to find DQE(0), a first-order linear extrapolation to DQE(u) was used (table 2). This
is because the measured value of DQE is often affected by low frequency artefacts present in
the image (Dobbins 2000).
In order to find a value for k, this equation was fitted to the observer threshold contrast
results for the centre set of air-kerma results (42.3 µGy), for detail sizes ranging from 2.0 mm
A comparison between objective and subjective image quality measurements for a FFDM system 2457

100.0 100.0
0.80 mm
limiting DQE=0.6
CTR

10.0 10.0

Ct (%)
Ct (%)

0.20 mm
1.0 1.0
limiting DQE=0.6
CTR

0.1 0.1
1.0 10.0 100.0 1000.0 1.0 10.0 100.0 1000.0

air kerma at detector (µGy) air kerma at detector (µGy)

(a) (b)

100.0

10.0
Ct (%)

1.0
0.13 mm
limiting DQE=0.6
CTR

0.1
1.0 10.0 100.0 1000.0
air kerma at detector (µGy)
(c)

Figure 8. Comparison of observer threshold contrast results (squares) and Rose threshold contrast
calculated using DQE(0) (circles) for (a) 0.80 mm, (b) 0.20 mm and (c) 0.13 mm diameter discs.
Dotted line shows result for ‘limiting’ DQE of 0.60.

down to 0.13 mm. This gave a value of k = 2.56 and therefore an approximate value of 2.5
was used for k for all calculations of CTR and CTS in this paper.
The model gives results within approximately 15% for the c-d curve with changing
detector air kerma—the spread between the low and high air-kerma results simply reflects the
changes in quantum noise in the image. The CDMAM test object is sensitive enough (in terms
of percentage contrast step between consecutive discs) to follow these changes in quantum
mottle.
Calculated and measured results were also within approximately 15% for disc diameters
down to 0.2 mm, however there is considerable discrepancy for diameters less than 0.20 mm.
This is examined more closely in figure 8, where the threshold contrast data for a particular
detail diameter have been plotted as a function of detector air kerma. In order to smooth the
measured c-d data, a second-order least-squares fit was made to all the c-d curves. Data plotted
in figure 8 are contrasts taken from the fitted curves rather than the raw threshold contrast result.
The dotted curve in figure 8(a) is the Rose threshold contrast (CTR) calculated for
DQE = 0.6 (we are using this as the approximate limiting value of the DQE) and 0.8 mm
2458 N W Marshall

100.0

threshold contrast (%) 10.0

1.0

11.9 µGy
41.0 µGy
342 µGy
theory
0.1
0.01 0.10 1.00 10.00
diameter (mm)

Figure 9. Observer c-d results at three detector air-kerma settings (symbols) and c-d curves
calculated using signal detection model (equation (11)) with observer visual response (O(u, v)) =
1 and system magnification m = 1.

detail diameter. The measured threshold contrast is well modelled by the theory of Rose
using DQE(0) with an average difference between measurement and theory of 10%. At the
two lowest air-kerma values studied, the DQE drops due to the presence of detector amplifier
noise and this clearly results in an increase in threshold contrast. As air kerma increases,
measured and calculated threshold contrast approach the limiting value determined by DQE
of 0.6. Figure 8(b) shows that similar results are found for the 0.2 mm diameter discs.
At the smallest disc diameters (0.13 mm and 0.10 mm), however, the difference between
the measured result and the contrast calculated using DQE(0) is approximately 40% (see
figure 8(c)). One possible reason for this is that the observer visual response (not included
in equation (6)) is starting to limit threshold contrast. There will be some influence of the
visual response (especially for small detail sizes) on the measured c-d results but the use of
carefully controlled viewing conditions and the software magnification function on the SCW
should keep this to minimum. Another reason might be that detection of such small details
uses a higher spatial frequency region of the DQE (ICRU 1996).
Threshold contrast follows the expected Rose relationship, even for high air-kerma values
at the detector. A previous study (Marshall et al 2001) has discussed the influence of fixed
pattern noise in the form of phosphor structure mottle (Kume et al 1986) on native digital
fluorography images. This can limit the threshold contrast resolution of an x-ray system at
high detector air-kerma values. It is clear that is not the case for the a-Se detector in this
system; as expected the a-Se x-ray converter does not suffer from the phosphor granularity
noise that can limit contrast resolution. Low frequency DQE remains close to 0.6, even at
high detector air-kerma values.

Contrast-detail results and signal detection theory model

Threshold contrast (CTS) was calculated using equation (11) for detector air-kerma values of
9.1, 42.3 and 331 µGy. Although this equation uses the 2D forms of MTF, NPS and DQE, the
A comparison between objective and subjective image quality measurements for a FFDM system 2459

100.0 100.0
0.80 mm
Ro se C% limiting DQE=0.6
signal detectio n theo ry
threshold contrast (%)

threshold contrast (%)

10.0 10.0

1.0 1.0

0.20 mm
Ro se C% limiting DQE=0.6
signal detectio n theo ry

0.1 0.1
1.0 10.0 100.0 1000.0 1.0 10.0 100.0 1000.0
air kerma at detector (µGy) air kerma at detector (µGy)
(a) (b)

100.0
threshold contrast (%)

10.0

1.0

0.13 mm
Ro se C% limiting DQE=0.6
signal detectio n theo ry

0.1
1.0 10.0 100.0 1000.0
air kerma at detector (µGy)
(c)

Figure 10. Comparison of observer threshold contrast data (squares) and signal detection model
(triangles) for (a) 0.80 mm, (b) 0.20 mm and (c) 0.13 mm diameter discs. Dotted line shows result
for ‘limiting’ DQE of 0.60. Signal detection results were calculated with observer visual response
(O(u, v)) = 1 and system magnification m = 1.

results demonstrated reasonably good isotropy and therefore just the results from the data line
direction were used to simplify the calculations. When calculating threshold contrast we set
system magnification m = 1 and O(u) = 1, assuming initially that observers could optimize
their viewing conditions and that the effect of the display/perception would be negligible.
A similar approach was adopted by Workman and Cowen (1993) for computed radiography
systems. Figure 9 plots the observed c-d data along with CTS calculated from equation (11)
with an observer spatial frequency response O(u) = 1, m = 1 and k = 2.5. Again, only
data for detector air-kerma values of 9.1, 41.0 and 331 µGy are plotted. Two points can be
made here. First, the calculated curves follow the measured data closely as air kerma at the
detector is changed. Second, the full spatial frequency range DQE is used in the calculations
and therefore somewhat better agreement is seen at small disc sizes compared to the use of
DQE(0) in the Rose theory calculations. For example, the difference between measured and
2460 N W Marshall

100.0

threshold contrast (%) 10.0

1.0

11.9 µGy
41.0 µGy
342 µGy
theory
0.1
0.01 0.10 1.00 10.00
diameter (mm)

Figure 11. Comparison of observer c-d data (symbols) and signal detection model calculated
with system magnification m = 1.38, typical observer visual response function O(u, v) taken from
Kelly (1979) and a viewing distance of 25 cm.

calculated data for the 0.13 mm disc at 342 µGy for the detection theory calculation is 26%
compared to a difference of 44% for the Rose theory result.
Figure 10 examines signal detection model results as a function of air kerma at the detector.
Figures 10(a) and (b) show the measured threshold contrast data along with calculated contrast
CTS plotted as a function of air kerma at the detector for 0.8 mm and 0.2 mm discs. For these
diameters, the difference between measured and calculated threshold contrasts was 8% on
average. Figure 10(c) plots the results for 0.13 mm diameter discs where the difference
between measured and calculated results is 20%. There are several possible reasons for this
discrepancy. For example, the MTF and NPS data were acquired under a low scatter geometry
while scattered radiation was present at the c-d acquisition stage. It is possible that the
observers are using a different detection threshold for the smallest discs. Another possibility
is that the observer’s visual system spatial frequency response is starting to influence the
results. We should also consider the possibility that internal visual system noise may be
affecting observer performance (Ishida et al 1984, Abbey and Bochud 2000) for detail sizes
less than 0.2 mm.
Finally, CTS was calculated using a spatial frequency response curve for a typical human
observer (Kelly 1979), with a viewing distance of 25 cm and a magnification factor m of 1.38
(magnification for an image displayed at the default size)—see figure 11. For these conditions,
the observer spatial frequency response used increased from 0.14 at 0.1 mm−1 to a value of
1.0 at 0.67 mm−1 and then fell to 1.70 × 10−4 at 5 mm−1. This gives the characteristic ‘bow’
often seen in c-d curves, especially for larger detail diameters. Agreement is now closer
at the smallest disc diameters, however, it should be noted that the observed c-d data were
acquired using the software magnification function and hence m should be 2.3. Setting m = 2.3
in equation (11) gives values for CTS that are too low at small diameters. There was no
significant difference (within experimental error) in the observed threshold contrast data
when measured with and without the software magnification tool on this system. Further
modification of equation (11) appears necessary in order to correctly model the effect of the
A comparison between objective and subjective image quality measurements for a FFDM system 2461

observer visual system and system display magnification on these c-d curves. These results
highlight the difficulty in using objective measurements such as NPS to model/predict absolute
values, even for the simple c-d task. The objective measurements are an evaluation of detector
performance alone, while the observer threshold contrast results include effects of observer
detectability, observer visual response, display magnification and processing, ambient viewing
conditions including display reflection and scattered radiation/grid performance

Conclusions

This study used objective and subjective measures to evaluate the image quality of an a-Se
digital mammography system. The objective analysis began with the generation of variance
images from flat field images. These images were useful in obtaining a visual impression
of the variance across the entire detector before proceeding to perform NPS analysis on the
images. The variance image also clearly identified a severe detector artefact and a line of
dead pixels not included in the system dead pixel map. Results from variance images were
then used to generate SNR images which showed the varying x-ray SNR across image due
to the heel effect, as would be expected for a typical quantum noise dominated x-ray system.
This variation in x-ray SNR should be taken into account when deriving performance limits
in testing protocols.
Both pre-sampling MTF and axially averaged NPS were reasonably isotropic. At the
sampling limit (7.1 mm−1), the NPS fell to approximately 60% and 40% of the NPS(0) value,
for the data line and gate line directions, respectively. DQE calculated from these parameters
was approximately 0.6 at low spatial frequencies. DQE was also found to change with air
kerma at the image receptor, increasing from 0.3 (at low spatial frequencies) at 9.1 µGy to
approximately 0.6 for air-kerma settings of 50 µGy and above.
Subjective image quality results in the form of c-d measurements were then obtained
at the same detector air-kerma settings used for the objective analysis. The measured c-d
results reflected the reduction in DQE found at the low detector air-kerma setting; c-d data
also followed the changes in detector air kerma/image even for the relatively high air kerma of
342 µGy/image. Two models (the Rose model and a version of signal detection theory) were
used to examine the measured c-d data. The Rose model (using DQE(0)) gave results within
approximately 15% for all the detector air-kerma values studied and for detail diameters down
to 0.2 mm. Similar agreement (approximately 15%) was also found between the measured
c-d data and signal detection theory results, which were calculated with an ideal human visual
response function and a system magnification of unity. Using the full spatial frequency DQE
improved the agreement between the calculated and observer results for detail sizes below
0.13 mm.
This study has demonstrated close correspondence, in functional terms, between objective
detector image quality parameters calculated directly from image data and subjective (contrast-
detail) image quality results. As QA physicists gain experience with objective image quality
analysis over a wider range of digital mammography systems, standards can be formed using
these objective parameters. Eventually, we may hope to perform quality assurance of the x-ray
detector using measured image quality data rather than a carefully controlled, but ultimately
limited subjective task such as the contrast-detail method.

Acknowledgment

I would like to thank Dr Julie Horrocks for her help in preparing this work.
2462 N W Marshall

References

Albert M and Maidment A D 2000 Linear response theory for detectors consisting of discrete arrays Med. Phys.
27 2417–34
Aufrichtig R 1999 Comparison of low contrast detectability between a digital amorphous silicon and a screen–film
based imaging system for thoracic radiography Med. Phys. 26 1349–58
Burgess A E 1999 The Rose model, revisited J. Opt. Soc. Am. A 16 633–46
Bunch P C 1999 Advances in high-speed mammographic image quality Proc. SPIE 3659 120–30
CEP Report 05084 2006 Full Field Digital Mammography Systems Hologic Selenia: A Technical Report (London:
HMSO)
Cohen G, McDaniel D L and Wagner L K 1984 Analysis of variations in contrast-detail experiments Med. Phys.
11 469–73
Cowen A R, Brettle D S, Coleman N J and Parkin G J 1992 A preliminary investigation of the imaging performance
of photostimulable phosphor computed radiography using a new design of mammographic quality control test
object Br. J. Radiol. 65 528–35
Cowen A R, Haywood J M, Workman A and Clarke O F 1987 A set of X-ray test objects for image quality control in
digital subtraction fluorography: I. Design considerations Br. J. Radiol 60 1001–9
Cranley K, Gilmore B J, Fogarty G W A and Desponds L 1997 Catalogue of diagnostic x-ray spectra and other data
IPEM Report vol 78 (York: Institute of Physics and Engineering in Medicine)
Cunningham I A 2000 Applied linear systems theory Physics and Psychophysics Vol 1 of Handbook of Medical
Imaging ed J Beutel, H L Kundel and R L van Metter (Bellingham: SPIE) pp 79–159
Cunningham I A and Shaw R 1999 Signal-to-noise optimization of medical imaging systems J. Opt. Soc. Am. A 16
1–13
Dainty J C and Shaw R 1974 Image Science—Principles, Analysis and Evaluation of Photographic-Type Imaging
Processes (London: Academic)
Dance D R and Day G J 1984 The computation of scatter in mammography by Monte Carlo methods Phys. Med.
Biol. 29 237–47
Dobbins J T III 2000 Image quality metrics for digital systems Physics and Psychophysics Vol 11 of Handbook of
Medical Imaging ed J Beutel, H L Kundel and R L van Metter (Bellingham: SPIE) pp 161–222
Dobbins J T III, Ergun D L, Rutz L, Hinshaw D A, Blume H and Clark D C 1995 DQE(f) of four generations of
computed radiography acquisition devices Med. Phys. 22 1581–93
EUREF Digital Addendum 2003 Addendum on Digital Mammography to Chapter 3 of the European Guidelines for
Quality Assurance in Mammography Screening (Version 1.0, November 2003) (Nijmegen, the Netherlands:
European Reference Organisation for Quality Assured Breast Screening and Diagnostic Services (EUREF))
Evans D S, Workman A and Payne M 2002 A comparison of the imaging properties of CCD-based devices used for
small field digital mammography Phys. Med. Biol. 47 117–35
Flynn M J and Samei E 1999 Experimental comparison of noise and resolution for 2 k and 4 k storage phosphor
radiography systems Med. Phys. 26 1612–23
Fujita H, Tsai D-Y, Itch T, Doi K, Morishita J, Ueda K and Ohtsuka A 1992 A simple method for determining the
modulation transfer function in digital radiography IEEE Trans. Med. Imaging 11 34–39
Giger M L, Doi K and Metz C E 1984 Investigation of basic imaging properties in digital radiography: 2. Noise
Wiener spectrum Med. Phys. 11 797–805
Hanson K M 1983 Variations in task and the ideal observer Proc. Soc. Photo-Opt. Instrum. Eng. 419 60–67
Hay G A, Clarke O F, Coleman N J and Cowen A R 1985 A set of X-ray test objects for quality control in television
fluoroscopy Br. J. Radiol. 58 335–44
Hillen W, Schiebel U and Zaengel T 1987 Imaging performance of a digital storage phosphor system Med. Phys.
14 744–51
ICRU 1996 Medical imaging—the assessment of image quality ICRU Report 54 (Bethesda, MD: International
Commission on Radiation Units and Measurements (ICRU))
IPEM 2005 Commissioning and routine testing of mammographic X-ray systems IPEM Report 89 ed A C Moore,
D R Dance, D S Evans, C P Lawinski, E M Pitcher, A Rust and K C Young (York: IPEM)
Ishida M, Doi K, Loo L-N, Metz C E and Lehr J L 1984 Digital image processing: effect on detectability of simulated
low-contrast radiographic patterns Radiology 150 569–75
Judy P F 1976 The line spread function and modulation transfer function of a computed tomographic scanner Med.
Phys. 3 233–6
Kelly D H 1979 Motion and vision II stabilized spatio-temporal threshold surface J. Opt. Soc. Am. 69 1340–9
Kume Y, Doi K, Ohara K and Giger M L 1986 Investigation of basic imaging properties in digital radiography: 10.
Structure mottle of II-TV digital imaging systems Med. Phys. 13 843–49
A comparison between objective and subjective image quality measurements for a FFDM system 2463

Maidment A D and Albert M 2003 Conditioning data for calculation of the modulation transfer function Med.
Phys. 30 248–53
Maidment A D A, Albert M, Bunch P C, Cunningham I A, Dobbins J T, Gagne R M, Nishikawa R M, Wagner R F
and Van Metter R L 2003 Standardization of NPS measurement: Interim Report of AAPM TG16 Proc.
SPIE 5030 523–32
Marshall N W, Kotre C J, Robson K J and Lecomber A R 2001 Receptor dose in digital fluorography: a comparison
between theory and practice Phys. Med. Biol. 46 1283–96
Metz C E, Wagner R F, Doi K, Brown D G, Nishikawa R M and Myers K J 1995 Toward consensus on quantitative
assessment of medical imaging systems Med. Phys. 22 1057–61
Moran P R 1982 A physical statistics theory for detectability of target signals in noisy images: I. Mathematical
background, empirical review and development of theory Med. Phys. 9 401–13
Rogge F, Bosmans H, Willems P and Marchal G 2003 Use of MTF calculation in global and local resolution
assessment in digital mammography Proc. SPIE 5030 896–907
Rose A 1948 The sensitivity performance of the human eye on an absolute scale J. Opt. Soc. Am. 38 196–208
Rowlands J A and Yorkston J 2000 Flat panel detectors for digital radiography Physics and Psychophysics Vol 1 of
Handbook of Medical Imaging ed J Beutel, H L Kundel and R L van Metter (Bellingham: SPIE) pp 223–328
Samei E and Flynn M J 2003 An experimental comparison of detector performance for direct and indirect digital
radiography systems Med. Phys. 30 608–22
Samei E, Flynn M J and Reimann D A 1998 A method for measuring the presampled MTF of digital radiographic
systems using an edge test device Med. Phys. 25 102–13
Siewerdsen J H and Antonuk L E 1998 DQE and system optimization for indirect-detection flat-panel imagers in
diagnostic radiology Proc. SPIE 3336 546–55
Siewerdsen J H, Antonuk L E, el-Mohri Y, Yorkston J, Huang W, Boudry J M and Cunningham I A 1997 Empirical
and theoretical investigation of the noise performance of indirect detection, active matrix flat-panel imagers
(AMFPIs) for diagnostic radiology Med. Phys. 24 71–89
Schnitzler A D 1973 Image-detector model and parameters of the human visual system J. Opt. Soc. Am 63 1357–68
Sones R A and Barnes G T 1984 A method to measure the MTF of digital x-ray systems Med. Phys. 11 166–71
Stierstorfer K and Spahn M 1999 Self-normalizing method to measure the detective quantum efficiency of a wide
range of x-ray detectors Med. Phys. 26 1312–9
Wagner R F and Brown D G 1985 Unified SNR analysis of medical imaging systems Phys. Med. Biol. 30 489–518
Williams M B, Mangiafico P A and Simoni P U 1999 Noise power spectra of images from digital mammography
detectors Med. Phys. 26 1279–93
Workman A and Cowen A R 1993 Signal, noise and SNR transfer properties of computed radiography Phys. Med. Biol.
38 1789–808
Yorker J G, Jeromin L S, Lee D L Y, Palecki E F, Golden K P and Jing Z 2002 Characterization of a full field digital
mammography detector based on direct x-ray conversion in selenium Proc. SPIE 4682 21–9
Zhao W, Ji W G, Debrie A and Rowlands J A 2003 Imaging performance of amorphous selenium based flat-panel
detectors for digital mammography: characterization of a small area prototype detector Med. Phys. 30 254–63