IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 23, NO.

9, SEPTEMBER 2004

1141

Registration of Real-Time 3-D Ultrasound Images of the Heart for Novel 3-D Stress Echocardiography
Raj Shekhar*, Member, IEEE, Vladimir Zagrodsky, Member, IEEE, Mario J. Garcia, and James D. Thomas
AbstractStress echocardiography is a routinely used clinical procedure to diagnose cardiac dysfunction by comparing wall motion information in prestress and poststress ultrasound images. Incomplete data, complicated imaging protocols and misaligned prestress and poststress views, however, are known limitations of conventional stress echocardiography. We discuss how the rst two limitations are overcome via the use of real-time three-dimensional (3-D) ultrasound imaging, an emerging modality, and have called the new procedure 3-D stress echocardiography. We also show that the problem of misaligned views can be solved by registration of prestress and poststress 3-D image sequences. Such images are misaligned because of variations in placing the ultrasound transducer and stress-induced anatomical changes. We have developed a technique to temporally align 3-D images of the two sequences rst and then to spatially register them to rectify probe placement error while preserving the stress-induced changes. The 3-D spatial registration is mutual information-based. Image registration used in conjunction with 3-D stress echocardiography can potentially improve the diagnostic accuracy of stress testing. Index TermsImage registration, mutual information, stress echocardiography, three-dimensional ultrasound imaging.

I. INTRODUCTION

TRESS echocardiography is a clinical procedure routinely used to diagnose myocardial ischemia. Early diagnosis of myocardial ischemia may save many lives by allowing narrowed coronary vessels to be reopened before the myocardium becomes irreversibly scarred. Stress echocardiography measures the hearts response to exercise or other forms of stress by comparing resting and active phases of the wall of the left ventricle (LV) with conventional two-dimensional (2-D) ultrasound images. A normal heart becomes hyperkinetic upon exercise; myocardial ischemia manifests itself as hypokinesis or akinesis [1]. Despite its frequent clinical utilization, conventional stress echocardiography has some fundamental limitations. Because stress-induced wall motion abnormalities are both transitory and time-dependent, the time window for acquiring images of the heart when under stress is extremely narrow (on the order of 1

Manuscript received September 3, 2003; revised April 19, 2004. This work was supported by the Whitaker Foundation research grant RG-01-0071. The Associate Editor resposible for coordinating the review of this paper and recommending its publication was J. S. Duncan. Asterisk indicates corresponding author. *R. Shekhar is with the Department of Biomedical Engineering (ND20), Lerner Research Institute, The Cleveland Clinic Foundation, 9500 Euclid Avenue, Cleveland OH 44195 USA (e-mail: shekhar@bme.ri.ccf.org). V. Zagrodsky is with the Department of Biomedical Engineering, Lerner Research Institute of The Cleveland Clinic Foundation, Cleveland OH 44195 USA. M. J. Garcia and J. D. Thomas are with the Department of Cardiovascular Medicine of The Cleveland Clinic foundation, Cleveland OH 44195 USA. Digital Object Identier 10.1109/TMI.2004.830527

min). A tradeoff then exists when using conventional 2-D ultrasound in conjunction with stress testing. One must acquire several cine loops (2-D image sequences spanning a complete cardiac cycle) from different physical locations to view as much of the heart as possible; doing so, however, degrades the accuracy of wall motion information because the individual cine loops often do not reect the same state of the stressed heart, and some loops may even fail to capture any abnormal wall motion. In practice, no more than three or four cine loops can be acquired in the time available. A second limitation of conventional stress echocardiography is that the 2-D views captured before and after stress may not correspond to the same cross-section of the heart. This limitation arises because of the sonographers inability to duplicate the location and orientation of the ultrasound probe between imaging sessions during a procedure that, by design, involves signicant patient movement. Clinically, the term foreshortening is used to describe this phenomenon, which causes the LV to appear shrunken longitudinally in the poststress image in comparison to the prestress orientation. It is important that the identical regions of the LV are compared prestress and poststress for an accurate assessment of wall motion abnormality. Real-time three-dimensional (RT3D) imaging [2][4], a new development in ultrasound image acquisition, could address the aforementioned limitations of conventional stress echocardiography and thus improve its diagnostic power. RT3D Ultrasound, for the rst time, has made it possible to image a beating LV in its entirety without gating, thus reducing image acquisition duration to the time of a single cardiac cycle (approximately 1-s or less). It is, therefore, natural to think of performing stress testing in conjunction with RT3D ultrasound a combination we call three-dimensional (3-D) stress echocardiography. This procedure addresses the rst limitation through rapid volumetric acquisition. Complete wall motion data for every part of the LV for a given stress level is gathered by RT3D ultrasound. Moreover, this procedure has the ability to capture any transient wall motion activity that may not be captured by all cine loops acquired at different times in conventional stress echocardiography. RT3D Ultrasound thus has the potential to collect complete and accurate wall motion information. Three-dimensional Stress echocardiography can also address the limitation of misaligned image planes. By registering prestress and poststress volumetric images retrospectively, the same anatomical plane from the prestress and poststress data can be presented to a physician for making a diagnosis. Conventional stress echocardiography cannot provide this capability because it does not collect enough data to permit image registration. Furthermore, as we have shown, the availability

0278-0062/04$20.00 2004 IEEE

1142

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 23, NO. 9, SEPTEMBER 2004

of volumetric data in 3-D stress echocardiography allows visualization of any chosen cross-section of the LV [5] and, therefore, a more comprehensive wall motion analysis than is currently possible. Our focus here is registration of prestress and poststress RT3D ultrasound image sequences. The specic issues unique to this effort are the temporal alignment of the frames (which are always 3-D in the context of RT3D ultrasound) of the two sequences and the rectication of ultrasound transducer probe placement error in the presence of stress-induced anatomical changes.

II. PRIOR WORK AND CURRENT REQUIREMENTS Relatively few investigations involving ultrasound image registration have been reported [6]. Reported cardiac ultrasound image registration studies are even fewer in number. A recent review of cardiac image registration methods [7] noted no work using ultrasound images, although Ledesma-Carbayo et al. [8] have since then reported estimating cardiac motion from 2-D echocardiography by frame-to-frame registration. The characteristically low signal-to-noise ratio of ultrasound, lack of clearly identiable features in the soft tissue organs imaged with ultrasound, and deformable nature of organs typically imaged with ultrasound have made registering ultrasound images comparatively more challenging. Because of these same factors, the external marker-based method cannot be applied to the registration of ultrasound images, and performing segmentation-based image registration is not expected to be accurate and convenient. The voxel similarity-based approach, which does not rely on external markers or identication of internal features, is a feasible, and arguably the best, paradigm for ultrasound image registration. The voxel similarity-based approach can be fully automated, which makes it particularly suitable for the cardiac application discussed here because cardiac applications like ours may require registration of 1020 3-D image pairs. Any manual step would make the overall registration task slow and tedious and the results operator-dependent, hence nonreproducible. Rohling et al. [9] have presented one of the rst examples of voxel similarity-based registration of ultrasound volumes. The measure of voxel similarity was cross-correlation. Volumetric images were reconstructed from multiple 2-D images acquired during a freehand sweep of a conventional ultrasound probe over the region of interest. Up to six volumetric images of the gall bladder taken from differing viewing angles were spatially compounded (averaged) following registration. Registration helped minimize patient-movement artifacts and errors in the creation of the 3-D images. A second example is the investigation of Meyer et al. [10], who registered 3-D ultrasound images of the breast using the mutual information measure [11][13] of voxel similarity. The objective was to register a pair of color ow and/or power Doppler images to create a difference image for serial monitoring of patients in response to chemotherapy or radiation therapy. If imaged from multiple directions, superimposition followed by registration allowed lling in of the ow

information missing in a single view. Starting with an approximate registration based on user-selected point landmarks, the investigators rened the registration using voxel similarity. Rigid-body, full afne and elastic transformations were performed and compared. The Meyer study demonstrated that mutual information-based registration could be extended to nonrigid transformation modes for registering 3-D images of deformable organs. Several examples of voxel similarity-based, nonrigid multimodality registration, with 3-D ultrasound as one of the two modalities, have been reported to assist image-guided surgery [14], [15]. A common problem in image-guided surgery is the lack of correspondence between the intraoperative anatomy and the anatomy captured by preoperative computed tomography (CT) or magnetic resonance (MR) images. A registration of intraoperative 3-D ultrasound with preoperative CT/MR can be used to update the preoperative images such that they match the intraoperative anatomy. To the best of our knowledge, we are the rst investigators to have reported registration of cardiac ultrasound volumes and any registration work involving RT3D ultrasound images [16]. We have identied a set of preprocessing steps to create a smooth mutual information surface, which allows an optimization algorithm to converge reliably on the maximum of the mutual information function and hence lead to successful registration. In extensive experiments, we have computed the capture range and accuracy of mutual information-based image registration for four transformation modes: rigid-body, rigid-body + uniform scaling, rigid-body + nonuniform scaling, and full afne. The capture range was found wide enough to permit fully automated registration in all four modes if the two volumes have roughly the same overall orientation. Furthermore, we found that subvoxel accuracy was achievable in the rigid-body and rigid-body + uniform scaling transformation modes; voxel-order accuracy was expected in the other two transformation modes. We use similar image preprocessing and registration steps in this current work. The medical imaging community has been quick to recognize the benets of using RT3D ultrasound, even though the RT3D ultrasound imaging modality is relatively new and still evolving. Ahmad et al. [17] have also recognized the potential advantages of performing stress testing with RT3D ultrasound. In recently published results from a clinical trial, these investigators compared the diagnoses made using conventional stress echocardiography and 3-D stress echocardiography. The registration of prestress and poststress ultrasound volumes, however, has not been discussed. The present image registration work holds the potential to make 3-D stress echocardiography, as reported by Ahmad et al., more accurate.

III. REGISTRATION OF PRESTRESS AND POSTSTRESS 3-D ULTRASOUND IMAGES A. Heart Motion and Temporal Alignment The heart rate (HR) between resting and active phases during stress testing is not the same. In fact, the increase in the HR is used as an indicator of the stress level. Clinically, peak stress is

SHEKHARet al.: REGISTRATION OF REAL-TIME 3-D ULTRASOUND IMAGES OF THE HEART FOR NOVEL 3-D STRESS ECHOCARDIOGRAPHY

1143

considered achieved when the HR becomes 85% of the age-predicted maximum. Since the rate of RT3D ultrasound acquisition stays xed, the number of frames in the poststress sequence is always smaller than that in the prestress sequence. The shape and size of the heart differ at different phases in the cardiac cycle. However, if the beating of the heart is regular, it is reasonable to assume that the overall pattern of shape and size variation does not change beat-to-beat. Furthermore, as the HR varies, the overall pattern remains the same; this pattern just stretches or compresses to match the new duration of the cardiac cycle. That this temporal scaling is not linear is well documented [18]. As the HR increases causing the duration of a cardiac cycle to shorten, a proportionately greater reduction in the duration of diastole than in the duration of systole takes place. This general relationship is captured by a piecewise linear temporal scaling function, as shown in Fig. 1. Given these observations, we handle heart motion by pairing prestress and poststress frames taken during the same cardiac phase, a process we call temporal alignment. Temporally aligned 3-D image pairs are then spatially registered sequentially. Temporal alignment includes determination of the inection points of the piecewise linear function and nding a matching prestress frame for each poststress frame using this temporal scaling function. Automatic interpretation of an electrocardiogram (ECG) remains a difcult problem [19], [20], so we pick the end-diastolic and end-systolic frames interactively by following the size variation of the LV cavity and examining the accompanying ECGs. Finally, we use nearest-neighbor interpolation to identify a matching prestress frame when a poststress frame does not exactly coincide with one, which typically is the case. Temporal alignment is described schematically in Fig. 1, which shows the prestress and poststress ECGs along with the locations of the original frames.

Fig. 1. Illustration of temporal alignment with the aid of prestress and poststress ECGs plotted along the horizontal and the vertical axes, respectively. A piecewise linear temporal scaling function allows phase-by-phase matching when the durations of a cardiac cycle differ. The case shown had 20 prestress and 12 poststress frames, and the matched pairs are indicated by cross marks. Due to the use of nearest-neighbor interpolation, the cross marks do not necessarily lie on the temporal scaling curve.

B. Sources and Nature of Misalignment The primary cause of misalignment in a temporally aligned frame pair is the ultrasound transducer probe placement error (called probe placement error henceforth). The probe placement error arises due to the human inability to duplicate the exact spot of the ultrasound probe on the chest and its precise angle before and after administering stress. An important observation, however, is that any error in duplicating the probe location is a 3-D translation, and any error in duplicating the probe orientation is a 3-D rotation. The net misalignment due to probe placement error therefore is a rigid-body transformation. Because stress echocardiography is not respiratory-gated due to obvious practical difculties, prestress and poststress data may originate from different breathing phases. The misalignment thus caused is believed to have a large translational component, although nonlinear misalignments may be present, especially when the apex of the heart presses against the diaphragm. A recently reported method to compensate for movement of the heart due to respiration in cardiac MRI modeled this movement as an afne transformation and showed that the nonrigid components were relatively small in comparison to translation in

the superior-interior direction [21]. Since cardiac MRI is performed over several respiratory cycles, compensating for even small nonrigid misalignment is important for achieving distortion-free 3-D images. RT3D ultrasound acquisition is faster than the respiratory rhythm and so we assumed the nonlinear misalignment to be negligible and grouped the rigid misalignment caused by possibly different breathing phases of prestress and poststress data acquisition, including the most signicant translational component, together with the probe placement error. Lack of respiratory gating, as well as variability caused by differences in probe pressure and the unavoidable factor of operator hand tremor, can be sources of additional misalignment, which may vary frame-to-frame. However, because imaging with RT3D ultrasound is extremely fast and a typical imaging session lasts less than a second, we expect these variations to be minor and therefore believe we can safely ignore them. Given the above conditions, it seems logical to register a pair of prestress and poststress 3-D images rigidly, but the process does not always produce the desired result because of the presence of stress-induced anatomical change, causing geometrical misalignment within a pair of temporally aligned 3-D images. Observing that the stress-induced anatomical change is central to making a successful diagnosis from stress echocardiography, the goal of any registration effort should be to eliminate probe placement error while preserving stress-induced anatomical change. Although the actual model of the stress-induced anatomical change can be quite complex, for the current problem we have modeled it as global (uniform) scaling. Therefore, the preferred transformation mode used for the registration problem here is the one involving rigid-body + global scaling. Note that this 7-parameter transformation can be decomposed uniquely into a rigid component and a scaling factor. A more complex nonrigid transformation would make extraction of the rigid component complicated and more challenging.

1144

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 23, NO. 9, SEPTEMBER 2004

Fig. 2. A synthetic example to illustrate our registration approach. (a), (b) Two images of an object (apple) with identical shape but different position, orientation and size; (c) the result of rigid-body registration, which is undesirable; (d) upon rigid-body + global scaling registration, the secondary object completely overlaps the primary; and (e) the result of undoing scaling following rigid-body + global scaling registration. Note that the secondary object recovers its original size, has the same orientation as the primary, and is also centered with respect to the primary.

C. Separation of Probe Placement Error and Stress-Induced Anatomical Change As stated before, the goal of our registration is to eliminate probe placement error (3-D translation and 3-D rotation) while preserving stress-induced anatomical change (global scaling). Our solution to this problem is to perform registration using the rigid-body + global scaling transformation mode. This process includes nding seven geometric transformation parameters (three translations, three rotations and one scaling factor). Both sources of misalignment contribute to the result of such a registration. Since we must preserve the stress-induced anatomical changes, we must undo the scaling performed by the above registration. A rigid-body + global scaling transformation matrix ( ) can be parameterized into seven elementary transformations using different matrix multiplication orders. The result of registration, however, is independent of the order. In the current application, we treat the overall transformation process as a rigid-body transformation ( ) followed by global scaling ( ). Although we perform rotations prior to translations, the specic order of elementary transformations to build the rigid-body transformation matrix is not germane to the present discussion. The described image registration hence transforms an arbitrary point to as follows: (1) Ideally, scaling needs to be undone in a local coordinate system whose origin coincides with the centroid of the LV in the prestress 3-D image. Undoing global scaling by premultiplying , produces the right-hand side of (1) by the inverse matrix, undesired translational misalignment error if the scaling factor is not unity and the original and local coordinates systems be the 3-D translation matrix that transforms a differ. Let given point in the original coordinate system (typically at one of the corners of the volume) to the local coordinate system. The desired transformation, therefore, is the following: (2) where is point transformed such that stress-induced anatomical change is preserved without the probe placement error. We compute the matrix assuming that the centroid of the LV in the prestress frame coincides with the center of the

image volume. Obviously, if the exact centroid of the LV is known as a result of segmentation or manual selection, then the above assumption becomes unnecessary and the exactly known transformation may be used in (2). We show the workings of our two-step registration (generalized global rescaling registration followed by unscaling) with the aid of a synthetic example in Fig. 2. Note that the rigid-body mode does not produce the desired registration here, but the two-step process does. D. Data Description and Registration Procedure Prestress and poststress data were obtained from two healthy volunteers and eight patients referred for stress echocardiography for various medical reasons. The patients were recruited on a rst-come, rst-served basis and each subject gave written informed consent. Of the eight patients, ve had normal LV wall motion, one had ischemia and abnormal wall motion in the left anterior descending coronary artery (LAD) region, one was suspected of having ischemia in the right coronary artery region, and the last subject had no ischemia but abnormal wall motion nonetheless due to an implanted pacemaker. All subjects exercised on a supine bicycle. In addition to collecting the usual 2-D images, 3-D ultrasound images were collected using a 2.5 MHz RT3D ultrasound scanner (Volumetrics, Inc., Durham, NC). The scanner produced a sequence of volumes, shaped approximately like a truncated pyramid, with 60 azimuth and elevation angular spans, at a frame rate of 25 Hz. The scan depth was kept at 140 mm for the data sets used in this study. The actual number of frames depended on the heart rate. The number of frames in the prestress and poststress sequences ranged between 1423 and 712, respectively. The volumes were median ltered using a 3 3 3 kernel, scan converted and subsequently spatially subsampled by a factor of two using a 2 2 2 uniform averaging kernel to create 64 64 256 resolution data sets with a voxel size equal to 2.2 mm 2.2 mm 0.55 mm. Subsampling was needed to reduce the computational burden and to complete registration in a reasonable time. Subsampling can be eliminated as computers become more powerful or if faster implementations [22], [23] are available. Temporal alignment, as described above, was carried out to match each poststress frame in each subject with a prestress frame with the closest cardiac phase. Upon temporal alignment, each 3-D image pair was registered using mutual information-based registration, assuming the underlying transformation mode to be rigid-body + global scaling. Briey, mutual

SHEKHARet al.: REGISTRATION OF REAL-TIME 3-D ULTRASOUND IMAGES OF THE HEART FOR NOVEL 3-D STRESS ECHOCARDIOGRAPHY

1145

information was computed using the histogram method and partial volume interpolation, and a variant of the downhill simplex method was used for optimization [24]. The details of this registration procedure have been published [16]. E. The Median Transformation The spatial registration process described above produces as many solutions as the number of frame pairs following temporal alignment. The global scaling factor varies signicantly because it represents stress-induced overall change in the size of the LV, which is expected to vary throughout the cardiac cycle in a stress study. The remaining rigid-body transformation parameters (3-D translation and 3-D rotation), according to our model, represent the probe placement error. Since the RT3D ultrasound acquisition lasts less than 1 s, the misalignment due to the probe placement error is expected not to vary among frame pairs. In practice, the misalignment varies slightly among the image pairs, which we attribute to several factors, such as the limited accuracy of registration, approximation of stress-induced anatomical changes by global scaling, breathing and perhaps the human inability to hold the ultrasound probe still even for a second. We, therefore, seek a unique set of rigid-body transformation parameters that are representative of the entire sequence, and do so by computing the multidimensional median, as described below. We dened a metric called average distance error earlier to measure the difference between transformations [16]. To compute it, the two transformations are applied to two unit cubes. The average distance error then is the average of the Euclidean distance between transformed locations of eight homologous vertices. To compute the multidimensional median, the average distance error is calculated for each pair of transformations, and the two transformations associated with the largest average distance error value are discarded. The process is repeated until one or two transformations remain, depending on whether the starting number of transformations is odd or even, respectively. If only one transformation remains, it is the median value. If two transformations remain, the median solution is the mid-point computed by decomposing the transformations using matrix logarithms [25]. F. Visualization and Validation As is the common practice, prestress and poststress views designating the same anatomical cross-section were presented side-by-side for interpretation. Upon image registration in 3-D stress echocardiography, we displayed a slice of the prestress 3-D image and next to it the corresponding slice of the poststress 3-D image, transformed rst by the median solution. Indeed, in 3-D stress echocardiography, one can interactively and arbitrarily change the anatomic cross-section to visualize and analyze the motion of any region of the LV wall [5]. Because of the lack of a gold standard, we used expert opinion to validate the image registration performed here. Digital cine loops showing the four-chamber view of the heart were created before and after registration for each of the 10 subjects. Prestress and poststress images were superimposed using different colors so that the effect of registration was discernible. Two experienced echocardiologists were asked to select the cine loop

TABLE I SUMMARY OF REGISTRATION RESULTS

Fig. 3. Time-dependence of the global scaling factor in a healthy (solid line) and a diseased (dashed line) subject. Note that in the healthy subject, the scaling factor is less than 1 during most of systole and greater than 1 during diastole, indicating a healthy hyperkinetic heart. In the diseased subject, the scaling factor hovers at the unity line throughout the cardiac cycle, indicating dysfunctional contraction of the LV.

that appeared to be better aligned between the two choices for each subject. IV. RESULTS , three rotations Three translations and a global scaling factor resulted from the registration of each temporally aligned frame pair. The translation and rotation parameters upon multidimensional median computation are presented in Table I for all 10 subjects. Larger values in a row indicate a greater starting misalignment. Smaller values indicate that the sonographer was successful to a great extent in duplicating the probe location and orientation. Such a coincidence is rare, but not unexpected. The global scaling factor, as dened here, approximates the ratio of poststress LV volume to prestress volume. The time dependence of this factor in a healthy subject and in an ischemic subject is shown in Fig. 3. Note that this factor is less than 1 during most of the systole and greater than 1 in the diastole for the healthy subject, an observation consistent with the fact that a healthy heart becomes hyperkinetic upon stress.

1146

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 23, NO. 9, SEPTEMBER 2004

Fig. 4. Side-by-side prestress and poststress long-axis views before (left column) and after (right column) registration in ve approximately equally spaced phases of the heart, each shown in a separate row. White arrows point to the inter-ventricular septum in the rst row and the free wall for the LV in the last row. The marked wall segments in the poststress views are tilted and shifted with respect to the same segments in the prestress views before registration. Note that these segments specically and the entire LV in general are better aligned following registration.

An aspect of hyperkinesis is larger stroke volume, which generally results from a larger end-diastolic volume and a smaller end-systolic volume. In the diseased subject, the scaling factor hovers at 1, indicating a heart that is not contracting when under stress as forcefully as a normal heart does. The result of image registration in 3-D stress echocardiography is shown in Fig. 4. Before and after registration views of prestress and poststress 3-D images corresponding to ve different cardiac phases are shown. This example corresponds to

the maximally misaligned case (Subject 5 in Table I) and shown is a long-axis view of the LV. A better spatial matching of the LV wall in all phases can be seen upon registration. The subject in question had an unusually narrow acoustic window, which led to an extraordinarily large rotational misalignment before and after stress. An extreme case like this illustrates the effectiveness of image registration clearly. The recovery of a relatively moderate degree of misalignment in a diseased case (Subject 3 in Table I) is shown

SHEKHARet al.: REGISTRATION OF REAL-TIME 3-D ULTRASOUND IMAGES OF THE HEART FOR NOVEL 3-D STRESS ECHOCARDIOGRAPHY

1147

Fig. 5. Three orthogonal side-by-side prestress and poststress views before (left column) and after (right column) registration in a subject diagnosed with ischemia. White arrows point to the free wall of the LV in prestress and poststress views after registration. The same arrows applied to the poststress views before registration point to noisy features, not the free wall. Despite poor denition of its walls poststress, note that the LV is better aligned following registration in all three views.

in Fig. 5. Three orthogonal views of the heart in systole are shown here. Despite the presence of wall motion abnormality, the registration is effective, as evident from the better correspondence of the LV wall before and after stress. The result of validation using expert opinion is encouraging and supports visual ndings. Both clinical experts correctly identied the registered data set in nine cases. In one case (Subject 2 in Table I), one expert correctly identied the registered data set, whereas the second called the effect of registration unnoticeable. This observation may be attributed to a small starting misalignment a maximum elementary translation of 1.6 mm and a maximum elementary rotation of 1.5 . The limits of visual inspection are reported to be 2 mm and 2 in the registration of MR and CT brain images [25]. These limits are expected to be higher here because of the poorer spatial resolution of the RT3D ultrasound used.

V. DISCUSSION Threedimensional stress echocardiography is a new application enabled by the emerging RT3D ultrasound imaging. Threedimensional stress echocardiography can offer simplied imaging protocol, shorter duration of the stress testing procedure, and, as demonstrated here, an opportunity to present to a reviewing physician prestress and poststress views showing matching anatomy. It is important for an accurate diagnosis that equivalent LV wall regions are compared, a condition

that conventional stress echocardiography often cannot meet. Additionally, the availability of volumetric images in 3-D stress echocardiography allows visualizing any region of the LV and hence performing a comprehensive wall motion analysis. Misalignment correction in stress echocardiography has not been investigated, perhaps because it could not be performed with 2-D images, and 3-D ultrasonography is a recent development. A few reported investigations come from the area of nuclear medicine. Slomka et al. [26] and Declerck et al. [27] have presented methods to align prestress and poststress 3-D singlephoton emission computed tomography perfusion images. The registration problem is simpler in stress nuclear medicine, in which poststress imaging is performed well after stress induction (40 min or more), when the heart has returned to the resting beating pattern. Geometrically, the heart appears the same in prestress and poststress images. The HR, too, is roughly the same. Clearly, these similarities do not hold true in 3-D stress echocardiography, and we have provided a method that compensates for them. Therefore, we regard our contributions as: 1) recognition of the major sources of misalignment in stress echocardiography and their modeling, and 2) development of a method to eliminate undesired misalignments while preserving the stress-induced effect. The 3-D motion and deformation of the heart are complex factors and not well understood. Alteration to these due to stress adds yet another level of complexity. Modeling the stress-induced anatomical change as a global scaling in our method is simplistic. However, we expect this model to develop further

1148

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 23, NO. 9, SEPTEMBER 2004

as our own understanding of the underlying deformation advances. A caveat here is how to separate the undesirable misalignment from the desirable ones. Modeling the stress-induced anatomic change using a more complicated transformation with many more degrees of freedom is possible, and such a transformation is likely to capture the stress-induced anatomical change more accurately. Doing so, however, complicates the problem of separating the probe placement error from the stress-induced anatomical change. It is also expected that the clinical interpretation will not change beyond a certain level of registration accuracy. Whether our method reaches that accuracy threshold will require further investigation. Assuming that the centroid of the LV is located in the center of the image is another limitation of our method, currently. To dene the centroid accurately, one needs to segment the endocardium of the LV, then substitute the assumed location of the centroid with its true location. Meanwhile, assuming that the centroid lies at the center of the 3-D image is reasonable because sonographers always attempt to center the LV within the imaged volume. Any error caused by this simplied assumption will result in only a minor translational shift; rotationally, prestress and poststress 3-D image will still be aligned. Removing rotational misalignment is more critical because even a small rotation can cause large displacements in the periphery of the image, where the LV wall appears. Overall, our method transforms the poststress 3-D image sequence rigidly using a single transformation. The transformed sequence appears as it would if the sonographer had performed imaging from a different probe location and orientation than the actual one. In this analogy also lies the logic for reducing individual registration solutions to a single solution. A single solution is equivalent to a xed probe location and orientation. Applying individual solutions, on the other hand, would additionally simulate a jitter in the probe location and orientation, the kind that may result from hand tremor an undesired effect irrespective of the origin. Because our method simply reorients the poststress 3-D image sequence, which might well have resulted naturally if the sonographer had acquired the images from a slightly different viewing angle, we regard our method as safe. The method also appears effective based on its evaluation on 10 subjects here. The spatial and temporal resolution of the Volumetrics RT3D ultrasound system we used is not as high as that of most modern 2-D ultrasound systems. The method we have proposed is independent of resolution, although more accurate and robust temporal and spatial registration is expected with 3-D images with improved image quality. Fortunately, RT3D ultrasound systems continue to improve, offering higher spatial resolution. Such advances should establish 3-D stress echocardiography as a routine clinical procedure and the method we have presented an integral part of it. In conclusion, the registration of prestress and poststress RT3D ultrasound image sequences is a difcult problem complicated by the motion and nonrigid deformation of the heart. Nonetheless, we have attempted to gain insights into the misalignment process associated with 3-D stress echocardiography and have presented a practical method to eliminate this misregistration while preserving clinically useful information. The

preliminary results of the application of our initial theoretical and algorithmic developments are encouraging, and our next step is a more rigorous clinical validation. This investigation, we believe, will instigate further cardiac image registration work as well as development of many more novel clinical applications built upon the unique strengths of the RT3D ultrasound imaging modality. ACKNOWLEDGMENT The authors acknowledge the staff of the Echocardiography Laboratory of the Department of Cardiovascular Medicine, The Cleveland Clinic Foundation, for their help with the collection of ultrasound images used in the study. REFERENCES
[1] J. K. Oh, J. B. Seward, and A. J. Tajik, The Echo Manual, 2nd ed. Philadelphia: Lippincott-Raven, 1999. [2] S. W. Smith, H. G. J. Pavy, and O. T. von Ramm, High-speed ultrasound volumetric imaging systemI: Transducer design and beam steering, IEEE Trans. Ultrason., Ferroelect., Freq. Contr., vol. 38, pp. 100108, Mar. 1991. [3] O. T. von Ramm, S. W. Smith, and H. G. J. Pavy, High-speed ultrasound volumetric imaging systemII: Parallel processing and image display, IEEE Trans. Ultrason., Ferroelect., Freq. Contr., vol. 38, pp. 109115, Mar. 1991. [4] J. A. Panza, Real-time three-dimensional echocardiography: An overview, Int. J. Cardiac Imag., vol. 17, pp. 227235, 2001. [5] R. Shekhar and V. Zagrodsky, Cine MPR: Interactive multiplanar reformatting of four-dimensional cardiac data using hardware-accelerated texture mapping, IEEE Trans. Inform. Techn. Biomed., vol. 7, pp. 384393, Dec. 2003. [6] J. B. Maintz and M. A. Viergever, A survey of medical image registration, Med. Image Anal., vol. 2, pp. 136, 1998. [7] T. Makela, P. Clarysse, O. Sipila, N. Pauna, Q. C. Pham, T. Katila, and I. E. Magnin, A review of cardiac image registration methods, IEEE Trans. Med. Imag., vol. 21, pp. 10111021, Sept. 2002. [8] M. J. Ledesma-Carbayo, J. Kybic, M. Desco, A. Santos, and M. Unser, Cardiac motion analysis from ultrasound sequences using nonrigid registration, in Medical Image Computing and Computer-Assisted Intervention MICCAI 2001: 4th Int. Conf., Proc., Utrecht, The Netherlands, Oct. 1417, 2001, pp. 889896. [9] R. N. Rohling, A. H. Gee, and L. Berman, Automatic registration of 3-D ultrasound images, Ultrasound Med. Biol., vol. 24, pp. 841854, 1998. [10] C. R. Meyer, J. L. Boes, B. Kim, P. H. Bland, G. L. Lecarpentier, J. B. Fowlkes, M. A. Roubidoux, and P. L. Carson, Semiautomatic registration of volumetric ultrasound scans, Ultrasound Med. Biol., vol. 25, pp. 339347, 1999. [11] W. M. Wells, P. Viola, H. Atsumi, S. Nakajima, and R. Kikinis, Multi-modal volume registration by maximization of mutual information, Med. Image Anal., vol. 1, pp. 3551, 1996. [12] F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, Multimodality image registration by maximization of mutual information, IEEE Trans. Med. Imag., vol. 16, pp. 187198, Apr. 1997. [13] J. P. Pluim, J. B. Maintz, and M. A. Viergever, Mutual-information-based registration of medical images: A survey, IEEE Trans. Med. Imag., vol. 22, pp. 9861004, Aug. 2003. [14] T. Arbel, X. Morandi, R. M. C. Comeau, and D. L. Collins, Automatic nonlinear MRI-ultrasound registration for the correction of intra-operative brain deformations, in Medical Image Computing and ComputerAssisted Intervention MICCAI 2001: 4th Int. Conf., Proc., Utrecht, The Netherlands, Oct. 1417, 2001, pp. 913922. [15] A. Roche, X. Pennec, G. Malandain, and N. Ayache, Rigid registration of 3-D ultrasound with MR images: A new approach combining intensity and gradient information, IEEE Trans. Med. Imag., vol. 20, pp. 10381049, Oct. 2001. [16] R. Shekhar and V. Zagrodsky, Mutual information-based rigid and nonrigid registration of ultrasound volumes, IEEE Trans. Med. Imag., vol. 21, pp. 922, Jan. 2002.

SHEKHARet al.: REGISTRATION OF REAL-TIME 3-D ULTRASOUND IMAGES OF THE HEART FOR NOVEL 3-D STRESS ECHOCARDIOGRAPHY

1149

[17] M. Ahmad, T. Xie, M. McCulloch, G. Abreo, and M. Runge, Real-time three-dimensional dobutamine stress echocardiography in assessment of ischemia: Comparison with two-dimensional dobutamine stress echocardiography, J. Amer. Coll. Cardiol., vol. 37, pp. 13031309, 2001. [18] G. M. Van Der Hoeven, P. J. Clerens, J. J. Donders, J. E. Beneken, and J. T. Vonk, A study of systolic time intervals during uninterrupted exercise, Br. Heart J., vol. 39, pp. 242254, 1977. [19] E. Tatara and A. Cinar, Interpreting ECG data by integrating statistical and articial intelligence tools, IEEE Eng. Med. Biol. Mag., vol. 21, pp. 3641, Jan.Feb. 2002. [20] B. U. Kohler, C. Hennig, and R. Orglmeister, The principles of software QRS detection, IEEE Eng. Med. Biol. Mag., vol. 21, pp. 4257, Jan.Feb. 2002. [21] D. Manke, K. Nehrke, and P. Bornert, Novel prospective respiratory motion correction approach for free-breathing coronary MR angiography using a patient-adapted afne motion model, Magn. Reson. Med., vol. 50, pp. 122131, 2003. [22] C. R. Castro-Pareja, J. M. Jagadeesh, and R. Shekhar, FAIR: A hardware architecture for real-time 3-D image registration, IEEE Trans. Inform. Techn. Biomed., vol. 7, pp. 426434, Dec. 2003.

[23] R. Shekhar, V. Zagrodsky, C. R. Castro-Pareja, V. Walimbe, and J. M. Jagadeesh, High-speed registration of three- and four-dimensional medical images by using voxel similarity, Radiographics, vol. 23, pp. 16731681, 2003. [24] V. Zagrodsky, R. Shekhar, and J. F. Cornhill, Multi-function extension of simplex optimization method for mutual information based registration of ultrasound volumes, Proc. SPIE Int. Soc. Opt. Eng., vol. 4322, pp. 508515, 2001. [25] I. N. Bankman, Handbook of Medical Imaging: Processing and Analysis. San Diego, CA: Academic, 2000. [26] P. J. Slomka, G. A. Hurwitz, J. Stephenson, and T. Cradduck, Automated alignment and sizing of myocardial stress and rest scans to threedimensional normal templates using an image registration algorithm, J. Nucl. Med., vol. 36, pp. 11151122, 1995. [27] J. Declerck, J. Feldmar, M. L. Goris, and F. Betting, Automatic registration and alignment on a template of cardiac stress and rest reoriented SPECT images, IEEE Trans. Med. Imag., vol. 16, pp. 727737, Dec. 1997.