Professional Documents
Culture Documents
2020 - Pérez Cambet - High Performance Ultrasound Simulation Using Monte-Carlo Simulation A GPU Ray-Tracing Implementation
2020 - Pérez Cambet - High Performance Ultrasound Simulation Using Monte-Carlo Simulation A GPU Ray-Tracing Implementation
SPIEDigitalLibrary.org/conference-proceedings-of-spie
ABSTRACT
Medical ultrasound has gained relevance in medical diagnosis, procedure guidance and emergentology. It is a
low cost, safe and non-invasive practice, while obtaining real-time results. Limitations on working hours and
ultrasound equipment have made simulation the main alternative for training, since it eliminates the need for
a real ultrasound machine and allows the modeling of a large number of case studies. Patient 11 from the
3D-IRCADb-01 CT-scans data-set was used to generate corresponding OBJ mesh files. A JSON file carries
relevant information of the scene and tissues involved in it. Monte-Carlo simulation over GPU is used to improve
ray tracing performance on a deterministic surface model. An improvement in ray tracing performance and
throughput is achieved by the presented model. In execution cases presented, a minimum global acceleration
of 30% is reached, while reducing ray tracing times by at least 80%. Monte-Carlo simulation allows a more
realistic image generation than the previous deterministic model. Low memory usage gives room to implement
the remaining stages of the process on GPU. A correct parametrization of tissues is required in order to achieve
improved image quality.
Keywords: Ultrasound simulation, path tracing, medical training simulation, Monte-Carlo simulation
1. INTRODUCTION
Medical ultrasound (US) has become a common practice for medical diagnosis, as a diagnostic tool, for procedure
guidance (during drains and punctures, among others), and emergentology. This is because it is a low cost, safe,
non-invasive practice, as it does not require punctures nor irradiation, while being capable of obtaining real-time
results economically, without having to relocate the patient, and allowing repetition if needed.1
The main disadvantage of this method is its dependence on operator skills. Given that this is no simple
task, an under-trained operator might misunderstand the produced image or produce low quality images that
are useless for diagnosis. It is of vital importance that the US operator understands the relation between the
two-dimensional cut observed and the real three dimensional anatomy, as well as properly interprets artifacts
present in US images produced by speckle noise and acoustic shadows coming from different tissue properties
and US frequency. This requires intensive and extensive user training to acquire the proper visual and manual
skills in the operation of such devices.
Advances in educational technology have put medical simulation as a state-of-the-art methodology for health-
care professionals training, since various limitations to mandatory work hours and US equipment affect the cor-
rect learning of this technique. Its importance is also because during a real emergency no risks can be taken by
putting the life of patients in the hands of under-skilled students. More so, US equipment may not be available
for training purposes, and unfrequent or rare complex pathological cases may be difficult to come by in smaller,
less populated cities, extending and limiting the training process. This is where ultrasound simulators come in
handy, since such study cases can be generated digitally and effectively train students and residents.
For a simulator to be of any use, it must reproduce clinical scenarios as realistically as possible, providing a
controlled environment to train a student’s response during critical or abnormal situations. It is usually helpful
to supply a handful of performance metrics to correctly asset a residents level of skill.
16th International Symposium on Medical Information Processing and Analysis, edited by Eduardo Romero
Natasha Lepore, Jorge Brieva, Marius Linguraru, Proc. of SPIE Vol. SIP220, SIP22000
© 2020 SPIE · CCC code: 0277-786X/20/$21 · doi: 10.1117/12.2576099
To produce a realistic output, tissue properties need to be correctly modeled. Among the parameters that
must be set, we can find the acoustic impedance and wave attenuation. Also, to generate speckle noise caused
by scatterers, a pair (µ, σ) that defines a normal distribution for scatterer amplitude, and a ρ parameter that
represents a tissue’s scatterer density, is needed. A sum of all parameters and their values for each material can
be found in table 2.
The volume model introduced in9 uses a CT volume image and simulates its interaction with acoustic waves
using a ray casting technique. This allows the representation of both reflections and wave attenuation to be
appreciated at every tissue interface. The former are produced when the involved materials have uneven acous-
tic impedance, and the strength of the reflection is given by Equation (1), where Z1,2 represent the acoustic
impedance of the current and collided tissues, respectively. The resulting transmitted energy (αT ) is calculated
as 1 − αR .
Z2 − Z1 2
αR = (1)
Z2 + Z1
This reflections are not specular, but behave differently depending to the angle between the incident wave
and the surface normal vector. Because of this, the signal received by the transducer is given by a Lambertian
scattering model. This causes the intensity to be dependent only on the incidence angle, and not the angle of
vision.
For a proper scattering emulation, a second image is generated from the CT volume using the real-time
speckle noise model presented in.11 In this way, view dependant scattering artifacts can be produced, being very
similar to those found in real ultrasound images. Both, the reflections and scattering images, are then combined
creating the final simulated image.
The deterministic surface model proposed in12 is based on a ray tracing algorithm, used to simulate ultrasound
wave behaviour. The model is implemented using C++, along with the Bullet3D14 library as a collision detector
between rays and meshes, and segment searching for further ray marching later on, and OpenCV15 for image
post-processing, which includes convolution, enveloping and scan conversion. This rays are traced from the
transducer to the scene, where organs are modeled with the meshes mentioned in Table 1. Each one of these
is paired with a set of properties that describe its mechano-acoustic features (Table 2), needed to simulate
both wave behaviour and dispersion. The refraction and reflection rays are generated according to Snell’s Law
equations, which states that the reflection ray will be given by
where ~i is the incidence vector, r is the ratio between the refraction indexes of the involved tissues, and c is the
cosine of the incidence angle. The intensity that each ray will carry is given by the Fresnel equation presented
previously in Equation (1) and its complementary equation.
Considering that this two rays will be created recursively at every ray-organ interface, it is easy to notice that
the amount of rays traced will increase exponentially at a rhythm of 2d , where d denotes the depth of recursion.
Usually, the maximum depth of recursion for a scene will vary between 8 to 12, since rays generated from a
point onwards will make little to no contribution to the image. The echoes received at the probe are composed
by a reflection term, calculated when a collision is found, and a back-scatter term, calculated using the method
presented in16 using
Unlike the work presented in,6 our implementation allows mesh intersection to model internal organ vessels.
This is done by using a stack with the material parameters. As the ray traverses the scene, the parameters of
the tissue assigned to each crossed mesh are stacked. The main caveat occurs when a ray is travelling inside
a vessel and hits a non-vascular organ. When this happens, the procedure follows depending on whether the
collided organ is the same as the next-to-the-top organ or not. If yes, this means that the ray is exiting that
organ while staying within the vessel, so we must remove the collided organ from the stack. Otherwise, a new
organ parameter must be stacked below the vessel, indicating that outside the vessel is within a new organ.
(c) (d)
Figure 1: Simulated output for each model. (a) corresponds to the deterministic model, whilst images (b) to
(d) are outputs for different amounts of samples per ray, going through 1 sample on the top left, to 5 and 10
samples on the second row.
great bottleneck in our pipeline, as for more than 5SPR the stage occupies more than 52% of the image generation
process. The post-processing stage is not affected by the greater amount of rays, since it only depends on image
resolution.
The latter two stages of the pipeline can be partially paralelized. However, Nvidia OptiX is not the best
selection for this. It is advisable to use other GPGPU libraries such as Nvidia CUDA or OpenCL, as the system
GPU would be idle otherwise. Even though this would carry an overhead provoked by data transfer from system
to GPU memory, it would be mitigated by the acceleration produced from GPU computing. RAM and VRAM
are no issue either, since all models consumed only up to 200MB of RAM and 330MB of VRAM, which negligible
in modern systems. This leaves a big room to use meshes with higher polygon count or to incorporate more
parameters to improve simulation quality.
Furthermore, Monte-Carlo allows a better modeling of tissue features. The previously mentioned roughness
aids when representing diffuse surfaces using a cosn distribution, being n the roughness of a given tissue. This
distribution will modify the real normal vector of the surface to simulate the micro-imperfections of the organ’s
wall, avoiding specular, unreal reflections at interfaces. Something similar happens with bone thickness. In this
(c) (d)
Figure 2: Simulated output for each model. (a) corresponds to the deterministic model, whilst images (b) to
(d) are outputs for different amounts of samples per ray, going through 1 sample on the top left, to 5 and 10
samples on the second row.
case, when a ray collides with a bone, it will slightly penetrate it and strongly attenuate due to its high absorption
coefficient. This attenuation is calculated according to Ie−xα , where x is the distance the wave penetrated into
the bone, and α is its absorption coefficient. Even though this can produce more realistic images, a problem
arises when the corresponding values are assigned to each tissue, as it is a manual, time-consuming task.
ACKNOWLEDGMENTS
This project was partly funded by PICT 2016-0116 - FONCYT - ANPCYT of Argentina. The NVidia Quaddro
P6000 used for this research was donated by the NVIDIA Corporation. The financial support of these institutions
is greatly appreciated.
REFERENCES
[1] Pinzón Vargas, S. and Moreno Carrillo, A., “Uso y capacitación en ecografı́a en el departamento de emer-
gencias,” 54(3), 353–360 (2013).
Monte-Carlo
Deterministic
1SPR 5SPR 10SPR
Ray tracing 686.2ms 33.0ms 90.3ms 141.4ms
Ray marching 299.0ms 57.2ms 297.6ms 576.1ms
Post-processing 180.4ms 173.1ms 173.8ms 172.0ms
Total 1,165.6ms 263.3ms 561.7ms 889.5ms
FPS 0.86FPS 3.80FPS 1.78FPS 1.12FPS
Rays/second 67.5K 103K 188.3K 241.9K
[2] Shams, R., Hartley, R., and Navab, N., “Real-time simulation of medical ultrasound from ct images,” in
[Medical Image Computing and Computer-Assisted Intervention – MICCAI 2008], Metaxas, D., Axel, L.,
Fichtinger, G., and Székely, G., eds., 734–741 (2008).
[3] Kutter, O., Karamalis, A., Wein, W., and Navab, N., “A gpu-based framework for simulation of medical
ultrasound,” in [Medical Imaging 2009: Visualization, Image-Guided Procedures, and Modeling ], Miga, M. I.
and Wong, K. H., eds., 7261, 396–404, International Society for Optics and Photonics (2009).
[4] Jensen, J., “Field: A program for simulating ultrasound systems,” Medical and Biological Engineering and
Computing 34(1), 351–352 (1996).
[5] Mattausch, O. and Goksel, O., “Monte-carlo ray tracing for realistic ultrasound training simulation,” in
[Eurographics Workshop on Visual Computing for Biology and Medicine ], Springer (October 2016).
[6] Mattausch, O., Makhinya, M., and Goksel, O., “Realistic ultrasound simulation of complex surface models
using interactive monte-carlo path tracing,” Computer Graphics Forum 37(1), 202–213 (2018).
[7] Mattausch, O. and Goksel, O., “Image-based reconstruction of tissue scatterers using beam steering for
ultrasound simulation,” Transactions on Medical Imaging (2017).
[8] Bahou, A., Tanner, C., and Goksel, O., “Scatgan for reconstruction of ultrasound scatterers using generative
adversarial networks,” 1674–1677 (04 2019).
[9] Rubi, P., Vera, E. F., Larrabide, J., Calvo, M., D’Amato, J. P., and Larrabide, I., “Comparison of real-time
ultrasound simulation models using abdominal CT images,” in [12th International Symposium on Medical
Information Processing and Analysis], Romero, E., Lepore, N., Brieva, J., Brieva, J., and and, I. L., eds.,
10160, 55 – 63, International Society for Optics and Photonics, SPIE (2017).
[10] D’Amato, J., Lo Vercio, L., Rubi, P., Vera, E., Barbuzza, R., del Fresno, M., and Larrabide, I., “Efficient
scatter model for simulation of ultrasound images from computed tomography data,” (2015).
[11] Külsgaard, H. C., “Desarrollo de herramientas de procesamiento de imágenes para simulación de ultra-
sonido.,” (2017).
[12] Rubi, P., Manterola, H. L., Fernandez Vera, E., Dı́az, A., and Larrabide, I., “Modelling vascular structures
in real-time abdominal ultrasound simulation,” Trabajo sin publicar.
[13] Vitale, S., Orlando, J. I., Iarussi, E., and Larrabide, I., “Improving realism in patient-specific abdominal ul-
trasound simulation using cyclegans,” International Journal of Computer Assisted Radiology and Surgery 15,
183–192 (Feb 2020).
[14] Coumans, E., “Bullet physics sdk,” (2013).
[15] Bradski, G., “The OpenCV Library,” Dr. Dobb’s Journal of Software Tools (2000).
[16] Meunier, J. and Bertrand, M., “Ultrasonic texture motion analysis: Theory and simulation,” IEEE trans-
actions on medical imaging 14, 293–300 (1995).
[17] Parker, S. G., Bigler, J., Dietrich, A., Friedrich, H., Hoberock, J., Luebke, D., McAllister, D., McGuire, M.,
Morley, K., Robison, A., and et al., “Optix: A general purpose ray tracing engine,” 29(4) (2010).