Intelligent Systems with Applications 18 (2023) 200203

Contents lists available at ScienceDirect

Intelligent Systems with Applications

journal homepage: www.journals.elsevier.com/intelligent-systems-with-applications

Fast MRI reconstruction using StrainNet with dual-domain loss on spatial

and frequency spaces
Worapan Kusakunniran a,∗ , Sarattha Karnjanapreechakorn a , Thanongchai Siriapisith b ,
Pairash Saiviroonporn b
a
Faculty of Information and Communication Technology, Mahidol University, 999 Phuttamonthon 4 Road, Salaya, 73170, Nakhon Pathom, Thailand
b
Department of Radiology, Faculty of Medicine Siriraj Hospital, Mahidol University, 2 Wanglang Road, Bangkok Noi, 10700, Bangkok, Thailand

A R T I C L E I N F O A B S T R A C T

Keywords: One of the main challenges to obtain a high throughput in the MRI process is a slow signal acquisition. This
Fast MRI process could be improved using a parallel imaging technique, where fewer raw data with multiple radio
Image reconstruction frequency (RF) coils are acquired simultaneously to reconstruct a final MR image. Nowadays, all multi-coil
Encoder-decoder
MRI machines have a parallel imaging technique for the image reconstruction. However, the parallel imaging
CNN
still cannot accelerate sufficiently to reduce the overall acquisition time. In another way, this paper proposes
Frequency domain
a solution relying on a deep convolution neural network (CNN) to generate high-quality reconstruction MR
images with higher acceleration factors. The proposed method, called StrainNet, performs the reconstructions by
encoding the under-sampled data (i.e., for the speeding up process) into high-level features. Then the important
part of the network, called Strainer, is applied to discard irrelevance information, and decodes remaining features
back to reconstruct MR images. The proposed network could be trained end-to-end with a newly presented loss
function, Dual-Domain Loss (DDL), combining both spatial and frequency losses. The experimental results are
based on the fastMRI dataset and show that StrainNet outperforms the competing methods for both 4- and 8-fold
accelerations.

1. Introduction coding (SENSE) Pruessmann et al. (1999), Generalized Autocalibrating

A non-invasive imaging technology makes Magnetic Resonance
Imaging (MRI) a widespread tool for diagnostic. A significant challenge
of the MRI is the overall processing time which could take a long time
inherently when compared with other imaging tools. Thus it leads to a
lower patient throughput. So, the current and ongoing MRI researches
also focus on increasing the speed of data acquisition and improving the
reconstruction process.
Modern MRI machines use a multi-coil system with multiple radio
frequency (RF) coils. This is because more than one RF coil can si-
multaneously record different parts of a target object rather than in
a standard sequential order. However, to do such a paralleled process,
specialized mathematical algorithms are required to combine multiple
under-sampled MR images, from multiple RF coil, into a single fully-
sampled MR image. That is where parallel imaging (PI) is used in
the MRI process. Three well-known PI techniques are SENSitivity En-
Partially Parallel Acquisitions (GRAPPA) Griswold et al. (2002), and
Compressed Sensing (CS) Lustig et al. (2008). Due to their dominant
performances, the SENSE and GRAPPA are currently used in modern
MRI machines.
An emerging machine learning approach to a variety of problems re-
cently makes a good promising technique for the MR image reconstruc-
tion problem Hammernik et al. (2016), Wang et al. (2016), Hammernik
et al. (2018). In addition, the excellent performance of deep learning
technology makes it a gold standard for machine learning tasks. So
the deep learning technique suits the MR image reconstruction prob-
lem well due to the expandable complexity structure of the network. As
also mentioned by Liang et al. (2020), deep learning based approaches
were shown to have a high potential for achieving a task of MRI recon-
struction.
Recently, several research works applied the deep learning technol-
ogy to the MR image reconstruction with a high acceleration factor and

*
Corresponding author.
E-mail addresses: worapan.kun@mahidol.edu (W. Kusakunniran), j.sarattha@gmail.com (S. Karnjanapreechakorn), thanongchai@gmail.com (T. Siriapisith),
pairash.sai@gmail.com (P. Saiviroonporn).

https://doi.org/10.1016/j.iswa.2023.200203
Received 14 December 2022; Received in revised form 31 January 2023; Accepted 10 February 2023
Available online 14 February 2023
2667-3053/© 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
W. Kusakunniran, S. Karnjanapreechakorn, T. Siriapisith et al.
indicated promising results Zbontar et al. (2018), Zhu et al. (2018).
However, large datasets are needed to compensate for the model com-
plexity and prevent an overfitting phenomenon from achieving high
quality reconstruction results at a high acceleration rate. Therefore, the
Facebook AI Research (FAIR) collaborated with NYU Langone Health
on a research project competition, fastMRI, to investigate the use of
Artificial Intelligence (AI) to make MRI acquisition faster. The fastMRI
competition provided large-scale raw MRI datasets with baseline results
Zbontar et al. (2018).
Based on our literature review, the remaining research gap of speed-
ing up the MRI reconstruction process using machine learning based
approaches is about a quality of reconstructed images, especially for the
8-fold accelerations. Details in a reconstructed image are missing, when
compared with a corresponding groundtruth image, as can be seen in
Fig. 17.
This paper introduces the deep convolutional neural network-based
solution to reconstruct MR images. The proposed network, Strain-
Net, is developed based on the encoder-decoder structure Cho et al.
(2014a, 2014b), Ronneberger et al. (2015). Depending on the prob-
lem domains, the encoder-decoder structure provides an easement to
scale the complexity in both encoder and decoder modules. It strives
to reconstruct an MR image at a high acceleration rate. The raw MR
data in the k-space contains a large amount of high-frequency signals,
in which the conventional encoder-decoder method cannot compre-
hend. The Strainer module of the proposed network is introduced to
address this challenge. It is located between every encoder and de-
coder pair. Along with the StrainNet, a new type of loss function is
also proposed, called dual-domain-loss (DDL), to maximize the network
performance. The proposed network is trained and evaluated using the
fastMRI dataset Zbontar et al. (2018).
The main contributions of this research paper are as follows. First,
the strainer module is introduced and added in every layer between
an encoder module and a corresponding decoder module. It helps in
reducing small-size artifacts in a reconstruction process. It also en-
hances an extraction of multiple scales features, where a complexity
of the network is not expanded. Second, a loss function is proposed to
combine losses from both domains of spatial and frequency domains.
It benefits in a training process, since both domains are related to the
reconstruction, where an original data is in a frequency space and an
output is reconstructed in a spatial space. Third, a number of param-
eters of the proposed network is significantly smaller, when compared
with an original encoder-decoder based network. The performances of
the reconstruction could be remained the same, but with faster training
and inferencing processes. Fourth, the proposed method could preserve
more details in reconstructed images, when compared with existing
methods, as shown in Figs. 13–18.
The rest of the paper is organized as follows. Background and related
work are described in section 2. The proposed method is introduced in
section 3. Experimental results are explained in section 4, where discus-
sion and conclusion are stated in sections 5 and 6, respectively.
2. Background and related work
2.1. Parallel imaging
The slow process of MR acquisitions is caused by a full amount of
raw data in the frequency domain, i.e., k-space. The data is needed to be
acquired line by line till the raw MR data contains an entire field of view
(FOV). Along with the old generation of single RF coil MRI machines,
these accumulate the total processing time of the MR acquisitions. Af-
ter the acquisition process, an MR image in the spatial domain can be
constructed by applying an inverse Fourier transform function ( −1 ) to
the MR data in the frequency domain. The basis MR image in the Carte-
sian space 𝑥 ∈ ℂM , i.e., spatial domain, is related to the k-space in the
Cartesian space 𝑘 ∈ ℂM via an equation below.
2
Intelligent Systems with Applications 18 (2023) 200203
𝑘 =  (𝑥) + 𝛼 (1)
where 𝛼 is the noise that can be measured by the MRI machine Baert
(2007), Sriram et al. (2020).
However, modern MRI machines contain a multi-RF coil system that
can scan different object parts concurrently. They acquired coils’ k-
space data which is then modulated by their own coils’ sensitivities
to compute MR signals. Each coil’s sensitivity is varied due to a dis-
tance between the coil and object location. So the acquired MR images,
which have already been transformed, will look similar to a set of di-
verse images with inhomogeneous brightness. The k-space data, which
is acquired by the 𝑖-th coil, is described below.
𝑘𝑖 =  (𝑆𝑖 𝑥) + 𝛼𝑖 , 𝑖 = 1, 2, ..., 𝑁 (2)
where 𝑖 is the 𝑖-th coil, 𝑁 is a total number of coils, and 𝑆𝑖 is the 𝑖-th
coil’s sensitivity matrix. Then every coiled-MR images are combined by
a reconstruction algorithm as below.
𝑁
∑
𝐼𝑟𝑒𝑐𝑜𝑛 = ( 𝑘𝑖 ) (3)
𝑖=1
where 𝐼𝑟𝑒𝑐𝑜𝑛 is a reconstructed MR image, and  is a reconstruction
algorithm.
Since the MRI machine can acquire k-space data simultaneously
with the multi-coil system, each coil’s k-space will have some redundant
information spread from neighboring RF coils. For this reason, all mod-
ern MRI machines could accelerate the acquisition process accordingly.
This process reduces a number of acquired k-space lines under-sampled
k-space, which leads to decreasing of the acquisition time. However,
when the under-sampled k-spaces are transformed back to the spatial
domain, this will result in MR images that contain alias artifacts Hamil-
ton et al. (2017). Hence, the reconstruction algorithm, , in equation
(3), has to combine all coils’ k-spaces. It will also have to reduce or
eliminate such artifacts by utilizing the spread of redundant data.
2.2. Image reconstruction
The state-of-the-art conventional reconstruction methods used in
modern MRI machines can be divided into two main categories: SENSE
and GRAPPA. These two reconstruction techniques use different types
of data as inputs.
2.2.1. SENSE
Sensitivity encoding for fast MRI (SENSE) is the reconstruction
method that operates in the spatial domain. So the acquired k-space
data is needed to be transformed into the spatial domain, before SENSE
is applied. However, SENSE requires another key input data: the coil’s
sensitivity map that can be measured by a pre-scanning process in the
MRI machine. This is because the MRI machine that uses SENSE for the
reconstruction could result in aliased MR images with superimposed
pixels. This phenomenon happens because reducing the sampling k-
space lines’ distance reduces the FOV and causes aliasing. Since each
coil’s superimposed MR image occurs with different weights according
to the coil’s sensitivity. Therefore, the coils’ sensitivity map can be used
for solving equation (4) in order to reconstruct a clean and complete MR
image Pruessmann et al. (1999), Baert (2007), Hamilton et al. (2017),
Blaimer et al. (2004).
𝐹 = 𝑖𝑛𝑣𝑒𝑟𝑡(𝑆) × 𝐴 (4)
where 𝐹 is the unfolded MR image matrix, 𝑆 is the coils’ sensitivity
map matrix, and 𝐴 is the aliased images matrix.
2.2.2. GRAPPA
Generalized autocalibrating partially parallel acquisitions or
GRAPPA is a method that operates in the local k-space data, unlike
SENSE method, by synthesizing missing k-space lines from redundant
W. Kusakunniran, S. Karnjanapreechakorn, T. Siriapisith et al.
Fig. 1. An overview structure of the encoder-decoder network.
information Griswold et al. (2002). Another difference from the SENSE
method is that the GRAPPA uses under-sampled k-spaces that preserve
central area data called Auto Calibration Signal (ACS). This ACS calcu-
lates a GRAPPA weight (𝑊 ) related to each missing k-space line. This
can be done because the central information in the k-space corresponds
to low spatial frequency information, which can be reconstructed into
the overall structure of MR image. Therefore, each missing k-space
line can be filled with a linear combination by utilizing the GRAPPA
weight and neighboring sampled k-space from all coils as a convolu-
tional equation as below Griswold et al. (2002), Baert (2007), Hamilton
et al. (2017), Blaimer et al. (2004).
𝐾𝑚 = 𝑊 ∗ 𝐾
where 𝐾𝑚 is the missing k-space data, and 𝐾 is the known k-space
data. After every missing coils’ k-space data is filled, the inverse Fourier
transform ( −1 ) is used for transforming all k-space images back to the
image spatial-space. Then, the coils’ MR images are combined into a
single MR image using a sum of squares method.
2.3. Encoder-decoder convolution neural network
In recent years, many types of deep convolution neural networks
have been adopted to solve challenging problems in medical related
fields as they can be scaled according to each problem’s size and
complexity. One standout network type is the encoder-decoder type.
The encoder-decoder convolution neural network consists of two major
parts: encoder and decoder. The encoder module performs as an image
feature extractor that enables the network to learn image’s features.
Then decoder module takes the extracted and known image features
to transform them back to a problem requirement, e.g., segmented ob-
ject and reconstructed image. Typically, the encoder-decoder network
contains four encoder and decoder modules but can be varied by the
problem’s complexity. This type of architecture makes the encoder-
decoder network robust for object segmentation, image reconstruction,
and super-resolution problems.
Ronneberger et al. Ronneberger et al. (2015) proposed the state-
of-the-art encoder-decoder convolutional neural network with a shape
Intelligent Systems with Applications 18 (2023) 200203
similar to the U character, called U-net, as shown in Fig. 1, for biomed-
ical image segmentation. U-net achieves very high-performances on
various biomedical image segmentation problems. Also, each encoder
module consecutively decreases an input image’s size by two times. This
process enables U-net to learn image features at multiple scales. Then
these extracted features are passed to each corresponding decoder mod-
ule at the same level of U-net. Each decoder module expands an input
image’s size by two times.
The U-net structure can achieve outstanding segmentation results,
especially on medical images. Many research works have improved
and extended the original U-net structure. Zhou et al. proposed the
improved version of U-net, called U-net++, for medical image segmen-
(5) tation Zhou et al. (2018). U-net++ contains a more number of dense
layers and a series of nested feature extractors, when compared with
the original U-net. So, the number of trainable parameters of U-net++
increases significantly, and U-net++ can outperform the original U-
net. Milletari et al. presented the encoder-decoder network in which
its structure looks similar to the V character instead of the U character
Milletari et al. (2016). V-net also contains encoder and decoder modules
in the same ways as U-net, but each module is designed for a volume
type input data, e.g., MRI volume. Gu et al. proposed the enhanced U-
net that changes original encoder modules to the pre-trained ResNet-34
for more efficient feature extraction Gu et al. (2019).
Since the emergence of a parallel imaging technique for the multi-
coil MR machine, there are many techniques for the reconstruction
process of both conventional techniques, including SENSE and GRAPPA,
and more advanced techniques that require additional input data Lustig
et al. (2008), Hammernik et al. (2016, 2018), Donoho (2006). How-
ever, not many researches apply the deep convolution neural network
in the MRI reconstruction process. For example, Zhu et al. presented the
CNN-based solution that can transform and reconstruct under-sampled
k-space data to fully-sampled MR images to produce promising recon-
structed MR images Zhu et al. (2018). Recently, the method proposed
in the fastMRI competition adopted U-net for MR image reconstruc-
tion on both knee and brain MR images Zbontar et al. (2018). These
researches open up for applying the deep CNN to the MR image recon-
struction. Therefore, this paper proposes StrainNet that also relies on
3
W. Kusakunniran, S. Karnjanapreechakorn, T. Siriapisith et al.
Fig. 2. An overview structure of the proposed StrainNet.
Fig. 3. An overview structure of the developed encoder module.
the encoder-decoder structure, similar to U-net, as a backbone with ad-
ditional Strainer modules. In addition, a Dual-Domain loss function is
introduced for a better performance in a training process.
3. Proposed method
3.1. StrainNet
In this paper, StrainNet is proposed based on an encoder-decoder
type of convolution neural network. It takes multi-coil under-sampled
MR images as input and outputs a single reconstructed MR image. Fig. 2
shows the architecture of StrainNet, which consists of three parts for
four levels in depth. First, the encoder part acts as a feature extractor
in a consecutive order. The encoder module also decreases an input
image’s size by half at each level, for the multi-scale feature extraction.
Second, the Strain part receives the extracted features from the en-
coder and filters out non-relevant features. Then, the remaining features
are passed to the third part, i.e., the decoder. The filtered features are
then decoded and expanded consecutively until a reconstructed image
has the same size as an input image. In this research work, StrainNet
has four layers in depth. So, the encoder, Strainer, and decoder all are
existed in all four layers, as illustrated in Fig. 2, where the same colors
are referred to the same levels.
3.1.1. Encoder module
The encoder module is a StrainNet’s feature extractor that takes
multi-coil MR images as input data. Each encoder module extracts
features and decreases input images’ sizes by half at every level for
4
Intelligent Systems with Applications 18 (2023) 200203
various-scaled features. The deeper network could generate more high-
level features since a quality of extracted features is related to a number
of feature extractors (i.e., filtering kernels) at each level. However, a
large number of filtering kernels leads to a large number of trainable
parameters, where the network can struggle to fit into the input data
samples. Eventually, the training process is affected by an overfitting
phenomenon. Therefore, the encoder module in each level is carefully
constructed based on a trade-off between the performance and the net-
work complexity. The encoding process can be summarized as in the
equation below.
𝑥𝑒𝑛𝑐𝑜𝑑𝑒,𝑖 = 𝑖 (𝑥𝑚𝑢𝑙𝑡𝑖𝑐𝑜𝑖𝑙 ) (6)
where 𝑖 is an encoder module in the network at 𝑖-th level and 𝑥𝑚𝑢𝑙𝑡𝑖𝑐𝑜𝑖𝑙
is a multi-coil under-sampled MR image.
As shown in Fig. 3, an encoder module structure displays a data
flow from each sub-module. Each encoder module consists of two sub-
module groups combined with 3x3 convolutions, normalizations, Leaky
ReLU activations, and dropouts. This structure is designed for network’s
simplification and scalability. If more trainable parameters are needed
in each level, a more sub-module group can be appended accordingly.
Moreover, a filter number in the convolution layer can be adjusted
based on the problem’s complexity. The normalizations help the mod-
ule to normalize extracted features into the same scale. The Leaky ReLU
activation transforms the extracted features from the linear into the
non-linear spaces for enhancing important features.
The dropout discards weakly extracted features. Finally, the input
image’s size is reduced by half for a smaller-scaled feature extraction,
W. Kusakunniran, S. Karnjanapreechakorn, T. Siriapisith et al.
Fig. 4. An overview structure of the proposed Strainer module.
using a max pooling. Then the extracted features are passed to the
Strainer module and the next encoder module.
3.1.2. Strainer module
The Strainer module is placed between the encoder and the decoder
parts inside StrainNet in every level, as shown in Fig. 2. The Strainer
module enables the network to select a good quality of extracted fea-
tures at various scales. Moreover, selected features are passed to the
decoder module as extra input data for the MRI reconstruction. The
feature selection process can be described below.
𝑥𝑠𝑒𝑙𝑒𝑐𝑡,𝑖 = 𝑖 (𝑖 (𝑥𝑚𝑢𝑙𝑡𝑖𝑐𝑜𝑖𝑙 ))
where 𝑖 is the Strainer module in the network at 𝑖-th level, and 𝑥𝑠𝑒𝑙𝑒𝑐𝑡,𝑖
is the selected features.
Fig. 4 demonstrates the structure of the Strainer module, which con-
sists of two major components. They are concatenated with residual
input data at the end. A first component is a group of multi-scales
feature pooling called Residual Multi-Kernal Pooling (RMP) Gu et al.
(2019). It acts as a strainer to discard low-quality features at multiple
scales. Therefore, RMP is adapted to help StrainNet to select good-
quality features from mixed and complex multi-scales features. RMP
structure consists of four max-pooling kernels: 2x2, 3x3, 5x5, and 6x6,
as shown in Fig. 5, for handling multi-scales features. Then, it is fol-
lowed by a 1x1 convolution for a dimension reduction. Selected features
are concatenated with the residual feature map at the end.
A second component is a Zoom-In/Zoom-Out. It consists of three
convolution layers with a residual concatenation. Zoom-In/Zoom-Out
can be separated into two sub-units. The first one is Zoom-In which
decreases feature size at the second convolution with a zooming factor.
Moreover, the second sub-unit is Zoom-Out, which increases the feature
size one step back. Then the features are passed to the third convolution
and concatenation, respectively. Zoom-In with a zooming factor enables
StrainNet to learn roughly basic features from the encoder module at a
lower level. Zoom-Out re-sizes the features to match the decoder mod-
ule’s required size at the same level.
Zoom-In/Zoom-Out unit’s flow helps the network to learn basic en-
coded features as additional data. Fig. 2 shows that the decoder module
at the 𝑖𝑡ℎ level receives the input features from the (𝑖 + 1)𝑡ℎ level that
has already been decoded. However, the basic undecoded features help
each decoder module to learn how to reconstruct an output image with
selected good features. A zooming factor is determined by how much
the feature size is decreased. This enables Zoom-In/Zoom-out to learn
more than one level of basic encoded features. Fig. 6 shows the diagram
of Zoom-In/Zoom-Out structure. For the simplicity of the experiment,
Zoom-In/Zoom-Out is used with a zooming factor of 2. This lets the
network use basic encoded features from one level below.
Intelligent Systems with Applications 18 (2023) 200203
3.1.3. Decoder module
The decoder module is a process that reconstructs an image from the
selected encoded features and increases the image’s size consecutively.
A decoding process can be described below.
𝑥𝑟𝑒𝑐𝑜𝑛,𝑖 = 𝑖 (𝑖 (𝑖 (𝑥𝑚𝑢𝑙𝑡𝑖𝑐𝑜𝑖𝑙 )) + 𝑖+1 ) (8)
where 𝑖 is the decoder module in the network at 𝑖-th level, and 𝑥𝑟𝑒𝑐𝑜𝑛,𝑖
is the reconstructed image. Equation (8) shows that features at the same
network level are processed through encoding, filtering, and decoding
steps. It also uses a reconstructed result from a level below as an input
of filtered features.
Fig. 7 displays the decoder module structure that consists of two ad-
ditional layers, including a pixel shuffle and a convolution layer, besides
the two groups of the encoder sub-modules. The pixel shuffle is partic-
ularly used for increasing the features’ sizes instead of a convolution
transpose or bilinear interpolation like other techniques Ronneberger
et al. (2015), Zhou et al. (2018), Gu et al. (2019). Then, the additional
convolutional layer is applied for a dimension matching. Finally, the
two groups of encoder sub-modules are used as the central reconstructor
unit. Since both the encoder module and the decoder module also use
the same encoder sub-module, it can prevent the network from strug-
gling to fit into the input dataset.
The pixel shuffle is proposed by Shi et al. Shi et al. (2016) for an
image and video super-resolution. Unlike the conventional techniques,
bi-cubic or linear interpolation, which usually uses a mathematical algo-
rithm, the pixel shuffle utilizes additional feature’s channels to increase
the feature size, as shown in Fig. 8. The pixel shuffle is performed by
taking features from multiple channels and shuffling them into a single
channel with a larger size. As shown in Fig. 6, as an example, 2x2-
features from 4 channels are shuffled into a 4x4-feature in a single
(7)
channel. So, this step could reduce a number of channels, but instead
increase a feature size in a single channel.
Therefore, it can achieve a higher quality reconstructed image than
interpolation methods or even a well-known transpose convolution.
This is because, in the training process, the upsampling process is also
trained by the network for the reconstruction process. Consequently,
the network can learn to select correct data from each channel to use as
supplementary data for expanding the features’ sizes. Also, the network
that uses the pixel shuffle tends to output an image with fewer artifacts
when compared with the other techniques.
3.2. Dual-domain loss
Fig. 9a and 9b show an under-sampled MR image and an artificial
blurred MR image that is affected by a motion blur in the horizontal
axis, respectively. It can be seen that the two images look close to each
other in terms of the superimpose pixel phenomenon. Because when
transforming the under-sampled k-space back to the spatial domain, it
generates an aliased MR image, in which some pixels are folded into
neighborhood areas. As well as the blurred image, a group of pixels is
linearly translated into the position of nearby pixels by blur parameters
Tiwari et al. (2013).
When magnitude spectrum images are created by applying the
Fourier Transform on both under-sampled and artificial blurred MR
images, as shown in Fig. 10a and 10b, they both have the same charac-
teristics. They are visible light vertical lines that represent the superim-
posed pixel phenomenon. However, the magnitude spectrum image of
blurred MR image contains equable spreading of light vertical lines, un-
like the magnitude spectrum image of under-sampled MR image with
random spreading of light vertical lines due to the under-sampling
process. These can distinguish the average magnitude intensity value
graph along both magnitude spectrum images’ horizontal axis (i.e., fre-
quency).
The graph of the artificial blurred MR image, as shown in Fig. 11b,
contains smoother lines between each peak than the under-sampled MR
5
W. Kusakunniran, S. Karnjanapreechakorn, T. Siriapisith et al. Intelligent Systems with Applications 18 (2023) 200203

Fig. 5. An overview structure of RMP.

Fig. 6. An overview structure of the proposed Zoom-In/Zoom-Out component.

Fig. 7. An overview structure of the decoder module.

Fig. 8. An overview structure of the pixel shuffle operation.

Fig. 9. Illustration of (a) the under-sampled MR image, and (b) the artificial
blurred MR image.

image (Fig. 11a). Each peak in the graph represents the visible light ver-
Fig. 10. Illustration of (a) the magnitude spectrum image of the under-sampled
tical lines in the magnitude spectrum images (Fig. 10a and 10b). When
MR image, and (b) the artificial blurred MR image.
counting the number of peaks in the graph of blurred MR image, it is
equal to the number of blur parameters used to generate the blurred

6
W. Kusakunniran, S. Karnjanapreechakorn, T. Siriapisith et al.
Fig. 11. Illustration of (a) the average magnitude intensity value along the
horizontal axis graph of the under-sampled spectrum MR image, and (b) the
artificial blurred spectrum MR image.
MR image (Fig. 9b), where the central peak can be counted as two
peaks Tiwari et al. (2013), Moghaddam and Jamzad (2007). Therefore,
the magnitude spectrum image can identify a motion blur and its pa-
rameters. Then a de-blurring process can be applied by constructing the
motion blur model from this knowledge Tiwari et al. (2013), Moghad-
dam and Jamzad (2007), Mayntx et al. (1999), Ji and Liu (2008).
However, the under-sampled MR image on both the magnitude spec-
trum image and its graph is more complex than the blurred image. So
the traditional modeling process, such as the de-blurring, cannot be ap-
plied. The presented evidence shows that both the aliased and blurred
MR images have similar spatial and frequency domain characteristics.
Instead of modeling an aliased frequency data for a de-aliased process
like the motion blur, this research proposes a new loss function to uti-
lize the sophistication of the deep convolutional neural network and
its ability to learn non-linearly complex features for training StrainNet,
called Dual-Domain Loss (DDL). DDL is a combination of two 𝐿1 losses
on spatial and frequency (i.e., magnitude spectrum) data spaces. DDL
equation is described below.
𝐷𝐷𝐿 = 𝛼1 𝐿1,𝑠𝑝𝑎𝑡𝑖𝑎𝑙 + 𝛼2 (𝜆𝐿1,𝑓 𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦 ) (9)
with
𝐿1 = ||𝑥𝑟𝑒𝑐𝑜𝑛 − 𝑥𝑔𝑡 ||1 (10)
where 𝑥𝑟𝑒𝑐𝑜𝑛 is the reconstructed MR image and 𝑥𝑔𝑡 is the ground truth
MR image, 𝜆 is a rescaling factor for 𝐿1,𝑓 𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦 and both 𝛼1 and 𝛼2
are weights of spatial and frequency 𝐿1 losses respectively (such that
𝛼1 + 𝛼2 = 1).
7
Intelligent Systems with Applications 18 (2023) 200203
Fig. 12. Examples of under-sampled k-space images for (a) SENSE and (b)
GRAPPA techniques and their corresponding under-sampled MR images (c) and
(d) respectively, after applying the inverse Fourier transform.
This loss function enables the network to learn differences between
spatial reconstructed and ground truth MR images. However, the net-
work can also learn the differences in the frequency data space. This
will fill the missing data in the magnitude spectrum image (i.e., visi-
ble light vertical lines). Eventually, it will lead to a decrease in aliased
artifacts in the reconstructed MR image.
Since the conventional-based MR image reconstruction techniques
are operated on different domains, where SENSE is on the spatial do-
main, and GRAPPA is on the frequency domain. Therefore, the under-
sampling processes of these techniques are varied. SENSE requires an
under-sampled k-space to be acquired by reducing the k-space lines
equally, as shown in Fig. 12a. In contrast, GRAPPA requires an acquired
k-space to be preserved in central areas (i.e., ACS lines) along with eq-
uispaced reducing k-space lines for calculating GRAPPA’s weights, as
shown in Fig. 12b.
Consequently, transformed MR images from these two techniques
are very diverse, as shown in Fig. 12c and 12d. Since the GRAPPA
technique typically outputs higher quality reconstructed MR images.
Therefore, most deep convolutional neural network techniques, includ-
ing the proposed StrainNet, use the under-sampled k-space similar to
the GRAPPA type Zbontar et al. (2018), Zhu et al. (2018). Therefore,
StrainNet has to learn to reconstruct the GRAPPA-typed under-sampled
MR images.
4. Experimental results
In this section, experiments are operated on the multi-coil knee MR
images from the fastMRI dataset Zbontar et al. (2018), Knoll et al.
(2020). The fastMRI dataset consists of fully sampled k-space data from
1,594 scans of four different MRI machines. All k-space data samples
are 15 channels type. The dataset consists of two pulse sequences: coro-
nal proton-density weighting with fat suppression (PDFS, 798 scans)
W. Kusakunniran, S. Karnjanapreechakorn, T. Siriapisith et al. Intelligent Systems with Applications 18 (2023) 200203

Table 1
Comparison between the proposed method and the existing networks performance for the multi-coil knee task on
the validation dataset.

Model Acceleration NMSE PSNR SSIM

PD PDFS PD PDFS PD PDFS

fastMRI network (Zbontar et al., 2018) 4-fold 0.0054 0.0112 37.58 36.39 0.9287 0.8655
U-Net (Ronneberger et al., 2015) 0.0046 0.0107 38.37 36.64 0.9354 0.8693
CE-Net (Gu et al., 2019) 0.2422 0.2015 20.96 23.64 0.6431 0.6756
U-Net++ (Zhou et al., 2018) 0.0222 0.0259 31.35 32.94 0.8387 0.8289
StrainNet 0.0044 0.0107 38.60 36.67 0.9373 0.8695

fastMRI network (Zbontar et al., 2018) 8-fold 0.0120 0.0181 34.12 34.23 0.8915 0.8286
U-Net (Ronneberger et al., 2015) 0.0087 0.0156 35.54 34.89 0.8997 0.8324
CE-Net (Gu et al., 2019) 0.2514 0.2045 20.79 23.55 0.6139 0.6658
U-Net++ (Zhou et al., 2018) 0.0256 0.0384 30.73 31.27 0.8164 0.7836
StrainNet 0.0078 0.0150 36.01 35.09 0.9048 0.8351

Table 2
Reconstruction results of the proposed method for each combination of
𝛼1 and 𝛼2 on the validation dataset.

𝛼1 𝛼2 NMSE PSNR SSIM

PD PDFS PD PDFS PD PDFS

0.0 1.0 0.3729 0.2022 19.00 23.69 0.3789 0.5468

0.1 0.9 0.0059 0.0118 37.19 36.15 0.9243 0.8611
0.2 0.8 0.0060 0.0119 37.09 36.11 0.9229 0.8607
0.3 0.7 0.0061 0.0120 37.02 36.07 0.9230 0.8607
0.4 0.6 0.0060 0.0118 37.11 36.13 0.9236 0.8609
0.5 0.5 0.0057 0.0115 37.39 36.26 0.9260 0.8627
0.6 0.4 0.0054 0.0114 37.62 36.33 0.9275 0.8631
0.7 0.3 0.0060 0.0118 37.13 36.15 0.9240 0.8615
0.8 0.2 0.0057 0.0116 37.38 36.24 0.9262 0.8628
0.9 0.1 0.0059 0.0117 37.19 36.19 0.9240 0.8618
1.0 0.0 0.0060 0.0118 37.12 36.16 0.9241 0.8618

and without fat suppression (PD, 796 scans). The knee images were
cropped with a size of 320 × 320 pixels. All experiments use random
type masked k-space with both 4-fold and 8-fold for simulating the MRI
machine’s acceleration process.
The proposed StrainNet is compared with three existing networks
of image reconstruction (see Table 1). The three networks used in the
comparisons are:
• U-Net Ronneberger et al. (2015)
• CE-Net Gu et al. (2019)
• U-Net++ Zhou et al. (2018)
All three networks are the state-of-the-art encoder-decoder types
of networks that present good performances in many fields, e.g., im-
age segmentation, image reconstruction, and image super-resolution.
U-Net is a pinnacle and baseline for the encoder-decoder network Ron-
neberger et al. (2015), where CE-Net is the enhanced version of the
U-Net’s encoder part by replacing the traditional encoder module with
the ResNet-34 He et al. (2016) and shows significantly improved results
Gu et al. (2019).
On the other hand, U-Net++ is the improved version of U-Net on
creating nested groups of the encoder-decoder module to form a more
complex structure. U-Net++ shows better results on the medical image
segmentation, when compared with U-Net Zhou et al. (2018). There-
fore, it is promising to compare StrainNet with these networks. All
experiments are trained for 100 epochs with RMSProp algorithm to
minimize L1-loss for the three networks and the Dual-Domain Loss
of StrainNet. Evaluation metrics used for measuring the performance
are normalized mean squared error (NMSE), peak signal-to-noise ratio
(PSNR), and structural similarity (SSIM).
Our experiment begins with finding optimum values for 𝛼1 and
𝛼2 . However, to speed up this optimization step, a smaller-sized ver-
sion of the proposed network (i.e., with the starting feature size of
1x320x320x32) is used in this process. This smaller version can signifi-
cantly reduce the training time while preserving the same architecture’s
benefits. The experimental results for each combination of 𝛼1 and 𝛼2 are
shown in Table 2.
In Table 2, it can be seen that the highest performance is obtained
when 𝛼1 =0.6 and 𝛼2 =0.4. Then, the full-sized version of the proposed
network is further investigated around this optimal point. It is then con-
cluded that, with the full-sized version, the best combination is when
𝛼1 =0.7 and 𝛼2 =0.3.
So, in the remaining experiments, the proposed StrainNet is trained
by multi-coils under-sampled MR images with the Dual-Domain Loss,
where (𝛼1 , 𝛼2 ) and 𝜆 are empirically set to (0.7, 0.3) and 0.05, respec-
tively. The three compared networks are trained by the root sum-of-
squares (RSS), combining multi-coil images into single MR images with
the 𝐿1 loss. Experimental results are shown in Table 1, which compares
StrainNet and the three networks with the other published fastMRI
baseline results Zbontar et al. (2018). All networks are trained with
the fastMRI’s training dataset and tested on the fastMRI’s validation
dataset. The fastMRI’s testing dataset is not used because the fastMRI
competition only provides the ground truth MR images (fully-sampled
MR images) on the training and validation datasets. Table 1 presents
the networks’ performances on three metrics with two types of pulse
sequences.
Examples of reconstructed MR images are shown in Fig. 13b - 14e
and 15b - 16e for 4-fold and 8-fold accelerations, respectively, with
both pulse sequences. Reconstructed images are shown with ground-
truth MR images for a comparison. Fig. 17b - 17f and 18b - 18f show
some zoomed-in regions for 4-fold and 8-fold accelerations respectively.
5. Discussion
The experimental results show the promising reconstruction perfor-
mances in both scenarios of 4-fold and 8-fold accelerations. Table 1

8
W. Kusakunniran, S. Karnjanapreechakorn, T. Siriapisith et al. Intelligent Systems with Applications 18 (2023) 200203

Fig. 13. Sample reconstructions for 4-fold undersampling with the PD pulse sequence. They are (a) the Ground Truth image, (b) the U-Net reconstructed image, (c)
the CE-Net reconstructed image, (d) the U-Net++ reconstructed image, and (e) the StrainNet reconstructed image.

Fig. 14. Sample reconstructions for 4-fold undersampling with the PDFS pulse sequence. They are (a) the Ground Truth image, (b) the U-Net reconstructed image,
(c) the CE-Net reconstructed image, (d) the U-Net++ reconstructed image, and (e) the StrainNet reconstructed image.

Fig. 15. Sample reconstructions for 8-fold undersampling with the PD pulse sequence. They are (a) the Ground Truth image, (b) the U-Net reconstructed image, (c)
the CE-Net reconstructed image, (d) the U-Net++ reconstructed image, and (e) the StrainNet reconstructed image.

Fig. 16. Sample reconstructions for 8-fold undersampling with the PDFS pulse sequence. They are (a) the Ground Truth image, (b) the U-Net reconstructed image,
(c) the CE-Net reconstructed image, (d) the U-Net++ reconstructed image, and (e) the StrainNet reconstructed image.

shows that StrainNet outperforms the fastMRI baseline network and the
other three networks in every evaluation metric on both 4-fold and 8-
fold accelerations. Table 1 also shows that the improved U-Net type
networks, including CE-Net and U-Net++, struggle to reconstruct the
under-sampled MR image. This can be caused by replacing a standard
encoder module with ResNet-34 in CE-Net and the nested groups of
an encoder-decoder module in U-Net++. These two networks present
higher performances on other problems. However, in the MR image
reconstruction problem, the traditional encoder-decoder module out-
performs the improved versions. Although StrainNet achieved slightly
higher performance on the 4-fold acceleration than U-Net, StrainNet
outperforms U-Net performance significantly for the 8-fold accelera-
tion. The zoomed-in images in Fig. 17b - 17f and 18b - 18f also indicates
that the reconstructed images from StrainNet preserve tiny details bet-
ter than other networks with a superior contrast. This can be useful
when diagnosing a disease. Besides, for a future work, alternative solu-
tions could be based on a type of generative adversarial network Sohan
and Yousuf (2020). Also, a quality of reconstructed images could be fur-
ther validated using a well-trained classification model, e.g. Aurna et al.
(2022). This could confirm that details of diseases are remained in re-
constructed images in some extent, if these images could be correctly
classified.
6. Conclusion
In this paper, StrainNet was proposed for the MRI reconstruction.
After masking k-space data, to simulate the acceleration process in the
MRI machine, the multi-coil under-sampled k-space was transformed

9
W. Kusakunniran, S. Karnjanapreechakorn, T. Siriapisith et al. Intelligent Systems with Applications 18 (2023) 200203

Fig. 17. Sample reconstructions for 4-fold undersampling with the zoomed-in region. They are (a) the Ground Truth image with highlighted zoomed area, (b) the
Ground Truth zoomed image, (c) the U-Net zoomed reconstructed image, (d) the CE-Net zoomed reconstructed image, (e) the U-Net++ zoomed reconstructed image,
and (f) the StrainNet zoomed reconstructed image.

Fig. 18. Sample reconstructions for 8-fold undersampling with the zoomed-in region. They are (a) the Ground Truth image with highlighted zoomed area, (b) the
Ground Truth zoomed image, (c) the U-Net zoomed reconstructed image, (d) the CE-Net zoomed reconstructed image, (e) the U-Net++ zoomed reconstructed image,
and (f) the StrainNet zoomed reconstructed image.

back to the spatial under-sampled MR images. Then, StrainNet was used
to combine multi-coils under-sampled MR images and reconstruct the
fully-sampled single MR image. The structure of StrainNet was based on
the well-known encoder-decoder type of CNN that performed well on
the reconstruction problems. StrainNet also included the multi-layers
pooling between encoder and decoder modules on every level besides
the traditional encoder-decoder module. This could help the network
eliminate irrelevant information while the under-sampled MR images
were being trained. This research also proposed a new type of loss
function, called Dual-Domain Loss (DDL), by adding the 𝐿1 loss on the
frequency data and the standard 𝐿1 loss on the spatial data.
Reconstruction results of the proposed network on a multi-coil knee
validation dataset could outperform U-Net and other encoder-decoder
networks in every metric, including NMSE, PSNR, and SSIM. StrainNet
can preserve small information that was usually difficult to reconstruct
from the reconstructed results, especially in the 4-fold acceleration.
Besides the MR image reconstruction, StrainNet can be adapted to
other problems, such as a de-blurring process of an image that could
be affected by a motion blur or different types of blurring. The DDL
adopted information from the frequency and spatial domains that two
hyper-parameters could control. Moreover, the blurring effects could be
explained in the frequency domain. Therefore, the proposed StrainNet
can learn to apply this information for the de-blurring process.
The proposed method has a potential to achieve a better perfor-
mance, especially on reconstructing details in a spatial domain from a
high frequency data. To do this, it is necessary to increase a complexity
of the proposed architecture. Therefore, it would need more computa-
tional resources and times for training the reconstruction model, since
the number of parameters could be expanded from approximately 500
million to 1.5 billion.
CRediT authorship contribution statement
Worapan Kusakunniran: Conceptualization, Data curation, For-
mal analysis, Methodology, Software, Validation, Writing – original
draft. Sarattha Karnjanapreechakorn: Conceptualization, Data cura-
tion, Formal analysis, Methodology, Software, Validation, Writing –
original draft. Thanongchai Siriapisith: Conceptualization, Data cu-
ration, Formal analysis, Methodology, Validation, Writing – review &
editing. Pairash Saiviroonporn: Conceptualization, Data curation, For-
mal analysis, Methodology, Validation, Writing – review & editing.

10
W. Kusakunniran, S. Karnjanapreechakorn, T. Siriapisith et al.
Declaration of competing interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
Data availability
No data was used for the research described in the article.
References
Aurna, N. F., Yousuf, M. A., Taher, K. A., Azad, A., & Moni, M. A. (2022). A classification
of MRI brain tumor based on two stage feature level ensemble of deep CNN models.
Computers in Biology and Medicine, 146, Article 105539.
Baert, A. (2007). Parallel imaging in clinical MR applications. Springer Science & Business
Media.
Blaimer, M., Breuer, F., Mueller, M., Heidemann, R. M., Griswold, M. A., & Jakob, P. M.
(2004). SMASH, SENSE, PILS, GRAPPA: How to choose the optimal method. Topics in
Magnetic Resonance Imaging, 15, 223–236.
Cho, K., Van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014a). On the properties
of neural machine translation: Encoder-decoder approaches. arXiv preprint, arXiv:
1409.1259.
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., &
Bengio, Y. (2014b). Learning phrase representations using RNN encoder-decoder for
statistical machine translation. arXiv preprint, arXiv:1406.1078.
Donoho, D. L. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52,
1289–1306.
Griswold, M. A., Jakob, P. M., Heidemann, R. M., Nittka, M., Jellus, V., Wang, J., Kiefer,
B., & Haase, A. (2002). Generalized autocalibrating partially parallel acquisitions
(GRAPPA). Magnetic Resonance in Medicine: An Official Journal of the International
Society for Magnetic Resonance in Medicine, 47, 1202–1210.
Gu, Z., Cheng, J., Fu, H., Zhou, K., Hao, H., Zhao, Y., Zhang, T., Gao, S., & Liu, J. (2019).
CE-net: Context encoder network for 2D medical image segmentation. IEEE Transac-
tions on Medical Imaging, 38, 2281–2292.
Hamilton, J., Franson, D., & Seiberlich, N. (2017). Recent advances in parallel imaging
for MRI. Progress in Nuclear Magnetic Resonance Spectroscopy, 101, 71–95.
Hammernik, K., Klatzer, T., Kobler, E., Recht, M. P., Sodickson, D. K., Pock, T., & Knoll,
F. (2018). Learning a variational network for reconstruction of accelerated MRI data.
Magnetic Resonance in Medicine, 79, 3055–3071.
Hammernik, K., Knoll, F., Sodickson, D., & Pock, T. (2016). Learning a variational model
for compressed sensing MRI reconstruction. In Proceedings of the international society
of magnetic resonance in medicine (ISMRM) (p. 1088).
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recogni-
tion. In Proceedings of the IEEE conference on computer vision and pattern recognition
(pp. 770–778).
11
Intelligent Systems with Applications 18 (2023) 200203
Ji, H., & Liu, C. (2008). Motion blur identification from image gradients. In 2008 IEEE
conference on computer vision and pattern recognition, IEEE (pp. 1–8).
Knoll, F., Murrell, T., Sriram, A., Yakubova, N., Zbontar, J., Rabbat, M., Defazio, A.,
Muckley, M. J., Sodickson, D. K., Zitnick, C. L., & Recht, P. M. (2020). Advancing
machine learning for MR image reconstruction with an open competition: Overview
of the 2019 fastMRI challenge. Magnetic Resonance in Medicine, 84, 3054–3070.
Liang, D., Cheng, J., Ke, Z., & Ying, L. (2020). Deep magnetic resonance image recon-
struction: Inverse problems meet neural networks. IEEE Signal Processing Magazine,
37, 141–151.
Lustig, M., Donoho, D. L., Santos, J. M., & Pauly, J. M. (2008). Compressed sensing MRI.
IEEE Signal Processing Magazine, 25, 72–82.
Mayntx, C., Aach, T., & Kunz, D. (1999). Blur identification using a spectral inertia tensor
and spectral zeros. In Proceedings 1999 international conference on image processing
(cat. 99CH36348) (pp. 885–889). IEEE.
Milletari, F., Navab, N., & Ahmadi, S. A. (2016). V-net: Fully convolutional neural net-
works for volumetric medical image segmentation. In 2016 fourth international con-
ference on 3D vision (3DV) (pp. 565–571). IEEE.
Moghaddam, M. E., & Jamzad, M. (2007). Motion blur identification in noisy images using
mathematical models and statistical measures. Pattern Recognition, 40, 1946–1957.
Pruessmann, K. P., Weiger, M., Scheidegger, M. B., & Boesiger, P. (1999). SENSE: Sensi-
tivity encoding for fast MRI. Magnetic Resonance in Medicine: An Official Journal of the
International Society for Magnetic Resonance in Medicine, 42, 952–962.
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for
biomedical image segmentation. In International conference on medical image comput-
ing and computer-assisted intervention (pp. 234–241). Springer.
Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang,
Z. (2016). Real-time single image and video super-resolution using an efficient sub-
pixel convolutional neural network. In Proceedings of the IEEE conference on computer
vision and pattern recognition (pp. 1874–1883).
Sohan, K., & Yousuf, M. A. (2020). 3D bone shape reconstruction from 2D X-ray images
using med generative adversarial network. In 2020 2nd international conference on
advanced information and communication technology (ICAICT) (pp. 53–58). IEEE.
Sriram, A., Zbontar, J., Murrell, T., Zitnick, C. L., Defazio, A., & Sodickson, D. K. (2020).
GrappaNet: Combining parallel imaging with deep learning for multi-coil MRI re-
construction. In Proceedings of the IEEE/CVF conference on computer vision and pattern
recognition (pp. 14315–14322).
Tiwari, S., Shukla, V. P., Singh, A., & Biradar, S. (2013). Review of motion blur estimation
techniques. Journal of Image and Graphics, 1, 176–184.
Wang, S., Su, Z., Ying, L., Peng, X., Zhu, S., Liang, F., Feng, D., & Liang, D. (2016). Accel-
erating magnetic resonance imaging via deep learning. In 2016 IEEE 13th international
symposium on biomedical imaging (ISBI) (pp. 514–517). IEEE.
Zbontar, J., Knoll, F., Sriram, A., Murrell, T., Huang, Z., Muckley, M. J., Defazio, A., Stern,
R., Johnson, P., Bruno, M., et al. (2018). fastMRI: An open dataset and benchmarks
for accelerated MRI. arXiv preprint, arXiv:1811.08839.
Zhou, Z., Rahman Siddiquee, M. M., Tajbakhsh, N., & Liang, J. (2018). UNet++: A nested
U-net architecture for medical image segmentation. In Deep learning in medical image
analysis and multimodal learning for clinical decision support (pp. 3–11). Springer.
Zhu, B., Liu, J. Z., Cauley, S. F., Rosen, B. R., & Rosen, M. S. (2018). Image reconstruction
by domain-transform manifold learning. Nature, 555, 487–492.