ACCESS3277543

This article has been accepted for publication in IEEE Access.
This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2023.3277543
Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier 10.1109/ACCESS.2017.DOI
Efficient Blind Source Separation

Method for fMRI using Autoencoder and
Spatiotemporal Sparsity Constraints
MUHAMMAD USMAN KHALID1 , BILAL A. KHAWAJA2 , (Senior Member, IEEE), MALIK
MUHAMMAD NAUMAN3 ,
1
College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University, Riyadh 11564, Saudi Arabia
2
Faculty of Engineering, Department of Electrical Engineering, Islamic University of Madinah, Madinah 42351, Saudi Arabia
3
Faculty of Integrated Technologies, Universiti Brunei Darussalam, Bandar Seri Begawan BE1410, Brunei
Corresponding author: Malik Muhammad Nauman (malik.nauman@ubd.edu.bn).
This work was supported by the Grant No: UBD/RSCH/URC/RG(b)/2020/018
ABSTRACT Diversity measures exploited by blind source separation (BSS) methods are usually based on
either statistical attributes/geometrical structures or sparse/overcomplete (underdetermined) representations
of the signals. This leads to some inefficient BSS methods that are derived from either a mixing matrix
(mm), sparse weight vectors (sw), or sparse code (sc). In contrast, the proposed efficient method, sparse
spatiotemporal BSS (ssBSS), avoids computational complications associated with lag sets, deflation
strategy, and repeated error matrix computation using the whole dataset. It solves the spatiotemporal
data reconstruction model (STEM) with l1 -norm penalization and l0 -norm constraints using Neumann’s
alternating projection lemma and block coordinate descent approach to yield the desired bases. Its specific
solution allows incorporating a three-step autoencoder and univariate soft thresholding for a block update
of the source/mixing matrices. Due to the utilization of both spatial and temporal information, it can
better distinguish between the sources and yield interpretable results. These steps also make ssBSS unique
because, to the best of my knowledge, no mixing matrix based BSS method incorporates sparsity of both
features and a multilayer network structure. The proposed method is validated using synthetic and various
functional magnetic resonance imaging (fMRI) datasets. Results reveal the superior performance of the
proposed ssBSS method compared to the existing methods based on mmBSS and swBSS. Specifically,
overall, a 14% increase in the mean correlation value and 91% reduction in computation time over the
ssICA algorithm was discovered.
INDEX TERMS Sparse representation, autoencoder, BSS, fMRI, activation maps, PCA, l0 -norm, l1 -norm
I. INTRODUCTION fMRI studies [6]–[12].

Over the last two decades, the statistical parametric mapping Blind signal extraction is a fundamental problem in many
(SPM) toolbox’s [1] general linear model (GLM) has been signal and image processing applications [13]–[18] that aims
extensively used for fMRI analysis [2]. This hypothesis- at revealing hidden structures in the multivariate data through
driven model requires prior knowledge about the experi- matrix decomposition [19]. While utterly blind to the mixing
mental paradigm, such as hemodynamic response function process, it makes some assumptions about the source signals
(HRF). However, the dynamics of the experiment are some- by imposing constraints on the data matrix. In this regard,
times hard to model due to HRF variability across subjects principal component analysis (PCA) assumes underlying sig-
[3] and the absence of stimulus for resting-state experiments. nals are uncorrelated [20], independent component analysis
Instead, the data-driven approaches offer a promising alter- (ICA) hypothesizes statistical independence [21], canonical
native due to their adaptability to both task-based activation correlation analysis (CCA) expects that signals are auto-
detection and resting-state functional connectivity analysis correlated [22], sparse dictionary learning (SDL) considers
[4], [5]. In this regard, some mmBSS methods owing to their that signals are overcomplete [23], partial least square (PLS)
computational efficiency, have been very consequential for infers that sources have high autocovariance [24], and gen-
VOLUME 4, 2016 1
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
eralized morphological components analysis (GMCA) [25] rithm with learning based regularization for hyperspectral
encourages morphological diversity of signals. imaging [42].
Specifically, mmBSS methods are based on either de- Some computationally efficient mmBSS methods, such as
terministic or stochastic approaches. Deterministic meth- regular PCA and CCA, perform poorly for BSS. On the other
ods impose geometrical assumptions on signals such as hand, sparse variants of BSS strive to improve separation
non-negative matrix factorization (NMF) [26] and spa- accuracy at the cost of the increased numerical burden. This
tial/temporal correlation [27]. In contrast, statistical meth- is due to their reliance on either deflation strategy, iterative
ods are concerned with the probabilistic distribution of the thresholding, repeated eigen/singular value decomposition
signals such as Bayes’ theorem assigns probability density (SVD), or error matrix computation of huge covariance/data
function as priors to sources and mixing coefficients [28], and matrices. This paper reached a compromise between the two
ICA explores the statistical properties of the source signals scenarios by separating all source signals simultaneously in a
through higher order statistics [21]. Lower interpretability transformed space through a sparse approach. Moreover, mo-
offered by statistical/geometrical assumptions is usually re- tivated by this discussion and the effectiveness of concurrent
solved by imposing sparsity as an additional constraint on feature extraction [37], a new method named ssBSS has been
the weight vectors, which results in sparse variants of BSS proposed for fMRI data. The main contributions are
methods (swBSS), as discussed in the next section. 1) A novel approach that attempts to solve the proposed
Alternatively, consider scBSS methods due to an over- model in a computationally efficient manner through an
complete representation of the sparse signals such as SDL, autoencoder, BSS theory exploitation, and continuous
GMCA, and sparse component analysis (SCA) [29] that but implicit training from the data.
naturally circumvent the interpretability problem but their 2) Utilizing Neumann’s successive projection lemma [43],
computational cost is enormous. On the other hand, spatial each of the two sub-problems from the penalized and
ICA (sICA) has been very popular due to its numerical constrained STEM model is solved in an alternating
simplicity, better performance, and fMRI data’s lower spatial least square manner.
variations [30]. Previously, it has been argued that inde- 3) A comprehensive validation of the proposed method
pendence is not adaptive for fMRI [31], and the sparsity and its comparison with existing mmBSS and swBSS
of components rather than independence is a more fruitful methods using synthetic and experimental data.
assumption. Nevertheless, it was recently concluded that 4) In-depth analysis using performance metrics such as
both sparsity and independence are reasonable assumptions similarity measure, convergence analysis, and numerical
for fMRI analysis [32]. More recently, a framework was cost.
developed that jointly exploited both source diversities to The rest of the paper is organized into five sections where
increase separation accuracy for fMRI [33]. section 2 highlights the related work, section 3 provides
Analyzing temporal and spatial information separately some background on sparse BSS methods, section 4 presents
had been the convention, whereas exploiting both features some direction on mathematical notations followed by the
concurrently may lead to better performance. Some mmBBS discussion on the proposed method and its related algorithm,
methods have attempted this strategy in their analysis. For section 5 produces experimental study, and section 6 contains
instance, ECG specific BSS technique used spatial ICA to the concluding remarks.
remove non-Gaussian interference followed by second-order
blind identification (SOBI) [34] step to suppress temporal II. RELATED WORK
Gaussian noise [35], a low-dimensional CCA approach for As shown in Fig. 1, BSS methods generally employ three
fMRI that incorporated straight forward image projection strategies. These are
into temporal BSS model to exploit autocorrelations of both 1) mmBSS: The whitening operation followed by a non-
temporal and spatial features concurrently [36], and an SDL sparse mixing/rotation matrix estimation.
based method for multi-subject fMRI analysis that attempted 2) swBSS: Utilization of full data matrix to estimate sparse
to utilize information from both dimensions in terms of weight matrix through deflation.
principal components and loadings [37]. 3) scBSS: Computation of the error matrix using full data
More recent application-specific BSS methods are also matrix to estimate dictionary and sparse code through
worth mentioning. For instance, a hybrid approach was pro- alternating minimization.
posed that combined a variational autoencoder and bandpass State-of-the-art swBSS methods that have been quite suc-
filtering for speech processing [38], an efficient BSS method cessful for neuroimaging data include sparse PCA (sPCA)
was proposed that utilized multi-channel Wiener filtering [45], sparse CCA (sCCA) [46], [47], sparse PLS (sPLS) [48],
and joint diagonalization [39] for audio signal processing [49], improve fast ICA (ifICA) [50], sparse spatial ICA using
[40], two smooth variants of NMF namely smooth successive entropy bound minimization algorithm (ssICA) [33], multi-
projection algorithm and smooth vertex component analysis task sparse model (MTS) [51], and sparse two dimensional
have been presented for hyperspectral unmixing [41], and CCA (s2DCCA) [52].
a novel BSS approach named semi-blind GMCA (sGMCA) Sparse PCA improved the interpretation of original prin-
was introduced that combined alternating least square algo- cipal components by reformulating eigen-vector decomposi-
2 VOLUME 4, 2016
FIGURE 1. A pictorial description of different blind source separation strategies. The rendered brain is obtained using the BrainNet Viewer [44].
tion as a penalized regression problem to obtain sparse load- RN ×V to be obtained by assembling time courses of length
ings [45]. Sparse CCA extends the sPCA strategy and adapts N across V voxels resulting in a K number of retrieved maps
the elastic net to CCA to get sparse canonical weights [46]. in the source matrix S = TY ∈ RK×V , where T is assumed
Sparse PLS discovered an improved relationship between to be a semi-orthogonal demixing matrix whose pseudo-
two datasets by a penalized matrix decomposition applied inverse is equal to its transpose. As an alternative, temporal
to cross matrix product to obtain sparse weight vectors [49]. BSS considers collecting mixture matrix as Y⊤ ∈ RV ×N to
Based on the l0 norm, the improved fast ICA method was obtain K number of source signals as T = SY⊤ ∈ RK×N ,
proposed by adding the sparsity constraint to the fast ICA where S is a semi-orthogonal demixing matrix.
algorithm [50]. The BSS model given by equation (1) can be extended to
A regularized and a sparse CCA employed penalized rank- a sparse model, which is formulated as [53]
1 matrix approximation on the product of the orthogonal
projectors of two sets of data to improve interpretability ||yi − T⊤ si ||22 +λf (si ), i = 1, . . . , V (2)
and performance of the canonical variates [47]. To simulta-
neously handle the sparse decomposition of mixed signals, where V is the number of voxels, ||yi − T⊤ si ||22 is a smooth
the BSS problem was reformulated as a multi-task sparse and convex data fidelity term and f (.) is a convex regulariza-
problem that exploited connection between tasks to achieve tion function that promotes sparsity, and λ controls the trade-
higher retrieval accuracy [51]. While preserving correlation off between sparsity and reconstruction error. This leads to
structure of fMR images, two variants of s2DCCA, regular- the extraction of source signals from the observed signals in
ized and sparse, were proposed to improve the interpretability an overcomplete manner; however, this model during sparse
of the projected variables [52]. coding decomposes each mixture signal in Y independently,
resulting in an increased learning complexity as shown in Fig.
III. BACKGROUND 2a. One can refer to [54] for a block update of T⊤ and [23]
According to the uniquely/overdetermined case of BSS the- for a sequential update.
ory, where more or an equal number of mixtures than sources Consider on the other hand deflation based sparse methods
are observed, let an observed vector y = [y1 , y2 , . . . , yN ]⊤ [46]–[49],P[55] that exploit a different fidelity term given
K
be a linear mixture of the unknown source vector s = by ||Y − k=1 t⊤ k 2
k s ||F , where K is the number of sources
[s1 , s2 , . . . , sk ]⊤ , i.e., y = T⊤ x, where T ∈ RK×N is a as shown in Fig. 2b. This results in reliance on estimating
transpose of the mixing matrix, and its size is N ≥ K. This one source signal at a time and applying deflation to the
can be more formally expressed for multiple sources as data matrix to remove the effect described by the current
sparse mixing vector. This creates learning inefficiency, just
Y = T⊤ S + E (1) like SDL algorithms given by equation (2), and lowers the
where E is the model error. When implementing (1) using retrieval accuracy because this sort of penalized matrix de-
ICA or CCA, prewhitening the data is the first step. This composition ignores the important relationship between the
is carried out to reduce the correlation between features source signals.
and dimensionality of the data. Only D < N principal The proposed methodology described in the next section
components are kept, followed by the estimation of mixing eliminates the shortcomings mentioned in the last two para-
matrix T⊤ ∈ RD×K as the second step as shown in Fig. 1. graphs. While performing source separation, the relationship
From a general BSS perspective, there are two possible among source signals was taken into account by continuously
outcomes i) spatial BSS would require mixture matrix Y ∈ updating all sources until convergence. This resulted in better
VOLUME 4, 2016 3
FIGURE 2. An algorithmic description of different sparse blind source separation approaches.
PK ⊤
interpretability of retrieved sources compared to the deflation where Ek = Y − i=1,i̸=k ti xi . This is another example
strategy. On the other hand, sparse assumption and reduced- of inefficient learning because it calls dataset L times for
dimension space allowed us to use equation (1) indirectly for each of the K dictionary atoms/sparse codes, creating an
the source separation problem, which resulted in a significant analytic complexity of 4LK and computational complexity
computational gain. of approximately LK(KN V + 3N V + ζ 3 + Kp N ) [56].
Because the proposed algorithm operates in a block manner,
IV. METHODS makes lesser calls to data, and performs lesser computa-
For the proposed method, equation (1) is extended and tions, the total complexity is brought down to approximately
refined to produce a STEM model. Specifically, consider 2LK(K 2 + N V ).
an example of fMRI blood-oxygen-level-dependent (BOLD)
time series, which consists of N scanned volumes and V A. SPATIOTEMPORAL AUTOENCODER
number of voxels arranged along the column direction in A spatiotemporal PCA can be achieved by performing SVD
Y = [y1 , y2 , . . . , yV ] ∈ RN ×V . It is assumed that due to on the data matrix Y to decompose it as
sparseness along the row direction, each signal in Y can be
represented as a linear combination of temporal sources from Y = Ω∆Γ⊤
T according to the sparse coefficient strength in each column where ∆ ∈ RM ×M consists of singular values on the diago-
of the spatial source matrix S. Suppose that the BOLD signal nal, and matrices Ω ∈ RN ×M and Γ ∈ RV ×M consists of M
at any voxel can be assumed smooth due to a neural response number of left and right singular vectors, respectively. Using
that is smoothed by the HRF [56]; then this can be accounted these singular matrices that contain the eigenvectors of the
for as T⊤ = Tp A. To achieve this decomposition, a sparsity temporal and spatial covariance matrices, one can obtain the
constraint is imposed on coefficient rows and corresponding reduced spatial and temporal feature matrices Xt ∈ RK×N
columns of the sparse representation matrix A ∈ RKp ×K and Xs ∈ RK×V by keeping only the first K < M singular
[57] as vectors with maximum variance as
2
min ∥Y − Tp AS∥F + λ ∥X∥1 ,
A,S
Xt = ∆Γ⊤ Y⊤ , Xs = Ω⊤ Y
sub.to. ∥Tp ak ∥2 = 1, ∥ak ∥0 ≤ ζ (3) The objective of PCA to capture information from the data
into a few components may result in severe noise contami-
where Tp ∈ RN ×Kp is the matrix containing DCT basis,
nation of significant components because the hemodynamic
and ak is its k-th column, K < Kp < N , ∥.∥0 is the
response signal of fMRI is usually weak compared to the
l0 norm that induces sparsity by counting the number of
noise signals [8]. One way to resolve this problem is to
non-zero elements, ∥X∥1 is the l1 norm on X and is given
PK PV k
order components by their autocorrelation profile rather than
as k=1 j=1 |xj |, and λ is the sparsity parameter that maximum variance using CCA, but producing corresponding
controls coefficient values in the k-th row. Associated l0 spatiotemporal correlated components is challenging. Due
norm constrained least square is not part of the regularized to this limitation and computational inefficiency associated
problem and is solved separately [58]. One way to solve this with SVD operation, an autoencoder was introduced as an
problem is by computing error matrix of full data [56], [59], alternative to PCA.
[60] to update the unknowns in a sequential manner as A three-step temporal autoencoder [61] has been deployed
2
{ak , xk } = arg min Ek − Tp ak xk for the proposed method. It is an extension of an extreme
ak ,xk F
learning machine equipped with a feedback mechanism to
4 VOLUME 4, 2016
TABLE 1. Algorithm for spatiotemporal dimensionality reduction TABLE 2. Algorithm for realizing l0 constraint
Given: Given:
Training data (Y ∈ RN ×V ) Matrix (T ∈ RP ×N )
Activation function (G) Parameters (Kp , ζ)
Constants (K, L)
Repeat for p = 1 to P :
1. Initialization: a) Compute correlation
vector
as
For a given number of hidden neurons, generate the initial orthonormal c ← sort T⊤ ⊤
p Xt up , descend
weights/biases f A/f b and temporal/spatial feature data f H/f F accor-
b) After sorting correlation vector in descending order keep ζ-largest
ding to equation 4
values as
2. Repeat for i = 1 to L:
a) Hidden nodes j A and j b are obtained in the decoding layer using is ← {cr }ζr=1
least squares and invertible activation function given by equation 5 c) Compute representation vector as
b) Spatiotemporal feature data is generated using hidden nodes f A ap,is ← (T⊤ p,is Tp,is )
−1 T⊤ X⊤ u
p,is t p
and f b from the previous layer and equation 6 d) Normalize as
ap ← ap /∥Tp ap ∥2
Output: f H and f F
Output: A
pull the output data back to hidden nodes to enrich and

accelerate the learning process. It is adapted for the proposed where the sigmoid transfer function is just a proposition, and
ssBSS case to learn the respective spatial features along with sinusoid can be used for the spatial case as well and vice-
† †
temporal ones. It is used as a dimensionality reduction step, versa, f H = (f Hf H⊤ + βK IK )−1f H and f F = (f Ff F⊤ +
−1f
and preferred over PCA-based whitening operation due to its βK I K ) F. In the third step, feature data is updated using
learning efficiency and overcomplete nature. It atleast can weights f A = j A, f C = j C and biases f b = j b1 ∈ R,
f
preserve the quantity of the input information, and subse- d = j d1 ∈ R from the decoding layer as
quent steps of the proposed method can improve quality.
It is a two-layer network where input is transferred to low-
f
H = G f A, f b, Y ⊤ ,

dimensional random feature space followed by the computa- f
F = G f C, f d, Y . (6)
tion of hidden nodes using invertible functions and, finally,
re-estimating low-dimensional feature space by pulling the Steps 2 and 3 are repeated several times until convergence;
weights of the decoding layer back to the hidden nodes. however, we found that the computation of feature data
For a given data matrix Y ∈ RN ×V , orthogonal random f
H just twice is sufficient to transform the data from high
weight matrix f A ∈ RK×V and bias vector f b ∈ RK of to low dimensional space while retaining the meaningful
the encoding layer, the first step constructs the first layer information from the observed data. The detailed description
of the network through the orthogonal random process for of these steps is given in Table 1 in the form of an algorithm.
(N, V ) number of input neurons and (K, K) number of
hidden neurons given as
B. SSBSS

f
H = G f A, f b, Y ⊤ ,
Using blind source separation theory, one can obtain the
f ⊤ ⊤f
Af A = I, f b b = 1, following pair of mixture models given as

f
F = G f H, Y . (4) X⊤ ⊤ ⊤
t =T U , Xs = WS (7)
where the unknowns U, W ∈ RK×P are the mixing matrices

where f H, Y = f H(f H⊤f H + βN IN )−1 Y, and G is any
whereas T ∈ RP ×N and S ∈ RP ×V are the source matrices,
general hidden neuron function such as sigmoid, wavelet,
and K ≥ P . For a non-orthogonal demixing matrices,
radial basis, and sinusoid. In the second step, invertibility
equation (7) and (1) can be manipulated together to obtain
of the activation function is exploited to obtain weights
j the unknowns W and U as
A ∈ RK×N , j C ∈ RK×V and biases j b ∈ RV , j d ∈ RN of
the decoding layer as W = Xs (Y⊤ Y + βV IV )−1 Y⊤ X⊤ ⊤
t (UU + βK IK )
−1
U,
U⊤ = (W⊤ W + βP IP )−1 W⊤ Xs (Y⊤ Y + βV IV )−1 Y⊤ X⊤
†
⊤
j
A = f H sin−1 (Y⊤ ) if G(.) = sin(.), t
†
1 ⊤ 1 where βK IK and βP IP are the Tikhonov regularization
j
C = −f F log −1 if G(.) = , terms to avoid overfitting and singularity of UU⊤ and W⊤ W,
Y 1 + e−(.)
r
⊤ respectively [55]. From the above equation, one can infer that
⊤
j
B = mse f H j A − sin−1 (Y⊤ ) , taking the inverse of the data’s covariance matrix is cumber-
r some. Therefore, it would be more convenient to indirectly
1 ⊤
j ⊤
D = mse f F j C + log −1 . (5) exploit the information from the observed mixture matrix
Y Y to estimate the source signals. Specifically, the source
VOLUME 4, 2016 5
TABLE 3. Algorithm for solving the minimization problem (3) mutually converging to the subspace formed by them. This
can be accomplished by letting T estimated from equation (8)
Given: to play its role in optimizing (9) and using S estimated from
Training data (Y ∈ RN ×V ) equation (9) in solving (8) through their mutual connection
Tuning parameters (λ1 , λ2 , λ3 , ζ, Kp )
Constants (K, P, L, l = 0, τ = 0.05) given by equation (1). Therefore, temporal sources obtained
from the above problem can then be plugged into equation
1. Initialization: (1) to obtain the spatial sources and vice versa.
a) Initialize T using normally distributed random numbers and
normalize its rows Due to l1 norm in (3), proximal splitting methods [62] are
b) Update S using equation (1) deployed to minimize functions that are not differentiable by
c) Initialize U and A with zeros splitting the cost function into a sum of convex functions that
d) Initialize Tp using DCT basis
2. Spatiotemporal Component Extraction: are minimized in an alternating manner. For instance, one
Decompose the data matrix using SVD or algorithm given in Table 1 way to approach this problem is through gradient descent,
3. Repeat while T⊤ − T⊤ ⊤
l F / Tl F > τ : giving rise to the thresholded Landweber problem given as
l ←l+1
T⊤l ←T
⊤
sn+1 = Mλ snk − T(T⊤ snk − yk ) ,
a) Using S update T through equation (1) k
b) Normalize each row of T λ
if update = ’block’ Mλ (z) = sgn(z) ◦ (|z|− )+
c) Fix A, and update U by solving 2
min X⊤ ⊤ 2 + λ ∥U∥ , using equation (12) where (z)+, sgn(.), and ◦ define the component-wise max
U t − Tp AU F 1 1
d) Fix U, and update A using algorithm given in Table 2 and between (0, z), the component-wise sign, and the Hadamard
compute T⊤ ← Tp A product, respectively [63]. For the proposed ssBSS method,
else the sparsity pattern,
for p ← 1 to P however, has been imposed
row-wise,
k,n+1 k,n ⊤ k,n
then s = Mλ s − tk (tk s − Y) . It is speculated

c) u⊤ ⊤ ⊤ ⊤ ◦ |a⊤ T⊤ E⊤ |− λ1p
p ← sgn ap Tp Ep p p p 2 +
d) ap ← arg minap E⊤
2 that similar conditions, that is, when T⊤ holds the finite basis
p up − Tp ap 2 sub.to. ∥ap ∥0 ≤ ζ
end injective property and s†k possesses a strict sparsity pattern,
end if linear convergence rate would be achieved as sk,n → s†k
e) Using T update S through equation (1)
f) Fix S, and update W by solving such that sk,n → s†k ≤ Cα where C > 0 and 0 < α < 1
min ∥Xs − WS∥2F + λ2 ∥W∥1 , using equation (16) [64].
W
g) Fix W, and update S by solving Similarly, it is also postulated that both mixing matrices
min ∥Xs − WS∥2F + λ3 ∥S∥1 , using equation (17) must hold i) strict sparsity pattern because none of their
S
columns can be completely zero for any case, and ii) both
Output: T and S source matrices S/T hold unique sources/basis that cannot
be constructed as a linear combination of the other two.
When a scenario such as a bad source occurs (whose spatial
estimates S and T obtained directly from the entire data source consists of all zeros), the respective temporal source is
matrix can be used to estimate mixing matrices and then re- replaced with the least represented normalized data element
estimate the source matrices. This approach will produce an from the mixture matrix. Before attempting an efficient so-
optimal solution by properly unmixing the PC/autoencoder lution, equation (8) can be re-written for a sequential update
components. For instance, by imposing sparsity on the matri- as
ces W, U, S, and T, the orthogonality condition of the mixing K
matrices, e.g. W† = W⊤ can be violated, and results far ⊤ 2
X
min E⊤ p − T a u
p p p F + λk1p |ukp |,
superior to PCA/autoencoder can be obtained. ap ,u⊤
p
k=1
Therefore, we extend this idea to efficiently solve the non- sub.to. ∥Tp ap ∥2 = 1, ∥ap ∥0 ≤ ζ
smooth constrained optimization problem given in equation
i⊤ ⊤
PP
(3). For this purpose, consider solving equation (3) using (7), where E⊤ p = Xt −
⊤
i=1,i̸=p t ui . Then the following
which breaks (3) into following pair of problems closed-form solution can be reached by solving this equation
2 with respect to the unknowns as shown in [56]
min X⊤
t − Tp AU
⊤
F
+ λ1 ∥U∥1 ,
A,U
λ1p
u⊤
p = sgn a⊤ ⊤ ⊤
T
p p pE ◦ |a ⊤ ⊤ ⊤
T
p p p E |− ,
sub.to. ∥Tp ap ∥2 = 1, ∥ap ∥0 ≤ ζ (8) 2 +
2
ap = arg min E⊤
p up − Tp ap 2
sub.to. ∥ap ∥0 ≤ ζ (10)
2 ap
min ∥Xs − WS∥F + λ2 ∥W∥1 + λ3 ∥S∥1 (9)
S,W
where λ1p = [λ11p , . . . , λk1p ]. Although this error matrix
According to Neumann’s alternating projection strategy [43], based solution in combination with equation (9) is not based
these two subproblems can be solved iteratively and alter- on a full data matrix, it is still only efficient for the synthetic
nately until the convergence of the unknowns, and they will data. As the size of the data increases, it becomes an expen-
ultimately result in an optimal solution of equation (3) while sive update, and therefore we shift our focus toward the block
6 VOLUME 4, 2016
FIGURE 3. A block diagram of the proposed method where the autoencoder performs dimensionality reduction and the rest of the blocks attempt to recover
underlying spatial and temporal sources.
FIGURE 4. The size of 8 different activation blobs controlled using spread parameter ρ to create none to significant spatial overlap.
update for all experimental data. Taking the Lagrangian of While disregarding penalization on mixing matrix, from (15)
equation (8) provides an update for W is given as
L(A, U) = Xt X⊤ ⊤ ⊤ ⊤
t − 2Xt UT + UTT U + λ1 ∥U∥1 (11) W = Xs S⊤ (SS⊤ + βP IP )−1 (16)
Ignoring penalization on the mixing matrix and after some To accommodate penalization on W, take into account the
manipulations, a block update for U can be obtained as soft thresholding given by wp = sgn(wp ) ◦ (|wp |− λ22 )+ .
Similarly, S, while ignoring penalization on its entries, can
U = Xt T⊤ (TT⊤ + βP IP )−1 (12) be updated as
Now taking penalization of U into account, univariate soft S = (W⊤ W + βP IP )−1 W⊤ Xs (17)
thresholding can be performed on it as up = sgn(up ) ◦
(|up |− λ21 )+ , and block update for T is obtained as and its soft thresholding is implemented as sp = sgn(sp ) ◦
(|sp |− λ23 )+ . Using equation (12, (14), (16), and (17)) a
T⊤ = X⊤ ⊤
t U(U U + βP IP )
−1
(13) block update for four unknowns U, A, W, and S can be
obtained by an alternating minimization approach where one
Using (13) an update for p-th column of A can be obtained variable is kept fixed while other one is updated. The related
as ap = (T⊤ p Tp )
−1 ⊤ ⊤
Tp Xt up . Using this solution of ap and optimization steps are given as an algorithm in Table 3 and in
imposing l0 norm constraint on it gives rise to the following terms of a flow chart in Fig. 3.
constrained problem given as
2 V. EXPERIMENTS
ap = arg min X⊤
t up − Tp ap 2
sub.to. ∥ap ∥0 ≤ ζ (14)
ap To validate the proposed ssBSS method, its evaluation is
carried out in this section. It consists of several compar-
Solution for (14) is ap,is = (T⊤p,is Tp,is )
−1 ⊤
Tp,is X⊤
t up , where ative studies implemented using a synthetic dataset, three
finding the indices set is using correlation based thresholded experimental fMRI datasets, and existing BSS algorithms
values [56], [65] is the main task as described in Table 2 in based on mixing matrices and sparse weights strategies. The
the shape of an algorithm. Equation (9) can be written as participating algorithms are introduced in Table 4 along with
⊤ ⊤ their code availability. These are JADE, sICA, sPCA, ifICA,
L(S, W) = X⊤ ⊤
s Xs − 2Xs WS + S W WS + λ2 ∥W∥1
ssICA, sPLS, sCCA, and the variants of the proposed method
+ λ3 ∥S∥1 (15) i) ssBSSP, ii) ssBSSA, and iii) ssBSSAS.
VOLUME 4, 2016 7
FIGURE 5. Mean values of a) mcTC and b) mcSM over 100 trails where sources are recovered with respect to GT-SMs.
FIGURE 6. For all variants of the proposed method, a) convergence rate, and b) the mean of the sum of the highest correlations between ground truth SMs/TCs
and recovered sources shown as a function of algorithm iterations.
The synthetic dataset was prepared using the Simtb tool- aged by trying their different combinations. The ones
box [66], the block-design fMRI dataset of a single subject that produced the best results in terms of the sum of
was acquired from the quarter 3 release of the Human Con- the correlation strength between the recovered and the
nectome Project (HCP) [67], [68], the resting state dataset ground truth sources were considered.
of a single subject was obtained from the first set of the 3) It was aimed to preserve most of the information in
quarter 11 release of the HCP [67], [68], and the event- the data when performing dimensionality reduction by
related dataset of a single subject was provided by the Korea PCA or autoencoder. For this purpose, trying different
Advanced Institute of Science and Technology (KAIST) and values and selecting the best one was preferred over the
has been previously used in other papers [69]–[71]. Using model selection approach such as Akaike’s Information
these datasets, all participating BSS algorithms are compared criterion [72].
to assess their ability to recover the ground truth. 4) While selecting the total number of DCT basis and their
Before commencing with the analysis, it is imperative to associated sparsity parameter for the proposed method,
mention the following specifics, which are common to all our approach relied on a bias-variance tradeoff to avoid
datasets: preserving too much information and ignoring essential
features in the temporal source. For this purpose, several
1) For the variants of the proposed method, the matrix T bases were tried between 60 and N before reaching an
was generated by drawing random numbers from the optimal number.
standard normal distribution when using both synthetic 5) For sPLS and sCCA, the second multivariate data set
and experimental datasets. was generated using a lagged version of SVD’s left
2) Each algorithm’s tuning parameter settings were man-
8 VOLUME 4, 2016
TABLE 4. A summary of the BSS algorithms that have been included in the comparative study.
Algorithms Approach Matlab code availability

JADE [39] Data whitening followed by the computation of fourth-order cumulants http://www2.iap.fr/users/cardoso/compsep_classic.html
and rotation matrix
sICA [21] Data whitening followed by the gradient descent method to estimate the https://research.ics.aalto.fi/ica/fastica/
demixing matrix
sPCA [45] Singular value decomposition followed by the estimation of sparse http://www2.imm.dtu.dk/projects/spasm/
weights
ifICA [50] Two-step optimization strategy i) maximize independence, ii) refine the https://bit.ly/3nGxxUl
results in the previous step using the sparsity constraint
ssICA [33] Three-step optimization approach i) initial estimate of the demixing https://zoisboukouvalas.github.io/Code.html
matrix, ii) refinement through orthogonal ICA, iii) further refinement
through non-orthogonal ICA
sPLS [49] Utilize rank-1 decomposition of the covariance matrix to estimate
sparse weights
sCCA [46] Utilize iterative partial least square and soft thresholding to estimate https://pypi.org/project/cca-zoo/
sparse weights
Proposed (ssBSSP) Undercomplete basis obtained through dimensionality reduction using
SVD/PCA followed by the block update of sparse mixing/source matri-
ces
Proposed (ssBSSA) Overcomplete basis obtained through dimensionality reduction using https://github.com/usmankhalid06/ssBSS_2023
autoencoder followed by the block update of sparse mixing/source
matrices
Proposed (ssBSSAS) Overcomplete basis obtained through dimensionality reduction using
autoencoder followed by the sequential update of temporal sparse mix-
ing/source matrix and the block update of spatial sparse mixing/source
matrix
singular vectors. The reason for including 4a is to create a scenario with

6) The number of underlying sources is unknown for ex- no spatial dependence but considerable sparseness of the
perimental fMRI data, so their selection was based on spatial sources. These sources were then utilized to generate
trial and error. Several numbers ranging between 30 and a synthetic
P8 dataset using a linear mixture model given as
60 were tested, and the ones that gave the best results Y = i=1 (tci + ψi )(smi + ϕi ), where the noise generating
were used for each algorithm. matrices Ψ ∈ R240×8 and Φ ∈ R8×22500 were produced
7) Because sICA and ssICA gave different results and using Gaussian distribution ∼ N (0, 0.6) and ∼ N (0, 0.01),
computation times when applied repeatedly on the same respectively, TC ∈ R240×8 contains the time courses, and
dataset, they were applied ten times on each experimen- SM ∈ R8×22500 contains the spatial maps obtained after
tal data. The run with best results was acknowledged, reshaping the image slices. Depending on the value of ρ, the
and the mean computation time over ten runs was con- resulting dataset Y was then generated and used for source
sidered. retrieval by all algorithms.
A. SYNTHETIC DATASET 2) Synthetic dataset source separation

1) Synthetic dataset generation To keep the convergence rate comparable, the total number of
In order to produce a dataset that mimics the experimental iterations was set to 30 for all algorithms. Unlike real fMRI
fMRI signals, the Simtb toolbox was employed. It allowed data, ground truth about the number of generating sources is
constructing eight distinct temporal and spatial sources for known for the synthetic dataset. Therefore, the same number
synthetic dataset generation. Each time source consisted of of components were trained for each algorithm. The total
240 points with a repetition time (TR) of 1 sec, and each number of components to be extracted was set to 8 for JADE,
spatial source comprised a square image of 150 × 150 sPCA, sPLS, and sCCA. Using SVD dimensionality of the
voxels. The spatial components were generated by these data was reduced to 16, from which eight components were
source IDs {3, 6, 8, 10, 22, 23, 26, 30}. Activations within the extracted using sICA, ssICA, ifICA, and ssBSSP. Whereas
image slices were created by combining Gaussian distribu- 250 components were kept from the autoencoder, and then
tions parameterized by location, orientation, and spread. The eight sources were retrieved for ssBSSA and ssBSSAS. The
spatial extent of these activations was controlled using the ability of the autoencoder to extract a set of overcomplete
spread parameter (ρ), which is the mean of the Gaussian basis is exhibited by reducing the data dimensionality to 250,
distribution ∼ N (ρ, 0.012 ). The spatial maps (SMs) with which is greater than the total number of time points. As the
none to significant spatial overlaps are shown in Fig. 4a-e, selection of tuning parameters was based on getting the best
and their respective time courses (TCs) are shown in Fig. 7, results in terms of correlation values, many values were tried
which are also considered as the ground truth (GT) TCs and before setting the sparsity parameter to 300 for sPCA, 30 for
SMs when retrieving the underlying sources. sCCA, and 5 for sPLS, and default parameter settings were
VOLUME 4, 2016 9
TABLE 5. Mean correlation values over 10 trials for ρ = 4.5.
Srcs JADE sCCA sPLS sICA ssICA ssBSSP ssBSSA

TC1 0.762 0.506 0.777 0.745 0.731 0.819 0.821
TC2 0.703 0.513 0.638 0.687 0.646 0.820 0.796
TC3 0.687 0.634 0.732 0.603 0.750 0.787 0.851
TC4 0.671 0.485 0.776 0.700 0.728 0.825 0.827
TC5 0.807 0.738 0.771 0.804 0.809 0.815 0.855
TC6 0.816 0.654 0.806 0.700 0.814 0.793 0.863
TC7 0.730 0.493 0.697 0.742 0.761 0.737 0.826
TC8 0.716 0.487 0.704 0.702 0.681 0.765 0.744
SM1 0.866 0.576 0.850 0.865 0.880 0.923 0.956
FIGURE 7. Activation maps with a different spatial dependence compared to SM2 0.761 0.631 0.815 0.780 0.862 0.839 0.928
Fig. 4 and the corresponding time courses. SM3 0.842 0.784 0.877 0.681 0.877 0.754 0.909
SM4 0.884 0.520 0.811 0.888 0.898 0.884 0.932
SM5 0.868 0.593 0.760 0.862 0.893 0.856 0.859
SM6 0.895 0.557 0.716 0.830 0.922 0.829 0.921
used for ifICA. The sparsity and smoothing parameter for SM7 0.827 0.659 0.814 0.815 0.774 0.896 0.885
ssICA was set to 0.7 and 105 , respectively. For ssBSSP, the SM8 0.835 0.632 0.641 0.831 0.843 0.914 0.912
sparsity parameters were set as λ1 = λ2 = 0.01 and ζ = 90, Mean 0.792 0.591 0.761 0.765 0.804 0.829 0.868
whereas λ1 = 2, λ2 = 3, and ζ = 60 for ssBSSA, and
λ1 = λ2 = 2 and ζ = 60 for ssBSSAS, and the total number
increased computational complexity. Moreover, compared to
of DCT basis was set to Kp = 150 and λ3 = 12 for all three
other algorithms, the proposed algorithms performed excep-
variants.
tionally in unveiling spatial maps when the sources were very
sparse, that is, ρ = 0.6. It is worth mentioning that ssICA
3) Synthetic dataset results
turned out to be a runner-up as the spatial dependence was
This section discusses simulation results to demonstrate the
increased, and JADE also accomplished impressive results
robustness and consistency of the proposed method to spatial
over all cases. Fig. 6 illustrates that all variants of the pro-
dependence. In this regard, the spread parameter ρ was varied
posed method have converged around the fifteenth iteration,
between 0.6 and 5 to gradually create activation overlaps
after which they also stopped accumulating the correlation
as shown in Fig. 4a-e and Fig. 7. Initially, five different
strength. After the fifteenth iteration, this strength was main-
activation instances were created using five different values
tained as the algorithms settled down after convergence to a
of ρ. For each activation instance, all algorithms attempted
local minimum.
to recover the underlying sources 100 times, where every
Later on, in order to obtain component-wise analysis and
time the dataset Y was generated using Gaussian distribution
keep computational analysis on equal footing to the real
∼ N (0, 0.6) and ∼ N (0, 0.01) , for spatial and temporal
fMRI datasets, the experiment was repeated for ρ = 4.5
noise respectively.
as shown in Fig. 7. For this experiment and the real fMRI
Underlying source TCs/SMs were recovered by correlat-
dataset in subsequent sections, ssBSSAS, due to its higher
ing every algorithm’s extracted sources with the ground truth
computational cost, ifICA and sPCA, due to their inferior
sources and keeping the indices of the highest correlation
results in terms of spatial maps, have been dropped from
values. These values were computed with respect to GT-SMs
further analysis. This experiment was repeated ten times
and saved as cTC/cSM. Except for JADE and ifICA, where
with a random spatial and temporal noise of variance 0.6
temporal sources were recovered first, all other algorithms
and 0.01, respectively. The mean of correlation values over
recovered spatial maps and then obtained corresponding tem-
ten trials is given in Table 5 where one can see that the
poral sources using least squares. For each of the 100 trials
block variant of the proposed method with components from
and five spatial overlap instances, the mean of the highest
autoencoder (ssBSSA) has outperformed all other algorithms
correlation values for eight spatiotemporal sources is saved
by recovering both spatial and temporal sources with the
as mcTC/mcSM, and their mean is plotted in Fig. 5. The
highest correlation values except the few occurrences. In this
convergence rate of all variants of the proposed method as
case, the total computation time in seconds over all trials
a function of algorithm iterations is plotted in Fig. 6a. The
is given in the top row of Table 9 with the lowest values
progression of the mean of the sum of correlation values
highlighted in bold.
between the recovered spatiotemporal sources and ground
truth sources over the algorithm iterations is shown in Fig.
6b. B. EXPERIMENTAL FMRI DATASET
It can be deduced from Fig. 5 that the proposed method’s 1) Experimental fMRI dataset preprocessing
variants achieved the highest source recovery performance. Excluding the resting-state dataset, which the HCP had
This performance, although decreased with increasing spa- already processed using their preprocessing pipelines, the
tial overlaps, it remained higher than all other competing block-design, and the event-related dataset were prepro-
algorithms. Among all the variants, the ssBSSAS algorithm cessed using the SPM-12 toolbox [1]. The preprocessing
seemed to have performed slightly better than ssBSSA for steps comprised realignment, normalization, spatial smooth-
some cases; however, this performance gain is at the cost of ing, and masking. More details about these steps can be
10 VOLUME 4, 2016
FIGURE 8. For the block-design dataset’s left toe (A) and tongue (B)
movement tasks, the most correlated temporal components with MHR
obtained using a) JADE, b) sICA, c) ssICA, d) ssBSSP, and e) ssBSSA. The
corresponding correlation values are given in Table 6.
TABLE 6. Correlation values of most correlated temporal components with

MHRs for six different tasks obtained using five BSS algorithms and two
variants of the proposed method. FIGURE 9. Thresholded activation maps for left toe (A) and tongue (B)
movement tasks at a random field correction p < 0.001 obtained using a)
JADE, b) sICA, c) ssICA, d) ssBSSP, and e) ssBSSA.
Algos VC LT LF RT RF T Mean
JADE 0.679 0.622 0.393 0.521 0.407 0.451 0.512
sICA 0.643 0.699 0.700 0.734 0.405 0.790 0.662 of the HCP [67], [68]. To map the primary motor cortex of
ssICA 0.716 0.791 0.593 0.738 0.501 0.744 0.681
sPLS 0.585 0.410 0.654 0.689 0.579 0.737 0.609 the brain experiment was run for 204 secs; the visual cues
sCCA 0.554 0.618 0.517 0.526 0.356 0.764 0.556 were presented to the subjects to tap their right or left fingers,
ssBSSP 0.733 0.742 0.705 0.555 0.571 0.734 0.673
ssBSSA 0.754 0.756 0.743 0.668 0.659 0.725 0.717 squeeze their right or left toes, or move their tongue. Subjects
were shown a three-second visual cue followed by a specific
movement task lasting 12 seconds. There were ten movement
found in [56]. To correct for motion artifacts, functional tasks, including left/right finger, left/right toe, and two tongue
images were realigned to the first image, then all images movements. Thus, there were a total of 13 blocks, including
were spatially normalized to a standard Tailarach template, three fixation blocks that lasted for 15secs. To obtain GT-
resampled to 2 × 2 × 2 mm3 voxels, and spatially smoothed TCs, six modeled HRF (MHRs) were produced by using the
using a 6 × 6 × 6 mm3 full-width at half-maximum (FWHM) convolution operation between the canonical HRF and task
Gaussian kernel followed by the masking step to remove stimuli related to 5 movement types {left toe (LT), left finger
any data outside the scalp. After the masking step, the (LF), right toe (RT), right finger (RF), tongue (T) and visual
four-dimensional datasets were reshaped and stored in a 2- type cue (VC)}. A Siemens 3 Tesla (3T) scanner was used
dimensional matrix Y to be considered as a whole brain to acquire fMRI images for each subject. The acquisition
dataset. This resulted in the size of Y to be 284 × 236115 parameters were TR = 0.72 secs, echo time (TE) = 33.1 ms,
for the block-design dataset, 311 × 166390 for the event- field of view (FOV) = 208 × 180 mm, flip angle (FA) =
related dataset, and 400×230367 for the resting state dataset. 52o , matrix size = 104 × 90, slice thickness = 2 mm with 72
As a second last step, temporal filtering was performed on contiguous slices, and 2 mm isotropic voxels, echo spacing
all three datasets, which consisted of a high-pass DCT basis = 0.58 ms, BW = 2290 Hz/Px, and 284 EPI volumes were
set filter to remove scanner-induced trends and a low-pass collected. The dataset of a single subject with ID 163129,
FWHM filter to remove physiological noise. The cutoff for aged between 26 and 30 years, was used in our analysis.
a DCT-based filter was set to 1/150 Hz for block design,
whereas 1/128 Hz for event-related and resting state datasets. 3) Block design dataset source separation
FWHM of 2 secs was used for block design, 1.5 secs for Many values of the tuning parameters were tested, and
event-related, and 1 sec for resting state dataset. These cutoffs their best combination in terms of the sum of correla-
were assigned according to the previous studies [69], [71]. As tion strength with the ground truth was considered. The
a final step, all column entries of Y were normalized to have total number of components to be estimated was set
zero mean and unit variance for all datasets. to 40 for sPLS, and sCCA. While performing dimen-
sionality reduction 40/60/50 components were kept from
2) Block design dataset PCA, and then 40/60/50 sources were uncovered using
The motor task 3T MRI unprocessed block-design dataset JADE/sICA/ssICA. Whereas 60/105 components were kept
used in this section was acquired from the quarter 3 release from PCA/autoencoder, and then 40/35 were extracted using
VOLUME 4, 2016 11
FIGURE 11. For the event-related dataset, default mode network retrieved by
keeping the most correlated spatial map with R4 RSN templates obtained from
a) JADE, b) sICA, c) ssICA, d) ssBSSP, and e) ssBSSA.
TABLE 7. Correlation values of most correlated temporal components with

finger tapping MHR and most correlated spatial map with few RSN templates
using five BSS algorithms and two variants of the proposed method. The best
values are highlighted in bold.
Algos FT R1 R2 R4 R9 R10 Mean

JADE 0.620 0.416 0.518 0.462 0.245 0.217 0.413
sICA 0.651 0.533 0.515 0.357 0.324 0.253 0.439
ssICA 0.637 0.489 0.468 0.451 0.314 0.258 0.436
sPLS 0.481 0.425 0.409 0.379 0.345 0.281 0.387
FIGURE 10. For the even-related dataset, thresholded activation maps for the sCCA 0.427 0.382 0.220 0.482 0.362 0.202 0.346
right finger tapping task at a random field correction p < 0.001 and ssBSSP 0.558 0.544 0.426 0.572 0.527 0.378 0.501
corresponding most correlated temporal source with MHR function obtained ssBSSA 0.645 0.523 0.565 0.583 0.450 0.426 0.532
using a) JADE, b) sICA, c) ssICA, d) ssBSSP, and e) ssBSSA. The respective
correlation values are given in Table 7.
algorithms for these plotted tasks, the recovered temporal
dynamics are comparable and spatial maps are distinct but
ssBSSP/ssBSSA. The total number of iterations was set to analogous. These results for the motor task dataset are very
30 for all algorithms. The sparsity parameter was set to 15 similar to what has been reported in other papers [56], [71],
for sCCA and sPLS. The sparsity and smoothing parameter [73].
for ssICA was set to 50 and 105 . For ssBSSP, the sparsity
parameters were set as λ1 = λ2 = 0.01, λ3 = 16, ζ = 50, 5) Event related dataset
and Kp = 60, whereas λ1 = 0.3, λ2 = 0.11, λ3 = 16, The motor task 3T MRI unprocessed event-related dataset
ζ = 50, and Kp = 60 were used for ssBSSA. used in this section was provided by KAIST [69]. During
the experiment, which was recorded for 650 secs, subjects
4) Block design dataset results were asked to perform a right finger tapping task to map the
Due to the absence of ground truth for spatial maps, we left motor area of the brain. The session began with an initial
decided to base our analysis on the BOLD time series via six preparation period of 30 secs and a resting period of 30 secs,
constructed MHRs. This was accomplished by correlating the followed by 40 repeated trials of the task and resting period.
MHRs with temporal components retrieved by all algorithms Here each trial’s duration is 28 secs, always starting with a
and saving the highest correlation values. These values are task period and ending with a rest. This was followed by an
mentioned in Table 6 with highest values highlighted in bold, additional 30 secs resting period. The interstimulus interval
and a couple of recovered temporal components (left toe and varied between 4 and 20 secs, with a mean of 12 secs within
tongue along with the MHRs in red) and their corresponding each repeated trial. The brain scans were acquired using the
spatial maps are given in Fig. 8 and 9, respectively. Only 3T ISOL Technologies MRI system Korea. The acquisition
two activation maps have been shown to avoid unnecessar- parameters were matrix size = 64×64, TR/TE = 2000/35ms,
ily increasing the paper length. The table shows that the FA = 80o , slice thickness = 4 mm with 24 contiguous slices,
proposed ssBSSA algorithm has the highest similarity with voxel size = 3.44×3.44×4 mm3 . A total of 325 EPI volumes
the ground truth overall. Fig. 9 shows that the variants of were collected, from which the first 15 preparation scans
the proposed (ssBSS) method, specifically ssBSSA, retrieved were discarded. The dataset of a single subject with ID s3sh
activation maps that are very specific to the motor area aged between 23 and 28 years approximately was used in our
compared to other algorithms. Although, ssBSSA has not analysis.
produced the highest correlation values compared to other
12 VOLUME 4, 2016
FIGURE 12. For the resting-state dataset, occipital pole visual (A) and lateral visual (B) network retrieved by keeping the most correlated spatial maps with R2 and
R3 RSN templates, respectively, obtained from a) JADE, b) sICA, c) ssICA, d) ssBSSP, and e) ssBSSA.
TABLE 8. Correlation values of the most correlated spatial component with ground truth, and the highest correlation values were saved.
few RSN templates using seven different algorithms, including the variants of
the proposed. These values are reported in Table 7 with highest values high-
lighted in bold, task-related (right finger tappig) recovered
Algos R1 R2 R3 R4 R9 R10 Mean time course and its associated spatial maps are plotted in
JADE 0.420 0.617 0.227 0.331 0.316 0.344 0.376 Fig. 10, and recovered default mode network (R4) is plotted
sICA 0.496 0.629 0.235 0.333 0.347 0.337 0.396 in Fig. 11. Overall, the ssBSSA algorithm has produced
ssICA 0.486 0.611 0.327 0.276 0.275 0.298 0.379
sPLS 0.455 0.551 0.394 0.329 0.269 0.254 0.375 the highest correlation values. Like the block-design dataset,
sCCA 0.273 0.284 0.231 0.316 0.141 0.209 0.242 task-related activation maps from all algorithms seem to have
ssBSSP 0.678 0.704 0.437 0.424 0.351 0.372 0.494
ssBSSA 0.640 0.712 0.448 0.485 0.380 0.374 0.507 recovered activations very specific to the motor area, and
all algorithms have successfully recovered the default mode
network. In both cases, the maps obtained using ssBSSA
6) Event related dataset source separation
seems to be more specific compared to other algorithms.
The same strategy used for the block-design dataset was The recovered activation maps are similar to earlier reported
adapted for event-related dataset analysis. We commenced by results for this dataset [37], [69]–[71], [75].
selecting the optimal number of sources and tuning parame-
ters by experimentation. To keep the comparison unbiased,
8) Resting state dataset
all algorithms were run for 30 iterations. For sPLS, and
sCCA, 40 components were estimated. For JADE, sICA The resting state dataset was obtained from the first set of
and ssICA, 40 components were estimated after retaining 3T MRI preprocessed quarter 11 release of the HCP [67],
the same number of components from PCA. Whereas for [68]. The same acquisition parameters as for the block-design
ssBSSP/ssBSSA, the dimensionality of the data was reduced dataset given in Section V-B2 were used. The experiment was
to 60/105 using PCA/autoencoder to extract 40 components run for 873 secs, resulting in a total of 1200 scans. This data
in both cases. For sCCA and sPLS, the sparsity parameter was acquired two times in a single session for two different
was set to 25 and 8, respectively. For ssICA, the smooth- phase encoding directions. Subjects were asked to keep their
ing/sparsity parameters were set to 105 /5. The sparsity pa- eyes fixated on a bright crosshair presented in a darkened
rameters were set to λ1 = λ2 = 0.025, λ3 = 11.5, ζ = 150, room, and the second run that consisted of left to right
and Kp = 300 for ssBSSP. On the other hand, λ1 = 0.5, phase encoding was considered for our analysis. The first
λ2 = 0.01, λ3 = 11.5, ζ = 150, and Kp = 300 were used 20 scans were discarded, the following 400 scans were kept
for ssBSSA. for analysis, and the remaining 780 scans were discarded.
Before applying temporal filtering, the preprocessed data
7) Event related dataset results downloaded from the HCP website also underwent spatial
This section deployed task-related MHR and some resting- smoothing using 6 × 6 × 6 mm3 FWHM Gaussian kernel.
state networks (RSNs: R1-R10) described in [74] as the The dataset of a single subject with ID 100206, aged between
ground truth. Using recovered components from each algo- 26 and 30 years, was used in our analysis.
rithm, both temporal and spatial sources correlated with the
VOLUME 4, 2016 13
FIGURE 13. For two variants of the proposed method, a) convergence rate, and b) the mean of the sum of highest correlations between ground truth
spatiotemporal sources and retrieved sources shown as a function of algorithm iterations.
9) Resting state dataset source separation TABLE 9. Total and mean computation time in seconds and correlation
values by each algorithm for four different datasets.
Similar to the last three datasets, all algorithms were iterated
30 times for an unbiased comparison. The number of sources Data sCCA ssICA JADE sPLS sICA ssBSSP ssBSSA
to be uncovered by each algorithm was decided by trying SD 107 70 2 15 7 25 33
numbers between 30 and 60. Different tuning parameter BD 450 531 415 211 134 33 21
ER 370 232 71 145 19 40 25
combinations were tested, and only the best were considered. RS 510 541 135 195 45 54 42
The number of components estimated using sPLS and sCCA Mean 359 344 156 142 51 38 30
was 40. Whereas the dimensionality of the data was re- SD 0.591 0.804 0.792 0.761 0.765 0.829 0.868
duced to 40/60/40, and the number of components estimated BD 0.556 0.681 0.512 0.609 0.662 0.673 0.717
ER 0.346 0.436 0.413 0.387 0.439 0.501 0.532
was set to 40/40/30 for JADE/sICA/ssICA. On the other RS 0.242 0.379 0.376 0.375 0.396 0.494 0.507
hand, the dimensionality reduction was set to 60/105 for
Mean 0.434 0.575 0.523 0.533 0.566 0.624 0.656
ssBSSP/ssBSSA, while only 40 components for both algo-
rithms were eventually retained. The best sparsity settings
for sCCA/sPLS were found to be 60/12. The best smooth- C. DISCUSSION
ing/sparsity parameters for ssICA were noted as 105 /10. The Some statistics about the proposed method and its overall
sparsity parameters were set to λ1 = λ2 = 0.01, λ3 = 13, comparison with other algorithms are illustrated in Fig. 13
ζ = 150, and Kp = 300 for ssBSSP. In contrast, λ1 = 0.2, and 14. The convergence rate of the block variants of the
λ2 = 0.2, λ3 = 13, ζ = 150, and Kp = 300 were used for proposed method for block-design (BD), event-related (ER),
ssBSSA. and resting-state (RS) datasets as a function of algorithm
iterations is plotted in Fig. 13a. The growth of the mean
10) Resting state dataset results of the sum of correlation coefficients between the retrieved
The resting state networks (R1-R10), as reported in [74], spatiotemporal and ground truth sources over the algorithm
were factored in as the ground truth. Six of these networks iterations is shown in Fig. 13b. To give an idea about the
were found to be recovered by all algorithms. The recov- sparseness of the mixing matrix for the ssBSSA algorithm,
ered components were correlated with six of these resting- its image with scaled colors for the event-related dataset is
state templates, and the highest correlation values were kept, drawn in Fig. 14a. The total computation time and correla-
which are reported in Table 8 with best values highlighted tion values by all participating algorithms for all datasets,
in bold. The recovered activation maps that were most cor- including the synthetic dataset (SD), are given in Table 9. The
related with R2 and R3 are plotted in Fig. 12. Overall, pictorial version of this performance comparison is provided
the ssBSSA algorithm has produced the highest correlation in Fig. 14b, where the mean computation time values have
values. All algorithms successfully recovered the activation been normalized to accommodate correlation values and
network for the occipital pole region; however, only ssICA, computation time in the same graph.
ssBSSA, and ssBSSP could correctly recover the ground Fig. 13a shows that apart from some fluctuations, both
truth for the lateral visual network. Maps from both ssBSSA variants of the proposed method converged for all experimen-
and ssBSSP were found to be more specific, with lower false tal datasets consistently; however, this consistency was more
positives compared to other algorithms. evident for ssBSSP. Fig. 13b shows the gradual development
of source retrieval where the ssBSSA has been shown to
14 VOLUME 4, 2016
FIGURE 14. a) Temporal mixing matrix (U) of the event-related dataset revealing sparseness of its entries, b) comparison of all participating algorithms in terms of
computational time and source retrieval accuracy.
have performed better than ssBSSP. After observing both into account the temporal smoothness and the spatial sparse-
subfigures, one can conclude that both algorithms for all three ness along the fMRI data matrix columns and rows, respec-
datasets started to settle down after the fifteenth iteration tively, is computationally intensive to solve; however, by
except for ssBSSA in case of the BD dataset. Fig. 14b manipulating the BSS theory, it was disintegrated into two
displays the overall performance by all algorithms where sub-optimization problems. Based on Neumann’s alternating
the variants of the proposed method have exhibited superior projection lemma, its solution is based on performing the
results in terms of computation time and source recovery block/sequential update of the source/mixing matrices from
strength, with ssICA being the runner-up in source recovery these two sub-problems in an iterative manner. This strategy
category and sICA in computation time. allowed us to solve the STEM model in a computationally
efficient manner. It was speculated and then found empiri-
VI. CONCLUSION cally that convergence of the proposed model is guaranteed
A new source separation method is proposed for fMRI if finite basis injective property and a strict sparsity pattern
data, and all possible settings for its solution are presented, are sustained. It produced results that were specifically su-
including thresholding, PCA, autoencoder, and discussion perior to strategies based on the combination of sparsity and
on convergence. The effectiveness of the proposed ssBSS independence.
method is demonstrated using synthetic and experimental
fMRI datasets. Its performance was consistent across exper- VII. ACKNOWLEDGMENT
iments, and its computational simplicity makes it favorable The block-design and resting-state fMRI data was provided
over swBSS and scBSS variants such as SOBI, JADE, sPCA, [in part] by the Human Connectome Project, WU-Minn Con-
sCCA, sPLS, sICA, ssICA, and ifICA. Its reduced com- sortium (Principal Investigators: David Van Essen and Kamil
putational complexity is in terms of analytical complexity Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes
associated with lesser calls to data and arithmetic complex- and Centers that support the NIH Blueprint for Neuroscience
ity associated with lesser operations required to solve the Research; and by the McDonnell Center for Systems Neu-
method [76]. It can be considered a promising alternative to roscience at Washington University. On the other hand, the
ICA, sparse ICA, and other swBSS-based neuroimaging data event-related fMRI data was provided by Jong Chul Ye of
analysis methods. Investigating the fluctuations exhibited in the Korea Advanced Institute of Science and Technology.
the convergence of the ssBSSA algorithm and using model The authors are grateful to Dr. Sohaib Z. Khan from Is-
selection criteria for the optimal number of PCA/autoencoder lamic University in Madinah, Saudia Arabia, for his constant
components can be pursued as part of future work. support and mentoring during this work.
Unlike conventional time-consuming strategies, such as
joint diagonalization of covariance matrices, deflation, inde- REFERENCES
pendent decomposition of each mixture signal during sparse [1] W. Penny, K. Friston, J. Ashburner, S. Kiebel, and T. Nichols, Statistical
Parametric Mapping: The Analysis of Functional Brain Images. Aca-
coding, and repeated error matrix computation of full data demic Press, 2007.
matrix, the proposed method concurrently exploited spatial [2] K. J. Friston, P. Jezzard, and R. Turner, “Analysis of functional MRI time-
and temporal information from reduced-dimension space in series,” Human Brain Mapping, vol. 1, no. 2, pp. 153–171, 1994.
[3] G. K. Aguirre, E. Zarahn, and M. D’esposito, “The variability of human,
a sparse manner, leading to better source recovery at a BOLD hemodynamic responses,” Neuroimage, vol. 8, no. 4, pp. 360–369,
reduced computational cost. The STEM model that takes 1998.
VOLUME 4, 2016 15
[4] M. U. Khalid and A.-K. Seghouane, “Constrained maximum likelihood [27] X. Zhuang, Z. Yang, and D. Cordes, “A technical review of canonical cor-
based efficient dictionary learning for fMRI analysis,” in IEEE Interna- relation analysis for neuroscience applications,” Human Brain Mapping,
tional Symposium on Biomedical Imaging (ISBI), 2014, pp. 45–48. vol. 41, no. 13, pp. 3807–3833, 2020.
[5] M. U. Khalid and A.-K. Seghouane, “A single SVD sparse dictionary [28] S. Moussaoui, D. Brie, O. Caspary, and A. Mohammad-Djafari, “A
learning algorithm for fMRI data analysis,” in IEEE Workshop on Statisti- Bayesian method for positive source separation,” in IEEE International
cal Signal Processing (SSP), 2014, pp. 65–68. Conference on Acoustics, Speech, and Signal Processing, vol. 5, 2004, pp.
[6] M. J. Mckeown, S. Makeig, G. G. Brown, T.-P. Jung, S. S. Kindermann, V–485.
A. J. Bell, and T. J. Sejnowski, “Analysis of fMRI data by blind separation [29] P. Georgiev, F. Theis, A. Cichocki, and H. Bakardjian, “Sparse Component
into independent spatial components,” Hum Brain Mapp, vol. 6, no. 3, pp. Analysis: A New Tool for Data Mining,” in Data Mining in Biomedicine,
160–188, 1998. 2007, vol. 8, pp. 91–116.
[7] A. H. Andersen, D. M. Gash, and M. J. Avison, “Principal component [30] V. D. Calhoun and T. Adalı, “Multisubject independent component analy-
analysis of the dynamic response measured by fMRI: A generalized linear sis of fMRI: A decade of intrinsic networks, default mode, and neurodiag-
systems framework,” Magnetic Resonance Imaging, vol. 17, no. 6, pp. nostic discovery,” IEEE Rev Biomed Eng, vol. 5, pp. 60–73, 2012.
795–815, 1999. [31] I. Daubechies, E. Roussos, S. Takerkart, M. Benharrosh, C. Golden,
[8] O. Friman, M. Borga, P. Lundberg, and H. Knutsson, “Exploratory fMRI K. D’Ardenne, W. Richter, J. D. Cohen, and J. Haxby, “Independent
analysis by autocorrelation maximization,” Neuroimage, vol. 16, no. 2, pp. component analysis for brain fMRI does not select for independence,”
454–464, Jun. 2002. Proc Natl Acad Sci U S A, vol. 106, no. 26, pp. 10 415–10 422, 2009.
[9] M. U. Khalid, A. Shah, and A.-K. Seghouane, “Adaptive 2DCCA Based [32] V. D. Calhoun, V. K. Potluru, R. Phlypo, R. F. Silva, B. A. Pearlmutter,
Approach for Improving Spatial Specificity of Activation Detection in A. Caprihan, S. M. Plis, and T. Adalı, “Independent Component Analysis
Functional MRI,” in International Conference on Digital Image Comput- for Brain fMRI Does Indeed Select for Maximal Independence,” PLOS
ing Techniques and Applications (DICTA), 2012, pp. 1–6. ONE, vol. 8, no. 8, p. e73309, 2013.
[10] A. Shah, M. U. Khalid, and A.-K. Seghouane, “Comparing causality mea- [33] Z. Boukouvalas, Y. Levin-Schwartz, V. D. Calhoun, and T. Adalı, “Sparsity
sures of fMRI data using PCA, CCA and vector autoregressive modelling,” and Independence: Balancing Two Objectives in Optimization for Source
in International Conference of the IEEE Engineering in Medicine and Separation with Application to fMRI Analysis,” Journal of the Franklin
Biology Society, Aug. 2012, pp. 6184–6187. Institute, vol. 355, no. 4, pp. 1873–1887, 2018.
[11] W. Lin, H. Wu, Y. Liu, D. Lv, and L. Yang, “A CCA and ICA-Based [34] A. Belouchrani, K. Abed-Meraim, J.-F. Cardoso, and E. Moulines, “A
Mixture Model for Identifying Major Depression Disorder,” IEEE Trans- blind source separation technique using second-order statistics,” IEEE
actions on Medical Imaging, vol. 36, no. 3, pp. 745–756, 2017. Transactions on Signal Processing, vol. 45, no. 2, pp. 434–444, 1997.
[12] M. A. Qadar and A.-K. Seghouane, “A Projection CCA Method for [35] F. Castells, J. Rieta, J. Millet, and V. Zarzoso, “Spatiotemporal blind source
Effective fMRI Data Analysis,” IEEE Trans Biomed Eng, vol. 66, no. 11, separation approach to atrial activity estimation in atrial tachyarrhyth-
pp. 3247–3256, 2019. mias,” IEEE Transactions on Biomedical Engineering, vol. 52, no. 2, pp.
258–267, 2005.
[13] M. U. Khalid and A.-K. Seghouane, “Improving functional connectivity
detection in fMRI by combining sparse dictionary learning and canonical [36] M. Li, Y. Liu, G. Feng, Z. Zhou, and D. Hu, “OI and fMRI signal
correlation analysis,” in IEEE International Symposium on Biomedical separation using both temporal and spatial autocorrelations,” IEEE Trans
Imaging, 2013, pp. 286–289. Biomed Eng, vol. 57, no. 8, pp. 1917–1926, 2010.
[37] M. U. Khalid and A.-K. Seghouane, “Multi-subject fMRI connectivity
[14] M. Pal, R. Roy, J. Basu, and M. S. Bepari, “Blind source separation: A re-
analysis using sparse dictionary learning and multiset canonical correla-
view and analysis,” in IEEE International Conference Oriental COCOSDA
tion analysis,” in IEEE International Symposium on Biomedical Imaging
Held Jointly with Conference on Asian Spoken Language Research and
(ISBI), 2015, pp. 683–686.
Evaluation (O-COCOSDA/CASLRE), 2013, pp. 1–5.
[38] H. D. Do, S. T. Tran, and D. T. Chau, “Speech Source Separation Using
[15] C. Ruan, D. Zhao, W. Jia, C. Chen, Y. Chen, and X. Liu, “A New Image
Variational Autoencoder and Bandpass Filter,” IEEE Access, vol. 8, pp.
Denoising Method by Combining WT with ICA,” Mathematical Problems
156 219–156 231, 2020.
in Engineering, vol. 2015, pp. 1–10, 2015.
[39] J. F. Cardoso, “High-order contrasts for independent component analysis,”
[16] A.-K. Seghouane and M. U. Khalid, “Learning dictionaries from correlated
Neural Comput, vol. 11, no. 1, pp. 157–192, 1999.
data: Application to fMRI data analysis,” in IEEE International Confer-
[40] N. Ito, R. Ikeshita, H. Sawada, and T. Nakatani, “A Joint Diagonalization
ence on Image Processing (ICIP), 2016, pp. 2340–2344.
Based Efficient Approach to Underdetermined Blind Audio Source Sepa-
[17] S. Hinderer, “Blind Source Separation of Radar Signals in Time Domain ration Using the Multichannel Wiener Filter,” IEEE/ACM Transactions on
Using Deep Learning,” in IEEE International Radar Symposium (IRS), Audio, Speech, and Language Processing, vol. 29, pp. 1950–1965, 2021.
2022, pp. 486–491.
[41] N. Nadisic, N. Gillis, and C. Kervazo, Smoothed Separable Nonnegative
[18] L. Y. Taha and E. Abdel-Raheem, “Blind Source Separation: A Perfor- Matrix Factorization. arXiv, 2021.
mance Review Approach,” in IEEE International Conference on Signal [42] R. Carloni Gertosio, J. Bobin, and F. Acero, “Semi-Blind Source Sepa-
Processing and Information Security (ICSPIS), 2022, pp. 148–153. ration with Learned Constraints,” Signal Processing, vol. 202, p. 108776,
[19] S. Choi, A. Cichocki, H.-M. Park, and S.-Y. Lee, “Blind source separation 2023.
and independent component analysis: A review,” Neural Inf. Process. Lett. [43] J. von Neumann, Functional Operators Volume 2: The Geometry of
Rev, vol. 6, 2004. Orthogonal Spaces. Princeton University Press, 1951.
[20] I. T. Jolliffe, Principal Component Analysis for Special Types of Data. [44] M. Xia, J. Wang, and Y. He, “BrainNet Viewer: A network visualization
Springer, 2002. tool for human brain connectomics,” PLoS One, vol. 8, no. 7, p. e68910,
[21] A. Hyvärinen and E. Oja, “Independent component analysis: Algorithms 2013.
and applications,” Neural Networks, vol. 13, no. 4, pp. 411–430, 2000. [45] H. Zou, T. Hastie, and R. Tibshirani, “Sparse Principal Component Analy-
[22] O. Friman, J. Cedefamn, P. Lundberg, M. Borga, and H. Knutsson, sis,” Journal of Computational and Graphical Statistics, vol. 15, no. 2, pp.
“Detection of neural activity in functional MRI using canonical correlation 265–286, 2006.
analysis,” Magn Reson Med, vol. 45, no. 2, pp. 323–330, 2001. [46] S. Waaijenborg, P. C. Verselewel de Witt Hamer, and A. H. Zwinderman,
[23] M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algorithm for de- “Quantifying the association between gene expressions and DNA-markers
signing overcomplete dictionaries for sparse representation,” IEEE Trans- by penalized canonical correlation analysis,” Stat Appl Genet Mol Biol,
actions on Signal Processing, vol. 54, no. 11, pp. 4311–4322, 2006. vol. 7, no. 1, 2008.
[24] A. Krishnan, L. J. Williams, A. R. McIntosh, and H. Abdi, “Partial [47] A. Aïssa-El-Bey and A.-K. Seghouane, “Sparse and smooth canonical cor-
Least Squares (PLS) methods for neuroimaging: A tutorial and review,” relation analysis through rank-1 matrix approximation,” EURASIP Journal
Neuroimage, vol. 56, no. 2, pp. 455–475, 2011. on Advances in Signal Processing, vol. 2017, no. 1, p. 25, 2017.
[25] J. Rapin, J. Bobin, A. Larue, and J.-L. Starck, “Sparse and Non-Negative [48] D. M. Witten, R. Tibshirani, and T. Hastie, “A penalized matrix decom-
BSS for Noisy Data,” IEEE Trans. Signal Process., vol. 61, no. 22, pp. position, with applications to sparse principal components and canonical
5620–5632, 2013. correlation analysis,” Biostatistics, vol. 10, no. 3, pp. 515–534, 2009.
[26] P. Paatero and U. Tapper, “Positive matrix factorization: A non-negative [49] J. M. Monteiro, A. Rao, J. Shawe-Taylor, and J. Mourão-Miranda, “A
factor model with optimal utilization of error estimates of data values,” multiple hold-out framework for Sparse Partial Least Squares,” Journal
Environmetrics, vol. 5, no. 2, pp. 111–126, 1994. of Neuroscience Methods, vol. 271, pp. 182–194, 2016.
16 VOLUME 4, 2016
[50] R. Ge, Y. Wang, J. Zhang, L. Yao, H. Zhang, and Z. Long, “Improved “Correspondence of the brain’s functional architecture during activation
FastICA algorithm in fMRI data analysis using the sparsity property of the and rest,” Proceedings of the National Academy of Sciences, vol. 106,
sources,” J Neurosci Methods, vol. 263, pp. 103–114, 2016. no. 31, pp. 13 040–13 045, Aug. 2009.
[51] B. Du, S. Wang, C. Xu, N. Wang, L. Zhang, and D. Tao, “Multi-Task [75] M. U. Khalid, “Dictionary learning algorithms for functional magnetic res-
Learning for Blind Source Separation,” IEEE Trans. on Image Process., onance imaging,” Ph.D. dissertation, The Australian National University,
vol. 27, no. 9, pp. 4219–4231, 2018. 2015.
[52] M. A. Qadar, A. Aïssa-El-Bey, and A.-K. Seghouane, “Two dimensional [76] Y. Nesterov, Introductory Lectures on Convex Optimization. Springer US,
CCA via penalized matrix decomposition for structure preserved fMRI 2004, vol. 87.
data analysis,” Digital Signal Processing, vol. 92, pp. 36–46, 2019.
[53] A. Hyvärinen, J. Hurri, and P. O. Hoyer, Natural Image Statistics a
Probabilistic Approach to Early Computational Vision. Springer-Verlag,
2009.
[54] K. Engan, S. Aase, and J. Hakon Husoy, “Method of optimal directions for
frame design,” in IEEE International Conference on Acoustics, Speech,
and Signal Processing, vol. 5, 1999, pp. 2443–2446.
[55] D. R. Hardoon and J. Shawe-Taylor, “Sparse canonical correlation analy-
sis,” Mach Learn, vol. 83, no. 3, pp. 331–353, 2011.
[56] M. U. Khalid, “Sparse Group Bases for Multisubject fMRI Data,” IEEE
Access, vol. 10, pp. 83 379–83 397, 2022.
[57] R. Rubinstein, M. Zibulevsky, and M. Elad, “Double Sparsity: Learning
Sparse Dictionaries for Sparse Signal Approximation,” IEEE Transactions
on Signal Processing, vol. 58, pp. 1553–1564, 2010. MUHAMMAD USMAN KHALID received the
[58] M. Nikolova, “Relationship between the optimal solutions of least squares M.Sc. degree in electrical engineering from the
regularized with $l_0$-norm and constrained by k-sparsity,” Applied and Royal Institute of Technology, Stockholm, in
Computational Harmonic Analysis, vol. 41, no. 1, pp. 237–265, 2016. 2010, and the Ph.D. degree in engineering and
[59] A.-K. Seghouane and A. Iqbal, “Basis Expansion Approaches for Regular- computer science from Australian National Uni-
ized Sequential Dictionary Learning Algorithms With Enforced Sparsity versity, Canberra, in 2015. He has previously
for fMRI Data Analysis,” IEEE Transactions on Medical Imaging, vol. 36, worked as a Visitor Academics with the Depart-
no. 9, pp. 1796–1807, 2017. ment of Electrical and Electronic Engineering,
[60] M. U. Khalid, A. Shah, and A.-K. Seghouane, “Sparse dictionary learning
The University of Melbourne, and as a Research
for fMRI analysis using autocorrelation maximization,” in Annual Inter-
Fellow at the School of Mathematics and Statis-
national Conference of the IEEE Engineering in Medicine and Biology
Society (EMBC), 2015, pp. 4286–4289. tics, The University of Melbourne. His research interests include sparse
[61] Y. Yang, Q. M. J. Wu, and Y. Wang, “Autoencoder With Invertible representation, blind source separation, numerical optimization, and medical
Functions for Dimension Reduction and Image Reconstruction,” IEEE imaging.
Transactions on Systems, Man, and Cybernetics: Systems, vol. 48, no. 7,
pp. 1065–1079, 2018.
[62] P. L. Combettes and J.-C. Pesquet, “Proximal Splitting Methods in Signal
Processing,” in Fixed-Point Algorithms for Inverse Problems in Science
and Engineering. Springer, 2011, pp. 185–212.
[63] W. J. Fu, “Penalized Regressions: The Bridge versus the Lasso,” Journal of
Computational and Graphical Statistics, vol. 7, no. 3, pp. 397–416, 1998.
[64] K. Bredies and D. A. Lorenz, “Linear Convergence of Iterative Soft-
Thresholding,” J Fourier Anal Appl, vol. 14, no. 5, pp. 813–837, 2008.
[65] R. Gribonval, H. Rauhut, K. Schnass, and P. Vandergheynst, “Atoms of
All Channels, Unite! Average Case Analysis of Multi-Channel Sparse
Recovery Using Greedy Algorithms,” J Fourier Anal Appl, vol. 14, no. 5,
pp. 655–687, 2008.
BILAL A. KHAWAJA (M’08-SM’18) received
[66] E. Erhardt, E. Allen, Y. Wei, T. Eichele, and V. Calhoun, “SimTB, a simu-
lation toolbox for fMRI data under a model of spatiotemporal separability,” the B.S degree in computer engineering from Sir
NeuroImage, vol. 59, pp. 4160–7, Dec. 2011. Syed University of Engineering and Technology,
[67] D. C. Van Essen, K. Ugurbil, E. Auerbach, D. Barch, T. E. J. Behrens, Karachi, Pakistan, in 2002, the M.Sc. degree in
and R. Bucholz, “The Human Connectome Project: A data acquisition communication engineering and signal processing
perspective,” Neuroimage, vol. 62, no. 4, pp. 2222–2231, Oct. 2012. from the University of Plymouth, Plymouth, U.K.,
[68] D. M. Barch, G. C. Burgess, M. P. Harms, S. E. Petersen, B. L. Schlaggar, in 2005, and the Ph.D. degree in Electrical En-
M. Corbetta, M. F. Glasser, S. Curtiss, S. Dixit, and C. Feldt, “Function in gineering from the University of Bristol, Bristol,
the human connectome: Task-fMRI and individual differences in behav- U.K. in 2010. From 2010 to 2016, he was an
ior,” Neuroimage, vol. 80, pp. 169–189, Oct. 2013. Assistant Professor at the Electronics and Power
[69] Kangjoo Lee, Sungho Tak, and Jong Chul Ye, “A Data-Driven Sparse Engineering Department, PN-Engineering College, National University of
GLM for fMRI Analysis Using Sparse Dictionary Learning With MDL Science and Technology (NUST), Karachi, Pakistan. In 2015, he was a
Criterion,” IEEE Trans. Med. Imaging, vol. 30, no. 5, pp. 1076–1089, May Visiting Postdoctoral Researcher with the Lightwave Systems Research
2011. Laboratory, Queens University, Kingston, Canada, involved in the Natural
[70] C.-M. Ting, A.-K. Seghouane, M. U. Khalid, and S.-H. Salleh, “Is First- Sciences and Engineering Research Council (NSERC)-Canada CREATE
Order Vector Autoregressive Model Optimal for fMRI Data?” Neural
Next Generation Optical Network (NGON) project on the characterization
Comput, vol. 27, no. 9, pp. 1857–1871, Sep. 2015.
and measurements of 25GHz RF signal generation optical comb sources.
[71] A.-K. Seghouane and A. Iqbal, “Consistent adaptive sequential dictionary
learning,” Signal Processing, vol. 153, pp. 300–310, Dec. 2018.
Currently, Dr. Khawaja is an Associate Professor with the Department
[72] H. Akaike, “A new look at the statistical model identification,” IEEE of Electrical Engineering, Faculty of Engineering, Islamic University of
Transactions on Automatic Control, vol. 19, no. 6, pp. 716–723, Dec. 1974. Madina (IUM), Madina, Saudi Arabia. His current research interests include
[73] S. Zhao, J. Han, J. Lv, X. Jiang, X. Hu, Y. Zhao, B. Ge, L. Guo, and T. Liu, Antennas / Antenna arrays design and characterization, Metamaterials, 5G
“Supervised dictionary learning for inferring concurrent brain networks,” and Next-generation wireless systems, RF/microwave and mm-wave sys-
IEEE Trans Med Imaging, vol. 34, no. 10, pp. 2036–2045, Oct. 2015. tems, and the use of Artificial Intelligence Techniques in these systems.
[74] S. M. Smith, P. T. Fox, K. L. Miller, D. C. Glahn, P. M. Fox, C. E. Mackay, Dr. Khawaja has authored and co-authored more than 60 peer-reviewed
N. Filippini, K. E. Watkins, R. Toro, A. R. Laird, and C. F. Beckmann, international journals and Conference papers.
VOLUME 4, 2016 17
MALIK MUHAMMAD NAUMAN is an accom-

plished academic and researcher with over 11
years of experience in Manufacturing, Energy En-
gineering, and Innovation and Entrepreneurship.
He has worked in various universities across the
globe, including in Turkey and Saudi Arabia, be-
fore joining the Faculty of Integrated Technologies
at Universiti Brunei Darussalam as an Associate
Professor and Deputy Dean. Dr. Nauman’s re-
search expertise extends beyond energy devices
and encompasses other areas, such as Image Processing and Shape Memory
Alloys. He has published numerous research papers and articles in reputed
international journals and conferences in these fields, in addition to his
contributions to 2D/3D printing of energy devices involving functional and
smart materials, Thin Film Solar Cells, Memristors, and other electronic
devices, as well as Inverse Estimation of Heat and Mass Transfer. Through
his active involvement in various academic and professional activities,
including organizing and participating in several workshops, seminars, and
conferences, both locally and internationally, Dr. Nauman has demonstrated
his unwavering commitment to academic excellence and the pursuit of
knowledge. Overall, his exceptional achievements and vast experience in
Mechatronics Engineering and related disciplines make him an invaluable
member of the academic community.
18 VOLUME 4, 2016

ACCESS3277543

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ACCESS3277543

Uploaded by

Copyright:

Available Formats

This article has been accepted for publication in IEEE Access.

Efficient Blind Source Separation

I. INTRODUCTION fMRI studies [6]–[12].

FIGURE 2. An algorithmic description of different sparse blind source separation approaches.

pull the output data back to hidden nodes to enrich and

Algorithms Approach Matlab code availability

singular vectors. The reason for including 4a is to create a scenario with

A. SYNTHETIC DATASET 2) Synthetic dataset source separation

TABLE 5. Mean correlation values over 10 trials for ρ = 4.5.

Srcs JADE sCCA sPLS sICA ssICA ssBSSP ssBSSA

TABLE 6. Correlation values of most correlated temporal components with

TABLE 7. Correlation values of most correlated temporal components with

Algos FT R1 R2 R4 R9 R10 Mean

MALIK MUHAMMAD NAUMAN is an accom-

You might also like