This article has been accepted for inclusion in a future issue of this journal.
Content is final as presented, with the exception of pagination.
IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING
Nonsmooth Nonconvex Optimization for
Low-Frequency Geosounding Inversion
Hugo Hidalgo-Silva, Member, IEEE, and Enrique Gómez-Treviño
Abstract—A study of the application of nonconvex regulariza-
tion operators to the electromagnetic sounding inverse problem is
presented. A comparison is presented among three nonconvex reg-
ularization algorithms: one smooth usually considered, two nons-
mooth, and a convex one, the total variation (TV) operator. One of
the nonsmooth nonconvex regularization methods is a novel imple-
mentation based on the Legendre–Fenchel transform and the Breg-
man iterative algorithm. The nonconvex regularization operator is
approximated by the convex dual, and the minimization is then
implemented considering the equivalence between the Bregman it-
eration and the augmented Lagrangian methods. The algorithm is
simple and provides for better models when applied to synthetic
data, than those obtained with TV, and other nonconvex smooth
regularizers. Results of the application to field data are also pre-
sented, observing that NS2 recovers a model in better agreement
with the truth, compared to those obtained with additional mag-
netometric resistivity data by other researchers.
Index Terms—Geosounding, nonconvex optimization, regular-
ization.
I. INTRODUCTION
O image the internal properties of the earth, geophysicists
T relay mostly on the interpretation of measurements made
on the surface. This applies for deep soundings of hundreds of
kilometers as for shallow studies of merely a few meters below
the surface. The electrical conductivity of rocks is often the
property of interest in these type of studies. For this reason, a
great number of electrical techniques have been developed to
infer the conductivity structure of the subsurface on the basis of
surface measurements.
One technique is based on electromagnetic induction by
means of an alternating current that is made to flow in a trans-
mitting coil. This current generates an alternating magnetic field
in the surrounding environment, which in turn induces an elec-
tromotive force both in the conductive ground and in a receiving
coil, e.g., [1]. The currents induced in the ground depend on the
conductivity distribution and affect the receiving coil accord-
ingly; thus, making it possible to obtain information about the
subsurface on the basis of purely surface measurements. Con-
trol on the depth penetration is accomplished by varying the
frequency of the current in the transmitting coil or the separa-
tion between the transmitting and receiving coils, or by varying
both frequency and separation.
Manuscript received September 29, 2015; revised December 15, 2015, Febru-
ary 9, 2016, and March 2, 2016; accepted March 19, 2016.
The authors are with the Centro de Investigación Cientı́fica y Educación
Superior de Ensenada, Ensenada 22860, México (e-mail: hugo@cicese.mx;
egomez@cicese.mx).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/JSTARS.2016.2549942
1939-1404 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
1
A particular version of the technique that works at very low
frequencies, at what is known as low induction numbers, is of
special interest from both practical and theoretical, e.g., [2] rea-
sons. At low induction numbers, the skin depth of the currents
induced in the ground must be much greater than the separation
between the coils. When this happens the depth of penetration
is independent of frequency. Depth discrimination is accom-
plished by varying solely the separation between the coils: the
greater the separation the deeper the penetration. Although in
practice this version of the electromagnetic technique is less ef-
fective than a combined frequency-separation approach, it can
be shown that the information provided by soundings at low in-
duction numbers is sufficient for a unique recovery of a vertical
conductivity profile [3]. Depth discrimination works very much
in the same way as in the familiar direct-current methods, where
the spacing between current and potential electrodes determines
the depth of penetration. The main difference, in practice, is that
electromagnetic induction allows for faster field surveys, since
it eliminates the need for material contact with the ground.
In one-dimensional (1-D), the surface measurements can be
exactly represented by a linear functional of the unknown con-
ductivity σ of the subsurface, e.g., [3]. In 2-D, the corresponding
functional is necessarily nonlinear, but it can be approximated
for many practical applications using a linear functional [4].
Apparent conductivity σa , a normalized version of the measure-
ments, can be expressed in 2-D as
 ∞  ∞
2|x2 − x1 |
σa = A2D (x, z, x1 , x2 )σ(x, z)dxdz
π 0 −∞
(1)
where A2D is presented on Section IV and x1 , x2 are the coor-
dinates of the transmitting and receiving coils, respectively.
The inverse problem posed by this equation differs from im-
age enhancement in several ways. The electrical conductivity
distribution within the integral sign represents the unknowns or,
equivalently, the output data. These output data make up the im-
age of the subsurface and ideally their size is infinite, although
for practical reasons, the size is considered finite. The input data
on the left-hand side are the measurements taken on the surface
of the ground, and typically, they are only a few. The inverse
problem in this case is more of constructing an image that does
not exist than of enhancing a preexisting one.
Tikhonov’s regularization method [5] is the standard tech-
nique applied to obtain models of the subsurface conductiv-
ity distribution from electric or electromagnetic measurements
(e.g., [6], [7]). The model proposed for the conductivity distri-
bution m is obtained by minimizing the functional
UT (m) = F (m) − d2H + λP (m). (2)
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
2 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING
In this formulation, F : M ⊃ D(F ) → H represents the di-
rect functional, applied to an element on the model space M, a
Banach space, and returning a member of the Hilbert space H
of data. The second term correspond to the Tikhonov’s stabi-
lizing functional, with P (m) =   m2M , maximum smooth-
ness functional the usual approach [7] and λ the regularization
parameter, controlling the tradeoff between smoothness of the
model and goodness of fit to the data, represented by d. Due
to the roughness penalizer inclusion, the model developed by
Tikhonov’s algorithm tends to smear discontinuities, a feature
that may be undesirable in situations where different rock types
simply face each other without a fuzzy transition. In an effort
to generate models with sharp boundaries, several techniques
have been applied to this problem: piecewise continuous for-
mulations as in [8] and [9], l1 norm minimization [10]–[12],
total variation (TV) penalizers in [13] and [14], among others.
In this paper, we will consider the discrete version of lin-
ear problem F (m) = K(m) = κm, with κ the kernel of a
Fredholm integral equation of the first kind
U (m) = Am − d2 + λP (m)
with A the matrix of integral kernel evaluations and P (m) the
discrete version of stabilizing operator. This kind of problems
appears in several geoscience areas, from atmospheric transmis-
sion models, ground penetration radar, applied electromagnet-
ics, among others, [15], [16]. A usually considered application
image deconvolution permits to develop fast algorithms due to
the characteristic form of the matrix A. Here, in our application
to low-frequency electromagnetic soundings, matrix A does not
have a circular form due to the slowly decreasing kernel.
An important requirement for the regularizer is to allow the
recovery of edges, and to smooth the homogeneous parts. As is
well known, TV [17] is now the standard approach to meet this
requirement. TV is a convex regularizer, ensuring existence and
uniqueness of a solution. In practice, TV tends to produce low-
contrasted solutions, and in the case of integral equations with
a slow-decaying kernel smooth staircase-type models. Several
enhanced versions of TV as well as other operators have been
developed by applying Bregman iterations [18]–[20]. The mod-
els developed by the Bregman versions present higher contrast,
but the staircasing effect remains for TV in the slow-decaying
kernel inverse problems. A new interpretation of TV considers
it as mixed norms of first degree directional derivative of the
image [21]. They define a family of higher order regularizers
named higher order total variation (HDTV), preserving strong
directional derivatives at a specific direction and attenuating
small ones at other directions. Although, HDTV suffers of a
high computational complexity compared to some state-of-the-
art TV methods.
Li et al. proposed a fast algorithm for TV image reconstruc-
tion based on alternating minimization [22]. The algorithm im-
plementation, called TVAL3 will be used to evaluate the model
reconstruction of geosounding data against modeling results
of nonconvex smooth and nonsmooth
 algorithms. The mini-
mization of (3) with P (m) = i Di m with Di m ∈ R2 the
discrete gradient of m at element i is considered in TVAL3
by alternating minimization. They introduce an auxiliary vari-
able wi to transfer Di m out of the nondifferentiable term  · ,
TABLE I
TVAL3 ALGORITHM
TVAL3 Algorithm
Input: d
Initialization: k = 0, m 0 = 0, d 0 = 0, ν i = 0,λ = 0
repeat
update m k + 1 minimizing (4)
update w ik + 1 using (5)
λk + 1 ← λk + μ(d − A m k + 1 )
ν i k + 1 ← ν ik + β (w k + 1 − D i m k + 1 )
k ←k+1
until (convergence)
Output m k
penalizing the difference between wi and Di m, yielding the
approximation scheme to (3)
μ
UT V A L 3 = Am − d2 − λt (Am − d)
2
 β
(3) + (wi  + Di m − wi 2 − νit (Di m − wi )) (4)
i
2
with β a penalty parameter and ν, λ the Lagrange multipliers.
For a m fixed, the minimizers wi for all i are obtained via
 
νi Di m − νi /β
wi = max Di m −  − 1/β, 0 . (5)
β Di m − νi /β
On the other hand, for fixed wi , they approximately minimize the
quadratic respect to m by taking one steepest descent step. After
each steepest descent step, they update wi and repeat process
until (5) is approximately solved within a prescribed tolerance.
The alternating minimizations algorithm is presented in Table I.
After the work of Geman and Geman [23], various nonconvex
regularizers have been also considered. Despite the numerical
complexities arising with nonconvex regularization, a variety of
applications have shown that nonconvex regularization supplies
much richer possibilities for the restoration of high-quality im-
ages with well-kept edges. A theoretical analysis was presented
in [24] and the numerical results are reported in numerous works
[25]–[28]. In [29], Nikolova et al. proved that minimizers of
problems like (3) with P (m) = φ(Di m) where φ(·) is non-
convex and nonsmooth at zero are composed of constant re-
gions surrounded by closed contours and neat edges. This result
motivates the application of this kind of regularization opera-
tors to the development of inversion algorithms. They propose
a continuation method minimizing a sequence of approximate
nonsmooth energies, the first of these strictly convex and the
last one the original energy to minimize.
Jung and Kang [30] incorporate nonsmooth nonconvex reg-
ularization with the L1 data fidelity term in order to perform
image restoration and segmentation. To overcome the numer-
ical difficulties of the nonconvex regularizer, they use a mul-
tistage convex regularization technique. They propose to solve
(3) by using a variable splitting scheme followed by the appli-
cation of augmented Lagrangian method. They need to solve
two optimization subproblems, one can be efficiently solved for
a convolution type problem, the other, nonconvex is solved by
using the multistage technique. In this paper, we propose two
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

HIDALGO-SILVA AND GÓMEZ-TREVIÑO: NONSMOOTH NONCONVEX OPTIMIZATION 3

TABLE II with nonconvex regularizers; from simulated annealing [32],

BREGMAN ITERATIVE ALGORITHM FOR (3) WHEN A LINEAR PROBLEM
F (m) = Am IS CONSIDERED
deterministic continuation in the form of graduated noncon-
vexity [33] to application of dual convex approximation [30],
[34], [35].
Bregman Iteration based Algorithm
The dual convex approximation is based on the Legendre–
Input: d Fenchel transformation, defined for a continuous not necessarily
Initialization: k = 0, m 0 = 0, d 0 = 0 differentiable function as
repeat
d k + 1 ← d + (d k − A m k ) f ∗ (u) = sup( x, u − f (x)). (9)
m k + 1 ← arg m in m λP (m ) + 12 A m − d k + 1 2 x∈H
k ←k+1
until (R M S < m in ) This transformation has been used to define functional approx-
Output m k imations [30], [34], [35] that allow to implement iterative algo-
rithms.

simpler approaches for the geosounding method in the lin- A. Smooth Algorithm
ear case of low-induction numbers. We consider the use of
A smooth nonconvex regularization algorithm was proposed
nonsmooth nonconvex regularizers presenting two algorithms,
by Charbonnier et al. [35], considering the following operator:
both using the convex dual approximation based on Legendre–  
Fenchel transform. One optimization is based on the variable PE P R = φ[(Dx m)k ] + φ[(Dz m)k ] (10)
splitting scheme and the other on directly solving the convex k k
dual by coordinate descent. Both algorithms are evaluated com-
where (Dx m)k is the finite difference implementation of first-
paring modeling results with those of one nonconvex smooth
order derivative at point k, and φ is the potential function. This
algorithm and with TV for synthetic and real data. All of the
operator has been considered on the vision community under
nonconvex implementations include some kind of Bregman
the name of “edge-preserving regularization” (EPR) [35], and
iteration.
applied to the resistivity inverse problem by Hidalgo-Silva et al.
[36]. Charbonnier et al. developed some properties that a poten-
II. BREGMAN ITERATIONS
tial function has to comply in order to obtain an EPR operator.
Bregman iterative regularization was introduced by Osher A particularly important nonconvex potential function initially
et al. [19] in image processing. They extended the Rudin et al. proposed by Geman and McClure [37] for image restoration is
[17] model into an iterative regularization model by using the
x2
Bregman distance. φ(x) = (11)
The Bregman distance based on a convex functional J(·) x2 + β 2
between points u and v is defined as with β > 0. Charbonnier et al. apply an iterative algorithm de-
veloped by Geman and Yang [25] called “half quadratic reg-
DJp (u, v) = J(u) − J(v) − p, u − v (6)
ularization” for the minimization of a modified version of the
where p ∈ δJ(v) is some subgradient in the subdifferential of original functional (3) with P = PE P R . The algorithm is aimed
J at the point v. to minimize the dual functional
Yin et al. [31] observed that the algorithm can also be repre-
UE∗ P R (m, bx , bz ) = Am − d2
sented as the original problem (3) but with dk +1 = d + (dk −

Amk ) as input, instead of d, obtaining the “add-back residual” +λ [φ∗ [(Dx )k , (bx )k ] + φ∗ [(Dz )k , (bz )k ]] (12)
version k

mk ← solve (3) with d := dk (7) with φ∗ defined such that

dk +1 ← d + (dk − Amk ). (8) φ(t) = inf φ∗ (t, w) (13)
w
The Bregman iterative algorithm applied to (3) is presented and bx , bz auxiliary variables introduced to ease the minimiza-
in Table II. A convergence termination criteria is established tion of the augmented energy functional
when fitness to data (RM S = √1N d − Am) reaches some
minimum, with A the matrix containing the discrete evaluation UE∗ P R (m, bx , bz ) = Am − d2
of the kernel κ. Osher et al. [19] presented convergence results 
+λ (bx )k (Dx m)2k + ψ[(bx )k
when H(m) generalizes the data fitness term 12 Am − d22 .
k

III. NONCONVEX REGULARIZATION +(bz )k (Dz m)2k + ψ[(bz )k ]] (14)

As mentioned before, despite the numerical complexities aris-
ing with nonconvex regularization, a variety of applications have
shown that nonconvex regularization supplies much richer pos-
sibilities for the restoration of high quality images with well-
kept edges. Several methodologies have been proposed to deal
ψ is a strictly convex and decreasing function such that φ(t) =
inf w (wt2 + ψ(w)), and√ for the case of potential function in
(11), ψ(w) = wβ 2 − 2 wβ + 1.
The half quadratic procedure is based on alternating mini-
mizations over m and b. Charbonnier et al. proved convergence
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING

TABLE III with ρ > 0. Vese and Chan [34] considered these functions as re-
BREGMAN EPR ALGORITHM
duced nonconvex approximations of Mumford–Shah functional.
They demonstrated, using the Legendre–Fenchel transform, that
Bregman EPR Algorithm
φ(t) can be represented using a convex decreasing function ψ
Input: d such that
Initialization: k = 0, m 0 = 0, d 0 = 0
repeat φ(t) = inf (v 2 t + ψ(v)), ∀t ≥ 0 (19)
v
d k + 1 ← d + (d k − A m k )

obtaining ψ1 (v) = (v −1)

update m k + 1 using 15 2

update b z , b x using 16 ρ and ψ2 (v) = ρ1 (v 2 − 2 log v − 1).

k ←k+1 They solve a functional like (3) with P (m) = φ(|∇m|), ap-
until (R M S < m in ) proaching the convex functional by a sequence of nonconvex
Output m k
functionals, implementing a discrete version similar to the TV
as in [17]. They consider the image restoration and segmentation
problem.
of this procedure when φ(t) possess the edge-preserving condi- Jung and Kang proposed to solve the minimization problem in
tions [35]. The convexity of ψ allows Hidalgo-Silva and Gómez- image restoration and segmentation by using a variable splitting
Treviño to easily incorporate the half-quadratic minimization in technique and the augmented Lagrangian method. They apply
the Bregman iteration-based algorithm [38]. They propose to a multistage convex relaxation method to handle the nonconvex
use a coordinate descent algorithm, iteratively solving for rth regularizer. Following a similar approach, we will consider one
element of m as algorithm for the linear problem considering the equivalence
 
j (dj − aj l ml )aj r + λn1 between the Bregman iteration and the augmented Lagrangian
mr = l =r 2 (15) [31], [39]. This algorithm, named NS1, is obtained with the
j aj r + λb
operator splitting version of (3)
with 
UN S 1 = 1/2Am − d2 + γ/2∇m − b2 + λ φ(|bi |)
n1 = mr −n x (bz )r + mr +n x (bz )r +n x + mr −1 (bx )r i
(20)
+mr +1 (bx )r +1 with bj a component of vector b, an auxiliary variable introduced
b = (bz )r + (bz )r +n x + (bx )r + (bx )r +1 to implement the split operation. Applying Bregman iteration to
both, m and b, the algorithm consists in realizing the iteration
and the minimum for bx , bz is at updates
φ (mr − mr −n x ) dk +1 ← d + (dk − Amk )
(bz )r =
2(mr − mr −n x )
dˆk +1 ← b + (dˆk − ∇mk )
φ (mr − mr −1 )
(bx )r = (16) and solve at iteration k
2(mr − mr −1 )
with φ the derivative of φ, aij = [A]ij , and considering m mk +1 , bk +1 ← min 1/2Am − dk 2
m ,b
as a vector representation of a 2-D model with nx elements in 
x coordinate. +γ/2(∇m − b) − dˆk 2 + λ φ(|bi |) (21)
The iterative procedure is implemented by using (15), (16), i
and (17) to solve the minimization steps for mr , bz , and bx . where dˆ is an auxiliary variable included for the Bregman type
This will be called Bregman EPR, presented in Table III, with iteration of the gradient operator, initially, d0 = 0, dˆ0 = 0.
the modeling results observed on next section, when comparing This equation is separately solved for m and b, considering
with other methods. coordinate descent, the solution for mr in the 1-D case is
Hidalgo-Silva and Gómez-Treviño [38] observed that the  
model developed by the Bregmanized version had a better ap- j (dj − aj l ml )aj r + γn2
k +1
mr = l =r2 (22)
proximation to synthetic models than the original algorithm. j aj r + 2γ

with
B. Non Smooth Regularizers
 n2 = (br + dˆr − br +1 − dˆr +1 + mr −1 + mr +1 )
Consider the minimization of (3) with P (m) = φ(|∇m|),
with φ a nonconvex nonsmooth function such as and for br
|t| bkr +1 = shrink(mr − mr −1 − dˆr , λvr /γ) (23)
φ1 (|t|) = (17)
1 + ρ|t|
with the shrink operator defined as shrink(x, α) = (|x| −
or α)+ sign(x). Solution for b is obtained from the dual convex ob-
1 tained applying Legendre–Fenchel transform (9).  For the case
φ2 (|t|) = log(1 + ρ|t|) (18) of φ1 (·), last term in (21) is replaced by λ( i vi2 |Dx m|i +
ρ
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

HIDALGO-SILVA AND GÓMEZ-TREVIÑO: NONSMOOTH NONCONVEX OPTIMIZATION 5

TABLE IV TABLE V
NONSMOOTH NONCONVEX NS1 ALGORITHM NONSMOOTH NONCONVEX NS2 ALGORITHM

NS1 Algorithm NS2 Algorithm

Input: d Input: d
Initialization: k = 0, m 0 = 0, d 0 = 0, dˆ0 = 0 Initialization: k = 0, m 0 = 0, d 0 = 0
repeat repeat
d k + 1 ← d + (d k − A m k ) d k + 1 ← d + (d k − A m k )
dˆk + 1 ← b + ( dˆk − ∇m k ) update m k + 1 using 25
update m k + 1 using 22 update v z , v x using 26 or 27
update b z , b x using 23 k ←k+1
k ←k+1 until (R M S < m in )
until (R M S < m in ) Output m k
Output m k

algorithm, respect to NS1, is that (26) may have several so-

1
ρ (vi− 1)2 ) for the horizontal components of the gradient, and lutions due to ϕ(·). In the implementation, we test for every
similarly for the vertical ones. The algorithm is presented in solution and select the one that produces the greatest descent in
Table IV. the error functional; thus, increasing the computing time.
Another algorithm, named NS2, can be developed for the
nonconvex nonsmooth operator considering, as in the EPR case, IV. 2-D GEOSOUNDING PROBLEM
the dual functional
As mentioned in Section I, in 1-D, the surface measurements
UN S 2 (m, b) = Am − d2 can be exactly represented by a linear functional of the unknown
  conductivity of the subsurface, e.g., [3]. In 2-D, the correspond-
+λ φ∗ [(Dx m)k ] + φ∗ [(Dz m)k ] (24) ing functional is necessarily nonlinear, but it can be approxi-
k k mated for many practical applications using a linear functional
[4]. Apparent conductivity σa , a normalized version of the mea-
with φ∗ the convex dual as defined in (20).
surements, can be expressed for a 2-D conductivity distribution
The half-quadratic procedure is also implemented on alter-
σ(x, z) as in (1), where A2D is for vertical dipoles
nating minimizations over m and v.
 ∞ 2
Now, the convexity of ψ allows us to easily incorporate the (y − (x − x1 )(x2 − x))dy
half-quadratic minimization in the Bregman algorithm (II). Min- A2D (x, z, x1 , x2 ) = (29)
0 [(r12 + z 2 ]3 [r22 + z 2 ]3
imization is implemented with a coordinate descent algorithm,
taking derivative respect to rth element of m and solving and for horizontal dipoles
   ∞
j (dj − l =r aj l ml )aj r + λn3 A2D (x, z, x1 , x2 ) = (Ex 1 Ex 2 + Ey 1 Ey 2 )dy (30)
mr =  2 (25) 0
j aj r
with
with
2 2z z
n3 = −ϕ(mr − mr −n x )(vz )2r + ϕ(mr +n x − mr )(vz )2r +n x Ey 1 = y(x − x1 ) − −
r14 r14 r12 + z2 r12 (r12 + z 2 )3 / 2
−ϕ(mr − mr −1 )(vx )2r + ϕ(mr +1 − mr )(vx )2r +1
2 2z z
Ey 2 = y(x − x2 ) − −
and the minimum for v is at r24 r24 r22 + z 2 r22 (r22 + z 2 )3 / 2
1
vr = (26) Ex 1 =
1 + ρ|(Dm)r |  
1 z 2y 2 y2 z 2y 2 z
for ψ1 and − − 4 + 2 2 +
r12 r12 r12 + z 2 r1 r1 (r1 + z 2 )3 / 2 r14 r12 + z 2
1
vr = (27) Ex 2 =
1 + ρ|(Dm)r |  
for ψ2 . There, aij = [A]ij , considering m as a vector represen- 1 z 2y 2 y2 z 2y 2 z
− − 4 + 2 2 +
tation of a 2-D model with nx elements in x coordinate. ϕ(x) is r22 r22 2
r2 + z 2 r2 r2 (r2 + z )2 3 / 2
r2 r22 + z 2
4

the subdifferential of |x| given by x1 is the location of the transmitting coil, x2 the loca-
sgn(x) if x = 0 tion of the receiving coil, r1 = (x − x1 )2 + y 2 and r2 =
ϕ(x) = (28) (x − x2 )2 + y 2 .
Co{−1, 1}, if x = 0.
The coordinate system is shown in Fig. 1.
A Bregman iterative procedure is obtained easily, by using (25), The conductivity variations along y, the strike direction, are
(26), and (27) to solve the minimization step for mr . The al- neglected. The kernel for vertical dipoles can be integrated ana-
gorithm is presented in Table V. A main disadvantage in this lytically along strike and evaluated in terms of elliptic functions
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING

Fig. 1. Axis configuration for the 2D modeling equation.

[4]. For horizontal dipoles, the integration is carried out numer-

ically. Notice that none of the above expressions depend on the
frequency of the electromagnetic field. The depth of penetration
of electromagnetic measurements at low induction numbers is
independent of frequency. In fact, the induced currents in the
ground are proportional to frequency, and so are the associated
fields and therefore the measurements. However, because the
normalization factors derived from the responses of a homo-
geneous medium are also proportional to frequency, the final
expressions are frequency independent [2].

V. EXPERIMENT WITH SYNTHETIC DATA

Results of application of Bregman EPR, NS1, NS2, and
TVAL3 to synthetic data are presented in Fig. 2, for this method
at low induction frequencies for the joint vertical plus horizontal
dipole configurations. The model is developed over an homoge-
neous mesh of size 67 in x and 64 in z. Regularization param-
eter λ is determined in the synthetic case by solving a smooth
(φ(x) = x2 ) problem first, testing for several values of λ, and
choosing the one corresponding to the model that fits the data to
a given RM S value. In the real data case, it can be determined
from the misfit condition Amλ − d = , with  some a priori
estimation of noise level of data. β and ρ are scaling parameters
Fig. 2. Results for the Synthetic Data obtained from model in (a), (b) Bregman
and selected according to the expected model variations. After EPR result, (c) TVAL3 result, (d) NS1 result, and (e) NS2 result.
several tests, β = 1 and ρ = 0.01 provided for the best models
for the considered synthetic data. In the case of γ for the NS1
algorithm, this is some kind of regularization parameter of the
operator splitting formulation. It should be large enough so that From the figures, it is observed that EPR and NS2 provide
b ≈ ∇m. The use of the Bregman iteration as an equivalence for a better approximation to the real model, with NS2 attaining
to augmented Lagrangian allows us to iterate among d, ˆ without greater contrast. NS1 presents a very low-contrasted model,
using large regularization parameter γ. unable to separate the two blocks in the structure.
A comparison with a model developed with smooth operator TV also recovers a very low-contrasted image with a shal-
2
φ(t) = t 2t+1 in the Bregman EPR is presented also in Fig. 2. lower presentation of the blocks in the model. The implementa-
Parameter λ was set to 0.01 for all algorithms, β = 1 for EPR, tion used is the augmented Lagrangian TVAL3 algorithm of Li
ρ = 0.1 in φ1 for NS1 and NS2, and γ = 0.01 for NS1. All al- et al. [22] with parameters: mu = 1000, beta = 25 , tol = 1E−3.
gorithms were implemented on a Bregman loop, stopping when Convergence curves are presented in Fig. 3 with plots of
∗ 2
RMS fitting error and approximation error mm−m 
k
RM S ≤ 0.002. ∗ 2 versus
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

HIDALGO-SILVA AND GÓMEZ-TREVIÑO: NONSMOOTH NONCONVEX OPTIMIZATION 7

TABLE VII
EFFECT OF λ AND γ ON ITERATIONS REQUIRED TO ATTAIN RM S = 0.002,
ALONG WITH APPROXIMATION ERROR, SHOWN IN PARENTHESES, FOR NS1

λ 0.0001 0.001 0.01 0.1 1.0 10.0

0.0001 12 (0.85) 9 (0.75) 8 (0.84) 12 (0.88) 51 (0.87) -

0.001 11 (0.78) 10 (0.82) 8 (0.79) 12 (0.87) 49 (0.87) -
0.01 11 (0.78) 9 (0.80) 9 (0.80) 32 (0.99) 48 (0.88) -
0.1 9 (1.04) 9 (0.85) 9 (0.8) - - -
1.0 9 (1.5) 9 (0.87) 9 (0.8) - - -
10.0 9 (1.6) 9 (0.87) 9 (0.8) - - -

TABLE VIII
PARAMETERS OF ALGORITHMS

Algorithm Parameters

TVAL3 μ, β ,tol
EPR λ, β ,min
NS1 λ, γ , ρ,min
NS2 λ, ρ,min

oped for lower values of λ, but one of the blocks attain a smaller
structure than the real one. On the other side, a smoother model
is recovered for λ > 0.01, failing to recover the two blocks in
Fig. 3. Convergence curves, RMS fitness (a) and Approximation Error, the structure.
(b) versus iteration number k. The performance for several values of λ and γ is presented
in Table VII for NS1. The numbers also represent the itera-
TABLE VI tions required and approximation error attained when algorithm
EFFECT OF λ ON ITERATIONS REQUIRED TO ATTAIN RM S = 0.002, ALONG run until RM S = 0.002. Results with (−) indicate that the al-
WITH APPROXIMATION ERROR, SHOWN IN PARENTHESES, FOR EPR AND NS2 gorithm stagnates at some RMS above the required, without
reaching the target. Regardless of the nonconvexity of opera-
λ EPR NS2 tors in EPR, NS1, and NS2, they did not showed a divergent
0.0001 9 (1.37) 9 (1.21)
behavior.
0.001 9 (0.82) 9 (0.73) All the models have a similar structure for a given γ, re-
0.01 8 (0.80) 10 (0.66) gardless the value of λ. The best performance is observed when
0.1 13 (0.84) 10 (0.77)
1.0 45 (0.84) 18 (0.82)
λ ∈ [0.001, 0.01], with more contrast observed when γ = 0.001.
10.0 203 (0.87) 71 (0.82) In this case, a more compact model is developed for lower
values of γ, and a smoother one is obtained for greater γ.
A summary of parameters required by the algorithms is pre-
sented in Table VIII. TVAL3 and NS1 need two regularization
iteration number k, with m∗ the real model used to generate
parameters [(μ, β) and (λ, γ), respectively] due to the augmented
the synthetic data. Notice that all algorithms behave similarly in
Lagrangian formulation. As for EPR and NS2, λ is the only reg-
RMS fitting, but NS2 present much better performance in terms
ularization parameter.
of approximation error. The best model is obtained at about
ten iterations for EPR and NS2, and eight iterations for NS1.
VI. APPLICATION TO FIELD DATA
Some artifacts appear on the model when the algorithm NS1 is
permitted to continue after that iteration. An application to field data is presented now, the data corre-
A study was conducted in order to observe the sensitivity spond to measurements taken at Las Auras, a site near the city
to the regularization parameters on the three nonconvex algo- of Tecate, México. The purpose of the surveys was to identify
rithms. Results are presented in Table VI for EPR and NS2, with sediment-filled faults affecting the construction of a dam. We
β = 1 and ρ = 1. are considering the vertical and horizontal dipole measurements
The Table shows how many iterations were needed and the taken at several points across strike for 10, 20, and 40 m separa-
corresponding approximation error obtained when the algo- tions as reported by Pérez-Flores et al. [4]. Applying maximum
rithms run until a RMS fitness of 0.002 was attained, for λ ∈ smoothness regularization, they observed the presence of two
[0.0001, 10.]. The best performance was obtained at λ = 0.01 main conductive zones, corresponding to sediment-filled faults.
with model reported in Fig. 2. A more compact model is devel- The two main conductors observed by them with the same data
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
8 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING
Fig. 4. Results for the field data (a) field data, (b) Bregman EPR result,
(c) TVAL3 result, (d) NS1 result, and (e) NS2 result.
appear at 20–40 m and 90–100 m in x, extending from 0 to
5 m in depth. In the same paper, they incorporate magnetomet-
ric resistivity data jointly with the vertical and horizontal data,
observing that the second conductive block extends in x from
80 to 100 m.
The data and the models obtained by EPR, NS1, NS2, and
TVAL3 are presented in Fig. 4 for the joint (vertical + horizon-
tal dipoles) inversion. The first three rows in Fig. 4(a) are the
horizontal dipole data, and the lower three correspond to the
vertical dipole data.
Parameter λ = 1 for EPR, NS1, and NS2, ρ = 10 for NS1
and NS2, γ = 0.1 for NS1. EPR, NS1, and NS2 were stopped
when RM S ≤ 0.6. For TVAL3, the same parameters as before
were considered, achieving an RM S = 0.57.
TV, NS1, and NS2 recover the block at 17–43 m, and another
one at 75–90 m in x. NS2 is the only one that develops a third
block at 95–105 m, which is in agreement with the results of
Pérez–Flores [4, Fig. 5], who observe a single block from 80 to
100 m, and mention the location of a fault below that block.
NS2 does not develop the large variations on the first layer
obtained by all other algorithms. Due to the regularization con-
sidered, the developed models are more compact than those
obtained by [4], with a shallower structure, from 3–7 m depth.
EPR obtains a model of several small blocks, uncharacteristic
of a sedimentary structure.
VII. CONCLUSION
A new algorithm for nonconvex nonsmooth optimization
(NS2) is presented, observing better performance when com-
pared with synthetic data with TV and other two nonconvex
methods. A study is presented on a empirical evaluation of
nonconvex nonsmooth algorithms applied to the low-induction
number geosounding method. Results of the application to field
data are also presented, observing that NS2 recovers a model in
better agreement with the truth, compared to those obtained with
additional magnetometric resistivity data by other researchers.
ACKNOWLEDGMENT
The authors would like to thank S. Méndez Delgado for pro-
viding the field data.
REFERENCES
[1] F. S. Grant and G. F. West, Interpretation Theory in Applied Geophysics.
New York, NY, USA: McGraw-Hill, 1965.
[2] S. Méndez Delgado, E. Gómez-Treviño, and M. A. Pérez-Flores, “Forward
modelling of direct current and low-frequency electromagnetic fields using
integral equations,” Geophys. J. Int., vol. 137, pp. 336–352, 1999.
[3] E. Gómez-Treviño, F. Esparza, and S. Méndez-Delgado, “New theoreti-
cal and practical aspects of electromagnetic soundings at low induction
numbers,” Geophysics, vol. 67, pp. 1441–1451, 2002.
[4] M. A. Pérez-Flores, S. Méndez-Delgado, and E. Gómez-Treviño, “Imag-
ing low-frequency and dc electromagnetic fields using a simple linear
approximation,” Geophysics, vol. 66, no. 4, pp. 1067–1081, 2001.
[5] A. Tikhonov and V. Arsenin, Solutions of Ill-Posed Problems. Washington,
DC, USA: V. H. Winston, 1977.
[6] R. L. Parker, Geophysical Inverse Theory. Princeton, NJ, USA: Princeton
Univ. Press, 1994.
[7] S. C. Constable, R. L. Parker, and C. G. Constable, “Occam’s inversion:
A practical algorithm for generating smooth models from electromagnetic
sounding data,” Geophysics, vol. 52, pp. 289–300, 1987.
[8] H. Hidalgo, J. L. Marroquı́n, and E. Gómez-Treviño, “Piecewise smooth
models for electromagnetic inverse problems,” IEEE Trans. Geosci. Re-
mote Sens., vol. 36, no. 2, pp. 556–561, Mar. 1998.
[9] I. M. Varentsov, “A general approach to the magnetotelluric data inversion
in a piecewise-continuos medium,” Izvestya Phys. Solid Earth, vol. 38,
pp. 913–934, 2002.
[10] F. J. Esparza and E. Gómez-Treviño, “Electromagnetic sounding in the
resistive limit and the Backus–Gilbert method for estimating averages,”
Geoexploration, vol. 24, pp. 441–454, 1987.
[11] F. Esparza and E. Gómez-Treviño, “1-D inversion of resistivity and in-
duced polarization data for the least number of layers,” Geophysics,
vol. 62, pp. 1724–1729, 1997.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
HIDALGO-SILVA AND GÓMEZ-TREVIÑO: NONSMOOTH NONCONVEX OPTIMIZATION
[12] H. Hidalgo, E. Gómez-Treviño, and M. A. Pérez-Flores, “Linear programs
for the reconstruction of 2-D images from geophysical electromagnetic
measurements,” Subsurface Sens. Technol. Appl., vol. 5, pp. 79–96, 2004.
[13] D. C. Dobson, “Recovery of blocky images in electrical impedance to-
mography,” in Inverse Problems in Medical Imaging and Nondestructive
Testing, Engl, Rundell, Louis, Ed. New York, NY, USA: Springer, 1997,
pp. 43–64.
[14] C. Vogel and M. Oman, “Fast, robust total variation-based reconstruction
of noisy, blurred images,” IEEE Trans. Image Process., vol. 7, no. 6,
pp. 813–824, Jun. 1998.
[15] F. Lenti, F. Nunziata, C. Estatico, and M. Migliaccio, “Analysis of re-
constructions obtained solving-penalized minimization problems,” IEEE
Trans. Geosci. Remote Sens., vol. 53, no. 9, pp. 4876–4886, Sep. 2015.
[16] T. Lin, Y. Zhang, L. Wan, Y. Qu, and J. Lin, “A para-whole space model for
underground magnetic resonance sounding studies,” IEEE J. Sel. Topics
Appl. Earth Observations Remote Sens., vol. 9, no. 1, pp. 264–271, Jan.
2016.
[17] L. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise
removal algorithms,” Physica D, vol. 60, pp. 259–268, 1992.
[18] M. Benning, Singular Regularization of Inverse Problems, Ph.D. disser-
tation, Dept. Comput. Sci. Math., Univ. Muenster, Münster, Germany,
2011.
[19] S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin, “An iterated reg-
ularization method for total variation based image restoration,” SIAM
J. Multiscale Model. Simul., vol. 4, pp. 460–489, 2005.
[20] H. Hidalgo-Silva and E. Gómez-Treviño, “Bregman iterative algorithms
for 2-D geosounding inversion,” Inverse Problems Sci. Eng., vol. 23, no. 6,
pp. 1085–1099, 2015.
[21] Y. Hu and M. Jacob, “Higher degree total variation (HDTV) regularization
for image recovery,” IEEE Trans. Image Process., vol. 21, no. 5, pp. 2559–
2571, May 2012.
[22] C. Li, W. Yin, and Y. Zhang, “User’s guide for TVAL3: TV minimization
by augmented lagrangian and alternating direction algorithms,” CAAM
Rep., vol. 20, pp. 46–47, 2009.
[23] S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and
the Bayesian restoration of images,” IEEE Trans. Pattern Anal. Mach.
Intell., vol. PAMI-6, no. 6, pp. 721–741, Nov. 1984.
[24] M. Nikolova, “Analysis of the recovery of edges in images and signals
by minimizing nonconvex regularized least-squares,” Multiscale Model.
Simul., vol. 4, no. 3, pp. 960–991, 2005.
[25] D. Geman and C. Yang, “Nonlinear image recovery with half-quadratic
regularization,” IEEE Trans. Image Process., vol. 4, no. 7, pp. 932–946,
Jul. 1995.
[26] D. Geman and G. Reynolds, “Constrained restoration and the recovery of
discontinuities,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 3,
pp. 367–383, Mar. 1992.
[27] M. Nikolova, “Markovian reconstruction using a GNC approach,” IEEE
Trans. Image Process., vol. 8, no. 9, pp. 1204–1220, Sep. 1999.
[28] M. Nikolova, M. Ng, S. Zhang, and W. Ching, “Efficient reconstruction
of piecewise constant images using nonsmooth nonconvex minimization,”
SIAM J. Imag. Sci., vol. 1, no. 1, pp. 2–25, 2008.
[29] M. Nikolova, M. Ng, and C. Tam, “Fast nonconvex nonsmooth mini-
mization methods for image restoration and reconstruction,” IEEE Trans.
Image Process., vol. 19, no. 12, pp. 3073–3088, Dec. 2010.
[30] M. Jung and M. Kang, “Efficient nonsmooth nonconvex optimization for
image restoration and segmentation,” J. Sci. Comput., vol. 62, pp. 336–
370, 2014.
[31] W. Yin, S. Osher, D. Goldfarb, and J. Darbon, “Bregman iterative
algorithms for l1 -minimization with applications to compressed sensing,”
SIAM J. Imag. Sci., vol. 1, no. 1, pp. 143–168, 2008.
[32] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated
annealing,” Science, vol. 220, no. 4598, pp. 671–680, 1983.
9
[33] A. Blake and A. Zisserman, Visual Reconstruction, vol. 2. Cambridge,
MA, USA: MIT Press, 1987.
[34] L. Vese and T. Chan, “Reduced non-convex functional approximations
for image restoration & segmentation,” Math. Dept., Univ. California Los
Angeles, Los Angeles, CA, USA, Tech. Rep. CAM97-56, 1997.
[35] P. Charbonnier, L. Blanc-Féraud, G. Aubert, and M. Barlaud, “Determin-
istic edge-preserving regularization in computed imaging,” IEEE Trans.
Image Process., vol. 6, no. 2, pp. 298–311, Feb. 1997.
[36] H. Hidalgo-Silva, E. Gómez-Treviño, J. L. Marroquı́n, and F. J. Esparza,
“Piecewise continuous models for resistivity soundings,” IEEE Trans.
Geosci. Remote Sens., vol. 39, no. 12, pp. 2725–2728, Dec. 2001.
[37] S. Geman and D. McClure, “Statistical methods for tomographic image
reconstruction,” in Proc. 46th Sess. Internat. Statist. Inst. Bulletin ISI, vol.
52, pp. 5–21,1987.
[38] H. Hidalgo-Silva and E. Gómez-Treviño, “Bregman iterative algorithms
for 2-D geosounding inversion,” Inverse Problems Sci. Eng., vol. 23,
pp. 1085–1099, 2015.
[39] E. Esser, “Applications of Lagrangian based alternating direction methods
and connections to split Bregman,” Comput. Appl. Math., Univ. California
Los Angeles, Los Angeles, CA, USA, Tech. Rep. CAM Rep. 09-31, 2009.
Hugo Hidalgo-Silva (M’15) received the B.Sc. de-
gree in electronics engineering from the Instituto
Tecnológico Regional de Chihuahua, Chihuahua,
México, in 1980, the M.Sc. degree in electronics and
communications from the Centro de Investigación
Cientı́fica y Educación Superior de Ensenada (CI-
CESE), Ensenada, México, in 1985, and the Doc-
tor en Ciencias degree in computer science from the
Centro de Investigación en Matemáticas, Guanajuato,
México, in 1999.
He has been a Faculty Member of CICESE since
1985. His interests include inverse problems and pattern recognition.
Enrique Gómez-Treviño received the B.Sc. degree
in physics from the Universidad Autónoma de Nuevo
León, San Nicols de los Garza, México, in 1974, and
the M.Sc. and Ph.D. degrees both in geophysics from
the University of Toronto, Toronto, ON, Canada, in
1977 and 1981, respectively.
He performed postdoctoral research at the Univer-
sity of Toronto during 1981–1982. He was a Member
of the faculty of the Universidad Autónoma de Nuevo
León, during 1982–1983, and since 1984, he has been
a Member of the faculty of Centro de Investigación
Cientı́fica y Educación Superior de Ensenada, Ensenada, Mexico. His interests
include modeling and inversion of potential and electromagnetic data.
Dr. Gómez-Treviño is a Member of SEG, EAEG, UGM, and AGU.