A supplement to a regularization method

for the proximal point algorithm

Satit Saejung

Journal of Global Optimization

An International Journal Dealing with
Theoretical and Computational Aspects
of Seeking Global Optima and Their
Applications in Science, Management
and Engineering

ISSN 0925-5001

J Glob Optim
DOI 10.1007/s10898-012-9931-2

1 23
Your article is protected by copyright and
all rights are held exclusively by Springer
Science+Business Media, LLC.. This e-offprint
is for personal use only and shall not be self-
archived in electronic repositories. If you
wish to self-archive your work, please use the
accepted author’s version for posting to your
own website or your institution’s repository.
You may further deposit the accepted author’s
version on a funder’s repository at a funder’s
request, provided it is not made publicly
available until 12 months after publication.

1 23
 Author's personal copy
J Glob Optim
DOI 10.1007/s10898-012-9931-2

A supplement to a regularization method

for the proximal point algorithm

Satit Saejung

Received: 5 May 2011 / Accepted: 25 May 2012

© Springer Science+Business Media, LLC. 2012

Abstract The purpose of this paper is to show that the iterative scheme recently studied
by Xu (J Glob Optim 36(1):115–125, 2006) is the same as the one studied by Kamimura and
Takahashi (J Approx Theory 106(2):226–240, 2000) and to give a supplement to these results.
With the new technique proposed by Maingé (Comput Math Appl 59(1):74–79, 2010), we
show that the convergence of the iterative scheme is established under another assumption.
It is noted that if the computation error is zero or the approximate computation is exact, our
new result is a genuine generalization of Xu’s result and Kamimura–Takahashi’s result.

Keywords Maximal monotone operator · Proximal point algorithm · Hilbert space

1 Introduction

The problem of finding a zero of a maximal monotone operator plays an important role in
many branches of applied sciences. One of interesting methods for solving this problem is
the proximal point method and it has been studied and developed by many researchers. We
mention the following one developed by Kamimura and Takahashi [2] in 2000 as follows:
(the related definitions and notion will be given in the next section)

Theorem 1.1 Let H be a real Hilbert space and T : H → 2 H be a maximal monotone

operator such that T −1 0 := {x ∈ H : 0 ∈ T x}  = ∅. Suppose that Jr is the resolvent of T
associated with the parameter r > 0, that is, Jr = (I + r T )−1 . Define an iterative sequence
{xn } by the following way:

S. Saejung (B)
Department of Mathematics, Faculty of Science, Khon Kaen University, Khon Kaen 40002, Thailand
e-mail: saejung@kku.ac.th

S. Saejung
The Centre of Excellence in Mathematics, Commission on Higher Education (CHE), Sri Ayudthaya Road,
Bangkok 10400, Thailand

123
 Author's personal copy
J Glob Optim

⎧
⎪
⎨ x1 = x ∈ H,
yn ≈ Jrn xn , (1)
⎪
⎩
xn+1 = αn x + (1 − αn )yn ,
where {αn } ⊂ (0, 1), {rn } ⊂ (0, ∞), and the approximate computation of yn satisfy the
following conditions

(A1) limn→∞ αn = 0 and ∞ n=1 αn = ∞;
(B1) limn→∞ rn = ∞; 
(C1) yn − Jrn xn  ≤ δn and ∞ n=1 δn < ∞.
Then the sequence {xn } converges strongly to PT −1 0 x, where PT −1 0 denotes the projection
onto T −1 0.
In 2006, Xu [8] proposed the other iterative scheme for this problem and proved the
following result.
Theorem 1.2 Let H, T, Jr be given as above. Define an iterative sequence {xn } by the fol-
lowing way:

x1 = x ∈ H,
(2)
xn+1 = Jrn (αn x + (1 − αn )xn + en ),
where {αn } ⊂ (0, 1), {rn } ⊂ (0, ∞) and the computation error sequence {en } ⊂ H satisfy
the following conditions
 ∞
(A2) limn→∞ αn = 0, ∞ n=1 αn = ∞ and  n=1 |αn − αn+1 | < ∞;
∞
(B2) r n ∈ [a, b] for some a, b ∈ (0, ∞) and n=1 |rn+1 − rn | < ∞;
∞
(C2) n=1 e n  < ∞.
Then the sequence {xn } converges strongly to PT −1 0 x.
Recently, Song and Yang [5] fixed a small gap in the proof of the preceding theorem and
showed that the conclusion of Theorem 1.2 remains true with the condition (B2) replaced by
 ∞
(B3) rn ∈ [a, ∞] for some a ∈ (0, ∞) and either ∞ n=1 |rn+1 − rn | < ∞ or n=1 |1 −
rn
rn+1 | < ∞.
Remark 1.3 The schemes (1) and (2) are the same. To see this, let us set
u n ≡ αn x + (1 − αn )xn + en ,

αn ≡ αn+1 .
Then it follows from (2) that xn+1 = Jrn u n and
u n+1 = 
αn x + (1 − 
αn ) xn+1 + en+1
=
αn x + (1 − 
αn ) Jrn u n + en+1
=
αn x + (1 − 
αn ) yn ,
where
1
yn ≡ Jrn u n + en+1 .
1 −αn
Notice that
1
δn := yn − Jrn u n  = en+1 
1 −αn

123
 Author's personal copy
J Glob Optim

and
∞ ∞ ∞
1
δn < ∞ ⇐⇒ en+1  < ∞ ⇐⇒ en+1  < ∞.
1 −αn
n=1 n=1 n=1

The last equivalence follows since limn→∞  αn = 0. However, Theorems 1.1 and 1.2 are of
independent interest because the condition on the parameters are different.

In this short paper, we will consider the iterative scheme introduced by Xu for the problem
of finding a zero of a maximal monotone operator. Using the new technique proposed by
Maingé [3], we also show that the convergence result is established under the different con-
ditions on the control parameters. These results are supplement to both Theorem 1.1 proved
by Kamimura and Takahashi, and Theorem 1.2 by Xu. In particular case, for example, the
computation is exact or the computation error is zero, our results are better than theirs.

2 Preliminaries and auxiliary results

In this section, let us recall some related definitions and some known facts. Throughout this
paper, let H be a real Hilbert space with inner product ·, · and its associated norm  · .
The strong and weak convergence are denoted by → and , respectively. The following
elementary fact are needed in our paper:
♦ x + y2 ≤ x2 + 2 y, x + y for all x, y ∈ H ;
♦ xn , yn  → x, y whenever xn  x and yn → y.
The projection of an element x ∈ H onto a closed convex subset C ⊂ H , denoted by PC x,
is a unique element PC x ∈ C such that
x − PC x = inf{x − y : y ∈ C}.
It is also known that x − PC x, y − PC x ≤ 0 for all y ∈ C, that is, the angle between
x − PC x and y − PC x is obtuse.
An operator T : H → 2 H is monotone if x − x  , y − y   ≥ 0 for all (x, y), (x  , y  ) ∈
G(T ) := {(z, w) ∈ H × H : w ∈ T z}. A monotone operator T : H → 2 H is maximal
if there exists no monotone operator S : H → 2 H such that G(T )  G(S). The following
assertion is an easy fact obtained directly from the definition above: for a maximal monotone
operator T : H → 2 H and (x, y) ∈ G(T ),
(x, y) ∈ G(T ) ⇐⇒ x − z, y − w ≥ 0 for all (z, w) ∈ G(T ).
A more interesting and deeper result which characterizes maximality was proved by Minty
[4]: A monotone operator A : H → 2 H is maximal if and only if the range of I + r T is the
whole space H for some (and hence for all) r > 0. For more detail about maximal monotone
operators, we refer to Takahashi’s book [6].
To deal with a maximal monotone operator, the concept of the resolvent is introduced
and it plays an important role in our problem. Let T : H → 2 H be a maximal monotone
operator and r > 0. The resolvent of T associated with r , denoted by Jr is defined by
Jr x = (I + r T )−1 x for all x ∈ H . The following three properties are interesting:
♦ (The resolvent identity) If r, r  > 0, then
r r
Jr  = Jr  I + 1 −  Jr  .
r r

123
 Author's personal copy
J Glob Optim

♦ Jr is firmly nonexpansive, that is,

Jrn u − Jrn v2 + u − Jrn u − (v − Jrn v)2 ≤ u − v2
for all u, v ∈ H ;
♦ The set of zeros of T and the set of fixed points of Jr coincide, that is,
0 ∈ T z ⇐⇒ z = Jr z.
To simplify the notion, the two sets above are denoted by T −1 0 and Fix Jr , respectively.
Notice that T −1 0 is closed and convex.
The following result is well known from fixed point theory for nonexpansive mapping.
Lemma 2.1 (Demiclosedness Principle, see e.g. [1]). Suppose that S : H → H is a nonex-
pansive mapping, that is, Sx − Sy ≤ x − y for all x, y ∈ H . If {yn } is a sequence H
such that yn  p ∈ H and yn − T yn → 0, then p ∈ Fix S.
In this paper, we denote the set of all positive integers by N.

Lemma 2.2 ([7]) Suppose that {an } ⊂ [0, ∞), {bn } ⊂ (−∞, ∞), and {αn } ⊂ (0, 1) are
sequences such that
♦ an+1 ≤ (1 − αn )an + αn bn for all n ∈ N;
∞
♦ n=1 αn = ∞;
♦ lim supn→∞ bn ≤ 0.
Then limn→∞ an = 0.

Lemma 2.3 ([3]) Suppose that {an } is a sequence of real numbers and there exists a subse-
quence {am j } of {an } such that am j +1 > am j for all j ∈ N. Then there exists a nondecreasing
sequence {n k } of positive integers such that n k → ∞,
an k ≤ an k +1 and ak ≤ an k +1
for all k ∈ N.

3 Main results

We present our main result which is supplement to Theorem 1.2 proved by Xu [8] (and also
its corrections proved by Song and Yang [5]).
Theorem 3.1 Let H be a real Hilbert space and T : H → 2 H be a maximal monotone
operator such that T −1 0 := {x ∈ H : 0 ∈ T x}  = ∅. Suppose that Jr is the resolvent of
T associated with the parameter r > 0. Define an iterative sequence {x n } by the following
way:

x1 = x ∈ H,
(3)
xn+1 = Jrn (αn x + (1 − αn )xn + en ),
where {αn } ⊂ (0, 1), {rn } ⊂ (0, ∞) and the computation error sequence {en } ⊂ H satisfy
the following conditions

(A1) limn→∞ αn = 0 and ∞ n=1 αn = ∞;
(B4) rn ∈ [a, ∞) for some a ∈ (0, ∞);

123
 Author's personal copy
J Glob Optim

en 
(C3) limn→∞ αn = 0.

Then the sequence {xn } converges strongly to PT −1 0 x, where PT −1 0 denotes the projection
onto T −1 0.

Remark 3.2 In the presence of (A1), let us note that

(C3)  (C2).

To see that (C3)  (C2), let αn ≡ √1

n+1
and en  ≡ n+1 . On the other hand, let α2n ≡
1

n+1 , α2n−1 ≡ , e2n  ≡ and e2n−1  ≡

1 1 1 1
(n+1)2 (n+1)2
, (n+1)2
. Then we have (C2)  (C3).

Remark 3.3 It is clear that (B2) ⇒ (B3) ⇒ (B4) and (A2) ⇒ (A1). In particular, if
en  ≡ 0, our Theorem 3.1 is a strict improvement of Theorem 1.2 proved by Xu [8] and the
recent one proved by Song and Yang [5]. In fact, let r2n−1 ≡ 1 and r2n ≡ n. It is clear that
∞
∞
(B4) is satisfied while (B3)
∞ is not. Because n=1 |r n+1 − r n | ≥ n=1 |r2n − r2n−1 | = ∞
and ∞ rn
n=1 |1 − rn+1 | ≥
r2n
n=1 |1 − r2n+1 | = ∞.

Before proving the result, we need the following lemma.

Lemma 3.4 Suppose that {u n } is a sequence in H such that u n  p and u n − Jrn u n → 0

where {rn } ⊂ [a, ∞) for some a > 0. Then p ∈ A−1 0.

Proof We may assume without loss of generality that limn→∞ rn = r ∈ [a, ∞].

Case 1: r < ∞. It follows from the resolvent identity and the nonexpansiveness of the
resolvent that
 
r r
Jrn x − Jr x = Jr x + 1− Jrn x − Jr x
rn rn
 
 r 
≤ 1 −  Jrn x − x
rn

for all x ∈ H . Hence u n − Jr u n → 0. Now it follows from the demiclosedness

principle that p ∈ Fix Jr = T −1 0.
Case 2: r = ∞. In this case the proof is extracted from that of [2, Theorem 1]. We make
use of the fact that T is maximal monotone. That is, to show that p ∈ T −1 0, it
suffices to show that u − p, v ≥ 0 for all u, v ∈ H with v ∈ T u. Let u, v ∈ H
be such that v ∈ T u. Since r1n (u n − Jrn u n ) ∈ T Jrn u n and T is monotone, we have

 
1
u − Jrn u n , v − (u n − Jrn u n ) ≥ 0.
rn

In particular, it follows since Jrn u n  p and rn (u n

1
− Jrn u n ) → 0 that
u − p, v ≥ 0. 


123
 Author's personal copy
J Glob Optim

Proof of Theorem 3.1 Put z = PT −1 0 x. Then z = Jrn z for all n ∈ N and hence we have the
following
xn+1 − z = Jrn (αn x + (1 − αn )xn + en ) − Jrn z

en
≤ αn x − z + + (1 − αn )(xn − z)
αn
en
≤ αn x − z + + (1 − αn )xn − z
αn
 
en
≤ max x − z + , xn − z
αn
 
en 
≤ max x − z + , xn − z .
αn
en 
We may assume by (C3) that there exists a constant M > 0 such that x − z + αn ≤M
for all n ∈ N. It follows by induction that
xn − z ≤ M
for all n ∈ N.
Next, we make use of the fact that each resolvent Jrn is firmly nonexpansive. In particular,
let yn ≡ αn x + (1 − αn )xn + en , we have
Jrn yn − z2 + yn − Jrn yn 2 ≤ yn − z2 .
We claim that if there exists a subsequence {xn k } of {xn } such that
lim inf (xn k +1 − z − xn k − z) ≥ 0,
k→∞
then
lim sup x − z, xn k − z ≤ 0.
k→∞
To see this, note that
xn k +1 − z2 = Jrnk yn k − z2
≤ yn k − z2 − Jrnk yn k − yn k 2
≤ (αn k x − z + (1 − αn k )xn k − z + en k )2 − Jrnk yn k − yn k 2 .
This implies that
Jrnk yn k − yn k 2 ≤ (αn k x − z + (1 − αn k )xn k − z + en k )2 − xn k +1 − z2 .
Since limn→∞ en  = limn→∞ αn = 0, we have
2
lim sup Jrnk yn k − yn k
k→∞
≤ lim sup((αn k x − z + (1 − αn k )xn k − z + en k )2 − xn k +1 − z2 )
k→∞
 
= lim sup xn k − z2 − xn k +1 − z2
k→∞
 
= lim inf xn k +1 − z2 − xn k − z2
k→∞
≤ lim inf 2M(xn k +1 − z − xn k − z) ≤ 0.
k→∞

123
 Author's personal copy
J Glob Optim

In particular, Jrnk yn k − yn k → 0 and hence Jrnk xn k − xn k → 0. We now extract a subse-

quence {xn kl } of {xn k } such that liml→∞ x − z, xn kl − z = lim supk→∞ x − z, xn k − z and
xn kl  p ∈ H . It follows from Lemma 3.4 that p ∈ A−1 0, and hence

lim sup x − z, xn k − z = x − z, p − z = x − PA−1 0 x, p − PA−1 0 x ≤ 0.

k→∞

This proves our claim.

Notice that

xn+1 − z2 ≤ (1 − αn )xn − z2 + 2αn x − z, αn (x − z) + (1 − αn )(xn − z) + en 

(4)
+ 2en (M + en ).

In fact, we have the following chain of inequalities

xn+1 − z2
= Jrn (αn x + (1 − αn )xn + en ) − z2
≤ αn (x − z) + (1 − αn )(xn − z) + en 2
≤ (1 − αn )(xn − z)2 + 2 αn (x − z) + en , αn (x − z) + (1 − αn )(xn − z) + en 
≤ (1 − αn )xn − z2 + 2αn x − z, αn (x − z) + (1 − αn )(xn − z) + en 
+ 2 en , αn (x − z) + (1 − αn )(xn − z) + en 
≤ (1 − αn )xn − z2 + 2αn x − z, αn (x − z) + (1 − αn )(xn − z) + en 
+ 2en (αn x − z + (1 − αn )xn − z + en )
≤ (1 − αn )xn − z2 + 2αn x − z, αn (x − z) + (1 − αn )(xn − z) + en 
+ 2en (M + en ).

We finally analyze the inequality (4) by considering the following two cases.

Case 1: There exists an n 0 ∈ N such that xn+1 −z ≤ xn −z for all n ≥ n 0 . In this case,
we have lim inf n→∞ (xn+1 −z−xn −z) = limn→∞ (xn+1 −z−xn −z) =
0 and hence it follows from our claim that
en 
lim sup 2 x − z, αn (x − z) + (1 − αn )(xn − z) + en  + 2 (M + en )
n→∞ αn
= 2 lim sup x − z, αn (x − z) + (1 − αn )(xn − z) + en 
n→∞
= 2 lim sup x − z, xn − z ≤ 0.
n→∞

If we set an ≡ xn − z2 and bn ≡ 2 x − z, αn (x − z) + (1 − αn )(xn − z) + en  +

2 eαnn  (M + en ), then an+1 ≤ (1 − αn )an + αn bn and lim supn→∞ bn ≤ 0. It
follows immediately from Lemma 2.2 that limn→∞ xn − z2 = 0.
Case 2: There exists a subsequence {xm j − z} ⊂ {xn − z} such that xm j +1 − z >
xm j −z for all j ∈ N. In this case, we apply Lemma 2.3 to obtain a nondecreasing
sequence {n k } of positive integers such that n k → ∞,

xn k − z ≤ xn k +1 − z, (5a)

xk − z ≤ xn k +1 − z (5b)

123
 Author's personal copy
J Glob Optim

for all k ∈ N. By discarding some repeated terms, we may consider {x n k − z}
as a subsequence of {xn − z}. It follows from (4) and (5a) that
xn k +1 − z2 ≤ (1 − αn k )xn k − z2
+ 2αn k x − z, αn k (x − z) + (1 − αn k )(xn k − z) + en k 
+ 2en k (M + en k )
≤ (1 − αn k )xn k +1 − z2
+ 2αn k x − z, αn k (x − z) + (1 − αn k )(xn k − z) + en k 
+ 2en k (M + en k ).
This implies that
xn k +1 − z2 ≤ 2 x − z, αn k (x − z) + (1 − αn k )(xn k − z) + en k 
en 
+2 k (M + en k ).
αn k
Notice that lim inf k→∞ (xn k +1 − z − xn k − z) ≥ 0. It follows from our claim
above that
lim sup x − z, αn k (x − z) + (1 − αn k )(xn k − z) + en k 
k→∞
= lim sup x − z, xn k − z ≤ 0.
k→∞

Consequently, by (5b) and (C3), we have

lim sup xk − z2 ≤ lim sup xn k +1 − z2
k→∞ k→∞

≤ lim sup 2 x − z, αn k (x − z) + (1 − αn k )(xn k − z) + en k 

k→∞

en 
+2 k (M + en k ) ≤ 0.
αn k
The proof is finished. 


Similarly, we obtain the following result whose proof is omitted.

Theorem 3.5 Let H, T, Jr be as Theorem 3.1. Define an iterative sequence {xn } by the
following way:
⎧
⎨ x1 = x ∈ H,
⎪
yn ≈ Jrn xn , (6)
⎪
⎩
xn+1 = αn x + (1 − αn )yn ,
where {αn } ⊂ (0, 1), {rn } ⊂ (0, ∞) and the approximate computation of yn satisfy the
following conditions

(A1) limn→∞ αn = 0 and ∞ n=1 αn = ∞;
(B4) rn ∈ [a, ∞) for some a ∈ (0, ∞);
(C4) yn − Jrn xn  ≤ δn and limn→∞ αδnn = 0.
Then the sequence {xn } converges strongly to PT −1 0 x.

Remark 3.6 As mentioned in Remark 3.2, we also have (C4)(C1) in the presence of (A1).

123
 Author's personal copy
J Glob Optim

Remark 3.7 It is clear that (B1) ⇒ (B4). In particular, if δn ≡ 0, then our Theorem 3.5
is a strict improvement of Theorem 1.1 proved by Kamimura and Takahashi [2]. In fact, let
rn ≡ 1, it follows that (B4) is satisfied but (B1) is not.

Acknowledgments The author is supported by the Centre of Excellence in Mathematics, the office of Com-
mission on Higher Education of Thailand. The author would like to thank the referee for the suggestions.

References

1. Goebel, K., Kirk, W.A.: Topics in Metric Fixed Point Theory. Cambridge Studies in Advanced Mathemat-
ics, vol. 28. Cambridge University Press, Cambridge (1990)
2. Kamimura, S., Takahashi, W.: Approximating solutions of maximal monotone operators in Hilbert spaces.
J. Approx. Theory 106(2), 226–240 (2000)
3. Maingé, P.-E.: The viscosity approximation process for quasi-nonexpansive mappings in Hilbert
spaces. Comput. Math. Appl. 59(1), 74–79 (2010)
4. Minty, G.J.: Monotone (nonlinear) operators in Hilbert space. Duke Math. J. 29, 341–346 (1962)
5. Song, Y., Changsen, Y.: A note on a paper “A regularization method for the proximal point algorithm”.
J. Glob. Optim. 43(1), 171–174 (2009)
6. Takahashi, W.: Introduction to Nonlinear and Convex Analysis. Yokohama Publishers, Yokohama (2009)
7. Xu, H.-K.: Iterative algorithms for nonlinear operators. J. Lond. Math. Soc. 66(1), 240–256 (2002)
8. Xu, H.-K.: A regularization method for the proximal point algorithm. J. Glob. Optim. 36(1), 115–125 (2006)

123