Acta Mathematica Scientia 2018,38B(6):1821–1832

http://actams.wipm.ac.cn

EXISTENCE OF MULTIPLE SOLUTIONS FOR A

FRACTIONAL p-LAPLACIAN SYSTEM WITH
CONCAVE-CONVEX TERM∗

Junhui XIE ( Ÿ) Xiaozhong HUANG ( ‘§) Yiping CHEN ( ±²)

Department of Mathematics, Hubei University for Nationalities, Enshi 445000, China
E-mail : smilexiejunhui@hotmail.com; huangzheren@126.com; chenyiping66@gmail.com

Abstract In this article, we study the existence of multiple solutions for the following sys-
tem driven by a nonlocal integro-differential operator with zero Dirichlet boundary conditions
 2α

 (−∆)sp u = a(x)|u|q−2 u + c(x)|u|α−2 u|v|β , in Ω,

 α + β


2β (0.1)
 (−∆)sp v = b(x)|v|q−2 v + c(x)|u|α |v|β−2 v, in Ω,

 α + β



u = v = 0, in Rn \Ω,

where Ω is a smooth bounded domain in Rn , n > ps with s ∈ (0, 1) fixed, a(x), b(x), c(x) ≥ 0
and a(x), b(x), c(x) ∈ L∞ (Ω), 1 < q < p and α, β > 1 satisfy p < α + β < p∗ , p∗ = n−ps np
.
By Nehari manifold and fibering maps with proper conditions, we obtain the multiplicity of
solutions to problem (0.1).
Key words fractional p-Laplacian system; Nehari manifold; multiple solutions
2010 MR Subject Classification 35S15; 58J05; 58J40

1 Introduction
Let Ω be a smooth bounded domain in Rn , n > ps with s ∈ (0, 1) fixed. We study the
multiplicity of solutions to the following fractional elliptic system with zero Dirichlet boundary
conditions 
 2α

 (−∆)sp u = a(x)|u|q−2 u + c(x)|u|α−2 u|v|β , in Ω,

 α+β

2β (1.1)
 (−∆)sp v = b(x)|v|q−2 v + c(x)|u|α |v|β−2 v, in Ω,

 α + β



u = v = 0, in Rn \Ω,
where a(x), b(x), c(x) ∈ L∞ (Ω) and a(x), b(x), c(x) ≥ 0, 1 < q < p and α > 1, β > 1 satisfy
np
p < α + β < p∗ , p∗ = n−ps is the fractional Sobolev critical exponent, and (−∆)sp is the
∗ Received September 2, 2017; revised April 23, 2018. This work was supported by the National Natural
Science Foundation of China (11761030), Hubei Provincial Natural Science Foundation of China (2017CFB352),
Doctoral Science Research Foundation of Hubei University for Nationalities (MY2013B019) and Youth Research
Foundation of Hubei Institute for Nationalities (MY2017Q023).
1822 ACTA MATHEMATICA SCIENTIA Vol.38 Ser.B

fractional p-Laplacian operator which is defined as

Z
|u(y) − u(x)|p−2 (u(y) − u(x))
(−∆)sp u(x) = 2 lim dy, x ∈ Rn .
ǫ→0 Rn \B (x)
ǫ
|x − y|n+ps
For the definition of fractional Laplacian operator, we follow some idea of [1–4].
Recently, fractional Sobolev spaces and the corresponding nonlocal partial equations are
widely applied to many subjects, such as anomalous diffusion, gradient potential theory, minimal
surfaces, optimization, phase transitions and water waves, one can see [5, 6] and their references.
Set a(x) = λ, b(x) = µ, c(x) ≡ 1, where λ and µ are both positive parameter. Then problem
(1.1) becomes the following fractional elliptic equation

 2α

 (−∆)sp u = λ|u|q−2 u + |u|α−2 u|v|β , in Ω,

 α + β

2β (1.2)
 (−∆)sp v = µ|v|q−2 v + |u|α |v|β−2 v, in Ω,

 α + β



u = v = 0, in Rn \Ω,
where 1 < q < p and α > 1, β > 1 satisfy p < α + β < p∗ . Chen and Deng [7] proved the
existence of multiple non-trivial solution of problem (1.2) under some condition on λ and µ.
In addition, many researchers investigated the fractional elliptic problems (1.2) with p = 2.
Wei and Su [8] obtained the existence and multiplicity of non-trivial solutions for the general
2n
nonlinearity mainly by the Mountain Pass lemma. When p = 2, q = 2∗s = n−2s and α + β = 2∗s ,
problem (1.2) becomes a non-local critical Laplacian system, in [9], the authors obtained the
existence of ground state solution.
Single equation involving fractional p-Laplacian had been investigated by some researchers,
such as 
 (−∆)s u = λ|u|q−2 u + |u|γ−2 u, in Ω,
p
(1.3)
 u = 0, in Rn \Ω,
np
where Ω is a bounded smooth domain in Rn , 1 < q < p and p < γ < p∗ = n−ps , Goyal and
Sreenadh [10] studied the existence and multiplicity of non-negative solutions to problem (1.3).
In order to illustrate our result, we first give some notations which will be used in the whole
paper. Let Ω be a smooth bounded domain in Rn , s ∈ (0, 1) and p > 1. We define the usual
fractional Sobolev space W s,p (Ω) endowed with the form
Z  p1
|u(x) − u(y)|p
kukW s,p (Ω) = kukLp(Ω) + n+ps
dxdy .
Ω×Ω |x − y|

Set Ωc be the complement of Ω in Rn . We define

 Z 
|u(x) − u(y)|p
X = u|u : Rn → R is measurable, u|Ω ∈ Lp (Ω), and dxdy < ∞ .
R2n \(Ωc ×Ωc ) |x − y|n+ps
The space X is endowed with the norm defined by
Z  p1
|u(x) − u(y)|p
kukX = kukLp(Ω) + dxdy .
R2n \(Ωc ×Ωc ) |x − y|n+ps
Then, we define
X0 = {u ∈ X : u = 0 a.e. in Ωc } (1.4)
No.6 J.H. Xie et al: EXISTENCE OF MULTIPLE SOLUTIONS 1823

equipped with the seminorm

Z  p1
|u(x) − u(y)|p
kukX0 = dxdy (1.5)
R2n \(Ωc ×Ωc ) |x − y|n+ps
with this norm, (X0 , k · kX0 ) is a Hilbert space with scalar product defined as
Z
|u(x) − u(y)|p−1 (v(x) − v(y))
hu, viX0 = dxdy. (1.6)
R2n \(Ωc ×Ωc ) |x − y|n+ps
Since u = 0 a.e. in Ωc , it is easy to see that
Z Z
|u(x) − u(y)|p |u(x) − u(y)|p
n+ps
dxdy = n+ps
dxdy.
R2n \(Ωc ×Ωc ) |x − y| R2n |x − y|

Hence, we just denote kukX0 by

Z  p1
|u(x) − u(y)|p
kukX0 = dxdy . (1.7)
R2n |x − y|n+ps
It can be seen from Theorems 6.5 and 7.1 in [5] that k ·kX0 and k ·kW s,p (Ω) are equivalent norms.
Moreover, by the result of [5], the embedding X0 ֒→ Lp (Ω) is continuous for any p ∈ [1, p∗ ] and
compact when p ∈ [1, p∗ ]. For further details on X, X0 and also for their properties, we refer to
[5] and the references therein.
Let X0 × X0 be the Cartesian product of two Hilbert spaces, which is a reflexive space
equipped with the norm
1
k(u, v)k = (kukpX0 + kvkpX0 ) p
Z Z  p1
|u(x) − u(y)|p |v(x) − v(y)|p
= n+ps
dxdy + n+ps
dxdy . (1.8)
R2n \(Ωc ×Ωc ) |x − y| R2n \(Ωc ×Ωc ) |x − y|

We say that (u, v) ∈ X0 × X0 is a weak solution of problem (1.1), if (u, v) ∈ X0 × X0 , and one
has
Z
|u(x) − u(y)|p−2 (u(x) − u(y))(φ(x) − φ(y))
dxdy
R2n \(Ωc ×Ωc ) |x − y|n+ps
Z
|v(x) − v(y)|p−2 (v(x) − v(y))(ψ(x) − ψ(y))
+ dxdy
R2n \(Ωc ×Ωc ) |x − y|n+ps
Z Z
2α
= (a(x)|u|q−2 uφ + b(x)|v|q−2 vψ)dx + c(x)|u|α−2 u|v|β φdx
Ω α + β Ω
Z
2β α β−2
+ c(x)|u| |v| vψdx. (1.9)
α+β Ω
Concerning with problem (1.1), our main results are the following.
Theorem 1.1 Let s ∈ (0, 1), n > ps and Ω be an open bounded domain in Rn . If
p p

a(x), b(x), c(x) ≥ 0, and 0 < ka(x)kLp−q p−q

∞ (Ω) + kb(x)kL∞ (Ω) < M , where

pq−p(α+β)
p − q α+β−p
p pq−p(α+β) p (α+β)(p−q−pq)−p2 q |Ω| (α+β)(p−q)
M =( ) (α + β − q) (p−q)(α+β−p) (α + β − p) p−q S (p−q)(α+β−p) p(α+β)
,
2
kc(x)kLα+β−p
t (Ω)

np
and t = np−(α+β)(n−ps) , then problem (1.1) has at least two solutions.
The rest of this article is organized as follows. We give some preliminaries in Section 2, and
the proof of Theorem 1.1 in Section 3.
1824 ACTA MATHEMATICA SCIENTIA Vol.38 Ser.B

2 Preliminaries
In this section, we give some notations about Nehari manifold and derive some preliminary
lemmas which will be used in the main conclusion.
Consider the following functional
Z Z
1 |u(x) − u(y)|p 1 |v(x) − v(y)|p
J(u, v) = dxdy + dxdy
p R2n \(Ωc ×Ωc ) |x − y|n+ps p R2n \(Ωc ×Ωc ) |x − y|n+ps
Z Z
1 2
− (a(x)|u|q + b(x)|v|q )dx − c(x)|u|α |v|β dx. (2.1)
q Ω α+β Ω
It is standard to see that (u, v) being a weak solution of problem (1.1) is equivalent to being
a critical point of the functional J(u, v). By a simple calculation, we have that J(u, v) ∈
C 1 (X0 × X0 , R), and
Z
′ |u(x) − u(y)|p−2 (u(x) − u(y))(φ(x) − φ(y))
hJ (u, v), (φ, ψ)i = dxdy
R2n \(Ωc ×Ωc ) |x − y|n+ps
Z
|v(x) − v(y)|p−2 (v(x) − v(y))(ψ(x) − ψ(y))
+ dxdy
R2n \(Ωc ×Ωc ) |x − y|n+ps
Z Z
2α
− (a(x)|u|q−2 uφ + b(x)|v|q−2 vψ)dx − c(x)|u|α−2 u|v|β φdx
Ω α + β Ω
Z
2β α β−2
− c(x)|u| |v| vψdx (2.2)
α+β Ω
for all (φ, ψ) ∈ X0 × X0 .
We will study critical points of the functional J(u, v) on the Hilbert space X0 × X0 . First
we denote the Nehari manifold by
N = {(u, v) ∈ X0 × X0 \{(0, 0)} : hJ ′ (u, v), (u, v)i = 0},
where h·, ·i is the scalar product on X0 × X0 given by (1.4). It is clear that all critical points of
J(u, v) must lie on N. So, we intend to study functional J(u, v) on N.
It is easy to find that (u, v) ∈ N if and only if
Z Z
q q
p
k(u, v)k − (a(x)|u| + b(x)|v| )dx − 2 c(x)|u|α |v|β dx = 0. (2.3)
Ω Ω

The Nehari manifold N is closely linked to the behavior of the functional ϕu,v : t → J(tu, tv)
for t > 0 which is
Z Z
tp p tq q q 2tα+β
ϕu,v (t) = J(tu, tv) = k(u, v)k − (a(x)|u| + b(x)|v| )dx − c(x)|u|α |v|β dx.
p q Ω α+β Ω
Those maps are known as fibering maps, which were introduced by Drábek and Pohozaev in
[11].
Lemma 2.1 Assume (u, v) ∈ X0 ×X0 \{(0, 0)}, then (tu, tv) ∈ N if and only if ϕ′u,v (t) = 0.
Proof The proof of this result can be obtained directly by the fact that ϕ′u,v (t) =
hJ ′ (tu, tv), (u, v)i. 
Owing to the above analysis, we know that the elements in N correspond to stationary
points of the maps ϕu,v . We notice that
Z Z
ϕ′u,v (t) = tp−1 k(u, v)kp − tq−1 (a(x)|u|q + b(x)|v|q )dx − 2tα+β−1 c(x)|u|α |v|β dx, (2.4)
Ω Ω
No.6 J.H. Xie et al: EXISTENCE OF MULTIPLE SOLUTIONS 1825

and
Z
ϕ′′u,v (t) = (p − 1)tp−2 k(u, v)kp − (q − 1)tq−2 (a(x)|u|q + b(x)|v|q )dx
Ω
Z
−2(α + β − 1)tα+β−2 c(x)|u|α |v|β dx. (2.5)
Ω
By Lemma 2.1, (u, v) ∈ N if and only if ϕ′u,v (1) = 0. Thus, for (u, v) ∈ N, by (2.5) we get
Z
ϕ′′u,v (1) = (p − 1)k(u, v)kp − (q − 1) (a(x)|u|q + b(x)|v|q )dx
Ω
Z
−2(α + β − 1) c(x)|u| |v|β dx
α
Ω
Z Z
p q q
= pk(u, v)k − q (a(x)|u| + b(x)|v| )dx − 2(α + β) c(x)|u|α |v|β dx
Ω Ω
Z Z
= (p − q) (a(x)|u|q + b(x)|v|q )dx + 2(p − (α + β)) c(x)|u|α |v|β dx
Ω Ω
Z
p
= (p − q)k(u, v)k − 2(α + β − q) c(x)|u|α |v|β
Ω
Z
= (p − (α + β))k(u, v)k + (α + β − q) (a(x)|u|q + b(x)|v|q )dx.
p
(2.6)
Ω
Next, we split N into three parts corresponding to local minimal, local maximal and points of
inflection, i.e.,
N+ = {(u, v) ∈ N : ϕ′′u,v (1) > 0};
N− = {(u, v) ∈ N : ϕ′′u,v (1) < 0};
N0 = {(u, v) ∈ N : ϕ′′u,v (1) = 0}.
Consequently, we prove the existence of solutions for problem (1.1) by exploring the existence
of minimizers of functional J(u, v) on N. The following lemma shows that local minimizers on
the Nehari manifold N are usually critical points of J(u, v).
Lemma 2.2 Suppose that (u0 , v0 ) is a local minimizer of J(u, v) on N, and (u0 , v0 ) 6∈ N0 .
Then (u0 , v0 ) is a critical points of J(u, v).
Proof The proof is the same as Theorem 2.3 in Brown-Zhang [12]. However, for com-
pleteness and the convenience of the reader, we give a detailed proof here.
If (u0 , v0 ) is a local minimizer of J(u, v) on N, then (u0 , v0 ) is a solution of optimization
problem
minimize J(u, v) subject to I(u, v) = 0,
where, I(u, v) = hJ ′ (u, v), (u, v)i. It follows from the standard minimization theory that there
exists a Lagrange multiplier λ ∈ R such that
J ′ (u0 , v0 ) = λI ′ (u0 , v0 ).
Therefore, hJ ′ (u0 , v0 ), (u0 , v0 )i = λhI ′ (u0 , v0 ), (u0 , v0 )i = λϕ′′u,v (1). Since (u0 , v0 ) 6∈ N0 , we
have ϕ′′u,v (1) 6= 0. Thus λ = 0. This completes the proof. 
In order to understand the Nehari manifold and fibering maps, we consider the functional
ψ : R+ → R defined by
Z
ψ(t) = tp−(α+β) k(u, v)kp − tq−(α+β) (a(x)|u|q + b(x)|v|q )dx.
Ω
1826 ACTA MATHEMATICA SCIENTIA Vol.38 Ser.B

We have the following results.

Lemma 2.3 Suppose (u, v) ∈ X0 × X0 \{(0, 0)}, ψ(t) Rsatisfies the following properties:
(α+β−q) Ω (a(x)|u|q +b(x)|v|q )dx 1
(i) ψ(t) has a unique critical point at tmax = [ (α+β−p)k(u,v)kp ] p−q ;
(ii) ψ(t) is strictly increasing on (0, tmax ) and strictly decreasing on (tmax , +∞);
(iii) lim ψ(t) = −∞.
t→+∞
Proof By a direct computation, we see that
Z
ψ ′ (t) = (p − (α + β))tp−(α+β)−1 k(u, v)kp − (q − (α + β))tq−(α+β)−1 (a(x)|u|q + b(x)|v|q )dx.
Ω
Setting ψ ′ (t) = 0, then there exists a real number tmax such that ψ ′ (tmax ) = 0 and ψ ′′ (tmax ) < 0
with  R  1
(α + β − q) Ω (a(x)|u|q + b(x)|v|q )dx p−q
tmax = .
(α + β − p)k(u, v)kp
Moreover, we have ψ ′ (t) > 0 for t ∈ (0, tmax ) and ψ ′ (t) < 0 for t ∈ (tmax , +∞). Finally, since
q < p, then (iii) holds. 
Before going on, we need to establish the following conclusions.
Lemma 2.4 (tu, tv) ∈ N+ if and only if ψ ′ (t) > 0, (tu, tv) ∈ N− if and only if ψ ′ (t) < 0.
Proof By direct computation, we deduce that for t > 0,
Z
ψ ′ (t) = (p − (α + β))tp−(α+β)−1 k(u, v)kp − (q − (α + β))tq−(α+β)−1 (a(x)|u|q + b(x)|v|q )dx.
Ω
Thus, if (tu, tv) ∈ N, then ϕ′u,v (t) = 0, which yields that,
Z Z
p−1 p q−1 q q α+β−1
t k(u, v)k − t (a(x)|u| + b(x)|v| )dx − 2t c(x)|u|α |v|β dx = 0
Ω Ω
and Z
ψ(t) = 2 c(x)|u|α |v|β dx. (2.7)
Ω
Consequently,
tα+β−1 ψ ′ (t) = ϕ′′u,v (t). (2.8)
Hence, (tu, tv) ∈ N+ if and only if ψ ′ (t) > 0, and (tu, tv) ∈ N− if and only if ψ ′ (t) < 0. 
With the above lemmas in hand, we can prove the following crucial lemma needed in the
proof of Theorem 1.1.
Lemma 2.5 Suppose (u, v) ∈ X0 × X0 \{(0, 0)}, there exist t1 , t2 such that t1 < tmax < t2
and (t1 u, t1 v) ∈ N+ and (t2 u, t2 v) ∈ N− , and we have
J(t1 u, t1 v) = inf J(tu, tv), J(t2 u, t2 v) = sup J(tu, tv).
0≤t≤tmax t≥0
R α β
Proof Noting that 2 Ω c(x)|u| |v| dx > 0, we derive that (2.7) has no solution if
Z
2 c(x)|u|α |v|β dx > ψ(tmax )
Ω
  p−(α+β) 
 q−(α+β)  R p−(α+β)
α+β−q α+β−q
p−q p−q ( Ω (a(x)|u|q + b(x)|v|q )dx) p−q
= −
α+β−p α+β−p p(q−(α+β))
k(u, v)k p−q
  p−(α+β) R p−(α+β)
p−q α+β−q p−q ( Ω (a(x)|u|q + b(x)|v|q )dx) p−q
= . (2.9)
α+β−q α+β−p p(q−(α+β))
k(u, v)k p−q
No.6 J.H. Xie et al: EXISTENCE OF MULTIPLE SOLUTIONS 1827

Moreover, considering (2.4) and (2.7), if a(x), b(x) and c(x) satisfy (2.9), then ϕ′u,v (t) > 0.
Otherwise, we have ϕ′u,v (t) < 0. Hence, (tu, tv) 6∈ N for any t > 0. On the other hand, if
Z
0<2 c(x)|u|α |v|β dx < ψ(tmax ),
Ω

then there exist t1 and t2 with t1 < tmax < t2 which yields that
Z
ψ(t1 ) = ψ(t2 ) = 2 c(x)|u|α |v|β dx,
Ω

and ψ ′ (t1 ) > 0 ψ ′ (t2 ) < 0. Due to (2.4) and (2.7), we know that ϕ′u,v (t1 ) = ϕ′u,v (t2 ) = 0.
Besides, we have ϕ′′u,v (t1 ) > 0, ϕ′′u,v (t2 ) < 0 as (2.8) holds. These facts imply that the fibering
map ϕu,v has a local minimum at t1 and a local maximum at t2 such that (t1 u, t1 v) ∈ N+ and
(t2 u, t2 v) ∈ N− , since ϕu,v (t) = J(tu, tv) and

J(t2 u, t2 v) ≥ J(tu, tv) ≥ J(t1 u, t1 v) for each t ∈ [t1 , t2 ],

J(t1 u, t1 v) ≤ J(tu, tv) for each t ∈ [0, t1 ].

Thus
J(t1 u, t1 v) = inf J(tu, tv), J(t2 u, t2 v) = sup J(tu, tv).
0≤t≤tmax t≥0

3 Proof of Theorem 1.1

In this section, we derive the main result by combining Nehari manifold, fibering maps and
some other analysis method.
np
Noting that α + β ∈ (p, p∗ ), where p∗ = n−ps is the fractional Sobolev critical exponent.
Let S denotes the best Sobolev constant for the embedding of X0 ֒→ Lα+β (Ω), which means,
for u ∈ X0 ,
Z  α+β
1

kukLα+β (Ω) = |u|α+β dx

Ω
Z  p1
1 1 |u(x) − u(y)|p
≤ S − p kukX0 = S − p dxdy . (3.1)
R2n \(Ωc ×Ωc ) |x − y|n+ps
p p
Lemma 3.1 Suppose a(x), b(x), c(x) ≥ 0, and 0 < ka(x)kLp−q p−q
∞ (Ω) + kb(x)kL∞ (Ω) < M , we
0
have N = ∅, where
pq−p(α+β)
 p − q  α+β−p
p
pq−p(α+β) p (α+β)(p−q−pq)−p2 q |Ω| (α+β)(p−q)
M= (α + β − q) (p−q)(α+β−p) (α + β − p) p−q S (p−q)(α+β−p)
p(α+β)
,
2 α+β−p
kc(x)k Lt (Ω)
np
and t = np−(α+β)(n−ps) .
p p
Proof We prove Lemma 3.1 by contradiction. Suppose 0 < ka(x)kLp−q p−q
∞ (Ω) + kb(x)kL∞ (Ω) <
0
M and we obtain N 6= ∅.
Then for (u, v) ∈ N0 , we have hJ ′ (u, v), (u, v)i = 0 and ϕ′′u,v (1) = 0. By (2.6), we obtain
Z
q − (α + β)
k(u, v)kp = (a(x)|u|q + b(x)|v|q )dx,
p − (α + β) Ω
1828 ACTA MATHEMATICA SCIENTIA Vol.38 Ser.B

and from (2.3), we obtain

Z Z
k(u, v)kp = (a(x)|u|q + b(x)|v|q )dx + 2 c(x)|u|α |v|β dx.
Ω Ω
Thus, Z
2(α+β−q)
k(u, v)k = p
p−q c(x)|u|α |v|β dx. (3.2)
Ω

By Young’s inequality, Hölder inequality and (3.1), we deduce that

Z
p 2(α + β − q)
k(u, v)k = c(x)|u|α |v|β dx
p−q
ZΩ  
2(α + β − q) α β
≤ (c(x)|u|)α+β + (c(x)|v|)α+β dx
p−q Ω α+β α+β
 Z  1t  Z  α+β
p∗
2(α + β − q) α ∗
≤ c(x)(α+β)t dx |u|p dx
p−q α+β Ω Ω
Z Z  α+β ∗

β 1 ∗ p
+ c(x)(α+β)t dx) t |v|p dx
α+β Ω Ω
2(α + β − q) − α+β α+β
≤ S p kc(x)kLt (Ω) k(u, v)kα+β , (3.3)
p−q
np
where t = np−(α+β)(n−ps) .
So, we have
  α+β−p
1
p−q α+β 1
k(u, v)k ≥ S p . (3.4)
2(α + β − q) kc(x)kα+β
Lt (Ω)

Besides, from (2.6) and Hölder inequality, we obtain

Z
p α+β−q
k(u, v)k = (a(x)|u|q + b(x)|v|q )dx
α+β−p Ω
 Z  p−q Z  p−q 
α+β−q p p p p
≤ a(x) p−q dx kukqLp(Ω) + b(x) p−q dx kvkqLp (Ω) . (3.5)
α+β−p Ω Ω

Owing to p < α + β < p∗ , we have

 Z  p−q Z  p−q 
α+β−q p p
q p p
p
k(u, v)k ≤ a(x) p−q dx kukLp(Ω) + b(x) p−q dx kvkqLp (Ω)
α+β−p Ω Ω
 Z  p−q  Z  p−q 
α+β−q p p
q p p
≤ a(x) p−q dx kukLα+β (Ω) + b(x) p−q dx kvkqLα+β (Ω)
α+β−p Ω Ω
 Z  p−q  Z  p−q 
α+β−q p p
− q p p q
≤ a(x) p−q dx S p kukX0 + b(x) p−q dx S − p kvkX0
α+β−p Ω Ω
α+β−q p p p−q α+β−q q
p |Ω| α+β S − p k(u, v)kq .
≤ (ka(x)kLp−q p−q
∞ (Ω) + kb(x)kL∞ (Ω) ) (3.6)
α+β−p
Thus, we have
  p−q
1

p−q α + β − q − pq α+β−q p p
1
k(u, v)k ≤ S |Ω| α+β (ka(x)kLp−q p−q
∞ (Ω) + kb(x)kL∞ (Ω) ) .
p (3.7)
α+β−p
Owing to (3.4) and (3.7), we get
p p  p − q  α+β−p
p
pq−p(α+β) p
ka(x)kLp−q p−q
∞ (Ω) + kb(x)kL∞ (Ω) ≥ (α + β − q) (p−q)(α+β−p) (α + β − p) p−q
2
No.6 J.H. Xie et al: EXISTENCE OF MULTIPLE SOLUTIONS 1829
pq−p(α+β)
(α+β)(p−q−pq)−p2 q |Ω| (α+β)(p−q)
×S (p−q)(α+β−p)
p(α+β)
= M,
kc(x)kLα+β−p
t (Ω)

p p

which contradicts 0 < ka(x)kLp−q p−q

∞ (Ω) + kb(x)kL∞ (Ω) < M. 
p
p−q
Lemma 3.2 J(u, v) is coercive and bounded from below on N for 0 < ka(x)k L∞ (Ω) +
p
p−q
kb(x)k ∞
L (Ω) < M.
Proof If (u, v) ∈ N, (2.1) and (2.3) yield that
Z Z
1 |u(x) − u(y)|p 1 |v(x) − v(y)|p
J(u, v) = dxdy + dxdy
p R2n \(Ωc ×Ωc ) |x − y|n+ps p R2n \(Ωc ×Ωc ) |x − y|n+ps
Z Z
1 2
− (a(x)|u|q + b(x)|v|q )dx − c(x)|u|α |v|β dx
q Ω α+β Ω
   Z
1 1 p 1 1
= − k(u, v)k − − (a(x)|u|q + b(x)|v|q )dx, (3.8)
p α+β q α+β Ω
combine with Hölder inequality, (3.1) and p < α + β, we get
Z Z  p−q
p
Z  p−q
p
p p
q
q q
(a(x)|u| + b(x)|v| )dx ≤ a(x) p−q dx |u|Lp (Ω) + b(x) p−q dx |v|qLp (Ω)
Ω Ω Ω
q α+β−q p p p−q
−p
≤S k(u, v)kq |Ω| α+β (ka(x)k p−q
L∞ (Ω) + kb(x)kLp−q
∞ (Ω) )
p . (3.9)
Then
   Z
1 1 p 1 1
J(u, v) = − k(u, v)k − − (a(x)|u|q + b(x)|v|q )dx
p α+β q α+β Ω
 
1 1
≥ − k(u, v)kp
p α+β
 
1 1 q α+β−q p p p−q
− − S − p k(u, v)kq |Ω| α+β (ka(x)kLp−q p−q
∞ (Ω) + kb(x)kL∞ (Ω) )
p . (3.10)
q α+β
Since 1 < q < p, if p p

0 < ka(x)kLp−q p−q

∞ (Ω) + kb(x)kL∞ (Ω) < M,

then J(u, v) is coercive and bounded from below on N. 

p
S p
Lemmas 3.1 and 3.2 guarantee that, if 0 < ka(x)k + kb(x)k < M, N = N+ N− ,
p−q
∞
L (Ω)
p−q
L∞ (Ω)
+ −
then J(u, v) is coercive and bounded from below on N and N . Thus we may define
c= inf J(u, v), c+ = inf J(u, v), c− = inf J(u, v).
(u,v)∈N (u,v)∈N+ (u,v)∈N−
p p
+
Lemma 3.3 (i) If 0 < ka(x)kLp−q p−q
∞ (Ω) + kb(x)kL∞ (Ω) < M, then c ≤ c < 0.
p p p
q p−q
(ii) If 0 < ka(x)kLp−q p−q
∞ (Ω) + kb(x)kL∞ (Ω) < ( p ) M, then c− > l0 for some positive constant
l0 depending on |Ω|, S, a(x) and b(x).
Proof (i) If (u, v) ∈ N+ , (2.6) imply that ϕ′′u,v (1) > 0, which means
Z
p−q p
k(u, v)k > c(x)|u|α |v|β dx.
2(α + β − q) Ω
Then
   Z
1 1 1 1
J(u, v) = − k(u, v)kp − 2 − c(x)|u|α |v|β dx
p q q α+β Ω
1830 ACTA MATHEMATICA SCIENTIA Vol.38 Ser.B
    
1 1 1 1 p−q
≤ − + − k(u, v)kp
p q q α+β α+β−q
(p − q)(α + β − p)
=− k(u, v)kp < 0. (3.11)
pq(α + β)
Consequently, according to the definition of c and c+ , we claim that c ≤ c+ < 0.
(ii) Let (u, v) ∈ N− , we have ϕ′′u,v (1) < 0, by (2.6), we obtain
Z
p−q
k(u, v)kp < c(x)|u|α |v|β dx. (3.12)
2(α + β − q) Ω

By Young’s inequality, Hölder inequality and (3.1), we have

Z Z  
α β α α+β β α+β
c(x)|u| |v| dx ≤ (c(x)|u|) + (c(x)|v|) dx
Ω Ω α+β α+β
α+β
≤ S− p kc(x)kα+β
Lt (Ω) k(u, v)k
α+β
. (3.13)

Then (3.12) and (3.13) ensure that

  α+β−p
1
p−q α+β 1 1
k(u, v)k > S p α+β−p α+β , (3.14)
2(α + β − q)
kc(x)kLα+β−p
t (Ω)

np
where t = np−(α+β)(n−ps) .
p p p
q p−q
Similar to the proof of Lemma 3.2, if 0 < ka(x)kLp−q p−q
∞ (Ω) + kb(x)kL∞ (Ω) < ( p ) M, we have
 
1 1
J(u, v) ≥ k(u, v)kq − k(u, v)kp−q
p α+β
  
1 1 q α+β−q p p p−q
− − S − p |Ω| α+β (ka(x)kLp−q∞ (Ω) + kb(x)k p−q
L∞ (Ω) ) p
q α+β
   α+β−p
p−q
1 1 p−q (α+β)(p−q)
≥ k(u, v)kq − S − p(α+β−p)
p α+β 2(α + β − q)
  
1 1 q α+β−q p p p−q
− − S − p |Ω| α+β (ka(x)kLp−q∞ (Ω) + kb(x)k p−q
L∞ (Ω) ) p
q α+β
≥ l0 > 0. (3.15)


−
Consequently, Lemma 3.3 (ii) indicates that J(u, v) is bounded from below on N . Then
there exists a minimizing sequence {un , vn } such that lim J(un , vn ) = inf J(u, v), this
n→∞ (u,v)∈N−
will be discussed in detail in Lemma 3.5. Next, we establish the existence of local minimum for
J(u, v) on N+ .
p p

Lemma 3.4 If 0 < ka(x)kLp−q p−q

∞ (Ω) + kb(x)kL∞ (Ω) < M, then the functional J(u, v) has a

minimizer (u1 , v1 ) in N+ and satisfies

(i) J(u1 , v1 ) = c = c+ < 0,
(ii) (u1 , v1 ) is a solution of problem (1.1).
Proof As J(u, v) is bounded from below on N+ , then there exists a minimizer sequence
{(un , vn )} ∈ N+ such that lim J(un , vn ) = c. It follows from Lemma 3.2 that J(u, v) is
n→∞
coercive and bounded from below on N, so {(un , vn )} is bounded in X0 × X0 . Then there exists
No.6 J.H. Xie et al: EXISTENCE OF MULTIPLE SOLUTIONS 1831

(u1 , v1 ) ∈ X0 × X0 , passing to a subsequence, that we still denote by {(un , vn )}, we may assume
that
un ⇀ u1 , vn ⇀ v1 weakly in X0 as n → ∞,
un → u1 , vn → v1 strongly in Lr (Ω) as n → ∞,
un → u1 , vn → v1 a.e. in Ω as n → ∞
for any 1 ≤ r < p∗ , the Brezis-Lieb Lemma [13] guarantees that there exists a function h(x) ∈
Lr (Rn ) such that
|un (x)| ≤ h(x), |vn (x)| ≤ h(x) a.e. in Rn .
Therefore, by Dominated Convergence Theorem we have that
Z Z
(a(x)|un | + b(x)|vn | )dx → (a(x)|u1 |q + b(x)|v1 |q )dx
q q
as n → ∞.
Ω Ω

Moreover, by Lemma 2.5, there exists a real number t1 such that (t1 u1 , t1 v1 ) ∈ N+ . Besides,
we show that {(un , vn )} → (u1 , v1 ) strongly in X0 × X0 . If not, then
k(u1 , v1 )k < lim inf k(un , vn )k.
n→∞
+
Consequently, for any {(un , vn )} ∈ N , we deduce that
Z
′ p−1 p q−1
lim ϕ (t1 ) = t1 k(un , vn )k − t1 (a(x)|un |q + b(x)|vn |q )dx
n→∞ un ,vn Ω
Z
−2tα+β−1
1 c(x)|un |α |vn |β dx
Ω
Z
> t1 k(u1 , v1 )kp − tq−1
p−1
1 (a(x)|u1 |q + b(x)|v1 |q )dx
Ω
Z
α+β−1
−2t1 c(x)|u1 |α |v1 |β dx
Ω
= ϕ′u1 ,v1 (t1 ) = 0, (3.16)
which means ϕ′un ,vn (t1 ) > 0 for n large enough. Noticing that (un , vn ) ∈ N+ , we claim that
ϕ′un ,vn (t) < 0 for t ∈ (0, 1)
and
ϕ′un ,vn (1) = 0 for all n.
Obviously, we deduce that t1 > 1. On the other hand, ϕ′u1 ,v1 (t) is decreasing on (0, t1 ), so
J(t1 u1 , t1 v1 ) ≤ J(u1 , v1 ) < lim J(un , vn ) = inf J(u, v).
n→∞ (u,v)∈N+

That is, (un , vn ) → (u1 , v1 ) strongly in X0 × X0 . This implies

J(un , vn ) → J(u1 , v1 ) = inf J(u, v) = c+ as n → ∞.
(u,v)∈N+

Indeed, (u1 , v1 ) is a minimizer of J(u, v) on N+ . By Lemma 2.2, it is standard to see that

(u1 , v1 ) is a solution of (1.1). 
p p

Lemma 3.5 If 0 < ka(x)kLp−q p−q

∞ (Ω) + kb(x)kL∞ (Ω) < M, then the functional J(u, v) has a

minimizer (u2 , v2 ) in N− and satisfies

(i) J(u2 , v2 ) = c = c− ,
(ii) (u2 , v2 ) is a solution of problem (1.1).
1832 ACTA MATHEMATICA SCIENTIA Vol.38 Ser.B

Proof Since J(u, v) is bounded from below on N− , there exists a minimizer sequence
un , ven )} ∈ N− such that
{(e
lim J(e vn ) = c− .
un , e
n→∞

By the same argument as that in the proof of Lemma 3.4, there exists (u2 , v2 ) ∈ X0 × X0 , we
extract a subsequence still denoted by {(e
un , ven ))}, we may assume that

en → u2 ,
u ven → v2 strongly in X0 × X0 as n → ∞.
p p
p−q p−q
Obviously, if 0 < ka(x)k L∞ (Ω) + kb(x)k L∞ (Ω) < M, then

un , ven ) → J(u2 , v2 ) =
J(e inf J(u, v) = c− as n → ∞,
(u,v)∈N−

similarly, (u2 , v2 ) is also a solution of problem (1.1). 

p
p−q
Proof of Theorem 1.1 By Lemmas 3.4, 3.5 and 2.2, we claim that if 0 < ka(x)k L∞ (Ω) +
p
kb(x)k p−q
< M, problem (1.1) has two solutions (u1 , v1 ) ∈ N+ and (u2 , v2 ) ∈ N− . Since
L∞ (Ω)
+T −
N N = ∅, it is clear that those two solutions are distinct from each other. This completes
the proof. 

References
[1] Ros-Oton X, Serra J. The Dirichlet problem for the fractional Laplacian: regularity up to the boundary. J
Math Pures Appl, 2014, 10: 275–302
[2] Ros-Oton X, Serra J. The Pohozaev identity for the fractional Laplacian. Arch Ration Mech Anal, 2014,
213: 587–628
[3] Servadei R, Valdinoci E. Mountain pass solutions for non-local elliptic operators. J Math Anal Appl, 2012,
389: 887–898
[4] Servadei R, Valdinoci E. The Brezis-Nirengerg result for the fractional Laplacian. Trans Amer Math Soc,
2015, 367: 67–102
[5] Di Nezza E, Palatucci G, Valdinoci E. Hitchhiker’s guide to the fractional Sobolev spaces. Bull Sci Math,
2012, 136: 521–573
[6] Bucur C, Valdinoci E. Non-Local Diffusion and Applications. Lecture Notes of the Unione Matematica
Italiana 20. Springer, 2016
[7] Chen W, Deng S. The Nehari manifold for a fractional p-Laplacian system involving concave-convex non-
linearities. Nonlinear Anal, 2016, 27: 80–92
[8] Wei Y, Su X. Multiplicity of solutions for non-local elliptic equations driven by the fractional Laplacian.
Calc Var Partial Differential Equations, 2015, 52: 95–124
[9] Guo Z, Luo S, Zou W. On critical sysyems involving fractional Laplacian. J Math Anal Appl, 2017, 446:
681–706
[10] Goyal S, Sreenadh K. A Nehari manifold for non-local elliptic operator with concave-convex non-linearities
and sign-changing weight function. Proc Indian Acad Sci, 2015, 125: 545–558
[11] Drábek P, Pohozaev S I. Positive solutions for the p-Laplacian: Application of the fibering method. Rov
Soc Edinburgh Sect A, 1997, 127: 703–726
[12] Brown K J, Zhang Y. The Nehari manifold for a semilinear elliptic equation with a sign-changing weight
function. J Differ Equ, 2003, 193: 481–499
[13] Brezis H. Analyse Fonctionelle. Théorie et Applications. Paris: Masson, 1983
[14] Chen W, Deng S. The Nehari manifold for nonlocal elliptic operators involving concave-convex nonlineari-
ties. Z Angew Math Phys, 2015, 66: 1387–1400
[15] W Chen, Xie J. Multiple solutions to the nonhomogeneous Kirchhoff type problem involving a nonlocal
operator. Diff Eq Appl, 2016, 8: 367–375
[16] Xu Y, Tan Z, Sun D. Multiplicity results for a nonlinear ellptic problem involving the fractional Laplacian.
Acta Math Sci, 2016, 36B: 1793–1803