Received November 18, 2019, accepted December 15, 2019, date of publication December 20, 2019,

date of current version December 31, 2019.

Digital Object Identifier 10.1109/ACCESS.2019.2961182

Multi-Level Coupling Network for Non-IID

Sequential Recommendation
YATONG SUN 1, GUIBING GUO 1, XIAODONG HE 2, AND XIAOHUA LIU 3
1 SoftwareCollege, Northeastern University, Shenyang 110819, China
2 JD AI Research, Beijing 100050, China
3 AINemo, Montreal, QC H4B 2L9, Canada

Corresponding author: Guibing Guo (guogb@swc.neu.edu.cn)

This work was supported in part by the National Natural Science Foundation of China under Grant 61972078 and Grant 61702084, and in
part by the Fundamental Research Funds for the Central Universities under Grant N181705007.

ABSTRACT Sequential recommendation has been recently attracting a lot attention to suggest users with
next items to interact. However, most of the traditional studies implicitly assume that users and items are
independent and identically distributed (IID) and ignore the couplings within and between users and items.
Although deep learning techniques allow the model to learn coupling relationships, they only capture a
partial picture of the underlying non-IID problem. We argue that it is essential to distinguish the interactions
between multiple aspects at different levels explicitly for users’ dynamic preference modeling. On the other
hand, existing non-IID recommendation methods are not well designed for the sequential recommendation
task since users’ interaction sequence causes more complicated couplings within the system. Hence,
to systematically exploit the non-IID theory for coupling learning of sequential recommendation, we propose
a non-IID sequential recommendation framework, which extracts users’ dynamic preferences from complex
couplings between three aspects, including users, interacted items and target items at feature level, entity level
and preference level. Furthermore, to capture the couplings effectively, we implement the framework on the
base of capsule network. We indicate that the basic ideas of capsule seamlessly suit our purpose of modeling
sophisticated couplings. Finally, we conduct extensive experiments to demonstrate the effectiveness of our
approach. The results on three public datasets show that our model consistently outperforms six state-of-the-
art methods over three ranking metrics.

INDEX TERMS Capsule network, coupling network, non-IID recommendation, sequential

recommendation.
I. INTRODUCTION
With the rapid development of personalized services,
the sequential patterns of user-item interactions have been
viewed as a valuable source of information to enhance tradi-
tional recommender system as user interest generally evolves
over time. For instance, on YouTube, the videos played (by
the same user) at continuous time slots could be correlated.
To better understand and characterize users’ dynamic pref-
erence, the task of sequential recommendation has attracted
much attention. It aims to recommend top-N items with which
a user is likely to interact in subsequent time steps.
However, most of the traditional sequential recommen-
dation methods implicitly assume that users and items are
independent and identically distributed (IID). Specifically,
they overlook the complex couplings within and between
The associate editor coordinating the review of this manuscript and
approving it for publication was Ruqiang Yan.
features, users and items, which could be crucial for extract-
ing users’ dynamic preferences. For instance, suppose there
is a user who likes comedy movies in general. Recently he
has watched several movies acted by Denzel Washington.
Then a traditional sequential recommender feels reasonable
to recommend Carbon Copy, a comedy acted by Denzel
Washington, to the user. However, this decision might not
be appropriate as Denzel Washington is a great action film
actor while not good at acting comedy movies. In other
words, the influence of Denzel Washington, comedy and
action should not be considered as independent to users’
preference. The feature coupling of Denzel Washington and
comedy should lead to a low recommendation score. Apart
from features, the interacted items by different users are
also usually coupled together by sharing similar or cor-
related properties. Furthermore, for sequential recommen-
dation, it is important to consider the couplings of users’
long- and short-term preferences. Whereas existing non-IID

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/
VOLUME 7, 2019 186247

recommendation approaches are only designed to model
users’ static preference and ignore the influence of users’
purchase sequences.
On the other hand, the sequential relationships in recom-
mendation tasks are different from those in natural language
processing (NLP) applications. First, there is no strict gram-
mar rules that govern the order of items. A user’s new inter-
actions are usually generated by sharing similar or correlated
features (feature-feature coupling) and items (item-item cou-
pling) with the user’s historical feedbacks. Therefore, extract-
ing couplings effectively is crucial for learning sequential
patterns from the roughly ordered interactions, which cannot
be solved properly by copying NLP solutions. Secondly,
the data of sequential recommendation is much sparser than
NLP tasks. Since the interaction sequences are generated
by different users and the model aims to make accurate
personalized predictions. Hence, it is necessary to take user-
item couplings into consideration for modeling each user’s
dynamic preference.
To capture the complex couplings of sequential recommen-
dation, we propose a non-IID sequential recommendation
framework as shown in Figure 1(c). It incorporates compre-
hensive couplings at three levels. To begin with, we adopt
user/item intra-couplings to capture the couplings between
each feature and form the representation of each entity (each
user and item). Then we extract users’ long-, short- and long-
short term preferences from the couplings between users and
their interacted items. Finally, each user’s dynamic preference
is composed of the coupling of her multi-term preferences.
The main contributions of this paper are as follows.
• We highlight the significance of modeling couplings
for sequential recommendation and propose a novel
Multi-level Coupling Network to fill the gap between
non-IID recommender system and sequential recom-
mendation. It considers comprehensive couplings within
and between multi-aspects at different levels for accurate
modeling of users’ dynamic preferences.
• We implement the framework on the base of a capsule
network. We indicate that the basic idea of capsule seam-
lessly suits our need of modeling sophisticated coupling
relationships. It can also avoid the intrinsic limitations
of widely used CNN- and RNN-based methods. In addi-
tion, we devise three routing strategies to model user-
item couplings with different influential level of user
information. To our best knowledge, this is the first work
to demonstrate the effectiveness of capsule network for
sequential recommendation.
• We conduct extensive experiments on three real datasets.
The results show that our approach significantly outper-
forms six state-of-the-art models. We also demonstrate
the effectiveness of each of our designs with comprehen-
sive ablation tests.
II. RELATED WORK
Our work synthesizes two tracks of research work, namely
Non-IID Recommendation and Sequential Recommendation
186248
Y. Sun et al.: MCN for Non-IID Sequential Recommendation
A. NON-IID RECOMMENDER SYSTEMS
The study of non-IID recommender systems was introduced
to cater for the non-IID nature of users and items [1]–[3],
which can be summarized into two categories.
Similarity-based methods adopt manually designed sim-
ilarity measures to capture the couplings in the system.
CMF [4] exploits coupled user/item similarity [5] to find
similar users and items. CPF [6] captures user/item inter-
actions by factorizing the similarity matrix with Poisson
factorization. Wang et al. [5] propose a similarity measure
for nominal attributes. It argues that couplings not only
exist between attributes, but also between the values of each
attribute. Besides, Wang et al. [7] bring up with a similar-
ity measure for numerical attributes. It extends the Pear-
son correlation coefficient [8] with a Taylor-like non-linear
expansion.
The basic framework of similarity-based methods can
be summarized as Figure 1(a). In general, similarity-based
methods capture coupling relationships hierarchically. It first
models feature-level couplings from inter- and intra- feature
interactions, which are further used to form the coupling
representation of users and items. Then, it captures entity-
level couplings by assuming that like-minded users/items also
share similar coupling representations. Finally, it matches
each user-item pair with their coupling representations to
approximate the rating matrix.
Neural network-based methods exploit deep neural net-
works to extract couplings from comprehensive interactions.
NCF [9] captures linear and non-linear couplings between
users and items with a MF and a multi-layer perceptron
(MLP) module respectively. ConvNCF [10] uses an outer
product to explicitly model pairwise interactions between
each feature dimension. CoupledCF [11] jointly learns
explicit and implicit couplings within and between users and
items with a CNN and a MLP respectively. PGCR [12] puts
forward a policy gradients for contextual recommendations
to capture high-order non-linear couplings of features in the
contexts. HERS [13] aggregates user-user/item-item relations
within a given context for coupling learning.
The general framework of neural network based methods is
summarized as Figure 1(b). Similar to similarity-based meth-
ods, they also extract couplings from comprehensive interac-
tions. The main difference is that it learns couplings implicitly
with a hierarchical neural network, instead of distinguishing
the couplings at different levels explicitly. In other words,
similarity-based methods are built upon a strong prior about
how the couplings are organized in the system, while neural
network based methods mainly learn the couplings from data.
However, both approaches have their limitations. To be
specific, the performance of similarity-based methods heav-
ily rely on its coupling assumptions, while neural network
based methods require high-quality datasets to support the
training of its expressive model. Moreover, existing non-
IID recommender frameworks are not well designed for
the sequential recommendation task. Regarding the dynamic
characteristic of user preference, the coupling relationship
VOLUME 7, 2019
Y. Sun et al.: MCN for Non-IID Sequential Recommendation
FIGURE 1. Basic frameworks of non-IID recommendation methods, where (a, b) show the frameworks of most existing work and (c) is our proposed
generic non-IID sequential recommendation framework.
will be more complicated within and between more aspects
at different levels as shown in Figure 1(c).
B. SEQUENTIAL RECOMMENDATION
The development of sequential recommendation can be
roughly summarized into two stages.
In the early stage, approaches utilize Markov chain (MC)
to model the sequential data. For instance, FPMC [14] learns
a transition cube where each slice is a user-specific transi-
tion matrix of an underlying MC based on users’ purchase
history. The model is trained by an adaption of the Bayesian
Personalized Ranking [15] framework so that it can bring
together the advantages of MC and matrix factorization (MF).
Fossil [16] shares a similar idea with FPMC while it replaces
the MF part with FISM [17]. The major problem of FPMC
and Fossil is that they assume each interacted item influences
users’ behavior linearly and independently, which is unable
to capture the couplings between interacted items. HRM [18]
partially addresses the problem by summarizing multiple
interacting factors through a nonlinear max pooling opera-
tion. However, this aggregation operation fails to capture the
combination patterns of local adjacent items, i.e., the feature
level couplings within and between items. Besides, they also
have difficulty in learning long-short interactive preference
effectively.
Recently, many deep learning methods have been pro-
posed to capture accurate sequential features, and achieved
state-of-the-art recommendation accuracy in comparison
with the traditional approaches. For instance, DREAM [19]
and GRU4Rec [20], [21] embed all the users’ historical
interactions into the final hidden state of RNN, so as to
represent the current preference. Other RNN based meth-
ods enhance the performance by integrating user personal
information [22], attention mechanism [23], [24] or other
VOLUME 7, 2019
contextual information [25]–[28]. Memory networks [29],
[30] also have be utilized to read and write a user’s dynamic
preference explicitly, which assumes the items/features that
are similar to the target item should be given more attention.
KSR [31] combines knowledge graph with memory network
to provide interpretability for sequential recommendation.
On the other hand, CNN also has been recognized as a
suitable structure to mine local sequential patterns of item
sequences. Specifically, Caser [32] embeds a user’s recent
interactions into an ‘image’ which is then processed by
convolution networks, so as to form the user’s short-term
preference. 3D-CNN [33] integrates additional side informa-
tion to enhance the performance of CNN-based framework.
However, CNN is good at detecting features but less effective
at exploring the spatial relationships among features due to
pooling operation. To solve this issue, NextItNet [34] adopts
a stack of holed convolutional layers to model long-range
dependencies without pooling. M3R [35] employs a mix-
ture of models, each with a different temporal range. These
models are combined by a learned gating mechanism capa-
ble of exerting different model combinations given different
contextual information. BERT4Rec [36] employs the deep
bidirectional self-attention to model user behavior sequences.
Although existing sequential recommendation methods
have achieved promising results, they are not able to explic-
itly distinguish the complicated couplings within the system.
Specifically, RNN-based methods update user’s dynamic
preference by directly interacting with the representation of
the next item, and thus lose the important hierarchical inter-
actions. For CNN-based methods, the feature level coupling
is partially captured by the convolution operation. But the
learned coupling types are limited by the shape and number
of filters. It also lacks the capacity of modeling entity- and
preference-level couplings explicitly. Furthermore, recent
186249

studies show that straightforward transformations for fusing
embeddings of different representations usually limits the
effectiveness [37], [38]. However, most existing approaches
adopt oversimplified strategies, like concatenation, to com-
bine users’ general (long-term) and short-term preference,
which may fail to model complex long-short interactive pref-
erence of users.
III. NON-IID SEQUENTIAL RECOMMENDATION
FRAMEWORK WITH CAPSULE NETWORK
A. NON-IID SEQUENTIAL RECOMMENDATION
FRAMEWORK
To solve the challenges of non-IID sequential recommen-
dations, we borrow the basic ideas of both similarity-based
methods and neural network-based methods, and construct
our non-IID sequential recommendation framework as shown
in Figure 1(c).
In particular, we capture the couplings explicitly with
a hierarchical architecture as similarity-based methods.
At first, we conduct feature interactions within each embed-
ding vector to form the coupling representations of each user
and her interacted items. Then we perform entity level user-
item, item-item interactions to build multi-term preferences.
Finally, each user’s dynamic preference is composed of the
coupling of her multi-term preferences. It is worth noting
that we do not incorporate the target item for hierarchical
coupling learning and the reason is two-fold. In the first place,
users’ dynamic preference should not be influenced by the
interactions in the future. Besides, the computation cost is
too huge to extract different dynamic preference embeddings
for every target item. At the same time, we implement the
hierarchical framework on the base of capsule network to
effectively learn the sophisticated couplings. The details of
the architecture will be illustrated in the next section.
B. CAPSULE NETWORK
In this paper, we realize the non-IID sequential recommen-
dation framework on the base of capsule network. In this
section, we will first analyze why dose capsule network fit
our requirements of modeling sophisticated couplings. Then
we will elaborate the technical details of its implementation.
Firstly, the main intuition of capsule suits our need of man-
aging expanded feature representations. Capsule is proposed
to extend the expressive capacity of a neuron. It replaces
each neuron with an activity vector which represents the
instantiation parameters of a specific type of entity such as
an object or an object part [39]. Similar to our requirements,
we expand the representation of each feature for coupling
learning [5], [7] as each feature is not only represented by
its own extent, but also its relation with other features. Then
it is reasonable to adopt capsule to arrange these expanded
representations for higher level couplings.
Secondly, the dynamic routing mechanism of capsule net-
work is suitable for arranging the part-whole relationships of
representations of our multi-level coupling network, where
186250
Y. Sun et al.: MCN for Non-IID Sequential Recommendation
Algorithm 1 Dynamic Routing Algorithm
Input: outputs of capsules at layer y, where the output
embedding of capsule i is denoted as mi
Output: the activity vectors of capsules at the
subsequent layer y + 1, where capsule j ’s
activity vector is denoted as v̂j
1 for each i ∈ y and j ∈ y + 1 do
2 m̂j|i ← W ij mi ; // build vote vectors
3 bij ← 0 ; // initialize weights
4 for T iterations do
5 for each i ∈ y and j ∈ y + 1 do
exp(bij )
6 cij ← P exp(b )
;
k ik
7 for each 
Pj ∈ y + 1 do
8 sj ← i cij m̂j|i ; // weighted sum
||sj ||2 sj
9 v̂j ← ; // squashing
1+||sj ||2 ||sj ||
10 for each i ∈ y and j ∈ y + 1 do
11 bij ← bij + m̂>
j|i v̂j ; // update
12 return all the v̂j at layer y + 1;
the higher-level couplings are built on top of the lower-level
couplings. Specifically, the transformation matrices between
capsules at different levels learn to convert lower-level fea-
tures to higher-level representations. On the other hand,
the dynamic routing between capsules at the same level learn
to highlight the salient common information and eliminate the
noises. This process is crucial for sequential recommendation
since the common ground of different aspects usually reflects
a user’s current interest. Whereas the accidental feedback can
confuse the model to learn the real preference.
Formally, the dynamic routing process is summarized in
Algorithm 1. The input to a capsule j at layer (y + 1) is
the outputs of the capsules at layer y. At line 2, the vector
sent from capsule i to capsule j is first transformed by
a matrix W ij ∈ RD×D . The generated vector m̂j|i is termed
as the vote vector from capsule i to capsule j . bij is the
log prior probability that capsule i should be coupled to
capsule j , which is initialized by 0. From line 5 to line 6, for
each capsule i at layer y, we adopt softmax to calculate the
probability that capsule i will be coupled with each capsule
j at layer y + 1, denoted as cij . Next, at line 8, for each
capsule j at layer y + 1, we calculate the weighted sum of
its received vote vectors, which is denoted as sj . Then at line
9, we squash sj into vector v̂j , the activity vector of capsule j .
|| · || denotes the L2 norm of a vector. In this way, the L2 norm
of v̂j is squashed into (0, 1), representing the activation degree
of capsule j . The direction of v̂j indicates the instantiation
parameters of capsule j . Next, at line 10 and line 11, all the
log prior probabilities (bij ) are updated w.r.t. the dot similarity
between m̂j|i and v̂j . Intuitively, the agreements between vote
vectors are highlighted while the anomaly information is
reduced during routing iterations. Figure 5(b) presents an
VOLUME 7, 2019
Y. Sun et al.: MCN for Non-IID Sequential Recommendation
FIGURE 2. Treat user as data point for routing.
example of a simple capsule network with four input embed-
dings and three output capsules.
With the help of dynamic routing, features can be con-
stituted hierarchically through the capsule network. For
instance, in computer vision tasks [40]–[43], the features are
learned in the form of edges, shapes and objects. In neural
language processing tasks [44], the features are learned in
the form of words, phrases and sentences. Wang et al. [45]
propose a capsule architecture for sentiment analysis, but
replace the routing process with an attention mechanism.
C. DYNAMIC ROUTING WITH USER INFORMATION
When modeling user-item couplings with a capsule network,
the straightforward intuition is to pose more attention to
the items which are similar to the user’s general interest.
However, the extent of user influence is hard to determine.
To investigate an appropriate manner of guiding routing with
user information, we devise three dynamic routing strategies
with different influential levels.
1) USER AS DATA POINT
During dynamic routing, the lower-level capsules can be
viewed as data points that will be clustered into several Gaus-
sian distributions. Therefore, the most straightforward way of
integrating user information is to treat users as the data points
in the same item space. That is, users will participate in the
routing procedure as items as shown in Figure 2. This strategy
has relatively loose effect on the routing process, since user’s
prediction vector will be deactivated when its vote strongly
disagrees with the items. In other words, if a user’s current
taste differs a lot from her general interest, the routing output
will be dominated by the user’s short-term preference.
2) USER AS ACTIVATION REGULARIZER
To pose more influence on routing process, we assume that
capsules agreeing with user’s general interest should be acti-
vated easier. In comparison with the first strategy, we make
the user influence unneglectable by regularizing the activa-
tion calculation. Formally, we add the following updating rule
of v̂j after line 9 of Algorithm 1:
v̂>
j m̂j|u
v̂j = v̂j · (1)
||v̂j || ||m̂j|u ||
VOLUME 7, 2019
FIGURE 3. Treat user as activation regularizer for routing.
FIGURE 4. Treat user as bias for routing.
where m̂j|u is the squashed user vote vector for capsule j ,
which is calculated as the same process of items. In this
way, we re-scale the activity vector of capsule j w.r.t. its
cosine similarity with user’s vote vector while keeping the
original direction as shown in Figure 3. We adopt cosine
similarity, instead of dot product since we mainly focus on
the similarity between their properties (orientation). In this
view, if a higher-level capsule’s activity vector is far from
the user’s long-term preference, then this capsule would be
difficult to activate. This strategy only restricts the activation
degree of higher-level capsules, but imposes no constraints
on their properties.
3) USER AS ROUTING BIAS
The third strategy is to treat user information as the bias of
all lower-level capsules’ votes for one higher-level capsule.
Formally, we modify the line 8 of Algorithm 1 as follows:
P
i cij m̂j|i
µj ← P (2)
cij
v i
m̂hj|i − µhj )2
uP
t i cij (P
u
h
sj ← + m̂hj|u (3)
i cij
where µj denotes the expectation of items’ vote vectors for
capsule j . The superscript h denotes the h-th dimension of
a vector. Then the new sj is constituted with the standard
deviation of items’ votes and the user’s general preference as
shown in Figure 4. Similar to the second strategy, the routing
procedure is forced to consider the influence of user infor-
mation, regardless of its agreement with other items. But it
restricts routing harder as it contributes to the calculation of
186251

output properties directly. When there are strong disagree-
ments between a user with her latest interacted items, their
properties will be fused together to form the user’s long-short
term preference.
IV. MULTI-LEVEL COUPLING NETWORK
In this section, we will first introduce the general framework
of our multi-level coupling network (MCN), followed by the
detailed realization of feature-, entity- and preference-level
couplings.
A. GENERAL FRAMEWORK
Before the illustration of our model’s architecture, we begin
with the basic notations. Suppose the systems involves a set
of users U = {u1 , u2 , . . . , uM } and a set of items I =
{i1 , i2 , . . . , iN }, where M and N denote the number of users
and items respectively. Each user u has interacted with a
sequence of items Hu = {iu1 , iu2 , . . . , iu|Hu | }. We use Hu,t =
{iut−1 , iut−2 , . . . , iut−L } to denote the previous L items before u
interacting t. Let pu ∈ RD and qi ∈ RD be the embeddings
of user u and item i, which are obtained from P ∈ RM ×D
and Q ∈ RN ×D , where D denotes the dimension size of each
embedding vector.
Figure 5 shows the overall architecture of MCN. The basic
idea of MCN is that the user’s dynamic interest should be
modeled from hierarchical couplings. Specifically, given a
user u, a target item t ∈ Hu , we can obtain the corresponding
latest L interacted items, denoted as i ∈ Hu,t . We first retrieve
their embedding vectors as the inputs and capture the cou-
plings from the interactions within and between them w.r.t.
the following process: 1) we model the couplings between
each feature within the same embedding vector with outer
product operation; 2) then we capture user-item, item-item
couplings from the interactions between their feature vectors
with capsules; 3) finally, we extract user dynamic preference
from the couplings between multi-term preferences with a
capsule network.
B. FEATURE COUPLING WITH OUTER PRODUCT
The idea of modeling feature couplings with outer product
is inspired from both similarity-based and neural network
based non-IID methods. Specifically, similarity-based mod-
els expand the feature representation as they assume the simi-
larity of two features are dependent on their relationships with
other features [4], [5], [7]. Neural network based methods
adopt CNN or MLP to extract higher level couplings from the
feature interaction map generated by outer product between
user and item embeddings [10], [11]. As a fusion of these
two ideas, we conduct outer product above each embedding
vector itself as follows:
C u = pu ⊗ pu , C i = qi ⊗ qi (4)
u
where ⊗ denotes the outer product operation. C ∈ RD×D
and C i ∈ RD×D are the output feature coupling representa-
tions of user u and item i (i ∈ Hu,t ) respectively. By doing
this, each feature is represented by a vector, including its
186252
Y. Sun et al.: MCN for Non-IID Sequential Recommendation
own extent value and the relationships with other features.
Then each user and item is modeled by a feature interaction
matrix, which is further used as the source of extracting
higher level coupling in the following layers. In addition,
we also inherit the advantages and avoid the limitations of
the two original solutions. On the one hand, the manually
designed similarities might not be flexible for various datasets
and can only be applied with explicit attribute data. On the
other hand, the deep learning methods directly use pairwise
feature interactions to model the couplings between entities
(users and items), which may lose some nuanced feature level
coupling relationships.
C. ENTITY COUPLING WITH CAPSULE NETWORK
After obtaining the feature interaction maps of user u and
her interacted items, we are able to model the higher entity
level couplings, i.e., the user-item couplings and item-item
couplings. In MCN, we adopt capsule network for entity
level couplings. Unlike other applications with capsule [44],
we only follow the design philosophy of capsule, instead
of adopting the original architecture proposed in [39], [46].
This is because the original version is designed for the tasks
with dense input as images or texts, which consists several
continuous capsule layers to extract more abstract represen-
tations. However, in our recommendation task, the inputs are
sparse user/item ids. Moreover, the observable feedback only
covers a very small fraction of the full rating matrix, which
is known as the issue of data sparsity. Consequently, it is
difficult to learn a huge amount of parameters with origin
capsule network.
Formally, for item-item couplings, we feed the same
dimension of row vectors of L interaction maps into the
same capsule. Thus the height of our capsule network will
be D. The output of each capsule is a D-dimensional vector,
representing the agreements between L interacted items at
this dimension. Besides, to enable the couplings for different
agreement, we pose K capsules for each feature dimension.
Thus, the depth of our capsule network is K . Figure 6 presents
a toy example with L = 4, D = 5, K = 3. The formal output
of item-item coupling at dimension d with the k-th capsule is
computed as follows:
d , Cd , . . . , Cd )
Okd = Routing(C t−1 t−2 t−L
(5)
where C t−l u
d denotes the row d of item it−l ’s coupling matrix
t−l
C . The procedure of Routing is summarized as Algo-
rithm 1. Then all the item-item coupling outputs at depth k
will form a matrix Ok ∈ RD×D . All K matrices at different
depth will form a D × D × K cube O, representing the short-
term preference of user u.
For user-item coupling, the computation process is similar.
The main difference is the additional input of user C ud with
three influential levels of routing strategies, which are sum-
marized in Section III-C. Formally, the output of user-item
coupling at dimension d with the k-th capsule is computed as
VOLUME 7, 2019
Y. Sun et al.: MCN for Non-IID Sequential Recommendation

FIGURE 5. Illustration of the proposed MCN model. The left subfigure shows the overall architecture of MCN, and the right subfigue presents a simple
example of the capsule network with four input embeddings and three output capsules.

coupling:
0 1 0K
Ekd = Routing(O1d , . . . , OK
d , O d , . . . , O d , Cd )
u
(7)
All the routing outputs Ekd at different feature dimension d
and depth k will also form a D × D × K cube. We flatten
the cube to a D-dimensional vector by adding the K matrices
together and then summarize the D vectors together.
K X
X D
eu = Ekd (8)
k=1 d=1
Hence, the final prediction for user u to target item t can
be estimated by the inner product of the user’s dynamic
preference and the item’s embedding.
FIGURE 6. A toy example of coupling learning with capsule network.
It includes L D × D interaction maps as inputs for a D × K capsule r̂ut = e>
u qt (9)
network, which outputs a D × D × K cube.
The whole model can be jointly trained by minimizing the
following pairwise cost function.
follows:
X
u qt − eu qt 0 )) + λ||2||F
2
arg min − log(logistic(e> >
(10)
2
k (u,t,t 0 )
O0 d = Routing(C t−1
d , Cd , . . . , Cd , Cd )
t−2 t−L u
(6)
where t 0 denotes the sampled negative item, λ is the coef-
Similar to O, O0
also denotes a D × D × K cube. Besides, ficient of the L2 norm regularizer, and 2 denotes the set of
with the help of user information, we are able to highlight model parameters.
the overlapping parts of user’s general interest and her latest
tastes with different influential degree of user information. V. EXPERIMENTS AND ANALYSIS
The generated cube O0 represents the long-short term prefer- In this section, we evaluate the proposed model on a wide
ence of the user. spectrum of real-world datasets to answer the following
research questions:
D. PREFERENCE COUPLING WITH CAPSULE NETWORK RQ1 Does the proposed multi-level coupling network
After obtaining user’s long-, short- and long-short term pref- achieve state-of-the-art performance?
erences, we are able to model the user’s dynamic preference RQ2 How does MCN’s key hyper-parameters influence its
as the agreements between them. For preference coupling, performance.
we adopt the same capsule network structure as entity level RQ3 Is each design of the proposed model effective.

VOLUME 7, 2019 186253


TABLE 1. Basic statistics of datasets.
A. EXPERIMENTAL SETTINGS
1) DATASETS
We evaluate our model in two different domains to demon-
strate its generality. In specific, Movielens1 is a dataset con-
taining user watching behaviors on different movies. Ama-
zon [47] is an e-commerce dataset, from which we choose
two categories–Electronics and Kindle–to cover different
data characters. We transformed the explicit ratings into
implicit data, where each entry is marked as 0 or 1 indicating
whether the user has rated the item. To collect sufficient user
preference, we select those users with at least 10 records
for experiments as [9], [29]. The statistics of the processed
datasets are shown in Table 1.
2) EVALUATION PROCEDURE
To evaluate the performance of recommendation, each of our
datasets is split into training/validation/testing sets. For each
user, we preserve the last two interactions to validation and
testing sets, while the rest interactions are used for training.
To reduce the expensive cost of ranking all items for each
user during evaluation, we follow the common strategy [9],
[11], [48] to randomly sample 99 items that have not been
interacted by the user, and to rank the target item among the
100 items. We adopt three well-known metrics to evaluate the
ranking quality, namely Precision, Recall, and Normalized
Discounted Cumulative Gain (NDCG). We omit their defini-
tions due to space limitations. Generally, higher metric values
indicate better ranking performance.
3) COMPARISON METHODS
As our work synthesizes non-IID Recommendation and
Sequential Recommendation, we have to adopt two groups
of baselines for each track of the research works respectively.
• Non-IID Recommendation: NCF [9] is the combi-
nation of a MF model and a multi-layer perceptron
(MLP), which captures the linear and non-linear user-
item couplings respectively. CoupledCF [11] adopts a
CNN and a MLP to learn the explicit and implicit user-
item couplings respectively. As we do not consider other
side information for recommendation, we take the outer
product of user and item embedding vectors as the input
of its CNN module.
• Sequential Recommendation: FPMC [14] is a tra-
ditional method for sequential recommendation. It is
a combination of matrix factorization and Markov
1 grouplens.org/datasets/movielens/1m/
186254
Y. Sun et al.: MCN for Non-IID Sequential Recommendation
chain, which learn users’ long- and short-term pref-
erence respectively. GRU4Rec [20] is a deep learn-
ing based recommendation method. It adopts RNN to
capture sequential dependencies and make predictions.
Caser [32] utilizes CNN to explore the users’ short-term
preference by capturing local sequential patterns of adja-
cent items, which is then fused with users’ general inter-
est for prediction. RUM [29] adopts memory network to
store external memories for sequential recommendation.
We take the item-level RUM for comparison.
B. COMPLEXITY ANALYSIS AND IMPLEMENTATION
For each user, the computation cost of MCN at feature cou-
pling layer is O((L + 1)D2 ), simplified as O(LD2 ). Next,
the computation cost of a dynamic routing procedure with
L D−dimensional input embeddings and K output capsules
with T iterations is denoted as O(LKD2 + T (LK + K (LD +
D)+LKD)), simplified as O(LKD(D+T )). For item-item cou-
pling, user-item coupling, and preference coupling, the input
embedding number are L, L + 1, and 2K + 1, respectively
with D capsule networks. Thus, the computation cost of entity
coupling and preference coupling is denoted as O(KD2 (L +
K )(T + D)). Note that the hyper-parameters K , L and T are
much less than D, and each user’s inferred dynamic pref-
erence embedding is consistent given different target item.
Therefore, The computation cost of matching M users with
N items is about O(MK (L + K )D3 + MND), reformulated as
O(MD(K (L + K )D2 + N )). In practice, K (L + K )D2 and item
number N are about the same order of magnitude, indicating
that the time complexity of MCN is approximately O(MND),
which is the same order of magnitude as traditional matrix
factorization methods like BPR [15]. On the other hand, exist-
ing non-IID models like NCF [9] and CoupledCF [11] feed
each user-item pair into a deep neural network to compute
matching score. Their overall computation cost of inference
is about O(MNDγ ), where γ denotes the depth of the network.
Empirically, BPR, MCN, NCF, and CoupledCF cost
around 120s, 260s, 590s, and 670s, for matching all user-
item pairs on Amazon-kindle dataset, respectively. As we
can see, MCN achieves comparable computation complexity
to traditional matrix factorization method BPR, being much
more efficient than existing non-IID approaches (NCF and
CoupledCF). It demonstrates that MCN is very scalable for
real applications. We implement MCN with TensorFlow on
a single NVIDIA GeForce GTX 1080 Ti GPU without any
pre-training. In practice, MCN can further benefit from multi-
GPU servers with pure C++ implementation and CUDA
acceleration.
Parameter Settings: All the models are trained by Tensor-
Flow2 using Adam optimizer [49] with a batch size of 256.
The optimal experimental settings for comparison methods
are determined either by our experiments or suggested by
previous works. Specifically, the learning rate and regular-
ization coefficient are commonly set to 0.001. The latent
2 www.tensorflow.org/
VOLUME 7, 2019
Y. Sun et al.: MCN for Non-IID Sequential Recommendation
TABLE 2. Performance comparison of MCN with other models, the best performance among all baseline methods is boldfaced. All the improvement of
MCN over the best baseline is signifcant at the level of 0.01.
FIGURE 7. Parameter analysis of MCN in terms of routing iteration
numberT and capsule number K .
dimension D is set to 50. The length of item sequence L is
empirically set as 5. For NCF and CoupledCF, we employee
three hidden layers for their MLP modules with tower struc-
tures of {64, 32, 16} for Movielens and {32, 16, 8} for the
other datasets. The filter shape of CoupledCF is set as (8,8)
with 8 channels for all datasets. For Caser, the number of
its horizontal filters is set as 16 for each height. And the
number of vertical filters is set as 4. For GRU4Rec, the hidden
layer size is set to 100. For MCN, the capsule number K for
each feature dimension and the iteration number of dynamic
routing T are set as 3. The influence of these two parameters
will be analyzed in the following experiments.
C. MODEL COMPARISON
In this part, we answer RQ1 by comparing the performance
of MCN with 6 state-of-the-art baselines on 3 benchmark
datasets. For clarity, we post results with the first routing
procedure (user as data point) for overall comparison. The
detailed analysis of different routing strategies will be dis-
cussed in the following tests. Table 2 shows the overall
performance of all models. It is noted that the two non-IID
baselines (NCF and CoupledCF) perform the worst than other
VOLUME 7, 2019
sequential recommendation models, shedding light on the
importance of modeling users’ dynamic preference in real-
world applications. In addition, NCF is inferior to CoupledCF
consistently, indicating that the outer product operation over
user/item embedding vectors is effective for capturing feature
level couplings. All the deep learning based models (RUM,
GRU4Rec, Caser) consistently outperform the traditional
method (FPMC), indicating that neural networks can bet-
ter capture the coupling relationships than traditional struc-
tures with over simplified IID assumptions. Even though,
the performance of deep neural models may vary in dif-
ferent datasets. Specifically, GRU4Rec outperforms RUM
and Caser on electronics dataset, while performing worse
on movielens and kindle datasets. It may be explained by
the fact that electronics dataset contains much more short-
term dependencies than the other datasets and GRU4Rec is
good at modeling users’ short-term preferences. On the other
hand, users’ preference towards movies and books is rela-
tively stable and thus requires fine grained coupling modeling
between users’ long- and short-term preferences. However,
GRU4Rec lacks the ability of distinguishing users’ multi-
term preferences explicitly as RUM and Caser. Although
RUM and Caser captures users’ long- and short-term inter-
ests separately, their straightforward fusion strategy cannot
capture fine grained couplings effectively. Furthermore, they
are incapable of modeling entity-level couplings explicitly.
After addressing the limitations of existing works, our
MCN model outperforms all the counterparts consistently
across all three datasets over three evaluation metrics. The
maximal relative percentage of improvements can be up
to 10% in electronics dataset, and often stays at 5% in
other datasets. It demonstrates the effectiveness of our prac-
tice in extracting users’ dynamic preferences from complex
couplings between three aspects, including users, interacted
items and target items at feature level, entity level and prefer-
ence level. The superiority of MCN also verifies that capsule
186255

TABLE 3. The relative performance decline of MCN variants w.r.t. the
original version. The worst performance is boldfaced.
network is effective for modeling sophisticated coupling rela-
tionships.
D. PARAMETER ANALYSIS
In this subsection, we aim to answer RQ2 by analyzing the
effect of routing iteration number T and the capsule number
K of MCN. The results are presented in Figure 7.
Recall that routing is used to highlight the salient common
features and eliminate the accidental noises of the inputs.
Therefore higher iteration number T will focus on more
specific fraction of the input and discard the other infor-
mation as noises. As shown in Figure 7 (a), the perfor-
mance achieves the best when T = 3, indicating that it
is helpful to exclude noises with routing. But discarding
too much information will hinder the extraction of accurate
features.
On the other hand, capsule number K controls the
complexity of the model by projecting the input vectors
into K different capsules for routing. As shown in Fig-
ure 7(b), the accuracy stops growing when K is greater
than 3 and 2 on Movielens and the other datasets respec-
tively. This observation indicates that it is redundant to con-
duct routing with too many capsules. In addition, denser
dataset is able to reach higher performance by training with
larger K .
E. MODEL ABLATION AND DISCUSSION
In this part, we aim to answer RQ3 by diving into an in-
depth model analysis. Specifically, we need to demonstrate
the effectiveness of capturing coupling relationship with the
hierarchical structure, capturing feature level coupling with
outer product operation, building hierarchical couplings with
capsule network, building user’s dynamic interest with three
terms of preference and modeling user-item coupling with
different routing strategies.
186256
Y. Sun et al.: MCN for Non-IID Sequential Recommendation
FIGURE 8. The training loss and test accuracy comparison of MCN,
MCNmatrix , and MCNvector . We conduct early stop if the test accuracy
doesn’t increase in 100 continuous iterations.
1) IMPACT OF HIERARCHICAL STRUCTURE
In the previous analysis, we assume the coupling relation-
ships exist at different levels in the non-IID sequential rec-
ommendation circumstance. And we implement the model
with the corresponding hierarchical architecture. To verify
our assumption and the structure, we reduce the entity and
preference level coupling layer into a single layer capsule
network and test its performance. The results are denoted
as no_hierarchy in Table 3. We observe that the hierarchical
architecture indeed improves the performance consistently.
2) IMPACT OF OUTER PRODUCT OPERATION FOR FEATURE
LEVEL COUPLING
Although there are some works on modeling user-item inter-
actions with outer product [10], [11], there is little study
on learning feature interactions within the same embedding
vector. As illustrated before, we aim to represent each feature
with its own extent and its relationship with other features
together. To investigate whether this operation is helpful,
the experiment is two-fold. On the one hand, we build the
MCNmatrix variant, which abandons the outer product opera-
tion and directly represents each user/item with an embedding
matrix. On the other hand, we build MCNvector to feed the
user/item embedding vector into capsules directly. That is,
the height of the capsule network is reduced to 1. As shown
in Table 3, both of the variants deteriorate significantly.
From Figure 8, we observe that MCNmatrix can reach much
lower training error in comparison with the original MCN
model, but its test accuracy does not increase correspond-
ingly. In other words, MCNmatrix suffers from over-fitting as it
incorporates too many redundant parameters. On the contrary,
MCNvector reaches higher loss and lower accuracy compared
with the raw version, i.e., it suffers from under-fitting since
VOLUME 7, 2019
Y. Sun et al.: MCN for Non-IID Sequential Recommendation

FIGURE 9. The visualization of MCNAtten and MCN with different routing iterations. We project the learned feature vectors into two-dimension with
PCA on movielens dataset. We pick a random user with her latest interacted items as the input of MCN and MCNAtten , and visualize the output of one
capsule of the user-item coupling network, which is denoted as the red point. Then we feed the user’s next interacted item into the same capsule and
get its prediction vector, which is denoted as the blue point. Finally we sample 1000 negative items and feed them into the same capsule, their
prediction vectors are denoted as the gray points.

its capacity of coupling learning is limited by the simplified TABLE 4. The relative performance decline of MCN variants for
preference level coupling learning.
structure. Additionally, it shows that MCNmatrix outperforms
MCNvector on movielens while vice versa on electronics. This
can be explained that the over-fitting of MCNmatrix becomes
worse on sparser datasets.

3) IMPACT OF CAPSULE NETWORK

As illustrated before, we adopt capsule network to capture
entity/preference level couplings as its basic idea perfectly
suits our requirements. To justify our analysis, we conduct the
following experiments. To begin with, we build the MCNMLP
variant, which replaces each capsule network in the origi-
nal architecture with a MLP. Namely, MCNMLP learns all
the entity and preference level couplings implicitly with the
MLPs. Secondly, we build the MCNAtten variant to replace
each capsule with a self-attention [50] network (i.e., Multi-
Head Attention), which arranges the weights of feature vec-
tors explicitly. From Table 3, we find that MCNAtten beats
MCNMLP and most of the baselines in Table 2, indicating that
it is helpful to learn the couplings explicitly.
However, MCNAtten cannot achieve the performance of
original MCN, which gives evidence of the superiority of
capsule network. This might be explained that self-attention
is good at learning the relationships between different fea-
tures, but lacks the ability to build higher level representations
as capsule does. In addition, self-attention cannot enhance
the salient common information explicitly. As shown in Fig-
ure 9(a) and (b), the predictions of MCNAtten and MCN with
1 routing iteration are both inaccurate. However, Figure 9(b)
(c) and (d) show that the feature vector of the input and the
target item are getting closer through the routing iterations.
Moreover, in self-attention network, all the queries share the
same transformation matrix, and thus loss the spatial infor-
mation about the inputs. The original paper solves this issue
by incorporating a position embedding for each input. While
capsule network poses different parameter matrix for differ-
ent feature dimensions and input positions, which gives more
flexibility for learning hierarchical couplings and avoids the
bother of position embedding.
4) IMPACT OF MULTI-TERM PREFERENCES
In the preference coupling layer of MCN, we consider three
terms of preference, namely, long, short and long-short term
preferences. To investigate the influence of each part, we con-
duct the ablation test by removing each of the preference and
test its performance decline. The results are shown in Table 4,
where −l, −s, and −ls denote deleting long-term, short-
term, and long-short-term preference respectively. We can
observe that all the variants are inferior to the original model,
indicating that it is necessary to incorporate all three terms
preference for preference coupling. Besides, −l performs the
worst among all the variants in most settings, indicating that
user’s general taste is crucial for the modeling of dynamic
preference. It also proves that the influence of long-term
preference might vanish during user-item coupling. Lastly,
−s possesses the smallest accuracy gain. It can be explained
by the fact that the long-short term preference captured by
user-item coupling is capable of compensating user’s short-
term interest to some extent.
5) IMPACT OF ROUTING STRATEGIES
Recall that to guide the routing procedure of capsule network
with user information, we devise three strategies with incre-
mental user influential levels: 1) user as data point; 2) user

VOLUME 7, 2019 186257


TABLE 5. The performance of MCN variants with different routing
strategies.
as activation regularizer; and 3) user as vote bias. We denote
these approaches as point, activ and bias respectively. The
results are presented in Table 5. Specifically, the accuracy
is increasing with the promotion of user influential levels
on movielens dataset, while decreasing on electronics and
kindle datasets, indicating that user influence is more reliable
with denser dataset. This pattern can also be observed from
Table 4 as the −l variant gets highest performance decline on
movielens dataset.
VI. CONCLUSION
In this paper, we proposed a novel multi-level coupling
network (MCN) for non-IID sequential recommendation.
It considered comprehensive couplings within and between
multi-aspects at different levels: 1) feature level couplings
within each user and her interacted items; 2) user-item and
item-item entity level couplings; and 3) long-, short- and
long-short- term preference couplings. We implemented the
framework on the base of capsule network as its basic ideas
are seamlessly suitable for sophisticated coupling learning.
We did detailed analysis about the advantages of adopting
capsule network. We conducted extensive experiments on
three real datasets. The results showed that our approach sig-
nificantly outperformed six state-of-the-art models. We also
conducted comprehensive ablation experiments to verify the
effectiveness of each design of the model.
REFERENCES
[1] L. Cao, ‘‘Non-iid recommender systems: A review and framework of rec-
ommendation paradigm shifting,’’ Engineering, vol. 2, no. 2, pp. 212–224,
2016.
[2] L. Cao, Y. Ou, and P. S. Yu, ‘‘Coupled behavior analysis with applications,’’
IEEE Trans. Knowl. Data Eng., vol. 24, no. 8, pp. 1378–1392, Aug. 2012.
[3] L. Cao, ‘‘Coupling learning of complex interactions,’’ J. Inf. Process.
Manage., vol. 51, no. 2, pp. 167–186, 2015.
[4] F. Li, G. Xu, and L. Cao, ‘‘Coupled matrix factorization within non-
iid context,’’ in Proc. Pacific–Asia Conf. Knowl. Discovery Data Mining
(PAKDD). New York, NY, USA: Springer, 2015, pp. 707–719.
[5] C. Wang, L. Cao, M. Wang, J. Li, W. Wei, and Y. Ou, ‘‘Coupled nominal
similarity in unsupervised learning,’’ in Proc. 20th ACM Int. Conf. Inf.
Knowl. Manage. (CIKM), 2011, pp. 973–978.
[6] T. D. T. Do and L. Cao, ‘‘Coupled Poisson factorization integrated with
user/item metadata for modeling popular and sparse ratings in scalable
recommendation,’’ in Proc. AAAI Conf. Artif. Intell. (AAAI), 2018, pp. 1–7.
[7] C. Wang, Z. She, and L. Cao, ‘‘Coupled attribute analysis on numerical
data,’’ in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), 2013.
186258
Y. Sun et al.: MCN for Non-IID Sequential Recommendation
[8] G. Gan, C. Ma, and J. Wu, Data Clustering: Theory, Algorithms, and
Applications, vol. 20. Philadelphia, PA, USA: SIAM, 2007.
[9] X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T.-S. Chua, ‘‘Neural
collaborative filtering,’’ in Proc. 26th Int. Conf. World Wide Web (WWW),
2017, pp. 173–182.
[10] X. He, X. Du, X. Wang, F. Tian, J. Tang, and T.-S. Chua, ‘‘Outer product-
based neural collaborative filtering,’’ in Proc. Int. Joint Conf. Artif. Intell.
(IJCAI), 2018.
[11] Q. Zhang, L. Cao, C. Zhu, Z. Li, and J. Sun, ‘‘CoupledCF: Learning
explicit and implicit user-item couplings in recommendation for deep
collaborative filtering,’’ in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), 2018,
pp. 3662–3668.
[12] F. Pan, Q. Cai, P. Tang, F. Zhuang, and Q. He, ‘‘Policy gradients for
contextual recommendations,’’ in Proc. 28th Int. Conf. World Wide Web
(WWW), 2019.
[13] L. Hu, S. Jian, L. Cao, Z. Gu, Q. Chen, and A. Amirbekyan, ‘‘Hers:
Modeling influential contexts with heterogeneous relations for sparse and
cold-start recommendation,’’ in Proc. AAAI Conf. Artif. Intell., vol. 33,
2019, pp. 3830–3837.
[14] S. Rendle, C. Freudenthaler, and L. Schmidt-Thieme, ‘‘Factorizing person-
alized Markov chains for next-basket recommendation,’’ in Proc. Int. Conf.
World Wide Web (WWW), 2010, pp. 811–820.
[15] S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme, ‘‘BPR:
Bayesian personalized ranking from implicit feedback,’’ in Proc. 25th
Conf. Uncertainty Artif. Intell. (UAI), 2009, pp. 452–461.
[16] R. He and J. McAuley, ‘‘Fusing similarity models with Markov chains for
sparse sequential recommendation,’’ in Proc. IEEE Int. Conf. Data Mining
(ICDM), Dec. 2016, pp. 191–200.
[17] S. Kabbur, X. Ning, and G. Karypis, ‘‘FISM: Factored item simi-
larity models for top-N recommender systems,’’ in Proc. 19th ACM
SIGKDD Int. Conf. Knowl. Discovery Data Mining (KDD), 2013,
pp. 659–667.
[18] P. Wang, J. Guo, Y. Lan, J. Xu, S. Wan, and X. Cheng, ‘‘Learning hierar-
chical representation model for nextbasket recommendation,’’ in Proc. Int.
ACM SIGIR Conf. Res. Develop. Inf. Retr. (SIGIR), 2015, pp. 403–412.
[19] F. Yu, Q. Liu, S. Wu, L. Wang, and T. Tan, ‘‘A dynamic recurrent model
for next basket recommendation,’’ in Proc. Int. ACM SIGIR Conf. Res.
Develop. Inf. Retr. (SIGIR), 2016, pp. 729–732.
[20] B. Hidasi, A. Karatzoglou, L. Baltrunas, and D. Tikk, ‘‘Session-based
recommendations with recurrent neural networks,’’ in Proc. Int. Conf.
Learn. Represent. (ICLR), 2016.
[21] B. Hidasi and A. Karatzoglou, ‘‘Recurrent neural networks with top-k gains
for session-based recommendations,’’ 2018, arXiv:1706.03847. [Online].
Available: https://arxiv.org/abs/1706.03847
[22] M. Quadrana, A. Karatzoglou, B. Hidasi, and P. Cremonesi, ‘‘Personal-
izing session-based recommendations with hierarchical recurrent neural
networks,’’ in Proc. 11th ACM Conf. Recommender Syst. (RecSys), 2017,
pp. 130–137.
[23] W. Pei, J. Yang, Z. Sun, J. Zhang, A. Bozzon, and D. M. J. Tax, ‘‘Interacting
attention-gated recurrent networks for recommendation,’’ in Proc. Int.
Conf. Inf. Knowl. Manage. (CIKM), 2017, pp. 1459–1468.
[24] J. Li, P. Ren, Z. Chen, Z. Ren, T. Lian, and J. Ma, ‘‘Neural attentive session-
based recommendation,’’ in Proc. Int. Conf. Inf. Knowl. Manage. (CIKM),
2017.
[25] C. Qiang, W. Shu, H. Yan, and L. Wang, ‘‘A hierarchical contex-
tual attention-based gru network for sequential recommendation,’’ 2017,
arXiv:1711.05114. [Online]. Available: https://arxiv.org/abs/1711.05114
[26] P. Sun, L. Wu, and M. Wang, ‘‘Attentive recurrent social recommendation,’’
in Proc. 41st Int. ACM SIGIR Conf. Res. Develop. Inf. Retr. (SIGIR), 2018,
pp. 185–194.
[27] Z. Sun, J. Yang, J. Zhang, A. Bozzon, L.-K. Huang, and C. Xu,
‘‘Recurrent knowledge graph embedding for effective recommenda-
tion,’’ in Proc. 12th ACM Conf. Recommender Syst. (RecSys), 2018,
pp. 297–305.
[28] R. Li, Y. Shen, and Y. Zhu, ‘‘Next point-of-interest recommendation with
temporal and multi-level context attention,’’ in Proc. IEEE Int. Conf. Data
Mining (ICDM), Nov. 2018, pp. 1110–1115.
[29] X. Chen, H. Xu, Y. Zhang, J. Tang, Y. Cao, Z. Qin, and H. Zha, ‘‘Sequential
recommendation with user memory networks,’’ in Proc. 11th ACM Int.
Conf. Web Search Data Mining (WSDM), 2018, pp. 108–116.
[30] G. Guo, S. Ouyang, X. He, F. Yuan, and X. Liu, ‘‘Dynamic item block and
prediction enhancing block for sequential recommendation,’’ in Proc. Int.
Joint Conf. Artif. Intell. (IJCAI), 2019.
VOLUME 7, 2019
Y. Sun et al.: MCN for Non-IID Sequential Recommendation

[31] H. Jin, X. Z. Wayne, D. Hongjian, W. Ji-Rong, and Y. C. Edward, ‘‘Improv- YATONG SUN is currently pursuing the M.S.
ing sequential recommendation with knowledge-enhanced memory net- degree in software engineering with Northeast-
works,’’ in Proc. 11th ACM Int. Conf. Web Search Data Mining (WSDM), ern University, China. Since September 2018, he
2018, pp. 108–116. has been an Algorithm Engineer with JD AI
[32] J. Tang and K. Wang, ‘‘Personalized top-N sequential recommendation via Research. His research interests include machine
convolutional sequence embedding,’’ in Proc. 11th ACM Int. Conf. Web learning, data mining, and recommender sys-
Search Data Mining (WSDM), 2018, pp. 565–573. tems. Specifically, he is focusing on incorporating
[33] T. X. Tuan and M. P. Tu, ‘‘3d convolutional networks for session-based
knowledge-based information into sequential rec-
recommendation with content features,’’ in Proc. 11th ACM Conf. Recom-
ommendation tasks.
mender Syst. (RecSys), 2017, pp. 138–146.
[34] F. Yuan, A. Karatzoglou, I. Arapakis, J. M. Jose, and X. He, ‘‘A sim-
ple but hard-to-beat baseline for session-based recommendations,’’ 2018,
arXiv:1808.05163. [Online]. Available: https://arxiv.org/abs/1808.05163
[35] J. Tang, F. Belletti, S. Jain, M. Chen, A. Beutel, C. Xu, and E. H. Chi,
‘‘Towards neural mixture recommender for long range dependent user
sequences,’’ in Proc. 28th Int. Conf. World Wide Web (WWW), 2019, GUIBING GUO received the Ph.D. degree in
pp. 1782–1793. computer science from Nanyang Technological
[36] F. Sun, J. Liu, J. Wu, C. Pei, X. Lin, W. Ou, and P. Jiang, ‘‘BERT4Rec: University, Singapore, in 2015. He is currently a
Sequential recommendation with bidirectional encoder representations Professor with the Software College, Northeastern
from transformer,’’ in Proc. Int. Conf. Inf. Knowl. Manage. (CIKM), 2019. University, China. His research interests include
[37] A. Fukui, H. P. Dong, D. Yang, A. Rohrbach, and M. Rohrbach, ‘‘Mul- recommender systems, deep learning, natural lan-
timodal compact bilinear pooling for visual question answering and guage processing, and data mining.
visual grounding,’’ in Proc. Empirical Methods Natural Lang. Process.
(EMNLP), 2016.
[38] H. Ben-Younes, R. Cadene, M. Cord, and N. Thome, ‘‘MUTAN: Multi-
modal tucker fusion for visual question answering,’’ in Proc. Empirical
Methods Natural Lang. Process. (EMNLP), 2017.
[39] S. Sabour, N. Frosst, and G. E. Hinton, ‘‘Dynamic routing between cap-
sules,’’ in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2017.
[40] D. Wang and Q. Liu, ‘‘An optimization view on dynamic routing between
capsules,’’ in Proc. ICLR Workshop, 2018. XIAODONG HE is currently the Deputy Manag-
[41] P. Afshar, A. Mohammadi, and K. N. Plataniotis, ‘‘Brain tumor type ing Director of the JD AI Research and the Head
classification via capsule networks,’’ 2018, arXiv:1802.10200. [Online]. of the Deep learning, NLP and Speech Lab, and
Available: https://arxiv.org/abs/1802.10200 the Technical Vice President of JD.com. He is
[42] R. LaLonde and U. Bagci, ‘‘Capsules for object segmentation,’’ 2018, also an Affiliate Professor with the University of
arXiv:1804.04241. [Online]. Available: https://arxiv.org/abs/1804.04241 Washington, Seattle, serves in Ph.D. supervisory
[43] M. T. Bahadori, ‘‘Spectral capsule networks,’’ in Proc. ICLR Workshop, committees. Before joining JD.com, he was with
2018. Microsoft for about 15 years, served as a Princi-
[44] M. Yang, W. Zhao, J. Ye, Z. Lei, Z. Zhao, and S. Zhang, ‘‘Investigat- pal Researcher and the Research Manager of the
ing capsule networks with dynamic routing for text classification,’’ in DLTC with Microsoft Research, Redmond,
Proc. Conf. Empirical Methods Natural Lang. Process. (EMNLP), 2018,
WA, USA.
pp. 3110–3119.
[45] Y. Wang, A. Sun, J. Han, Y. Liu, and X. Zhu, ‘‘Sentiment analysis by
capsules,’’ in Proc. Int. Conf. World Wide Web (WWW), 2018.
[46] H. Geoffrey, S. Sara, and F. Nicholas, ‘‘Matrix capsules with EM routing,’’
in Proc. Int. Conf. Learn. Represent. (ICLR), 2018.
[47] J. McAuley and J. Leskovec, ‘‘Hidden factors and hidden topics: Under-
XIAOHUA LIU is currently a Researcher with
standing rating dimensions with review text,’’ in Proc. 7th ACM Conf.
AINemo. He is also a Postdoctoral Research Fel-
Recommender Syst. (RecSys), 2013, pp. 165–172.
[48] Y. Koren, ‘‘Factorization meets the neighborhood: A multifaceted collab- low with the University of Montreal. Prior to
orative filtering model,’’ in Proc. 14th ACM SIGKDD Int. Conf. Knowl. that, he was a Project Lead and a Researcher
Discovery Data Mining (KDD), 2008, pp. 426–434. with Microsoft Research Asia, Natural Language
[49] D. Kingma and J. Ba, ‘‘Adam: A method for stochastic optimization,’’ in Computing Group, and JD AI Research. His
Proc. 3rd Int. Conf. Learn. Represent. (ICLR), 2015, pp. 1–15. research interests include big data processing,
[50] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, social content mining, NLP for ranking, and
L. Kaiser, and I. Polosukhin, ‘‘Attention is all you need,’’ in Proc. Adv. machine learning.
Neural Inf. Process. Syst. (NIPS), 2017, pp. 5998–6008.

VOLUME 7, 2019 186259