Recommender Systems

Collaborative Filtering
1. User-based Recommendation[1]

input:

rating matrix: R ∈ℝ N×M

where rui is the rating of user u for item i .

example:

The Matrix Titanic Die Hard Forrest Gump Wall-E

John 5 1 ? 2 2

Lucy 1 5 2 5 5

Eric 2 ? 3 5 4

Diane 4 3 5 3 ?

hypothesis:

1
r ui =
̂

rvi
|Ni (u)| ∑
v∈Ni (u)

where Ni (u) is the set of k users most similar to u that have rated i .

weights:

∑ v ∈Ni (u) wuv rvi

r ui =
∑
̂

|wuv |
v ∈Ni (u)
cosine similarity:

∑∈ i Iuv
rui rvi

‾∑‾‾‾‾‾‾‾‾‾‾‾‾‾‾
∑∈ ‾
wuv = cos(xu , xv ) =
2 2
r r
√ i∈ Iu ui i Iv vi

Pearson correlation:

∑∈ –μ –μ i Iuv
(rui u
)(rvi v
)

‾∑‾‾‾‾‾‾‾‾‾‾‾‾
– μ ‾ ‾∑‾‾‾‾‾‾‾‾‾‾‾‾
–μ ‾
wuv = PC(u, v) =
2 2
(rui u) (rvi v)
√ ∈
i Iuv ∈ √ i Iuv

issues with Pearson correlation[2].

normalization:

−1
∑ v ∈Ni (u) wuv h(rvi )
r ui = h
∑
̂

( |wuv | )
∈Ni (u)
v

mean-centering:

h(rui ) = rui −μ u

∑ ∈Ni (u) wuv (rvi −μ v)

μ
v
r ui = u +
∑
̂

|wuv |
∈Ni (u)
v

2. Item-based Recommendation[1]

input:

rating matrix: R ∈ℝ N×M

where rui is the rating of user u for item i .

hypothesis:

∑∈ j Nu(i)
wij ruj
r ui =
∑∈
̂

|wij |
j Nu(i)
normalization:

−1
∑∈ j Nu(i)
wij h(ruj )
r ui = h
∑∈
̂

( |wij | )
j Nu(i)

3. Matrix Factorization(SVD)[4,5]

input:

rating matrix: R ∈ℝ N×M

where rui is the rating of user u for item i .

hypothesis:

R = PQ
̂

T
r ui = pu qi
̂

cost function:

min (rui −p T
u qi )
2

q, p ∑
(u,i) ∈K

normalization:

min (rui −μ−p T

u qi )
2

q, p ∑
(u,i) ∈K

add bias:

min (rui −μ−b −b −p u i

T
u qi )
2

q, p ∑
∈K
(u,i)

regularization:

min (rui −μ−b −b −p u i

T
u qi )
2
+ λ(||p u
||
2 2
+ ||qi ||
2 2
+ bu + b )
i
q, p ∑
∈K
(u,i)

stochastic gradient descent[4]:

← + η ⋅ [( −μ− − − T
) −λ⋅ ]
 bu ← bu + η ⋅ [(r − μ − b − b − p q ) − λ ⋅ b ]
ui u i
T
u i u

←
bi b + η ⋅ [(r
i −μ−b −b −p q)−λ⋅b]
ui u i
T
u i i

pu ← pu + η ⋅ [(r −μ−b −b −p q)⋅q −λ⋅p ]

ui u i
T
u i i u

qi ← qi + η ⋅ [(r −μ−b −b −p q)⋅p −λ⋅q]

ui u i
T
u i u i

cost function 2:

C = cui (pui −x T
u yi )
2
+ λ( ||xu ||
2
+
2
||yi || )
∑ ∑ ∑
u,i u i

where

pui =
{ 0,
1, rui > 0
and cui = 1 + αr ui
rui = 0

alternating least squares[5,6]:

∂C
= −2 cui (pui −x T
yi )yi + 2 xu λ
∂x
u
∑
u
i

= −2
∑
cui (pui −y T
i
xu )yi + 2 xu λ
i

= −2Y T
C
u
p(u) + 2Y
T
C
u
Y xu + 2 xu λ
∂C
∂x
= 0 ⇒ (Y
T
C
u
Y + λI)x u = Y
T
C
u
p(u)
u

⇒ xu = (Y
T
C
u
Y + λI )− 1
Y
T
C
u
p(u)

∂C
= −2 cui (pui −x T
yi )xu + 2 yi λ
∂y
u
∑
i
u

= −2X T i
C p(i) + 2X
T i
C X yi + 2 yi λ
∂C
∂y
= 0 ⇒ (X
T
C X +
i
λI)y i
= X
T i
C p(i)
i

⇒ yi = (X
T i
C X + λI )− 1
X
T
C p(i)
i

Reference

1. Recommender Systems Handbook:

 http://www.cs.bme.hu/nagyadat/Recommender_systems_handbook.pdf

2. Similarity Functions for User-User Collaborative Filtering:

http://grouplens.org/blog/similarity-functions-for-user-user-collaborative-
filtering/

3. Matrix Factorization: A Simple Tutorial and Implementation in Python:

http://www.quuxlabs.com/blog/2010/09/matrix-factorization-a-simple-
tutorial-and-implementation-in-python/

4. Matrix Factorization and Neighbor Based Algorithms for the Netflix Prize
Problem: http://primeranks.net/yeti/Temp/reco/nmf_nn_netflix.pdf

5. Collaborative Filtering for Implicit Feedback Datasets:

http://labs.yahoo.com/files/HuKorenVolinsky-ICDM08.pdf

6. Analytic solution for matrix factorization using alternating least squares:

http://math.stackexchange.com/questions/1072451/analytic-solution-for-
matrix-factorization-using-alternating-least-squares

7. Recommender Systems: Collaborative Filtering and other approaches:

https://www.youtube.com/watch?v=bLhq63ygoU8

8. Netflix Prize: http://www.netflixprize.com/