Recommender Systems -

Personalized Recommendation
Prof. Rajib L. Saha
Personalized Retail

Indian School of Business

Personalized Radio

Indian School of Business

Personalized News

Indian School of Business

Amazon.in > Your orders > Your Recommendation
 Recommender Designs
• Content-based Recommenders: Find other
products with similar attributes (Pandora)
• Collaborative Filtering (CF): Use information
on what others buy/like (Amazon, Netflix)
– Item-to-Item CF
– User-similarity based CF
• Hybrid Systems (Spotify)
• Social Network based Recommenders
– LinkedIn will attempt to connect two people with
lots of common friends (say 50 or more)

Indian School of Business

Content-based Recommender
Find similar items based on distance between items (e.g., Pandora)
Distortion
in guitar

Rhythmic syncopation

Dreamy Drawbacks?
lyrics

Indian School of Business

 If using attributes to find similarities

• Find Nearest Neighbor(s) based on

distance (dissimilarity)
– Can use following Distance measures to
identify neighbors
• Euclidean distance
• Manhattan distance

– However, if all attributes are not available,

distance should be normalized by the
number of pairs of available attributes to
compute the distance.
 Collaborative Filtering
• If Person A has the same opinion as
Person B on an issue, A is more likely
to have B's opinion on a different
issue x, when compared to the opinion
of a person chosen randomly
 Traditional Collaborative
Filtering
• Customer as a p-dimensional vector of
items
– p: the number of distinct catalog items
– Components: bought (1)/not bought (0);
ratings; rated (1)/not rated(0).
• Find Similarity between customers
n customers X p items

These entries
could be
numbers
(ratings) as
well
 Cosine-based similarity
A.B
Cos(A,B) =
|A|∗|B|
• A: (a1, a2, …, aN)
• B: (b1, b2, …, bN)
• A.B: a1*b1+a2*b2+…+aN*bN
• |A|: (a12+a22+…+aN2)1/2 ; |B|: (b12+b22+…+bN2)1/2

1 2 3 4
Example: A 3 5 0 1
B 1 4 0 0
3∗1+5∗4+0∗0+1∗0
Cos(A,B)= =0.94
(32+52+02+12)1/2∗(12+42+02+02)1/2
Correlation-based Similarity

1 2 3 4
A 3 5 0 1
B 1 4 0 0

Covariance (A,B)
CorrAB =
𝑆𝑡𝑑𝑒𝑣 𝐴 ∗ 𝑆𝑡𝑑𝑒𝑣(𝐵)
n customers X p items

While computing
similarity between
Persons 1 & 2, Item 2’s
rating cannot be
included, since Person 2
hasn’t bought Item 2.

For binary data

(bought/didn’t buy), it is
NOT an issue.
Ideas for normalizing ratings
• Divide the ratings column by the
frequency (the number of customers
who have purchased or rated the item)

• It can make less well-known items

more visible
 Once similar, what item(s) to
recommend?
• The item that hasn’t been bought by the
user yet
• You may create a list of multiple items to
be considered for recommendation
• Finally, recommend the item he/she is
most likely to buy
– Rank each item according to how many
similar customers purchased it
– Or rated by most
– Or highest rated
– Or some other popularity criteria
 Negatives
• Memory-based / Lazy-learning
– What if recommendation is done based
on shopping cart (ride on users’ impulse
buying behavior)
– When does the recommendation engine
compute the “recommendation”?
• Computation-intensive
How to reduce computation?
• Randomly sample customers
• Discard infrequent buyers
• Discard items that are very popular or
very unpopular

• Cluster and compute similarity within

cluster
 Search-based Methods
• Based on previous purchases
– Books of the same/similar authors
– DVD titles of the same director
– Products that are identified by similar
keywords
A more sophisticated variation of the search-
based methods:
Item-to-Item Collaborative Filtering

• Cosine similarity among items

– Item being the vector
– Customers as components of the vector

• Correlation similarity among items

– Correlation of ratings of Items I & J
where users rated both I & J
 While computing
similarity between Items
1 & 2, Person 2’s rating
cannot be included,
since Person 2 hasn’t
bough Item 2
 Scalability and Performance of
Item-to-Item Collaborative Filtering
• Computation-expensive, however,
similar-items table is computed offline
• Online component: lookup similar
items for the user’s purchases &
ratings
• Dependent only on how many titles
the user has purchased or rated
 Disadvantages of Item-based
Collaborative Filtering
• Less diversity among items, compared to
users’ taste, therefore the
recommendations are often obvious

• When considering what to recommend to

a user, who purchased a popular item,
the association rules and item-based
collaborative filtering might yield the
same recommendation, whereas the
user-based recommendation will likely
differ.
 Runtime vs. Quality of
Recommendation
• Recommend while the customer is
browsing
vis-à-vis
• Recommend better but later
Collaborative Filtering (CF)
Item-Item CF (recommendation based on a similarly
rated item)

User-User CF (recommendation based on

a user with similar “persona”)

Product 1 Product 2 Product 3 Product 4

User 1 4 3 2 4 Product 1 Product 2 Product 3 Product 4
User 2 4 5 5 User 1 4 3 2 4
User 3 2 2 4 User 2 4 5 5
User 4 3 5 2 User 3 2 2 4
User 4 3 5 2

Indian School of Business

 Recommender Systems
• Recommender Systems use data on purchases, product ratings,
product characteristics and user profiles to predict which
products are best suited to a particular customer
– Most common design is a collaborative filter
– “Customers who bought this item also bought….”
– “People like you bought…”

Value to • Learn about new products

Consumers: • Sort through myriad choices

• Convert browser to buyers

Value to • Cross-sell
Firms: • Increase loyalty

Indian School of Business

 Long Tail

Source: Longtail.com

Supply-side drivers:
•centralized warehousing with more offerings
•lower inventory cost of electronic products

Demand-side drivers:
•search engines
•recommender systems
 What happens to the diversity
• Total sale of niche items go up in
absolute terms
• So does the sale of popular items
• However, market share of niche
products goes down (Why?)
– While niche products benefit, popular
products benefit even more
– ‘Rich-get-richer’ phenomena

Indian School of Business

 Hybrid Solutions
• Spotify started as a collaborative filtering approach
• Evolved to becoming hybrid -- content from crawling web and
machine learning to get tempo, tonality, loudness
• Hybrid designs (e.g., Spotify Discover) combine the “simplicity” of
collaborative filtering with the “fairness” of content-based designs

Indian School of Business

• How is friend suggestion on Facebook,
or a suggestion for connection on
LinkedIn or a match on a dating
platform is different?
– Number of mutual connections
– Similar interests - homophily

Indian School of Business

Image source: http://www.zinmobi.com/permission-
marketing-food-retailers/
Association Rules vs. Recommender Systems

Market basket analysis Recommender Systems

(Association rules) (Collaborative filtering)
• Finds many baskets • Finds items which
that contain the same have a large fraction of
items their baskets in
• Need lots of baskets common
• Used for generating • Number of baskets
impersonal, common unimportant
strategies • Used for
• Useful for large personalization
physical stores (chains, • Useful for online
supermarkets) recommendation (long-
tail)
 Amazon Recommender System
• Non-personal, based on sales statistics
– Best sellers, promotional, etc.

• Recommendations based on browsing history,

personal

• Recommendations based on Association Rules,

non-personal

• Personalized recommendation: Sign in > Account

> Your Recommendations
• Personalized recommendation over email
 A Critical Limitation of
Collaborative Filtering
• Cold Start:
– How to create recommendation for
• New users?
• New items?
 How to address Cold Start?
• Approaches to address cold start with new users:
– Popular items (get quick reaction of the users)
– Demographically relevant items
– Browsing history
– Secondary source of data --- social network, subscription
– Netflix – start with rating a few movies
• Approaches to address cold start with new items:
– Recommend to random users/ or some selective users
based on certain criteria
– How about offering the product to influential people in the
social network
Issues with Rating Matrix-based
Recommender Systems
• Person i likes Adventure Movie 1 (AM1) but has never
watched or rated Adventure Movie 2 (AM2); whereas
Person j has watched AM2 but not watched/rated AM1.
• What happens to the similarity between Persons i & j?

– The ratings of AM1 and AM2 are not even included while
computing similarity!

• Rating matrices are huge, and usually sparse. What is

the implication on computational burden on dealing
with the rating matrix?
Recommendation using SVD
R= UΣVT R : Rating Matrix
Nxn

UN x r: User-feature matrix

Vn x r: Item-feature matrix
 X U ∑ VT
1 0 0 0 2 0 0 1 0 4 0 0 0 0 1 0 0 0
0 0 3 0 0 0 1 0 0 0 3 0 0 0 0 1 0 0
= x x
0 0 0 0 0 0 0 0 -1 0 0 2.24 0 0.45 0 0 0 0.89
0 4 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0

k=3
0 0 1x 4 0 0x 0 1 0 0 0= 1 0 0 0 2 Compressed size
0 1 0 0 3 0 0 0 1 0 0 0 0 3 0 0 4*3+3+3*5=30 bytes
0 0 0 0 0 2.24 0.45 0 0 0 0.89 0 0 0 0 0
1 0 0 0 4 0 0 0
 Dealing with New Users
R= UΣVT
• 𝑟𝑖 = ith row of rating matrix = item ratings of user i
• 𝑢𝑖 = ith row of user-feature matrix = feature ratings of
user i

• 𝑟𝑖 = 𝑢𝑖 Σ 𝑉 𝑇 {dimension: 1xn = 1xr rxr rxn}

• 𝑟𝑖 𝑉 = 𝑢𝑖 Σ 𝑉 𝑇 V = 𝑢𝑖 Σ
• 𝑟𝑖 𝑉 Σ −1 = 𝑢𝑖 Σ Σ −1 = 𝑢𝑖

• 𝑢𝑛𝑒𝑤 = 𝑟𝑛𝑒𝑤 𝑉 Σ −1
– Let the new users rate a few items and use those partial
ratings to compute feature ratings
 Dealing with missing values
before applying SVD
• Impute the missing values in the
Rank matrix with user mean or item
mean
• If the rank matrix is already
normalized (mean-subtracted),
missing values can be simply zeroes
Some guidelines on evaluating a
recommender system?
• Divide the user-product rating matrix
to training and test randomly
• Predict Rating[user, product] for each
cell in the test user-product rating
matrix using one of the methods (as
shown in the next page)
• Calculate RMSE/MAE/MAPE
Calculate R[user, product]=?
– User-user collaborative filtering
• Predict rating of User u for Product p in test data
• Say, p has been bought by 𝑢𝑖 , 𝑢𝑗 , and 𝑢𝑘 and rated as 𝑟𝑖 , 𝑟𝑗 , and 𝑟𝑘
in training data
• Say, sim(u, 𝑢𝑖 )=𝑠𝑖 , sim(u, 𝑢𝑗 )=𝑠𝑗 , and sim(u, 𝑢𝑘 )=𝑠𝑘
• R[u,p]=(𝑟𝑖 *𝑠𝑖 +𝑟𝑗 *𝑠𝑗 +𝑟𝑘 *𝑠𝑘 )/(𝑠𝑖 +𝑠𝑗 +𝑠𝑘 )

– Item-item collaborative filtering

• Predict rating of Product p for User u in test data
• Say, u has bought by 𝑝𝑖 , 𝑝𝑗 and rated as 𝑟𝑖 , 𝑟𝑗 in the training data
• Say, sim(p, 𝑝𝑖 )=𝑠𝑖 , sim(p, 𝑝𝑗 )=𝑠𝑗
• R[p,u]=(𝑟𝑖 *𝑠𝑖 +𝑟𝑗 *𝑠𝑗 )/(𝑠𝑖 +𝑠𝑗 )
 Vulnerability of Recommender
Systems
• Recommender accuracy and neutrality may be
compromised
– Malicious users may try to push or kill a product by
using fake accounts
– Inconsistent ratings
– Some mechanism to establish integrity is necessary
• Privacy of users’ ratings or preferences can be
compromised
– In some systems it may be desired that users do not
get to know each other’s opinions
– Use of less transparent algorithm is less vulnerable
to hack
– SVD can act as a shield to transparency