Received June 5, 2018, accepted June 27, 2018, date of publication July 5, 2018, date of current version August

7, 2018.
Digital Object Identifier 10.1109/ACCESS.2018.2853107

Recommendation to Groups of Users Using the

Singularities Concept
FERNANDO ORTEGA 1 , REMIGIO HURTADO 2,3 , JESÚS BOBADILLA 2,

AND RODOLFO BOJORQUE 2,3

1 U-tad:Centro Universitario de Tecnología y Arte Digital, 28290 Las Rozas, Madrid, Spain
2 Department of Information Systems, Universidad Politécnica de Madrid, 28031 Madrid, Spain
3 Department of Computer Science, Universidad Politécnica Salesiana, Cuenca 010102, Ecuador

Corresponding author: Fernando Ortega (fernando.ortega@u-tad.com)

ABSTRACT Recommendation to a group of users is a big challenge for collaborative filtering. The
recommendations to groups of users arise from the convenience of being able to recommend a group of users
about products or services that satisfy the entire group. In this paper, we propose the similarity measure
SMGU, tailored for collaborative filtering recommendations to groups of users. This similarity measure
combines both numerical and non-numerical information. Numerical information is weighted attending
to the rating singularity of the group members. This paper focuses on the assumption that the singularity
of the ratings cast by the users of the group is relevant information for finding suitable neighbors. For
each item, we consider that a rating is singular for a group or for a user when that rating is different
from the majority of the rating cast by the other users. Non-numerical structural information can be
considered as valuable to match group preferences with neighbors preferences. Experiments have been run
using open recommender systems data sets. Compared with representative baselines, results show accuracy
improvements when the proposed method is used. Additionally, this paper provides a section devoted to
the experiments reproducibility issue. Finally, this paper opens opportunities to face new challenges in the
recommendation to a group of users: explanation of recommendations, determination of reliability measures,
and improvement of accuracy, novelty, and diversity results.

INDEX TERMS Recommendation to groups, group of users, collaborative filtering, recommender systems,
singularity.

I. INTRODUCTION RS based on content information [3] carry out recom-

This section is divided into three subsections: 1) Fundamental
concepts of RS: recommendation to individual users, 2) Rec-
ommendation to groups of users: Objectives and particu-
larities, and 3) General explanation and motivation of the
proposed method for recommending to groups of users.
A. RECOMMENDATIONS TO INDIVIDUAL USERS
Recommender Systems (RS) [1], [2] allow to mitigate part of
the Internet information overload problem. From the point of
view of an RS user, based on his past preferences, the system
automatically recommends a series of items (movies, books,
music, electronics, clothing, etc.) that are available and that
the user has not consumed. The RS can make recommen-
dations based on various types of information sources; the
most common ones are: content-based, demographic, collab-
orative, social, and context-aware.
mendations in the following way: if the user to whom you
wish to recommend (active user) has liked a product or ser-
vice, the RS recommends similar products or services: e.g.
if the active user bought a historical novel, it is likely to be
recommended a history book or a historical novel book that
he has not bought or read. One of the biggest drawbacks
of the content-based approach is the lack of novelty of its
recommendations.
The RS based on demographic information [4] make
recommendations based on the products consumed by demo-
graphically similar users to the active user (age, genre, loca-
tion, etc.). The main drawbacks of the demographic-based RS
are: 1) Complete demographic data is not usually available,
and 2) There is too much variability in the preferences of each
demographic group.
The social-based RS [5], [6] make use of relationships
between users: likes, dislikes, follows, etc. Social-based RS

2169-3536
2018 IEEE. Translations and content mining are permitted for academic research only.
VOLUME 6, 2018 Personal use is also permitted, but republication/redistribution requires IEEE permission. 39745
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept
recommend to active users based on the preferences of their
closest social network. The main problem with this approach
is that most of the existing datasets do not contain enough
social information. Finally, context-aware RS [7], [8] are
usually associated with the Internet of Things (IoT), where
context information is collected: GPS coordinates, RFID
information, credit card data, etc.
Collaborative Filtering (CF) RS [1], [9] usually offer the
best recommendation results. Their operation is as follows:
the active user is recommended items that have not been con-
sumed and that have been positively rated by users who have
preferences similar to those of the active user. That is, infor-
mation is extracted from all existing users (hundreds of thou-
sands or millions) and based on that information, the active
user is recommended. Normally, information is structured
as a matrix that stores the preferences (explicit or implicit)
of each of the users about the set of items. These matrices
(efficiently saved in datasets) are enormously sparse because
a typical user has only been able to consume or rating a very
small subset of the set of available items (thousands or tens
of thousands).
There is a wide variety of approaches to extract the most
relevant information from the sparse collaborative filtering
matrices. The traditional approach was the KNN algorithm
(K Nearest Neighbors) [10], [11], where the most similar
K users (neighborhood) are searched for each active user;
subsequently, items not consumed by the active user that has
been highly valued or consumed by its neighborhood are rec-
ommended. The previous approach is classified as memory-
based [1], [9]: information to recommend is obtained directly
from the data. The explained process is called user-based; it
is also possible to carry out an item-based recommendation,
obtaining neighborhood sets of each item.
Currently, collaborative filtering RS are usually designed
by using the model-based [1], [9] approach: A model is
created from the data, and subsequently recommendations
are obtained from the model. The RS most used model-
based method is the Matrix Factorization (MF) [6], [12]:
The sparse ratings matrix is compressed into two dense factor
matrices (one matrix containing the users information and
another matrix containing the items information). One of the
matrices has users x factors size, and the other matrix has
items x factors size. The number of factors is usually small
(10 to 40), and then the size of each of the two matrices is
much smaller than the size of the original information (users
x items). This compressed information contains the essence
of the original information, coded in factors that are called
hidden because its meaning (the concept they encode) is not
known. The prediction and recommendation process from
this model usually improves the quality obtained through
memory-based approaches.
Finally, it is important to highlight two important concepts:
1) The more information a RS gathers, the better results it
can provide; that’s why hybrid RS [13] are usually designed
in commercial RS (typically: collaborative + content +
demographic), and 2) There are model-based RS different
39746
from the MF approach, although their use is not extended:
fuzzy approaches [14], evolutionary algorithms (ants, swarm,
etc.) [15], Bayesian methods [16], clustering [17], etc.
B. RECOMMENDATION TO GROUPS OF USERS
While traditional recommendations are individual (they are
made for each active user), this paper focuses on recommen-
dations to groups of users. The recommendations to groups of
users arise from the convenience of being able to recommend
a group of users about products or services that satisfy the
entire group. A classic example is the recommendation of
a movie to a family or a group of friends. Sometimes the
design of the recommendation to groups method requires
the use of an aggregation policy [18]. The most commonly
used policies [19], [20] are: a) The last misery policy, which
avoids recommendations that satisfy a majority, but are not
satisfactory for a minority, and b) The weighted aggregation
policy, where it is attempted to maximize overall satisfaction,
without taking into account variations in the satisfaction of
the group’s components.
Although the recommendation to groups of users may
seem like a simple generalization of the recommendation to
individual users, it is more complex: a) When a memory-
based approach is chosen, there are several possibilities to
approach the problem: You have to choose between making
an aggregation in the recommendation phase [21], [22], in the
prediction phase [23], [24] or in the neighbors obtainment
phase [39]. It is also possible to establish a virtual user that
represents the group [25], [26], b) If we opt for a model-
based approach, we find that most of the models used in
recommendation to individual users are not generalizable for
recommendation to groups of users. Researchers adapt the
existing models [27], [28], or they create new approaches
[29], [30], c) The data sparsity negative effect is greater when
there are more restrictions (recommendation to groups of
users) than when there are less restrictions (recommendation
to individual users), and d) Methods with which results are
measured are less universal in recommendations to groups of
users: recommendation policies, sizes and distributions of the
groups, etc.
Response times in the recommendation to groups of
users is more critical than in the individual recommenda-
tions because it is necessary to perform additional opera-
tions. When the memory-based approach is used, the sooner
the aggregation is made, the faster the results will be
obtained [25]. Model-based approaches usually generate
results very quickly; on the other hand, a great learning time
is required to generate the model.
Fig. 1 shows the different options available to implement
the recommendation to groups of users (when using the
memory-based approach). Gray boxes show the usual phases
of a recommendation made with the KNN algorithm. The
four options (a to d), which flow horizontally, represent the
strategies followed to make recommendations to groups of
users. Option a) Merged recommendations, uses the lists of
items recommended to each user of the group and generates
VOLUME 6, 2018
F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept
FIGURE 1. KNN-based approaches for recommending to groups of users. Figure taken from [25] has been extended.
a unified list. Option b) Merged predictions uses the most
convenient predictions lists for each user in the group and
generates a unified list of predictions (there are more predic-
tions than recommendations). Option c) Related neighbors,
starts from the neighborhood of each user of the group and
generates a neighborhood that represents the entire group.
Finally, option d) Related users, creates a virtual user that
represents all the group’s users: from the virtual user a tradi-
tional individual recommendation can be made.
C. PROPOSED APPROACH FOR RECOMMENDATION
TO GROUPS
In section III, the proposed method for making recommen-
dations to groups of users is explained in detail. In this
subsection, the main ideas are presented, and the decisions
taken are motivated. The first decision was to choose between
a model-based or a memory-based method. Our experience
covers both fields: [25] provides a memory-based approach,
while [27] provides a model-based approach. The model
developed in [27] does not admit significant improvements,
whereas the solution provided in [25] does have a promising
route to improve results, both from [25] and [27].
Published approaches usually implement aggregation of
neighborhoods, or predictions and recommendations merging
(options b, c and d in Fig. 1). The solution proposed in [25]
provides an innovative metric (UGSM) that allows you to find
the existing similarity between any user of the RS and the
group of users to which you want to recommend: option a)
in Fig. 1. The proposed method is simple and also achieves
significantly better execution times than methods based on
options b), c), d) in Fig. 1. We test its effectiveness by
comparing it with the UGSM itself, with the model-based
method from [27] and with several baselines of very recent
publication.
The fundamental idea on which the improvement of the
proposed method is based, with respect to UGSM, has been
VOLUME 6, 2018
taken from the concept used in [31], where the singularity
of the ratings is exploited: the ‘‘singular’’ ratings are granted
more important than the ‘‘usual’’ ratings; e.g. in a movie
that receives a huge amount of negative ratings (or absence
of ratings), a positive rating from a user is very relevant to
find her neighborhood. The singularity concept has generated
important accuracy improvements when it has been applied to
several RS open datasets.
An important principle supporting the proposed similarity
measure to recommend to groups of users is the generaliza-
tion of the singularity concept. The underlying idea is the
following: we know that two users are similar if they present
singular ratings in common; e.g.: both users usually listen to
an unpopular and poorly rated song. Using the same reason-
ing: if a group of users shares unusual ratings with a neighbor
candidate, it is very likely that she is a suitable neighbor
candidate. The condition of singularity is less likely to occur
in the recommendation to groups (e.g.: a group of friends who
like a very unpopular movie). This lower probability helps us
to choose neighbors that fit very well with the preferences of
the group. In summary: the concept of singularity fits well
with the particularities of the recommendation to groups of
users.
The rest of the paper is structured as follows: section II
summarizes the related work in the recommendation to group
of users’ sub-field. Section III defines and explains the pro-
posed method. Section IV exposes the design of experi-
ments, section V shows the results obtained and discuss them.
Section VI provides the conclusions of the paper and suggests
future works. Finally, we list references, many of them of very
recent publications.
II. RELATED WORK
The CF-based recommendation to groups field presents a
wide variety in the way it has been focused. Mainly it
can be classified as follows: a) KNN-based approaches,
39747
 F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept
and b) Model-based approaches. Each of the above
approaches admits a subdivision. In this section we will
review the most relevant bibliography in the field, group-
ing the referenced papers according to the solution they
implement.
A. KNN-BASED APPROACHES
The KNN-based approaches directly use the ratings matrix to
make recommendation to groups. Fig. 1 shows: a) its taxon-
omy, b) the stages involved in the process of recommendation
to groups, and c) the existing alternatives. Finally, we provide
the sections names in which publications are usually classi-
fied in this subfield.
Each of the grey areas in Fig. 1 represents a stage in the
CF RS process using KNN. Step1 (similarity metric) makes
use of similarity metrics to obtain the similarity between
each user belonging to the group w.r.t. the full set of users
(cases a to c), or to obtain the similarity between the entire
group of users w.r.t. the full set of users (case d). In step 2
(neighborhoods) the neighbors of each user from the group
are used to obtain a unified set of neighbors (case c). Step 3
(predictions) generates the prediction of the group of users
based on the individual predictions of each user (case b);
aggregation methods are used. Finally, step 4 (recommen-
dation) obtains the recommendation of the group of users
based on the recommendations made to each of its users
(case a).
Based on [25], [32] the set of steps shown in Fig. 1 clas-
sify the most commonly used methods of recommendation
to groups of users. From [20], steps 1 and 2 correspond
with the ‘‘Aggregated model strategy’’ [33], [34], whereas
steps 3 and 4 correspond with the ‘‘Aggregated prediction
strategy’’ [18]. Below is the most relevant KNN-based related
work:
1) MERGED PREDICTIONS
Methods to suggest music and movies are provided by [23].
They generate group recommendations using different
methods: merging recommendations made for individuals,
aggregation of individuals’ ratings, and construction of group
preference models. To investigate the use of aggregated group
data in CF recipe recommendations, [24] analyzes the impact
of switching strategies, data aggregation heuristics, and group
characteristics on the performance of recommendations.
A RS for tourism [35] faces the group recommendation
eliciting out of the individual personal recommendations
through aggregation and intersection. The Choicla group
decision support environment [36] advances the state of the
art by supporting decision scenarios in a domain-independent
fashion. Most of the recommendation to group of users’
research papers are designed merging predictions. To merge
social interaction predictions [37] enhance mutual awareness
among group members. Attribute reduction-based mining
method [38] has been proposed to efficiently select the long-
tail user groups.
39748
2) MERGED RECOMMENDATIONS
Polylens [21] uses an algorithm that merges users’ recom-
mendation lists and sorts the merged list according to the
principle of least misery. They realize this merge algorithm
is unlikely to work well for large groups. The effectiveness
of group recommendations is obtained in [22]; they aggre-
gate the individual lists of recommendations produced by a
collaborative filtering system, and finally, they compare the
effectiveness of individual and group recommendation lists
using normalized discounted cumulative gain.
3) RELATED NEIGHBORS
This is the least used approach in the scientific literature.
In [39] the set of neighbors of the group of users are obtained
by unifying the sets of neighbors of the individual users.
This work proposes the intersection of a large number (k) of
neighbors of each user of the group.
4) RELATED USERS
A group recommendation similarity metric is proposed
in [25]. They show that the system performance improves
notably when the aggregation is performed in an earlier
stage of the collaborative filtering process. CATS [26] is
a conversational collaborative group RS designed to help a
group of up to four users. They take into account both prefer-
ences: the individual’s and the group’s preferences to merge
users from a group. A common strategy is to first merge all
user profiles to construct a common user profile, and then
using a recommendation approach to generate a common
program recommendation list for the group according to
the merged user profile [40]. A useful recommendations to
groups of users application provides personalized multime-
dia content for group users by taking care of the majority’s
preferences [41]. This paper’s main contributions are the user
identification and profile aggregation, followed by the user
profile merging algorithm.
B. MODEL-BASED APPROACHES
The Model-based approaches make a model from the ratings
matrix. Later, using the model, they obtain the recommenda-
tion to groups. The main types of models that the related work
shows are: Matrix factorization, graph approaches, fuzzy
technologies and clustering preprocessing methods; other
models and results are also abstracted.
1) MATRIX FACTORIZATION
Some SVD-based group recommendation methods are pro-
posed in [28]; they aggregate ratings of group members. Their
methods are divided into two categories: SVD-based aggre-
gation profiles (related users) and aggregation predictions
methods (merged predictions). The purpose of paper [42]
is to facilitate academic group activities in big data-based
library systems by recommending satisfying articles for aca-
demic groups. The main contribution is its scalable par-
allel implementation. Different matrix factorization-based
VOLUME 6, 2018
F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept
recommendation to groups approaches are proposed in [27];
they explain three original methods to map the group of users
to the latent factor space, and they compare the proposed
methods in three different scenarios: when the group size is
small, medium and large. Social information is used in [43];
this work combines a matrix factorization model to estimate
unknown ratings with a social network analysis to evidence
possible social influence.
2) GRAPH APPROACHES
An original group recommendation approach addresses the
problem of generating and then recommending an artworks
sequence for a group of visitors within a museum [29]. They
define the museum artworks as a graph. The recommendation
system suggests a route that maximizes visitors’ satisfaction,
recommending to individual users and then aggregate the
recommendations in a single group recommendation (merged
recommendations). A random walk with restart method is
proposed in [44]; they represent the relationships among
users, groups, and items as a tripartite graph. Based on
the tripartite graph it is possible to predict the relevance
degrees between groups and unrated items by comprehen-
sively detecting their relationships. The method from [45]
models information with a heterogeneous graph and con-
siders the recommendation problem as a query-dependent
node proximity problem. They propose a general graph-based
model, called HeteRS, to recommend on Event-based social
networks.
3) FUZZY TECHNOLOGIES
A new method for group decision making using group recom-
mendations based on interval fuzzy preference relations and
consistency matrices is presented in [30]. Cheng et al. [46]
propose a new autocratic decision-making method using
group recommendations based on ranking interval type-2
fuzzy sets. A group decision making with multi-granular
hesitant fuzzy linguistic information is presented in [47];
authors propose a model-based decision-making method.
In [48] it is considered using fuzzy logic for mod-
eling student clusters. As the representation of each
group, they assume fuzzy numbers connected with learner
attributes.
4) CLUSTERING PREPROCESSING METHODS
Group recommender systems can be designed to
automatically detect groups of users by clustering them [49];
the number of cluster respect a constraint on the maxi-
mum number of recommendation lists that can be produced.
An extensive model for group recommendations [50] exploits
recommendations for items that similar users to the group
members liked in the past. They make a pre-partition of
users into clusters and they use the cluster members for
recommendations. The clustering process [51] usually make
recommendations for users produced with respect to the
preferences of their cluster members without extensively
searching for similar users in the whole user base.
VOLUME 6, 2018
5) OTHER MODELS AND METHODS
Using ontological concepts, Ben Ahmed et al. [52] pro-
vide a semantic multidimensional group recommendation.
Meta-learning techniques and aggregation strategies [53] are
used in order to select group recommendation strategies.
Using stochastic methods, [54] paper makes recommen-
dations based on link-structure analysis in a probabilis-
tic manner, and they provide group recommendations in
social media systems. Knowledge flow mining [55] has
been used as model to make group recommendations for
task-based groups. A Hybrid model, merging content-based,
collaborative filtering, and knowledge-based solution for
groups of users has been used in [56]. Social Networks
based models play an important role in recommendation
to groups of users when social information is included in
datasets, [57] propose a social-aware group recommendation
framework that jointly utilizes both social relationships and
social behaviors to infer a group’s preference, and also to
model the tolerance and altruism characteristics of group
members.
III. PROPOSED METHOD
This section explains the proposed method for recommenda-
tion to groups of users. In the first subsection, we explain
previous concepts which are directly related to the proposed
design. In the second subsection, we expose the general
principles on which our recommendation to groups similarity
measure is based. The third subsection details the mathe-
matical formalism and relates it to the explained design.
Finally, an illustrative running example based on a datatoy
is provided.
A. PREVIOUS CONCEPTS
The proposed similarity measure uses the singularity concept
[31] and the combination of non-numerical and numerical
information concept [58]. The singularity concept hypothesis
is that CF results can be improved by taking contextual infor-
mation, drawn from the entire body of users. The singularity
main idea lies in the fact that the contribution of an item to
the similarity assigned to two users ought not to be considered
as absolute (which is what happens with traditional metrics),
but rather as relative to the vote awarded to this item by
the rest of the users in the system. Following the singularity
reasoning expressed through to its extremes, if all of the
users who have voted for an item have cast the same vote,
it is difficult to consider this item as a factor of similarity
between two users. In the opposite case scenario, if only
two users have voted differently to the rest for one item, this
represents a very great singularity which should be translated
into a very great similarity for this item. Equations from [31]
define SPi as the singularity of the relevant votes concerning
the item i, and SNi as the singularity of the non-relevant
votes concerning the item i. Relevant votes in MovieLens
are 4,5, whereas non-relevant votes are 1,2,3. SPi and SNi are
39749
 F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept
FIGURE 2. Information sources used to design the proposed method.
defined as:
#{u ∈ U |ru,i ∈ {4, 5}}
SPi = 1 −
#U
i #{u ∈ U |ru,i ∈ {1, 2, 3}}
SN = 1 −
#U
Where U is the set of users and ru,i is the vote cast for user
u to the item i.
The combination of non-numerical and numerical informa-
tion concept was proposed in [58]. Authors present a metric
which combines the numerical information of the votes with
independent information from those values, based on the
proportions of the common and uncommon votes between
each pair of users. The numerical information is processed
using the Mean Squared Differences (MSD) similarity mea-
sure, whereas Jaccard is used to deal with the non-numerical
information. The resulting metric JMSD is simple, it provides
good accuracy and performance, and it has been highly cited:
JMSD(u,v) = Jaccard(u,v) * (1-MSD(u,v)).
B. METHOD DESIGN
Starting from the concepts shown in the previous section,
we will show the constitutive elements of the proposed simi-
larity measure for groups of users. We are going to make use
of Fig. 2; our metric is constituted from three fundamental
elements: a) Numerical information (ratings), contemplated
in element 5 of Fig. 2, b) Non-numerical (structural) informa-
tion, contemplated in element 1 of Fig. 2, and c) Weightings
of the importance of the numerical information, contemplated
in elements 2, 3 and 4 of Fig. 2. We will call SMGU to the
proposed similarity measure (Singularity-based Measure for
Groups of Users).
1) Proportion of matching items: Jaccard measure pro-
vides the existing proportion of votes cast by two users:
the active user and each neighbor candidate. JMSD sim-
ilarity measure [58] shows a significant improvement
39750
FIGURE 3. Proportion of matching items and item unpopularity.
in the accuracy obtained when Jaccard is applied to
the KNN-based CF RS. Our proposed SMGU similarity
measure extends the JMSD Jaccard behavior in two
aspects:
a) Jaccard extension to groups of users: Jaccard,
applied to two users, can be defined as the ratio
between the number of common cast ratings and
the total number of cast ratings. The number of
common cast ratings can be obtained using the set
intersection operation, whereas the total number
of cast ratings can be obtained using the set union
operation. When a group of users is involved,
union and intersection operations can be made as
shown in the top of Fig. 3: each neighbor candi-
date rating is compared with the corresponding
group rating.
b) Incorporation of the item unpopularity: SGMU
incorporates the concept ‘‘singularity of items’’
[31], expressed as the absence of popularity of an
item for the users. The idea behind this concept is
that the most singular items provide better infor-
mation to determine the similarity between two
users. By way of example, if two users agree on
a film that has not been widely voted and known,
it is very likely that they will have similar tastes
among themselves, and different to the tastes of
other users. If two users agree on a well-known
movie, such as ‘‘Avatar’’, this information will
not be enough to match both users, since there
will be a large number of users who are also
paired based on their assessment of that popular
item. Following this reasoning, item 1 from the
bottom of Fig. 3 is more unpopular (less voted)
than item 2 because item 1 contains less ratings
than item 2. Our proposed similarity measure will
VOLUME 6, 2018
F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept

In [58], Mean Squared Differences (MSD) is set as a

suitable similarity measure in this context because the
differences of votes between users are highly penalized.
Our SMGU proposed similarity measure for groups of
users will also use MSD.
FIGURE 4. Users and groups singularity.
C. METHOD FORMALIZATION
Next, equations that define the proposed SMGU similarity
give more importance to the KNN neighbors that measure are established. This section is structured, numeri-
match, in item 1, with the group of users. cally, according to the values shown in Fig. 2. Finally, equa-
2) Users and neighbors singularity: this parameter gives tions are unified, showing the SMGU final result.
greater importance to the less popular ratings in each 1) Proportion of matching items
item. We define the singularity of a rating r for an Let’s define pi as the lack of importance of the item i
item i as the ratio between ratings different of r and based on its ratings. Here, the parameter meaning is:
the totality of the ratings issued in i. As an example, ‘‘unpopularity of the item’’.
in Fig. 4, for item 3, the singularity of rating 3 (vote=3)
is much greater than the singularity of rating 5; in Fig. 4 #{u ∈ U |ru,i6=• }
pi = 1 − (1)
item 2, the singularity of rating 5 is greater than the #U
singularity of rating 2. To understand the underlying where • means not voted, U is the set of users and ru,i
idea, we can regard to an item that has received many is the rating of user u to item i.
positive votes (e.g. ratings 4 or 5 in ‘‘Star Wars’’): Let’s define xG,u as the similarity of the group G and
two users casting ‘‘singular’’ (negative) rating to this the user u based on the non-numerical information of
item will be more related among them than two users the ratings:
casting ‘‘not singular’’ (positive) ratings. In the same P
way, an item that has been mostly poorly rated will i∈{I ∩I } pi
xG,u = P G u (2)
provide us with more relevant information of similarity i∈{IG ∪Iu } pi
between two users who have voted it positively. Finally,
where IG is the set of ratings voted for at least one user
the singularity of the rating cast by the user u to the
of the group G and Iu is the set of ratings voted by the
item i is defined as the ratio between ratings to the item i
user u.
different to ru,i and the totality of ratings issued in i:
2) Users and neighbors singularity:
e.g. From Fig. 4, user 10 has a big singularity in item 3,
Let’s define su,i as the singularity of the user u rating
whereas users 0, 2, 5 and 8 have a little singularity in
on the item i (ru,i ). This value indicates the singularity
item 3. From the singularity aspect, regarding to item 3
of the user’s vote for item i compared to the votes cast,
from Fig. 4, users u3 and u9 are more similar than users
in item i, by all other users.
u0 and u2 .
3) Number of the users in a group voting item i: the pro- #{v ∈ U |rv,i 6= ru,i ∧ rv,i 6= •}
su,i = , ∀ ru,i 6 = •
posed similarity measure modulates results according #{v ∈ U |rv,i6=• }
to the available information quantity; e.g.: according (3)
to Fig. 4 item 2, results from group <u0 , u1 , u2 , u3 >
will have less importance than results from group 3) Number of the users in a group voting item i:
<u5 , u8 , u9 >, since results from the first group are taken Let’s define Gi as the subset of users of the group G
using a single rating, whereas results from the second that have rated item i.
group are taken using three ratings.
4) Group singularity: the singularity of the ratings cast Gi = {u ∈ G|ru,i 6= •} (4)
for users belonging to a group can be considered as an 4) Group singularity:
extension of the user’s singularity. From Fig. 4, if we Let’s define sG,i as the singularity of the group G
consider <u0 , u2 , u3 , u5 > as a group of users, regarding ratings on the item i. This singularity is computed with
to item 3 their ratings are not too much singular, since the geometric mean of the users’ singularities in the
they have voted the usual value (5). If we consider group.
<u6 , u10 > as a group of users, their ratings are singular sY
regarding to item 3. sG,i = #Gi su,i , ∀ su,i 6= • (5)
5) Numerical differences: as in the KNN process for indi-
u∈Gi
vidual users, to determine the set of neighbors for
a group of users it is necessary to make use of the 5) Numerical differences:
ratings numerical information. We must choose KNN Let’s define 1G,u,i as the squared difference of
neighbors that have similar tastes to those of the group. the ratings of the group G users, for item i, with

VOLUME 6, 2018 39751

 F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept
respect to the rating of the neighbor user u, for
item i.
P 2
g∈Gi (rg,i − ru,i )
1G,u,i = , ∀ ru,i 6 = • (6)
#Gi
While step 1 provides the non-numerical (structural) infor-
mation, combining steps 2 to 5 into a single equation pro-
vides the numerical information. Let’s define yG,u as the
similarity between group G and user u, based on the numer-
ical information of the ratings. What we do is to use Mean
Squared Differences, weighting its values with three param-
eters: a) candidate to neighbor singularity for each item, b)
group singularity for each item, and c) number of votes in the
group for each item:
i∈{IG ∩Iu } su,i sG,i #Gi 1G,u,i
P
yG,u = 1 − P (7)
i∈{IG ∩Iu } su,i sG,i #Gi
Finally, the proposed SMGU similarity measure combines
non-numerical information from (2) and numerical informa-
tion from (7). α ∈ [0, 1] is a parameter that swings the impor-
tance of non-numerical information of ratings vs. numerical
information.
α
SMGU (G, u) = xG,u y1−α
G,u (8)
D. RUNNING EXAMPLE
In this section, we show a running example based on a data-
toy. From this example we will show, step by step, the opera-
tion of the proposed similarity measure for recommendation
to groups of users. Also, as ‘‘additional material’’ to the paper,
we provide a completely programmed spreadsheet containing
this same running example. In this way, we make it possi-
ble for anyone to modify the ratings matrix and check its
effect both for the final and for the partial results. Likewise,
researchers can modify the α parameter to calibrate its impact
on the similarity measure.
On top of Table 1 we show the ratings matrix that we have
designed. Users <u1 , u2 , u3 > will be the group G of users to
recommend. To simplify the analysis, we have used only two
possible ratings values: zero and one. By observing ratings
carefully, we can verify that users u6 and u8 are those that
contain more similar votes to those of group G, and therefore,
a priori, they are candidates to belong to the G neighborhood.
On the other hand, users u5 and u7 contain the votes that are
most different from those of group G: their similarity with the
group is expected to be low.
Table 1, ‘‘Proportion of matching items’’ shows the
‘‘unpopularity’’ pi of each item: item i3 has only received
one vote; It is the most unpopular item. Items i2 and i4 have
received eight votes and they are the less unpopular items.
Using the unpopularity values, xG,u performs the Jaccard
function between the group of users G and each neighbor
candidate u. Section 2 shows su,i : the singularity of the user’s
vote for item i compared to the votes cast, in item i, by all
other users. We have marked, in bold style, the most singular
ratings; the most representative rating is the vote of user u5
39752
for item i2 : it is the only rating ‘‘zero’’ of the nine votes
cast in this item. The opposite situation can be observed in
item i11 : any of its three ‘‘one’’ votes gets a null singularity,
due to the absence of diversity in the votes. Section 3 in
Table 1 shows the group G singularity sG,i . This singularity is
computed using the geometric mean of the users’ singularities
in the group; in our running example, the highest sG,i value
corresponds to the highest singularity value su,i from the
users of the group: s2,1 = 0.60. Section 4 in Table 1 just
shows the number of users in G who have voted item i; e.g.:
i12 = 2, since u1 and u3 have voted it. We can see that,
in this example, item i7 is particularly relevant to the SMGU
proposed similarity measure, since it has high values of su,i ,
sG,i and #Gi .
Section 5 shows results of the mean squared differences
computed according to the equation 1G,u,i . As expected,
users u6 and u8 provide the smallest error values, whereas
users u5 and u7 report the biggest errors. Combining error
values with singularity parameters we obtain the numeri-
cal factor yG,u . Finally, combining yG,u with the structural
information xG,u , and applying the α factor we obtain the
SMGU similarity measure final results; in all cases: structural,
numerical and final results, the expected neighbors u6 and u8
have been chosen, whereas u5 and u7 have been dismissed.
IV. EXPERIMENTS DESIGN
This section shows the experiments design and the obtained
results. Experiments are focused on: a) finding the opti-
mal parameters for the proposed method, and b) compar-
ing the proposed method with the state of the art ones.
Sub-section IV-A describes the environment designed to per-
form all the experiments; sub-section IV-B defines de quality
measures used to control the goodness of the recommenda-
tions; finally, sub-section IV-C & IV-D explain the experi-
ments carried out to test the proposed model.
A. EXPERIMENTS SET UP
The experiments have been divided into two phases: (1) find-
ing the optimal parameters of the proposed methods to each
tested dataset, (2) comparing the proposed method with state
of the art, to make CF recommendations to group of users. All
the experiments have been carried out using public datasets
widely used by RS research papers. We have selected Movie-
Lens [59], BookCrossing [60] and FilmTrust [61] datasets.
Table 2 contains the most relevant properties about these
datasets. Experiments have made use of the Original Software
Publication (OSP) framework [62].
Tested datasets do not include any social information to
test the proposed method using real groups of users, so we
have generated synthetic groups. Users have been grouped
randomly, but those users which have more training ratings
in common have a higher probability to be grouped together.
To perform the experiments, we have split users and items
into test and training sets. To avoid fluctuations due to the
random selection of training users, test items and test groups,
we perform each experiment using 10-folds Monte Carlo
VOLUME 6, 2018
F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept

TABLE 1. Running example from a datatoy.

#TPλG
cross-validation. Table 3 contains the main parameters of recallGλ = (10)
each dataset, chosen for the experiments execution. #TGλ

B. QUALITY MEASURES Where TP, FP and T denote the true positive, false positive
In order to evaluate the quality of recommendations to groups and expected recommendations sets, respectively:
of users, we define precision and recall for the group G as:
FPλG = i ∈ LG |aG,i < λ

(11)
#TPλG TPλG = i ∈ LG |aG,i ≥ λ

(12)
precisionλG = (9)
TPλG ∪ FPλG TGλ = i ∈ I |aG,i ≥ λ


# (13)

VOLUME 6, 2018 39753

 F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept

TABLE 2. Main properties of the datasets used in the experiments. TABLE 4. Tested model parameters.

TABLE 5. Configuration of the experiment to optimize the proposed

method parameters.
TABLE 3. Main parameters for testing each dataset.

Where aG,i is the proportion of users belonging to the

group G that liked item i, LG is the set of items recommended
to the group G, and λ is a parameter that controls the flexibil-
ity of the fail recommendations to one or more members of
users that belongs to the group. Finally, we will denote R−MAE as the averaged R−MAE
to each tested group of users.
# g ∈ G|r̂g,i ≥ θ

λ
P
aG =  (14) R−MAEG
R−MAE = G (20)

# g ∈ G|r̂g,i 6 = •
#G
Where r̂u,i is the test rating of the user u to the item i, and C. MODEL PARAMETERS OPTIMIZATION
θ is a threshold to consider a rating as like or dislike; e.g.: for The proposed similarity metric contains two parameters: α,
a range of ratings 1, . . . , 5, a reasonable θ value could be 4 that controls the importance of non-numerical rating informa-
(4, 5: liked; 1, 2, 3: did not like). tion against numerical rating information, and k, that fixes the
Finally, we will denote precision@λ and recall@λ as the KNN number of neighbors for the group. A proper adjustment
averaged precision and recall to each tested group of users. of these parameters, for each dataset, is required in order to
λ maximize the quality of recommendations.
P
G precisionG
precision@λ = (15) In this experiment, we will evaluate the proposed method
P #G λ for different combinations of both α and k parameters. Table 4
G recallG contains the tested values for each parameter. To select the
recall@λ = (16)
#G optimal configuration of these parameters we will: a) measure
To evaluate the quality of predictions we define R−MAE the rm ae, precision and recall, b) recommend 10 items for
as the mean absolute prediction error of the recommended each parameters configuration and, c) select the best one.
items. Let L̂uG the subset of items recommended to the group Table 5 contains the remaining parameters required in this
G that has been rated (in test) by user u: experiment.

L̂uG = i ∈ LG |r̂u,i 6 = •

(17) D. MODEL PERFORMANCE
To measure the performance of the proposed model we
We define R−MAEuG as the prediction error for the user u will compare it with state of the art group recommenda-
w.r.t. the item recommended to the group G: tion methods. The baselines selected for this comparison
are UGSM [25], MFGU [27], C&P [32], RWR − M [20]
P  
i∈L̂uG r̂u,i − pG,i
 
G and RANK [22]. Some of these recommendations methods
R−MAEu = (18)
#L̂uG require different parameters to work. We have configured that
parameters in order to maximize the quality of the recommen-
We define R−MAE as the averaged error of the predictions
dation in each dataset. Table 6 contains the parameters of each
performed to the group G:
model in each experiment.
R−MAEuG We will evaluate these methods using the previously
P
R−MAEG = u∈G (19) defined quality measures: precision, recall and R−MAE.
#G
39754 VOLUME 6, 2018
F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept
TABLE 6. Parameters used in the experiments to run baseline methods.
FIGURE 5. α parameter and number of neighbors impact on the quality results obtained by using the proposed method.
The experiment has been carried out for two group sizes:
a) from 2 to 5 users, to test the performance of each recom-
mendation method on small groups of users (families, cou-
ples, small group of friends, etc.), and b) from 6 to 10 users,
to test the performance of each recommendation method over
large group of users (teams, bands, large group of friends,
etc.). Furthermore, we have tested the experiment for mul-
tiple λ values (1.0, 0.75 and 0.5) in order to measure the
overall satisfaction of the group within the recommendations
performed. Each measure has been evaluated for different
numbers of recommendations (from 5 to 20).
V. RESULTS
This section is divided into three sub-sections: 1) Explains
the experiments carried out to adjust the parameters involved
in the proposed method, 2) Shows the prediction and
VOLUME 6, 2018
recommendation results obtained using the proposed method,
then compares them with those offered by processing the
baselines, and 3) Discusses the observed main strengths and
drawbacks.
A. PARAMETERS SETTING
Before testing the proposed method it is necessary to deter-
mine the alpha value that we will use in equation (8). That
is, we must choose an α value that weights the importance of
the structural (non-numerical) information of the votes with
respect to the numerical information. The lower the value of
α, the greater weight we give to the numerical information
and the less weight we give to the non-numerical information.
Fig. 5 implements Table 4 and summarizes the combined
impact that the α values and the numbers of neighbors pro-
duce on each selected quality measure and for each of the
39755
 F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept
considered datasets. In all the graphs the best results are
coded using the yellow color (low values in R_MAE and
high values in Precision and Recall). The main conclusions
from the results shown in Fig. 5 are: 1) The optimal α and
number of neighbors values depend, to a large extent, on the
nature of the data; that is: the dataset we are using, 2) The
structural (non-numerical) information helps to improve the
quality results: when α is zero the quality results get worse,
and 3) Generally, it is appropriate to assign low α values: we
should give more importance to the numerical information
than to the non-numerical information.
The values of α and number of neighbors exposed
in Table 5 have been selected based on the results shown
in Fig. 5. It is interesting to observe, as expected, how the
size of the dataset influences the appropriate value of the
number of neighbors parameter; e.g.: in the largest dataset
(MovieLens) the impact is very small, whereas in the smallest
dataset (FilmTrust) there is a big difference in the quality of
the results obtained by applying different neighborhood sizes.
It is also very interesting to observe, in FilmTrust, the fol-
lowing behavior: there is a tendency to obtain the best quality
results when establishing a direct correlation between α and
number of neighbors. That is, as we increase the size of
the neighborhood, we must also increase the α value. This
means that the best neighbors (the first ones, the most similar
ones) are best selected using numerical information: low α.
However, the more ‘‘forced’’ neighbors (the last ones, the less
similar ones) are better selected using structural information:
high α. Our proposed method can compensate this effect
by raising or by decreasing the value of α. The R_MAE
and Precision graphs, in MovieLens, also show this trend,
although in a less marked way due to the much larger size
of this dataset: It would be necessary to use a greater number
of neighbors to appreciate this trend with greater intensity.
B. COMPARATIVE RESULTS
First of all, we will show the prediction quality results
making use of the quality measure R_MAE ‘‘Mean abso-
lute prediction error of the recommended items’’. Fig. 6
shows the obtained results for the three tested datasets:
MovieLens, FilmTrust and BookCrossing. The used baselines
are: UGSM [25], MFGU [27] and C&P [32]. Baselines
RWR−M [20] and RANK [22] are not designed to predict; we
will use them to test the recommendation quality measures
Prediction and Recall. Experiments parameters are shown
in Table 6. Experiments have been carried out for different
group sizes: left side correspond with small groups of users
(groups containing from two to five users) and right side
correspond with large groups of users (groups containing
from six to ten users).
The most relevant conclusions obtained from Fig. 6 are:
1) The quality offered by the baseline similarity measures
vary depending on the dataset used and the number of users
of the groups: in some cases, UGSM exceeds MFGU , while
on other occasions the opposite occurs. C&P tends to be
the baseline that provides the worst results. 2) The proposed
39756
similarity measure SMGU achieves the best prediction results
in all cases. SMGU and MFGU behave very well when
applied to large datasets. 3) In general, the quality of the
prediction increases as the size of the group decreases: this
is the expected result, due to the greater diversity of the users
that make up the larger groups.
Once the superiority of the proposed method, in predic-
tion quality, has been verified, we carry out the experiments
that measure the recommendation quality; for this, we test
precision@λ and recall@λ, and we establish the following λ
values: 0.5, 0.75, 1.0. As an example, the value 0.75 means
that there is a proportion of 75% of users, belonging to each
group, that like the recommended items. The recommenda-
tion quality measures have been tested using all the consid-
ered baselines for groups of users: UGSM [25], MFGU [27],
C&P [32], RWR − M [20] and RANK [22]. Experiments
parameters are shown in Table 6. Fig. 7 shows the recom-
mendation quality results. Each function in Fig. 7 is obtained
through the results of ten recommendations. Experiment has
been performed for small groups from 2 to 5 users (left side
of the figure) and large groups from 6 to 10 users (right side
of the figure).
Fig. 7 shows the proposed similarity measure superiority
for the majority of the Prediction quality results (the higher
the value, the better the result). Specifically, in the larger
dataset: MovieLens, the improvement of results offered by
SMGU is clearly visible. MFGU is the baseline that, in large
datasets, best competes with the proposed group-based simi-
larity measure; this is because, being a model-based method,
its design is better suited to large volumes of data. The pro-
posed similarity measure does not always get the best results
from Recall; specifically, in the MovieLens dataset, the Rank
baseline method improves to SMGU . As expected, the small
groups results tend to be better than the large groups results,
due to the greater variation of users in large groups.
The set of experiments performed for each dataset, for each
baseline and for the proposed similarity measure have been
replicated in each of the quality measures: precision@0.5,
recall@0.5, precision@1.0 and recall@1.0. The qualita-
tive results are similar to those explained in this section
(precision@0.75 and recall@0.75), so the resulting fig-
ures are not included. These figures are stored online,
in the journal site, as ‘‘Additional Material’’. The quantitative
results show the expected behavior: the more demanding the
requested quality is (i.e.: the higher the λ value), the lower the
precision and recall results obtained.
The proposed SMGU mostly improves the results of
the best baseline MFGU : R_MAE, Precision and Recall.
However, the Recall of SMGU is not always the most appro-
priate (Fig. 7). In short, our proposed method obtains bet-
ter Precision than Recall. The question that arises is the
importance and meaning of each of these recommendation
quality measures. Simplifying, we can associate Precision
with an absolute quality value, and Recall with a relative
quality value: the first one measures the success with respect
to a fixed number of recommendations, while the second
VOLUME 6, 2018
F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept

FIGURE 6. Mean absolute prediction error of the recommended items in MovieLens, FilmTrust and BookCrossing datasets. Tested for small groups (left
side) and large groups (right side). Proposed method: SMGU . Baselines: UGSM [25], MFGU [27] and C &P [32]. Experiments parameters are shown
in Table 6. The lower the value, the better the result.

one measures the success with respect to the number of
cases in which it is appropriate to recommend. Precision
focuses on quantities (accuracy, effectiveness) while Recall
focuses on qualities (it rewards complicated hits). In short,
we can determine that the proposed SMGU method and
the main MFGU baseline are accurate (a large number
of recommendations are correct), although they are not
always the ones that provide the most complex recommen-
dations; e.g.: by using MovieLens, Rank gets good Recall
results.
Fig. 7 functions show Precision and Recall results when
the number of recommendations range from two (left side of
each function) to ten (right side of each function). We can
observe that, for the MovieLens dataset, SMGU provides the
best Recall results when the number of recommendations
is low. When the number of recommendations is not low,
Rank provides better Recall results. MGFU does not improve
our proposed similarity measure neither in Precision nor in
Recall.
C. DISCUSSION
The SMGU proposed similarity measure to recommend to
groups of users has important advantages, but also some
drawback that should be analyzed and given the importance
it deserves. The main advantage of SMGU is its superiority
in prediction and also in recommendation, measured, respec-
tively, with the R_MAE and Precision quality measures.
Additional advantages of SMGU are: a) It maintains the best

VOLUME 6, 2018 39757

 F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept

FIGURE 7. Recommendation results: Precision and Recall with a proportion of 75% of users belonging to the group that liked the recommended items.
MovieLens, FilmTrust and BookCrossing datasets. Number of recommendations: 10. Tested for small groups (left side) and large groups (right side).
Proposed method: SMGU . Baselines: UGSM [25], MFGU [27], C &P [32], RWR − M [20] and RANK [22]. Experiments parameters are shown in Table 6. The
higher the value, the better the result (both precision & recall).

results for different sizes of the groups, b) It improves the
baselines results when applied to datasets with different sizes
and diverse sparsities, and c) It recommends adequately both
when a small number of recommendations is required and
when a larger number is needed.
An interesting feature of SMGU is its ability to weight the
importance of the numerical information of ratings with the
non-numerical information. This weighting mechanism also
serves to adapt to the number of neighbors selected in the
CF recommendation process and to maintain high levels of
accuracy. The concept of singularity is used intensively in
SMGU . This concept has proven to provide good results in the
recommendation to individual users. In this paper, it is shown
that the concept of singularity fits well with the particularities
of the recommendation to groups of users.
SMGU is a memory-based similarity measure. Most of
the published papers offer memory-based solutions to the
recommendations to groups of users, but the MFGU method
is model-based. The memory-based versus model-based dis-
cussion in the recommendation to groups of users is the same
as in the recommendation to individual users: each approach
has its advantages and disadvantages. However, while in rec-
ommendation to individual users the model-based methods
usually offer better accuracy, in our case the memory-based
SMGU similarity measure improves the results of the MFGU
method.

39758 VOLUME 6, 2018

F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept
VI. CONCLUSIONS
Recommendation to groups of users is a challenging research
open area. It is important to find suitable information sources,
coming from ratings, in order to improve accuracy. This paper
focuses on the assumption that the singularity of the ratings
cast by the users of the group is a relevant information for
finding suitable neighbors. Results confirm this hipothesis:
accuracy has been improved both for prediction results and
for recommendation results.
The proposed similarity measure to recommend to groups
of users combine structural information with singularities-
weighted numerical information. Singularities weights
improve accuracy just using the regular numeric ratings
values. For each processed item, we provide three different
parameters to weight numerical information: 1) Singularity
of the group ratings, 2) Singularity of each neighbor rating,
and 3) Number of users in the group rating the item. Analo-
gously, non-numerical information is weighted attending to
the popularity of items.
Results show a large accuracy improvement of the pro-
posed method compared to the KNN baselines, both for
prediction and for recommendation. Compared to the matrix
factorization-based baseline, the proposed method shows
more adjusted improvements. Therefore, our method is pre-
sented as the currently most appropriate to meet diverse
objectives and restrictions: a) KNN-based recommender sys-
tems for groups of users, b) Systems providing continuously
updated recommendations, c) Explanation of recommenda-
tions to the group due to the superiority of KNN approaches
to explain recommendations against model based ones.
As future works we propose: a) To improve the proposed
similarity measure by incorporating new elements of simi-
larity processing between groups and neighbors candidates.
b) Explaining recommendations to the group according to
both the similarity values and the singularity weights, c) To
study beyond accuracy quality results (such as: novelty, diver-
sity or reliability) for each member of the group, and d) To
incorporate singularity weights into the matrix factorization
process to improve matrix factorization-based recommenda-
tion to groups of users.
REFERENCES
[1] J. Bobadilla, F. Ortega, A. Hernando, and A. Gutiérrez, ‘‘Recommender
systems survey,’’ Knowl.-Based Syst., vol. 46, pp. 109–132, Jul. 2013.
[2] J. Lu, D. Wu, M. Mao, W. Wang, and G. Zhang, ‘‘Recommender sys-
tem application developments: A survey,’’ Decis. Support Syst., vol. 74,
pp. 12–32, Jun. 2015.
[3] M.-L. Wu, C.-H. Chang, and R.-Z. Liu, ‘‘Integrating content-based fil-
tering with collaborative filtering using co-clustering with augmented
matrices,’’ Expert Syst. Appl., vol. 41, no. 6, pp. 2754–2761, 2014.
[4] M. Y. H. Al-Shamri, ‘‘User profiling approaches for demographic recom-
mender systems,’’ Knowl.-Based Syst., vol. 100, pp. 175–187, May 2016.
[5] J. Tang, X. Hu, and H. Liu, ‘‘Social recommendation: A review,’’ Social
Netw. Anal. Mining, vol. 3, no. 4, pp. 1113–1133, 2013.
[6] J. Yu, M. Gao, W. Rong, Y. Song, and Q. Xiong, ‘‘A social recommender
based on factorization and distance metric learning,’’ IEEE Access, vol. 5,
pp. 21557–21566, 2017.
[7] N. M. Villegas, C. Sánchez, J. Díaz-Cely, and G. Tamura, ‘‘Characteriz-
ing context-aware recommender systems: A systematic literature review,’’
Knowl.-Based Syst., vol. 140, pp. 173–200, Jan. 2018.
VOLUME 6, 2018
[8] Z. Yang, B. Wu, K. Zheng, X. Wang, and L. Lei, ‘‘A survey of collaborative
filtering-based recommender systems for mobile Internet applications,’’
IEEE Access, vol. 4, pp. 3273–3287, 2016.
[9] Y. Shi, M. Larson, and A. Hanjalic, ‘‘Collaborative filtering beyond the
user-item matrix: A survey of the state of the art and future challenges,’’
ACM Comput. Surv., vol. 47, pp. 3:1–3:45, May 2014.
[10] J. Bobadilla, F. Ortega, A. Hernando, and G. G. de Rivera, ‘‘A similarity
metric designed to speed up, using hardware, the recommender systems
k-nearest neighbors algorithm,’’ Knowl.-Based Syst., vol. 51, pp. 27–34,
Oct. 2013.
[11] J. Bobadilla, F. Ortega, A. Hernando, and Á. Arroyo, ‘‘A balanced memory-
based collaborative filtering similarity measure,’’ Int. J. Intell. Syst.,
vol. 27, no. 10, pp. 939–946, 2012.
[12] X. Guan, C. T. Li, and Y. Guan, ‘‘Matrix factorization with rating com-
pletion: An enhanced SVD model for collaborative filtering recommender
systems,’’ IEEE Access, vol. 5, pp. 27668–27678, 2017.
[13] T. K. Paradarami, N. D. Bastian, and J. L. Wightman, ‘‘A hybrid rec-
ommender system using artificial neural networks,’’ Expert Syst. Appl.,
vol. 83, pp. 300–313, 2017.
[14] S. P. Perumal, K. Arputharaj, and G. Sannasi, ‘‘Fuzzy family tree similarity
based effective e-learning recommender system,’’ in Proc. 8th Int. Conf.
Adv. Comput. (ICoAC), Jan. 2017, pp. 146–150.
[15] T. Horváth and A. C. P. L. F. de Carvalho, ‘‘Evolutionary computing in
recommender systems: A review of recent research,’’ Natural Comput.,
vol. 16, pp. 441–462, Sep. 2017.
[16] A. Hernando, J. Bobadilla, and F. Ortega, ‘‘A non negative matrix factoriza-
tion for collaborative filtering recommender systems based on a Bayesian
probabilistic model,’’ Knowl.-Based Syst., vol. 97, pp. 188–202, Apr. 2016.
[17] J. Bobadilla, R. Bojorque, A. H. Esteban, and R. Hurtado, ‘‘Recommender
systems clustering using Bayesian non negative matrix factorization,’’
IEEE Access, vol. 6, pp. 3549–3564, 2018.
[18] J. Masthoff, Group Modeling: Selecting a Sequence of Television Items
to Suit a Group of Viewers. Dordrecht, The Netherlands: Springer, 2004,
pp. 93–141.
[19] J. Masthoff, Group Recommender Systems: Combining Individual Models.
Boston, MA, USA: Springer, 2011, pp. 677–702.
[20] S. Feng and J. Cao, ‘‘Improving group recommendations via detecting
comprehensive correlative information,’’ Multimedia Tools Appl., vol. 76,
pp. 1355–1377, Jan. 2017.
[21] M. O’Connor, D. Cosley, J. A. Konstan, and J. Riedl, PolyLens: A Recom-
mender System for Groups of Users, Dordrecht, The Netherlands: Springer,
2001, pp. 199–218.
[22] L. Baltrunas, T. Makcinskas, and F. Ricci, ‘‘Group recommendations with
rank aggregation and collaborative filtering,’’ in Proc. 4th ACM Conf.
Recommender Syst. (RecSys), New York, NY, USA, 2010, pp. 119–126.
[23] I. A. Christensen and S. Schiaffino, ‘‘Entertainment recommender systems
for group of users,’’ Expert Syst. Appl., vol. 38, no. 11, pp. 14127–14135,
2011.
[24] S. Berkovsky and J. Freyne, ‘‘Group-based recipe recommendations: Anal-
ysis of data aggregation strategies,’’ in Proc. 4th ACM Conf. Recommender
Syst. (RecSys), New York, NY, USA, 2010, pp. 111–118.
[25] F. Ortega, J. Bobadilla, A. Hernando, and A. Gutiérrez, ‘‘Incorporating
group recommendations to recommender systems: Alternatives and per-
formance,’’ Inf. Process. Manage., vol. 49, no. 4, pp. 895–901, 2013.
[26] K. McCarthy, M. Salamó, L. Coyle, L. McGinty, B. Smyth, and P. Nixon,
‘‘CATS: A synchronous approach to collaborative group recommenda-
tion,’’ in Proc. Florida Artif. Intell. Res. Soc. Conf. (FLAIRS), 2006,
pp. 86–91.
[27] F. Ortega, A. Hernando, J. Bobadilla, and J. H. Kang, ‘‘Recommending
items to group of users using matrix factorization based collaborative
filtering,’’ Inf. Sci., vol. 345, pp. 313–324, Jun. 2016.
[28] X. Hu, X. Meng, and L. Wang, ‘‘Svd-based group recommendation
approaches: An experimental study of moviepilot,’’ in Proc. 2nd Challenge
Context-Aware Movie Recommendation (CAMRa), New York, NY, USA,
2011, pp. 23–28.
[29] S. Rossi, F. Barile, C. Galdi, and L. Russo, ‘‘Recommendation in museums:
Paths, sequences, and group satisfaction maximization,’’ Multimedia Tools
Appl., vol. 76, pp. 26031–26055, Dec. 2017.
[30] S.-M. Chen, S.-H. Cheng, and T.-E. Lin, ‘‘Group decision making systems
using group recommendations based on interval fuzzy preference relations
and consistency matrices,’’ Inf. Sci., vol. 298, pp. 555–567, Mar. 2015.
[31] J. Bobadilla, F. Ortega, and A. Hernando, ‘‘A collaborative filtering simi-
larity measure based on singularities,’’ Inf. Process. Manage., vol. 48, no. 2,
pp. 204–217, 2012.
39759
 F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept
[32] L. Boratto, S. Carta, and G. Fenu, ‘‘Investigating the role of the rating
prediction task in granularity-based group recommender systems and big
data scenarios,’’ Inf. Sci., vol. 378, pp. 424–443, Feb. 2017.
[33] S. Berkovsky, J. Freyne, and M. Coombe, ‘‘Aggregation trade offs
in family based recommendations,’’ in Advances in Artificial Intelli-
gence, A. Nicholson and X. Li, Eds. Berlin, Germany: Springer, 2009,
pp. 646–655.
[34] J. Freyne and B. Smyth, ‘‘Cooperating search communities,’’ in Adaptive
Hypermedia and Adaptive Web-Based Systems, V. P. Wade, H. Ashman,
and B. Smyth, Eds. Berlin, Germany: Springer, 2006, pp. 101–110.
[35] I. Garcia, L. Sebastia, and E. Onaindia, ‘‘On the design of individual and
group recommender systems for tourism,’’ Expert Syst. Appl., vol. 38,
no. 6, pp. 7683–7692, 2011.
[36] M. Stettinger and A. Felfernig, ‘‘Choicla: Intelligent decision support for
groups of users in the context of personnel decisions,’’ in Proc. ACM
RecSys IntRS Workshop, 2014, pp. 28–32.
[37] Y. Chen and P. Pu, ‘‘Cofeel: Using emotions to enhance social interaction in
group recommender systems,’’ in Proc. Workshop Tools Technol. Emotion-
Awareness Comput. Mediated Collaboration Learn. Alpine Rendez-Vous
(ARV), 2013, pp. 1–2.
[38] J. J. Jung, ‘‘Attribute selection-based recommendation framework for
short-head user group: An empirical study by movielens and IMDB,’’
Expert Syst. Appl., vol. 39, no. 4, pp. 4049–4054, 2012.
[39] J. Bobadilla, F. Ortega, A. Hernando, and J. Bernal, ‘‘Generalization of
recommender systems: Collaborative filtering extended to groups of users
and restricted to groups of items,’’ Expert Syst. Appl., vol. 39, no. 1,
pp. 172–186, 2012.
[40] Z. Yu, X. Zhou, Y. Hao, and J. Gu, ‘‘TV program recommendation for
multiple viewers based on user profile merging,’’ User Model. User-
Adapted Interact., vol. 16, no. 1, pp. 63–82, 2006.
[41] Y. Zhiwen, Z. Xingshe, and Z. Daqing, ‘‘An adaptive in-vehicle multimedia
recommender for group users,’’ in Proc. IEEE 61st Vehicular Technol.
Conf. (VTC-Spring), vol. 5, May 2005, pp. 2800–2804.
[42] Y. Liu, L. Yang, J. Sun, Y. Jiang, and J. Wang, ‘‘Collaborative matrix
factorization mechanism for group recommendation in big data-based
library systems,’’ Library Hi Tech, vol. 36, no. 3, pp. 458–481, 2018.
[43] I. Christensen and S. Schiaffino, ‘‘Matrix factorization in social group
recommender systems,’’ in Proc. 12th Mexican Int. Conf. Artif. Intell.,
Nov. 2013, pp. 10–16.
[44] Y. Bao, H. Fang, and J. Zhang, ‘‘TopicMF: Simultaneously exploiting
ratings and reviews for recommendation.,’’ in Proc. 28th AAAI Conf. Artif.
Intell., vol. 14, pp. 2–8, Jul. 2014.
[45] T. A. N. Pham, X. Li, G. Cong, and Z. Zhang, ‘‘A general graph-based
model for recommendation in event-based social networks,’’ in Proc. IEEE
31st Int. Conf. Data Eng., Apr. 2015, pp. 567–578.
[46] S.-H. Cheng, S.-M. Chen, and Z.-C. Huang, ‘‘Autocratic decision making
using group recommendations based on ranking interval type-2 fuzzy
sets,’’ Inf. Sci., vols. 361–362, pp. 135–161, Sep. 2016.
[47] W. Yu, Q. Zhong, and Z. Zhang, ‘‘Multi-granular hesitant fuzzy linguistic
group decision making with incomplete weight information,’’ Syst. Eng.,
vol. 38, no. 3, pp. 777–785, 2018.
[48] K. Myszkorowski and D. Zakrzewska, ‘‘Using fuzzy logic for recom-
mending groups in e-learning systems,’’ in Computational Collective Intel-
ligence. Technologies and Applications, C. Bǎdicǎ, N. T. Nguyen, and
M. Brezovan, Eds. Berlin, Germany: Springer, 2013, pp. 671–680.
[49] L. Boratto and S. Carta, ‘‘Art: Group recommendation approaches for
automatically detected groups,’’ Int. J. Mach. Learn. Cybern., vol. 6,
pp. 953–980, Dec. 2015.
[50] E. Ntoutsi, K. Stefanidis, K. Nørvåg, and H.-P. Kriegel, ‘‘Fast group
recommendations by applying user clustering,’’ in Conceptual Modeling,
P. Atzeni, D. Cheung, and S. Ram, Eds. Berlin, Germany: Springer, 2012,
pp. 126–140.
[51] I. Ntoutsi, K. Stefanidis, K. Norvag, and H.-P. Kriegel, ‘‘gRecs: A group
recommendation system based on user clustering,’’ in Database Systems
for Advanced Applications, S. Lee, Z. Peng, X. Zhou, Y. S. Moon,
R. Unland, and J. Yoo, Eds. Berlin, Germany: Springer, 2012,
pp. 299–303.
[52] E. Ben Ahmed, W. Tebourski, W. Ben Abdessalem Karaa, and F. Gargouri,
‘‘SMART: Semantic multidimensional group recommendations,’’ Multi-
media Tools Appl., vol. 74, pp. 10419–10437, Dec. 2015.
[53] A. Zapata, V. H. Menéndez, M. E. Prieto, and C. Romero, ‘‘Evaluation and
selection of group recommendation strategies for collaborative searching
of learning objects,’’ Int. J. Human-Comput. Stud., vol. 76, pp. 22–39,
2015.
39760
[54] H.-N. Kim and A. E. Saddik, ‘‘A stochastic approach to group recommen-
dations in social media systems,’’ Inf. Syst., vol. 50, pp. 76–93, Jun. 2015.
[55] C.-H. Lai, ‘‘Applying knowledge flow mining to group recommendation
methods for task-based groups,’’ J. Assoc. Inf. Sci. Technol., vol. 66, no. 3,
pp. 545–563, 2015.
[56] T. D. Pessemier, J. Dhondt, and L. Martens, ‘‘Hybrid group recommenda-
tions for a travel service,’’ Multimedia Tools Appl., vol. 76, pp. 2787–2811,
Jan. 2017.
[57] L. Sun, X. Wang, Z. Wang, H. Zhao, and W. Zhu, ‘‘Social-aware video rec-
ommendation for online social groups,’’ IEEE Trans. Multimedia, vol. 19,
no. 3, pp. 609–618, Mar. 2017.
[58] J. Bobadilla, F. Serradilla, and J. Bernal, ‘‘A new collaborative filtering
metric that improves the behavior of recommender systems,’’ Knowl.-
Based Syst., vol. 23, no. 6, pp. 520–528, 2010.
[59] F. M. Harper and J. A. Konstan, ‘‘The movielens datasets: History and con-
text,’’ ACM Trans. Interact. Intell. Syst., vol. 5, pp. 19-1–19-19, Dec. 2015.
[60] Z. Zaier, R. Godin, and L. Faucher, ‘‘Evaluating recommender systems,’’ in
Proc. Int. Conf. Automated Solutions Cross Media Content Multi-Channel
Distrib., Nov. 2008, pp. 211–217.
[61] J. Golbeck and J. Hendler, ‘‘FilmTrust: Movie recommendations using
trust in web-based social networks,’’ in Proc. 3rd IEEE Consum. Commun.
Netw. Conf. (CCNC), vol. 1, Jan. 2006, pp. 282–286.
[62] F. Ortega, B. Zhu, J. Bobadilla, and A. Hernando, ‘‘CF4J: Collaborative
filtering for java,’’ Knowl.-Based Syst., vol. 152, pp. 94–99, Jul. 2018.
FERNANDO ORTEGA was born in 1988. He
received the B.S. degree in software engineering
and the Ph.D. degree in computer sciences from
the Universidad Politécnica de Madrid, Madrid,
in 2010 and 2015, respectively. He is currently
an Assistant Professor with the Deparment of
Engineering, U-tad: Centro Universitario de Tec-
nología y Arte Digital. His main research fields
are machine learning, data analysis, and artificial
intelligence. He has published several papers in the
most relevant international journals. He also actively collaborates in various
projects with technology companies.
REMIGIO HURTADO was born in 1989. He
received the B.S degree in systems engineer-
ing from the Universidad Politécnica Salesiana,
Ecuador, in 2012, the master’s degree in informa-
tion and software technology from the Instituto
Tecnológico y de Estudios Superiores de Monter-
rey, Mexico, in 2014, and the master’s degree in
computer science and technology from the Univer-
sidad Politécnica de Madrid, Spain, in 2017.
He is currently a Lecturer with the Universi-
dad Politécnica Salesiana. His research interests include the recommender
systems and natural language processing. He is currently a member of the
Research Team in Artificial Intelligence and Assistive Technology, his team
is collaborating with the Universidad Politécnica de Madrid.
VOLUME 6, 2018
F. Ortega et al.: Recommendation to Groups of Users Using the Singularities Concept
JESÚS BOBADILLA received the B.S. degree in
computer science from the Universidad Politéc-
nica de Madrid and the Ph.D. degree in com-
puter science from the Universidad Carlos III
de Madrid. He is currently a Lecturer with the
Department of Applied Intelligent Systems, Uni-
versidad Politécnica de Madrid. He is a habitual
author of programming languages books working
with McGraw-Hill, Ra-Ma, and Alfa Omega pub-
lishers. His research interests include information
retrieval, recommender systems, and speech processing. He was in charge of
the FilmAffinity.com Research Team, involved in the collaborative filtering
kernel of the website. He has been a Researcher with the International
Computer Science Institute, Berkeley University, Berkeley, CA, USA.
VOLUME 6, 2018
RODOLFO BOJORQUE was born in 1982. He
received the B.S. degree in systems engineer-
ing from the Universidad Politécnica Salesiana,
Ecuador, in 2006, the master’s degree in secu-
rity information and communications technologies
from Universidad Autónoma de Barcelona, Spain,
in 2015, and the master’s degree in computer sci-
ence and technology from Universidad Politécnica
de Madrid, Spain, in 2017.
He is currently a Lecturer with the Universidad
Politécnica Salesiana, Ecuador. His research interests includes the machine
learning techniques and intelligent systems. He is currently a member of
research team in artificial intelligence and assistive technology, his team is
collaborating with the Universidad Politécnica de Madrid.
39761