Mestrado Integrado em Engenharia Informtica

Projecto Integrador

Public Display Applications

Pedro Albuquerque Santos
Rui Neves Madeira
Prof. Nuno Correia

March 19th, 2014

Monte de Caparica, Almada, Portugal

Why am I here?

I have finished my Masters last December

During my dissertation I developed a public display
application

I may have some experience to share with you about the

system I have built

I can give you some tips and answer some questions you may
have about this kind of system

I will present the system I have developed

Pointing out any similarities that it has with CampusTV

After that, I am open to discuss any doubts you may have

2

Motivation

Testing the personalization framework that was the main

focus of my Masters' dissertation
But also the realization that:

We live surrounded by mobile devices

While large situated displays are also present

e.g., smartphones, tablets, notebooks

e.g., TVs, computer screens, advertising and information displays

Such displays can be leveraged to improve the user

experience and to develop and improve interaction
mechanisms
3

Motivation

There is a growing body of research around the area of

public displays

It remains a complex and system-specific effort

Strong focus on advertising content

There is an opportunity to leverage the ubiquity of

personal devices and public displays

Provide useful and personalized information to users

Adding real-time interactivity to increase the system

usefulness

Which is not a major requirement of CampusTV

Related Work

Benefits of large displays

e.g., peripheral awareness, productivity, immersiveness

How people interact with public displays

Interaction techniques, input sources and ways to abstract them

Bluetooth scanning has been used to sense user presence and to

allow some degree of interaction, due to its low entry barrier

Other input sources include SMS and e-mail

There has also been some focus on group-based

personalization and content broadcasting based on user
preferences

Related Work

M. Finke, A. Tang, R. Leung, and M. Blackstock. Lessons Learned: Game Design for Large Public Displays. In:
Proceedings of the 3rd International Conference on Digital Interactive Media in Entertainment and Arts. DIMEA 08.
ACM, 2008, pp. 2633.
N. Kaviani, M. Finke, S. Fels, R. Lea, and H. Wang. What Goes Where?: Designing Interactive Large Public Display
Applications for Mobile Device Interaction. In: Proceedings of the First International Conference on Internet Multimedia
Computing and Service. ICIMCS 09. ACM, 2009, pp. 129138.

With that in mind...

FCT4U was born

(Semi-)public situated display based system

Provides useful and contextual information to people nearby in an

university campus:

e.g., students, teachers, researchers, etc.

Any similarity with CampusTV is pure coincidence

A personalization approach to enhance the user experience

was applied to it
A user study to assess and validate the system's purpose and
interaction modes was conducted at CITI
It was presented at the UIC-2013 last December
7

Principles & Goals

Situated displays can be:

Public, shared, interactive or proactive/personalized

FCT4U combines all of these categories

It aims to:

Provide information to users in the surrounding space while sharing the

public display between up to 3 users

Expand information, by using mobile and public displays simultaneously

Promote sharing and interaction among users (e.g., short messages

and content sharing)

CampusTV's public displays are mainly public and shared, with

less focus on interactiveness and proactiveness/personalization

FCT4U System

The system is composed of:

Mobile application that runs on smartphones and tablets

Android 2.3.3+ was chosen as the mobile platform due to its current
market share

Application running on a situated display

JavaFX-based application to support multiple platforms

This was also where the content manager was placed for simplicity
(the prototype used a single public display)

JPA for data storage, using MySQL as the back-end

The applications share the same web-based UI

Built with HTML5, JavaScript and CSS

It offers re-usability, portability and adaptability to different form factors

FCT4U System

Bluetooth was used to sense users' presence

It offers a cheap and easy way of sensing the proximity of other

devices/users

BlueCove (JSR-82 implementation) is used to implement a

presence sensing routine

There were attempts to use the RSSI to determine the approximate

distance from the display

It was the public display that sensed the mobile devices around

For CampusTV, I advise you to do the opposite so that you do not have to
register each user's Bluetooth MAC Address

As in: T. Kubitza, S. Clinch, N. Davies, and M. Langheinrich. Using Mobile Devices to

Personalize Pervasive Displays. In: SIGMOBILE Mob. Comput. Commun. Rev. 16.4
(2013), pp. 2627.

10

FCT4U System

Communication between the public display and mobile devices is

done through the WebSocket protocol

It allows bi-directional communication, while bypassing common network

restrictions

A central accessible node that works as a commnication hub is needed

For that, a Java-based framework was used: https://github.com/Atmosphere

It supports end-to-end encryption when run over TLS

It can be used by pure web applications running on modern browsers

Some older browsers, such as Android's WebView, do not support WebSockets.

In such cases, a native application is needed to deal with WebSockets connections.

A good option for CampusTV if you WANT to add bi-directional

communication between the public displays and mobile devices

e.g., allowing real-time interaction and/or pushing data among devices

11

Architecture

P2MUCA means Personalization Platform for Multimodal Ubiquitous Computing Applications

For CampusTV you would REALLY need to decouple the FCT4U Engine
from the Public Display Application

Multiple public displays would communicate with the FCT4U Engine through some
kind of mechanism (e.g., Java RMI, WebServices, REST, WebSockets, etc.)

The messages sent by the mobile devices through the communication hub, would
then be delivered to the FCT4U Engine or to the corresponding Public Display
Application, depending on where (certain?) features are implemented

12

P MUCA OAuth 2.0

13

Test Intallation

14

User Interface
Mobile Application
Public Display Application

15

Widgets

16

Widgets

17

Widgets

18

Mobile Application

It extends the public display

Allows users to view information while away from the public display

The latest information received from the display is cached

Provides a safe place to show privacy sensitive information and to set

preferences (including connecting to Facebook)

Works as an input device

It is used to associate the mobile device's Bluetooth MAC address

with a certain user

On FCT4U the display reacted to the presence of the users, not the
other way around

This is not needed if the mobile devices are the ones to scan for the
public displays, with MAC addresses known beforehand

19

Messaging & Facebook

20

(Optional) Location Tracking

It allows us to automatically determine where someone

lives, works and lunches

Density Clustering (DBSCAN algorithm)

Geo-fencing of public transport hubs is used to

determine the preferred transportation mode

21

Personalization

P2MUCA

Major focus of my MSc dissertation and also part of

Rui Neves Madeira's PhD

Functionalities are exposed through an OAuth 2.0

protected HTTP API and a companion website is
available for configuration

Uses a generic personalization model

It can be configured through an XML configuration file or

a web-based GUI

22

Personalization Model

Resources represent interaction streams, preferences, options or

contextual information
They can be aggregated into parameters using formulas

e.g., 0.2*messagesRead+0.3*messagesReceived+0.5*messagesSent

Personalizations define multiple options from the combination of multiple

parameters
The framework also supports context segmentation
If you WANT to have some personalization on CampusTV, you may follow
similar ideas in a more system-specific effort

23

User Study

A user study was conducted at CITI

The main focus was to evaluate if the idea of

system like FCT4U was well accepted

Comparing different system interaction modes was

also a major concern

Personalization evaluation was not taken into

consideration

It was a short term test to present the system to potential

users

24

User Study

The public display application was deployed on a 46 touch-screen

Multiple Android devices were used for the mobile application

Participants went trough 3 scenarios

Passive, mobile application interaction and touch-screen interaction

A questionnaire was answered after each scenario

9 test subjects who work at CITI

Aged between 26 to 39 years (average of 30.5)

2 females

8 PhD students and a post-PhD researcher

2/3 have mobile devices have Android 2.3.3+ devices

25

User Study Results

7 out of 9 users gave positive scores to the system's

usefulness (median of 6)
They also thought that system could save them time and that
it would be even better if was already tailored for their needs
Passive Mode

Answers were mostly positive

System responsiveness was considered adequate by most users

The information provided by the passive mode was considered just above
adequate (median of 5)

The first widgets were the ones seen that users saw
(i.e., Map, Weather, Lunch Menus)

26

User Study Results

Mobile Application Interaction

It gave users control of what is shown on screen

It was considered useful and easy to use

(answers with medians of 6)

Answers related with ease of use, user effort and

recovering from mistakes got medians of 5 or 6

But, in general, they considered the application

pleasant to use and quick to master
(medians of 6 and 7)
27

User Study Results

Mobile Application Specific Features

Regarding the short messages services the

reactions were mixed

Users also did not consider the system very

collaborative, even though some recognize its
potential to evolve in that direction

Those are two areas to improve in the future

28

User Study Results

Touch Screen Interaction

It was not as well accepted as the mobile application,

getting slightly lower scores on the same questions

It may be due to people not being used to use such a large touchscreen in an upright position
Maybe they did not see an advantage over the mobile application
Nevertheless, 7 out of 9 users considered this mode easy to use

Also, it is not practical when the application is not deployed

within arms reach

e.g., the displays recently installed on DI's building

29

User Study Results

Users preferred to use the public display alone

But they were able to cope with sharing it

System UI was rated with a a mean of 5.33

There were no negative scores, but most (6 out of

9) gave neutral or just slightly positive scores

This showed us that there is still room for improvement

Other more specific metrics had higher scores

30

User Study Results

Widgets

Favorite widget: Lunch Menus

High rated widgets: Map and Weather

Lower rated widgets: Messages and News

Users preferred the passive mode

All scenarios had similar scores

But most users said that they would use the passive mode
and one of the other active interaction modes

Personalization should increase their satisfaction

31

User Study Results

Privacy Sensitive features

7 out of 9 users were apprehensive about

Location tracking
Bluetooth visibility

If mobile devices are the ones that scan for displays this is
mitigated

Users want control over the information they share

But are interested in smart features

A compromise must be found

32

Conclusions and Future Work

This presentation followed the development of FCT4U:

A (semi-)public situated display based system that proactively shows

information to nearby users in an university campus, while providing
interactivity through a mobile application

Some of FCT4U's design decisions and architecture may be used as the

basis for similar systems (e.g., CampusTV)

The findings from the user study gave us some insight about the user
receptiveness to this type of systems

It would be interesting to see how users will react to your systems

A long-term user study must be conducted to help fine tune the

personalization model configuration
In general, we should continue to add features, improve existing ones
and explore new ideas

33

Thank you for your attention

Download Presentation

Contact: pe.santos@campus.fct.unl.pt

Anything else you want to discuss?

34