Week 8

Data analysis, Interpretation

and Presentation
Agus Cahyo Nugroho, S.Kom, M.T
Informatics Universitas Ciputra
2018
Quantitative vs Qualitative Data

 Quantitative Data
Can be easily translated into numbers.
 Qualitative Data
Difficult to measure/count.
Quantitative vs Qualitative Data

 All information capture methods discussed may create

qualitative or quantitative data.
 Any qualitative data can be translated into numbers.
With more or less trouble.
Quantitative vs Qualitative Data

 Numbers usually present clearer results

However, they can lead to misinterpretation
Ex: analysing interviews regarding a new product.
Count the number of times the product is mentioned.
 Ex: stating the 50% of users are happy with the product
when only 4 users were inquired
 Always present the context!
The First Step

 Transcribe interviews
Find the most relevant / important parts
 Questionnaires / closed questions
Remove false answers
Filter data into sets
Ex: segment by age
 Observation
 Synchronize data
 Summary

Raw Data Ex Qualitative Ex Quantitative Initial Steps

Interviews Audio, notes Responses to Age, job Transcription
video open questions
Questionnaires Written, online Responses to Years of Clean up data
database open questions Experience Cluster data
Observation Notes, photos, Descriptions of Time spent on a Transcription
audio, video, Behaviours task Synchronization
data logs,
written
Simple Quantitative Analysis

 Use averages and percentages

 Three types of averages
Mean, Median, Mode
 Ex: Data: {2,3,4,6,6,7,7,7,8}
Median = 6, Mode =7, Mean = 5,56
 Ex: Data: {2,2,2,2,450}
 Present standard deviation
Simple Quantitative Analysis

 Place data in tables

Distinguish “don't know” from “did not answer”
How to present enumerations?
User Job User Lecturer Programmer Other

1 Lecturer 1 x

2 Programmer 2 x x
Questions Influence the Type of Analysis

 “How do you felt about the system?”

 If the sample is too big analysing this can be a problem.
Many different answers.
It will be hard to summarize.
Questions Influence the Type of Analysis

 “In your opinion the system was amusing, irritating or

neither?”
User Amusing Irritating Neither
A 1
B 1
...
Z 1
Total 14 5 7

 14 out of 26 (54%) found the system amusing.

Questions Influence the Type of Analysis

 “In your opinion the system was amusing:”

1)Strongly agree; 2)Agree; 3)Neither; 4)Disagree; 5)Strongly disagree

User SA A N D SD
A 1
B 1
C 1
...
Z 1
Total 5 7 10 1 3

 4 out of 26 (15%) disagree that the system is amusing.

Simple Quantitative Analysis

 Find Outliers
Values that are significantly different.
May represent noise in the data.

Study / analyze these special cases.

Comparing Two Systems

 Bar graphs help comparing.

Was the system helpful?

Example – Star Wars Galaxies

 Online computer game

Star Wars Galaxies
Example – Star Wars Galaxies

 Goal: Identify patterns of interaction

 Data logs, in-game videos and ethnography
Data from 26 days (21 hours a day)
5493 unique players
 Two important locations
Starport and Cantina (Coronet City)
Example – Star Wars Galaxies
 Interactions of two main categories
Gestures (e.g. smile, greet, clap)
Public messages
Gesture Cantina (%Total) Gesture Starport (%Total)
Smile 18,13 Thank 15,95
Cheer 9,57 Bow 12,29
Clap 7,77 Wave 9,81
Wave 6,27 Flail 8,17
Wink 4,22 Smile 7,89
Grin 3,72 Nod 7,03
Nod 3,23 Salute 2,48
Bow 3,22 Pet 1,95
Example – Star Wars Galaxies
Example – Star Wars Galaxies
 Gestures made and received - Cantina
Size of circle → number of messages
Example – Star Wars Galaxies
 Gestures made and received - Starport
Size of circle → number of messages
Example – Star Wars Galaxies
 Some Results
Most of the players do not interact much.
Some players send many messages but do not use gestures.
 In the cantina
On average 1 gesture made, 1 gesture received and 4
messages exchanged in the Starport.
Few gestures but more public messages – it is the trading
spot.
Qualitative Analysis: Remarks

 If you only have a few records it is more important to

analyse the individual than trends.
Show tabular data.
Don't throw away the raw data.
Qualitative Analysis

 Using frameworks creates structured data.

Ex: observation frameworks
 Three simple methods :
1. Identifying recurring patterns.
2. Categorizing data.
3. Analyzing critical incidents.
Identifying Recurring Patterns

 Goals of the study provide orientation.

Major themes and minor themes.
Ex: Study the usability of train travel website.
Major theme: comment on the station stops.
Minor theme: comment on the company logo.
 Keep a description of themes.
 Define its granularity.
Refine the list, but not too much.
Identifying Recurring Patterns

 Use affinity Diagrams

Categorizing Data

 Tag categories to descriptions.

Words, sentences, paragraphs.
 Use categorization criteria.
May be based on a scheme.
May emerge from recurring patterns (themes).
Test the emergent scheme properly.
Give the scheme to different people.
Compute the inter-rater reliability (% of agreement).
Categorizing Scheme (Rens, 1997)

 Interface Problems
I1 - Show evidence of Dissatisfaction.
I2 - Show evidence of Uncertainty.
I3 - Show evidence of Surprise.
I4 - Show evidence of Physical Discomfort.
I5 - Show evidence of Fatigue.
I6 - Show evidence of Difficulty in Seeing particular aspects.
Categorizing Scheme (Rens, 1997)

Interface Problems
I7 - Show evidence of Having Problems Achieving
a Goal.
I8 - Show evidence that the user Made an Error.
I9 - Show evidence that the user was Unable to
Recover from an Error.
I10 - Suggestions made by the user.
Categorizing Scheme (Rens, 1997)

 Content Problems
C1 - Show evidence of Dissatisfaction
C2 - Show evidence of Uncertainty
C3 - Show evidence of Misunderstanding
C4 - Suggestions made by the user
Example

 I'm thinking that it it's just a lot of information to absorb from

the screen. I just don't concentrate very well when I'm looking at
the screen. I have a very clear idea of what I've read so far...
but it's because of the headings. … It would still be nice to see it
on a piece of paper, because it is a lot of text to read. … There
is so much reference to all those previously said. ...