Saarland University Proseminar Human-Computer Interaction Antonia Scheidel !

May 14th, 2009

USABILITY I
Introducing Usability Metrics

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

1 3

Tullis & Albert: Chapters 1 + 2

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

2 3

Overview
What is Usability? Why does Usability matter? Usability Metrics Data Types Statistical Procedures Data Presentation
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

Usability - Denition
ISO 9241-11: the extent to which a product can be used by specied users to achieve specied goals with effectiveness, efciency and satisfaction in a specied context of use

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

Why does Usability matter?

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

Why does Usability matter? (cont.)

Because the design of a product should not Disqualify groups of people from using the product Keep people from using the product for its / their intended purpose Be the cause for inconvenience or frustration for the user
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

Measuring Usability
" Measure effectiveness, efciency and user satisfaction. Two ways to go: Trust your own judgement / gut feeling / design department Take advantage of usability metrics when evaluating a new product
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

Measuring Usability
" Measure effectiveness, efciency and user satisfaction. Two ways to go: Trust your own judgement / gut feeling / design department Take advantage of usability metrics when evaluating a new product
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

Why use Usability Metrics?

Intuition is important, but data are better Compare usability of two products Classify the magnitude of a problem Make predictions about the actual use of your product Provide management with facts and gures
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

Usability Metrics
Desired qualities: Metrics should be Observable (task completion - Y/N) Quantiable Example: 90% of the users are able to complete a set of tasks in less than one minute.
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

Designing a Usability Study

1. Select Participants: Are they representative? Separate them into gender, age (..) groups Decide on sampling strategy 2. Choose Sample Size: How much is enough?
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

10

Designing a Usability Study II

3. Within-Subjects or Between-Subjects? One user - many tasks Multiple groups - one task each 4. Counterbalancing: Randomize! (Dont stick to one task order)
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

11

Know your Data

Four types of data: Nominal (categories) Ordinal (ranks) Interval Ratio
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

12

Nominal Data
Unordered categories Examples: Men and women Windows and Mac Users Users from France, Germany, the UK
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

13

Ordinal Data
Ordered categories " rankings Example: Severity ranking # poor # fair # good # excellent Differences between measurements are not meaningful!
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

14

Interval Data
Differences between measurements are meaningful! Example: poor # # # # excellent No natural zero

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

15

Ratio Data
Just like interval data, but: absolute zero point (ratio!) Examples: height, weight, ... (task completion) time (user) age
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

16

How can you use your Data?

Different types of data " different kinds of statistical operations available
s im

Nominal & Ordinal: Carry out !2 tests, compute frequencies

Interval & Ratio: Descriptive statistics, t-test, ANOVA, compute correlation

p li

ed

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

17

Statistical Tests: Chi-Square (!2)

For nominal and ordinal data Compare expected results to observed results Used to determine: goodness of t (in)dependence of variables
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

18

!2-Test: Example
Are hair colour and eye colour independent variables?

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

19

Descriptive Statistics

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

19

Descriptive Statistics

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

19

Descriptive Statistics

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

19

Descriptive Statistics

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

19

Descriptive Statistics

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

19

Descriptive Statistics

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

19

Descriptive Statistics

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

19

Descriptive Statistics

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

20

Comparing Means...
... based on Independent Samples Between-Subjects design: different groups, one task Example: Find out whether there is a signicant difference in the time men or women spend using our product. t-test: Two Samples Assuming Equal Variances
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

21

t-Test: Example
If this value is below a certain threshold (here: 0.05)

p-value

Conclude that the difference is statistically signicant.

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

22

Comparing Means...
... based on Paired Samples Within-Subjects design: One group, different tasks Example: Find out if your participants signicantly prefer one design over another t-test: Paired Two Samples for Means Like before: Evaluate p-value
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

22

Comparing Means...
... based on Paired Samples Within-Subjects design: One group, different tasks Example: Find out if your participants signicantly prefer one design over another t-test: Paired Two Samples for Means Like before: Evaluate p-value
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

23

What if there are more than two Variables?

Solution: ANalysis Of Variance (ANOVA) Compare variances between and within groups Example: Given three designs: Is there a signicant effect due to the different designs? Then: Use ANOVA results to carry out t-tests
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

24

Single Factor ANOVA: Example

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

25

Correlation
Or: How strong is the relationship between two variables?
R2 = 0,58

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

26

Presenting your Results - I

Bar Graphs

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

27

Presenting your Results - II

Line Graphs

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

28

Presenting your Results - III

Scatterplots

R2 = 0,58

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

29

Presenting your Results - IV

Pie Charts

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

30

Presenting your Results - V

Stacked Bar Charts

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

31 3

Summary
Usability matters. Measuring usability will convince people to keep on funding your project. Choose your participants wisely. Know your data. Use the right tests. Impress people with presentations.
Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I

Intro

Usability

Usability Metrics

Data Types

Statistical Procedures

Data Presentation

Thank you for your attention!

Antonia Scheidel ! Proseminar HCI ! May 14th 2009 ! Usability I