MSE UCD Unit3 WS1718

Dr.
Verena Seibert-Giller
User Centered Design

Lecture 4
© 2014 Seibert-Giller
2
Types of Evaluations
Without users With users

(expert based)
Usability inspection
Usability tests
3
Today‘s contents
▪ Types of Evaluations
▪ What is a Usability inspection?

– What kind of inspections / reviews are there?
– How do they work?
▪ What Usability testing?

– What kind of tests are there?
– How does it work?
▪ Very little theory, then practical examples

▪
4
Today‘s contents



▪
5
What is a Usability inspection ?
▪ A usability inspection (or review) is an evaluation of a user
interface carried out by usability experts (without users)
▪ It is a valid and low cost method to support a user centred
design process for most product developments.
▪ In case of very specialized domain knowledge you might
require the support of domain experts.
▪ Even experienced usability experts do not just “look at the

system as they feel” when an in depth evaluation is
required!
6
Today‘s contents



▪
7
Usability Inspection Methods
▪ There are various methods available, they vary in some
details.
▪ Two major categories of Usability inspection methods
1. Heuristic based evaluations (evaluating against specified

heuristics)
2. Task oriented walkthroughs
8
1. Heuristic Evaluation
10 Heuristics by Jacob Nielsen (originally invented the H.E.)
20 Heuristics by Susan Weinschenk and Dean Barker
2. Cognitive walkthroughs
By Wharton
By Spencer
9
By Wharton
By Spencer
10
Heuristic evaluation
▪ Usability professionals evaluate the user interface in order to
identify potential usability problems.
▪ Each evaluator goes through the interface several times and
inspects the views and dialogue elements and evaluates
them against defined list of specified heuristics
– E.g. Consistency (detailed list later)
▪ Each evaluator conducting the evaluation alone.

▪ Then they aggregate and prioritize their results.
11
Number of evaluators (by Jacob Nielsen)
Number of evaluators recommended: 3-5
12
Heuristic evaluation
▪ Potential problems:
– Task orientation of the user interface is not clearly

represented in this method.
– It requires a lot of practice to understand and apply the

respective heuristics in a efficient and valid manner.
13
By Wharton
By Spencer
14
Jabob Nielsen‘s Heuristics
1. Visibility of system status
– The system should always keep users informed about what is
going on, through appropriate feedback within reasonable
time.
2. Match between system and the real world

– The system should speak the user's language, with words,
phrases and concepts familiar to the user, rather than
system-oriented terms. Follow real-world conventions,
making information appear in a natural and logical order.
15
3. User control and freedom
– Users often choose system functions by mistake and will
need a clearly marked "emergency exit" to leave the
unwanted state without having to go through an extended
dialogue. Support undo and redo.
4. Consistency and standards

– Users should not have to wonder whether different words,
situations, or actions mean the same thing. Follow platform
conventions.
16
5. Error prevention
– Even better than good error messages is a careful design
which prevents a problem from occurring in the first place.
Either eliminate error-prone conditions or check for them and
present users with a confirmation option before they commit
to the action.
6. Recognition rather than recall

– Minimize the user's memory load by making objects, actions,
and options visible. The user should not have to remember
information from one part of the dialogue to another.
Instructions for use of the system should be visible or easily
retrievable whenever appropriate.
17
7. Flexibility and efficiency of use
– Accelerators—unseen by the novice user—may often speed
up the interaction for the expert user such that the system
can cater to both inexperienced and experienced users.
Allow users to tailor frequent actions.
8. Aesthetic and minimalistic
– Dialogues should not contain information which is irrelevant
or rarely needed. Every extra unit of information in a dialogue
competes with the relevant units of information and
diminishes their relative visibility.
18
9. Help users recognize, diagnose, and recover from errors
– Error messages should be expressed in plain language (no
codes), precisely indicate the problem, and constructively
suggest a solution.
10. Help and documentation

– Even though it is better if the system can be used without
documentation, it may be necessary to provide help and
documentation. Any such information should be easy to
search, focused on the user's task, list concrete steps to be
carried out, and not be too large.
19
By Wharton
By Spencer
20
Heuristics by Weinschenk and Barker
▪ Analysed various available heuristics and principles and
came up with the following 20 heuristics
21
▪ 1. User Control: heuristics that check whether the user has enough
control of the interface.
▪ 2. Human Limitations: the design takes into account human limitations,
cognitive and sensorial, to avoid overloading them.
▪ 3. Modal Integrity: the interface uses the most suitable modality for
each task: auditory, visual, or motor/kinesthetic.
▪ 4. Accommodation: the design is adequate to fulfill the needs and
behavior of each targeted user group.
▪ 5. Linguistic Clarity: the language used to communicate is efficient and
adequate to the audience.
▪ 6. Aesthetic Integrity: the design is visually attractive and tailored to
appeal to the target population.
22
▪ 7. Simplicity: the design will not use unnecessary complexity.
▪ 8. Predictability: users will be able to form a mental model of how the
system will behave in response to actions.
▪ 9. Interpretation: there are codified rules that try to guess the user
intentions and anticipate the actions needed.
▪ 10. Accuracy: There are no errors, i.e. the result of user actions
correspond to their goals.
▪ 11. Technical Clarity: the concepts represented in the interface have
the highest possible correspondence to the domain they are modeling.
▪ 12. Flexibility: the design can be adjusted to the needs and behaviour
of each particular user.
▪ 13. Fulfillment: the user experience is adequate.
23
▪ 14. Cultural Propriety: user's cultural and social expectations are met.
▪ 15. Suitable Tempo: the pace at which users works with the system is
adequate.
▪ 16. Consistency: different parts of the system have the same style, so
that there are no different ways to represent the same information or
behavior.
▪ 17. User Support: the design will support learning and provide the
required assistance to usage.
▪ 18. Precision: the steps and results of a task will be what the user
wants.
▪ 19. Forgiveness: the user will be able to recover to an adequate state
after an error.
▪ 20.Responsiveness: the interface provides enough feedback
information about the system status and the task completion.
24
By Wharton
By Spencer
25
Cognitive Walkthrough
▪ First a task analysis is required as basis for further
activities
– Each task has to be split up in it’s subtasks
▪ Then designers and developers of the product walk
through the tasks – step by step / subtask by subtask - as
a group.
▪ While doing so they keep asking themselves questions,
normally the questions shown on the next slide.
– Originally 4 questions by C. Wharton
– Streamlined 2 questions by Spencer
26
Cognitive Walkthrough
▪ Potential problems:
– The evaluators not necessarily know how to perform
the task themselves, so they stumble through the
interface trying to discover the correct sequence of
actions -- and then they evaluate the stumbling
process.
– No real users are tested on the system. The
walkthrough might identify more problems than you
would find in a single user test session.
27
By Wharton
By Spencer
28
Cognitive Walkthrough - Wharton
1. Will users be trying to produce whatever effect the action
has?
See the need for the respective subtask?
2. Will users see the control (button, menu, switch, etc.) for the
action?
3. Once users find the control, will they recognize that it
produces the effect they want?
4. After the action is taken, will users understand the feedback
they get, so they can go on to the next action with
confidence?
29
By Wharton
By Spencer
30
Cognitive Walkthrough - Spencer
1. Will the user know what to do at this step?
2. If they do the right thing, will they know they did it?
31
Now it is your turn
▪ You will try out the methods and we will compare and
discuss your results.
▪ www.wienerlinien.at: (mobile) Planning a trip
▪ In 2 groups
– ½ of you takes the heuristics from Jacob Nielsen
– ½ of you takes Whartons walkthrough questions
– work in teams of 2-3 students
32
Today‘s contents



▪
33
What Usability testing?
▪ ... is a structured way to let users interact with a system to
identify potential problems in the user interface.
▪ It is normally carried out in a single user session –

sometimes there are reasons to test 2 persons in a
combined session
▪ ... is a valid method to support a user centered design

process for any product development
▪ The respective user carries out (prepared) tasks and is

observed and questioned while doing so.
34
What kind of tests are there?
Testing can range from to
Rapid „do it In depth, resource High end studies,

yourself“ testing intensive industrial mainly for science
tests
These are perfectly fine for most systems in These are sometimes
the end user market! required for safety critical
systems
The more sophisticated it gets, the more equipment and

infrastructure you need.
35
Lots of cameras and screens
People observing closly
Copy right semantic web company, prag
© 2014 Seibert-Giller Copy right u-sentric, belgium

36
Eye tracking
Tracking the users eye
movements in order to learn
even more. Also carried out
here in Technikum Wien, there
is a seminar about it.
Copy right e voc insights, US
Meanwhile users don‘t need to

wear such head mounted
equipment any more….
Copy right Univ. Nebraska-Lincoln
37
Testing mobile appliances
Copy right Experience dynamics, US
Copy right Harry Brignull
38
Discussing testing variations
▪ Steve Krug shows a very reasonable approach to testing for
non-usability-professionals in his book „Rocket surgery
made easy“ (see next slides)
▪ His Big Honkin‘ Test opposed to a Do it your self testing
Highly structured, resource Informal testing done by

intensive laboratory test the developers or designers
39
Big Honkin‘ Test Do it your self testing
Steve Krug „Rocket surgery made easy“ 40

Steve Krug „Rocket surgery made easy“
41
Steve Krug „Rocket surgery made easy“
42
Organization of a Do It Yourself Test 1/3
Define what you want to learn about your system!
Problems in understanding, Time on task too long,
information/content presentation ok, …
▪ Generally you need a separated workplace or room

– Only if it is a system that cannot be brought there, then you
test wherever possible (e.g. system in industrial plant)
▪ If possible video beam the user and the system (face/voice
and screen) to another room where people shall sit
comfortably and watch
– If keyboard input is crucial it would be great to see that too
– Screen share software would also do
43
▪ Plan warming up questions, maybe a questionnaire
– Age, profession, domain knowledge, computer literacy
(amount of hours per day, mostly used applications, …)
▪ Organize materials such as paper & pens for you to take
notes- maybe screenshots to plot or draw on
▪ Develop the tasks the user shall carry out
– Clearly identify before hand on what aspects of the system
feedback is required at that stage e.g. structure/navigation,
content, wording, task flow..)
44
Developing tasks
▪ Each task has to
– target /check a concrete design question
– be like a task in reality
– have clear result (e.g. not just the opening of the right page)
▪ E.g. ÖBB “the ticket costs 17euro” not “here I find the ticket
prices”
– be texted without using the terms/words represented in the
user interface
▪ E.g. You want to travel to Paris Disneyland with your family (kids 10 and
13 years) for the weekend of the 31. Oct. You want to optimize your stay
by arriving Friday evening and leaving latest on Sunday night. What is
the cheapest flight you can get?
45
▪ Invite / recruit your users
▪ Carefully schedule the users, consider how much time per
user will you need
▪ Schedule a project team meeting right after the tests are
finished
▪ Be prepared to moderate this meeting and step out of it
with clear ToDo’s for the User Interface
46
Documenting results
▪ Document what is necessary for the project team to later
check or get back to specific questions
– Very basically describe the users problems and inputs
– Clear ToDos for the User Interface
– Why did the team decide upon these issues as they did?
▪ A structured email can be just fine!
47
Avoiding Bias
▪ The problems of your test person is what you are looking
for in order to optimize the system
– If you help, you get no insights
▪ So don‘t help and don‘t give hints!
▪ Watch out for your body language!
▪ Don‘t show that you are impatient!
▪ If a test person “is lost” and wants to give up:

– Verbally rephrase the task using other words
– Give (predefined) minor directions
▪ look on the upper left corner
48
Questions?
49

MSE UCD Unit3 WS1718

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

MSE UCD Unit3 WS1718

Uploaded by

Copyright:

Available Formats

Dr.

User Centered Design

Without users With users

▪ What is a Usability inspection?

▪ What Usability testing?

▪ Very little theory, then practical examples

▪ What is a Usability inspection?

▪ What Usability testing?

▪ Very little theory, then practical examples

▪ Even experienced usability experts do not just “look at the

▪ What is a Usability inspection?

▪ What Usability testing?

▪ Very little theory, then practical examples

1. Heuristic based evaluations (evaluating against specified

2. Task oriented walkthroughs

▪ Each evaluator conducting the evaluation alone.

– Task orientation of the user interface is not clearly

– It requires a lot of practice to understand and apply the

2. Match between system and the real world

4. Consistency and standards

6. Recognition rather than recall

10. Help and documentation

▪ www.wienerlinien.at: (mobile) Planning a trip

– work in teams of 2-3 students

▪ What is a Usability inspection?

▪ What Usability testing?

▪ Very little theory, then practical examples

▪ It is normally carried out in a single user session –

▪ ... is a valid method to support a user centered design

▪ The respective user carries out (prepared) tasks and is

Testing can range from to

Rapid „do it In depth, resource High end studies,

The more sophisticated it gets, the more equipment and

Copy right semantic web company, prag

© 2014 Seibert-Giller Copy right u-sentric, belgium

Copy right e voc insights, US

Meanwhile users don‘t need to

Copy right Univ. Nebraska-Lincoln

Copy right Experience dynamics, US

Copy right Harry Brignull

▪ His Big Honkin‘ Test opposed to a Do it your self testing

Highly structured, resource Informal testing done by

Steve Krug „Rocket surgery made easy“ 40

Steve Krug „Rocket surgery made easy“

Steve Krug „Rocket surgery made easy“

▪ Generally you need a separated workplace or room

▪ A structured email can be just fine!

▪ If a test person “is lost” and wants to give up:

You might also like