Contents

1) The RFP – Before the solution! ....................................................................................................... 2

2) Stakeholder mapping ...................................................................................................................... 3
3) Connect Stakeholders ..................................................................................................................... 4
4) Map Context ................................................................................................................................... 4
5) Define User groups ......................................................................................................................... 5
6) Identify User Group aspects (Interview the team, do research) .................................................... 7
7) Desk research .................................................................................................................................. 7
8) Workshopping ................................................................................................................................. 7
9) Build a research wall ....................................................................................................................... 8
10) Visit users .................................................................................................................................... 8
11) Process User interview - Persona’s ............................................................................................. 9
12) User research methodologies (Tips & tricks) ............................................................................ 11
13) User tasks (Preparation for Usability tests) .............................................................................. 15
14) Run Usability tests..................................................................................................................... 15
15) Get the team to observe ........................................................................................................... 17
16) Analyzing UX Research results .................................................................................................. 18
17) Draft hypothesis for the issues ................................................................................................. 19
18) ( order might need to be revised) Validating solutions based on behavior ............................. 20
19) Set UX metrics to see how design measures up against needs of Users .................................. 21
20) The User Experience Debrief meeting ...................................................................................... 23
21) Putting research into action...................................................................................................... 24
Guidelines ............................................................................................................................................. 26

TO ADD

Idea generation

Problem statements

Need analysis vs Value analysis

Product strategy Canvas

Prototyping

Story Mapping

Diary

Value hypothesis
 SWOT

North Star

Lean MVP Canvas

Wireframing
Clickable mock-ups

1) The RFP – Before the solution!

Challenge the problem or RFP that presented
a. Prepare your stakeholders for the discussion by informing them of the topic, goal,
information necessary to make the meeting productive
b.

c. Ask: What particular problem have you identified that you think a new “X” will
solve?
d. Guiding the discussion
i. Don’t focus on one problem (generate many)
ii. As the list of problems unfold, ask: “What else are you hoping X will
achieve?”
iii. I noticed nobody mentioned anything about “Y” Is that an issue for you?
iv. If you would make progress against the things on this list, would you have a
solution that exactly meets your expectations?
v. Develop evidence: “How do you know these are problems?”

vi. Categorize The results: “No evidence”, “soft evidence”, “Presumed

evidence”, and “Hard evidence”
vii. Explore the evidence
viii. Qualify if the problem is worth fixing by proposing to Imagine the scenario
with the potential solution removed
ix. Context and constraints
 Ask: “what you’ve described is clearly a serious problem, how long has it
been going on?”
“Have you tried anything to solve this problem before?
“Did something stop you from succeeding in the past?

Depending on the answers ask:

“So what’s different now?” What’s changed?”
If we try again now, will we still hit the same obstacles?
e. Set a North star
And near stars
i. A north star is a concrete, confident statement of a goal your going to
achieve
ii. A near star is a realistic goal that will help you get closer to your North star

2) Stakeholder mapping
a. Brainstorm on target Audience (New product/Service development => See
inspiration phase IDEO)
b. Map all user groups
i. Define Mainstream users
ii. Define Extremities
iii. Define Experts
iv. Define every involved stakeholder

f. Plan your Logistics (Localize target audience)

1. Where to find them
2. When to connect with them
3. What do you do together
4. What to gather
ii. Identify your “Experts”
1. Ask in-depth questions
g. Create a list of stakeholders
i. Identify Business
ii. Identify Dev
iii. Identify Target Audience
iv. Identify Users
h. Map their background / Pressure points /
i. Map stakeholders in the project context (as well as organizational)
i. Identify constraints
ii. Timeline
iii. Budget
iv. History
j. Gather research wants and needs and
Identify requirements for research (viewed through the eyes of stakeholders)
3) Connect Stakeholders
a. Immerse yourself in the context (Of the user / stakeholders / Problem scenario)
b. Connect Dev with other stakeholders
c. Connect Business With Dev

4) Map Context
a. Immerse yourself in the Context of the product
b. Map Goal of product
c. Functionality of Product
d. Seek analogo
e. How the team & stakeholders have done this so far
f. The organisation
i. Is criticism rejected?
ii. Is criticism welcomed?
iii. Is criticism solicited?
iv. Is criticism demanded?
g. Research (deconstruct) the construct that is your product and the standards it
should meet

h. Measure something
 i. Shake out the issues

5) Define User groups

a. writes 5 different user groups on 5 stickies
(e.g. a photography app could have: Food enthusiasts, business travelers,
“instagrammers”)

v. Facilitation exercise:

vi. Map stakeholders on the “Grid”

vii. What can “Hard to get” users teach us that “easy to get” users can’t
 viii. Classify User groups and their skills by asking what they do online and how
they go about it?.
- Low digital skills (Not so digitally active ..
- Advanced digital skills

What do they do alone? What do they do with assistance?

a. Select the group that will be “The design target”

b. Pre-screen The User groups
- Ask questions to validate if they are a viable target audience
- (For B2C) Find out if they make the decision to buy something or their friend does)
- Screen-out the participants that are not shy if you need them being “Loud”

Test your pre-screener on participants you don’t want

Send them an email/letter that is formal to make whatever your going to do more
real and tangible
Provide participants enough information to know what to expect but without
mentioning the specifics

Build the pre-screener with questions that are “Flexible”

- Recruit for Behaviors!

- Don’t make it obvious what you’re looking for
6) Identify User Group aspects (Interview the team, do research)
a. Goals
b. Motivation
c. Abilities
d. Identify the sweet spot for your UX diagram using the Venn diagram
(Environment, Goal, User) => Users trying to achieve goals in context
It tells us about the User, how he’s trying to achieve his goals, as well as the
influence of the environment

7) Desk research
a. Examine prior research done internally
b. Examine prior research done externally
(Solutions to problems, research on specific problems, …
ix. Done by charities
x. Government
xi. Universities
xii. Found in Google Scholar
xiii. Institutions known for it (Gartner, Career websites, … )
c. Research existing research on Users (Web analytics / Call center data)
xiv.
d. Take a close look at the quality
xv. Ask: Does it properly measure/judge/capture human behavior

8) Workshopping
a. Create assumption personas
i. ask the development team to imagine the most useful, actionable research
results possible. What would they tell us? How would we use them?
ii. Make sure the persona is seen for what it is, a tool that is a work in progress,
not a finished product.
iii. Make sure it serves the purpose without limiting it’s use for other purposes:
For design this is understanding user behavior while for marketing it is a
persona to which you can sell.
 b. Map the user environment (Tools and more)

9) Build a research wall

a) Post your Assumption Personas
b) Create an information Radiator (Usability dashboards / One-page test plan (Sharing
your results)

10) Building an interview guide

1. Define the interview type
2. Define the interview goal (See lean data)
3. Define if the interview is connected/related to Customer Promise (Lean data)
i. What benefits do you intend to deliver?
ii. And for who?
iii. How will you know you’ve delivered on it?
iv. Define Question set (Through “Desired action” Framework)
v. https://www.surveymonkey.com/mp/sample-size-calculator/
4. Follow guideline as set
i. Draft 5 -7 questions
Each questions only focuses on one aspect?
Questions are not leading
See interview guide doc for more info
5.

11) Define the user group

- Focus on people with lower capabilities or challenges focusing on issues that

trouble people with a lower literacy rate or skillset in the topic you are researching
(More competent people will experience less problems
- Looking to solve problems applicable to 33% of the users: Research 5 users to have
an 85% chance of extrapolating all the issues troubling 33% of users. If it affects less
than 33% you won’t find it.
- By interviewing 385 “Random” (Random=Everyone has an equal likelihood being
selected to participate in the study) people you can get similar results as to asking 1
million users with only 5 % Extra error/deviations
E.G. If 60 % Says they are willing to pay 50 dollars subscription we can definitely
achieve an accurate estimation that indicates 55% to 65% of the users will be willing
 to pay this much.
- Looking to solve problems applicable to 10% of the users: Research 18 users to
have an 85% chance of finding that problem
- Ask these 5 participants to do plenty of “Tasks” the more tasks done, the larger the
problems found.

a. Continue with steps 5 and 6)

BE AWARE THAT:
Representativeness does not work well with Agile
- Is not Agile
- Stifles innovation
- You have to be able to define everything up front (Not agile)
- Requiers interviewing too many people (Not enough time or budget for this)
- we don’t know our Users/Clients/Buyers yet

b.

12) Visit users

a. Set the scene
b. Explain the purpose
c. Validate the shot list with the interviewee
d. Take pictures

e.

13) Process User research/interviews - Persona’s

1. Share Inspiring Stories & Learnings
 i. Make a List
ii. Identify Any Holes
iii. Share Stories
iv. Capture What Everyone Is Saying
v. Hang Your Notes on the Wall
2. Cluster Into Themes
i. Find the "Gems"
ii. As a Group, Cluster Information Into Themes
iii. Find Supporting Evidence
iv. Refine Your Cluster s
v. Write Headlines
3. Create Insight Statements
i. Write Your Design Challenge

Theme: Payments

Insights:

 Kids often don't use clean toilets (or any toilets at all)
because they don't have any money to pay.
 People often only have money to pay for toilets at the start
of the month after they have been paid.
 Many residents find themselves without money to make a
payment at the exact time they need to use the toilet.

ii. Create “How Might We” Questions

4. PROPERLY SCOPING HMW QUESTIONS
i. How to:
1. TOO NARROW
2. TOO BROAD
3. JUST RIGHT
ii. Examples

State Your HMW Questions

Insight: People often only have money to pay for toilets at the start of the
month after they have been paid.

How might we create more flexible payment options for toilet use?

Insight: People learn best from their friends or family members, but this isn't
how clean hygiene information is disseminated now.
 How might we involve the community in creating and disseminating
information or campaigns about hygiene?

5. Have some desk research done before going to the user

6. Clarify: Goals / Abilities / Motivations

14) User research methodologies (Tips & tricks)

a. To remove biases and make sure research is validated
- Decide on a data logging procedure
- Assign a “Blind” data logger
- Record objective data “Task completion”,
- Double-check your data coding, data entry,
- Let someone read/evaluate your research
- Remove all potential bias: E.g. fake participants that might skew results
- Remove unjustified/false positivity regarding the potential success of a project
- Let stakeholders
- Don’t try and find out if they like it: Try and find out if they can use it and if they’ll use it
b. Prepare, ask:
- If we didn’t run the test, what would be the mail issue?
-
c. Usability tests preparations

Start with a usability test to

- The business objectives (What is the business trying to achieve with this thing and does
the user even want to do this with this app?)
- Define the key user group
- Figure out the key tasks
- Flush out stakeholder

(Can people use what we are giving them?/ Have we designed the right thing? / Did we
solve the right problem?)

d. 6 Types of Usability test tasks

- Scavenger hunt
You know what users want to do
=> Ask users to do something that has one ideal answer
(Tests if users can find the best way to complete the task)

- The reverse Scavenger hunt

You show the users that want to look for x type of product, the end result of the task
you want them to complete and follow them through the journey.
e.g. an image of a shopping cart with a travel bag in it and then let them find the travel
bag.
- Self generated tasks
You don’t know users want to do, or rather you’re not sure
=> Ask them what they expect to do with the product, then show it to them
 (Challenges them on the requirements for what they want to be able to do and what
they are expecting being able to do )
- Part self generated tasks
You define an overall goal for the users and what you want them to do

- “Skin in the game” tasks

You give them e.g. a voucher and give them the freedom to do what they want.

- Troubleshooting
Define a problem and a starting page from which they should start to solve

e. 5 common mistake

1) Talking too much

2) Explaining the design
3) Answering questions
4) Interviewing, rather than testing
(Do not engage in giving confirmation of any kind; eye contact or similar)
5) Soliciting opinions and referencing

f. Observations
(Arrange multiple developers to observe)
i. Ethnography research (Tornburg 4 questions)
Function

Design ethnography vs Traditional

Weeks VS Months/Years
Gain design insights VS Understand culture
Observe and interview VS live with
Just enough to test risky assumptions VS In great detail
Shared with team VS posted in academic journals
ii.
- What is the behavior for (How does it help the user survive and reprod)

iii. Research the problem

Proximate causation
- How does the behavior work?

Origin (How we learned it now

- How did the behavior develop (How did it come to be? Stimuli /
Behavior?
 History of Growth (What was the instigator in history, after and now,
- How did the behavior Evolve? (What has influenced how it
developed?)

e. Pop-up interviews
f. Field observations
i. Build rapport with the users
Let them demonstrate their work/Their activities
Tell me about the first time you started doing this activity
What’s got you interested in … ?
Maybe you could give me a tour of your place?
ii. Transition from a traditional interview to a master-apprentice model
iii. Observe
iv. Interpret
v. Summarize
vi. Build a shotlist and let them review it
g. Context Interviews

Context interviews focus on finding out why people want something, not what they
want. A good contextual interview helps validate your riskiest assumptions
vii. Draft a discussion guide
viii. Capture Motivations: What is the user trying to get done
 ix. Capture: How does the user do it at the moment
x. Capture: What are the pain points / Happy moments with the current
process

Don’t believe in what people tell you, believe in what they show you that they
do

Observe people in how they achieve their goal

Divide people in types

4 to 6 types
Start with 20 users and then refine them

See a context interview as a type of scaffolding

h. Field visits
(Who are our users and what are they trying to do?)
i. Do guerilla research

j.

15) User tasks (Preparation for Usability tests)

k. Ask the team to identify the users top 10 tasks
l. Ask users to prioritise the list (In person or through a survey)
m. Compare
n. ?? Ask Users their own top 10 of user tasks?
o. Turn tasks into usability tests (Zie 15)

16) Run Usability tests

p. First rank priority of tests
xi. How many Users are affected by the problem?
xii. Will users be bothered by the problem repeatedly?
xiii. How should you interpret severity levels?
 xiv.

q. Prepare
xv. Create a document for “Informed Consent”
(The participant knows what to expect, the stress it might cause or things
that might be requested of him or her: e.g. Open about non typical things
(Set users at ease)
 xvi. Guide the participant through the informed consent, ‘highlight’ specific
elements as you check them off individually as you go through:
1. I understand my participation is voluntary
2. I agree to my screen being recorded
3. I agree to my video session being recorded
4. I confirm I understand the purpose of this study
(This document has enough, but not too many items to check off and is
supported by an in-face introduction)
r. For different applications/products
xvii. Test the current product/prototype
xviii. Test the competitor
xix. Run a remote test with friends/family or others
s. Things to know about Usability expert reviews
xx. Use more than 1 expert reviewer
xxi. 1 reviewer gets at most 60% of the usability issues
xxii. 5 reviewers gets 100% of the issues that apply to 33% of the users
xxiii. Using more reviewers is bound to beget different views and different
experience
xxiv. Use technology specific guidelines
xxv. Make sure to have the experience necessary to judge that which is being
reviewed
t. Predicting product design
xxvi. If you know the user goals you can predict the product design
xxvii. If you know why the user is using the product you should be able to predict
the tasks the user will be looking to perform
xxviii. If you know the tasks, you should know where most users are most likely to
lookmost important features that the user will be looking to

17) Get the team to observe

u.
 v. Communicating results to the team members not present

18) Analyzing UX Research results

1. Empathy mapping observed behavior ( Human Centered design)
(A map that is composed of all the information we’ve gathered from all users
through observation, usability testing and interviews.)
 This should include
i. Pains
ii. Gains
2. Process UX research observations
i. Eliminate duplicates
ii. Add the frequency with which each observation happen (How many
participants experienced this
iii. Remove observations that don’t highlight / lead of strengths or weaknesses
of Usability
iv. Create an affinity diagram (Group observations)
5. Plan a Usability test in a timeframe that works for your observers
6. Observe a user performing his testing tasks
7. Have several observers take notes individually with regards to the
Usability poblems
8. Place the Stickies on a whiteboard, as they occur
9. Engage in group affinity sort

v. Process each insight into a sentence with a point of view (Insight

statements)
E.G. : “Users don’t use the same name for things as we do”
(Make them provocative and evoke a strong reaction)
vi. Dote vote on high priority
vii.

19) Draft hypothesis for the issues

 viii. Ask: What’s the smallest simplest change we can make?”

w. Confirm the facts (Find independent confirmation of the facts)

x. Ask:

y. Encourage debate
ix. Is the evidence empirical
x. Does the evidence flow logically
xi. Was the evidence interpreted correctly?
z. Remember that authorities can be wrong
aa. Develop more than one idea (Multiple hypothesis and let data decide)
bb. Keep an open mind
cc. Measure things ( Quantify what you can)
dd. Test every link
ee. Apply Occam’s Razor (When confronted with 2 valid options choose the simpler)
ff. Make sure your hypotheses are testable and can return valid answers
gg. Conduct experiments “Carefully designed and controlled experiments are key”

20) ( order might need to be revised) Validating solutions based on

behavior
hh. Hypothesis testing with storyboards
 ii. Hypothesis testing with a paper prototype
jj. Hypothesis testing with a wizard of OZ
kk.

21) Set UX metrics to see how design measures up against needs of

Users
ll. Write User Stories
mm. Set concrete metrics
xii. Example: Percentage of people who can successfully do .. X .. action
xiii. Clarify what the difference is between achieving 70% and achieving 80%
E.G. is there a quantitative difference in turnover rate?
xiv. D

xv. Lab-based measurement versus Metric based

Lab-based = A/B testing doesn’t give a clear reasoning as to why something
is better or worse
xvi. Is there Coverage error? (Someone that is left out due to cookies… )
xvii. Non response error?
Did your research reach your intended audience or did a part of your target
audience not respond due to the setup/design not being representative and
thus not replied to by your target audience. E.G. X amount of users click
your survey away, instead of replying (either because of unclarity of the
design or technical difficulty… )
 xviii.

xix.
xx. Key metrics to use
10. Ideal sampling size (70% of target audience has replied. Achieving
replies from 1066 = representative number for a target audience of
1 million)
11. Select your target audience from your User Base
12. Time on task
13. Check if a cost or profit loss can be attributed
- Cost: Time of employees spent on an action
- Profit loss: Lacking speed => decreases customer turnover
rate from 80% to 75%= 5%
- Cost: Operating cost
xxi. Expose people to “Real data” that shows a problem and brainstorm with
them on how to solve this instead of discussing what they think Users want
xxii.

2) Putting User needs research in action with the development team

a. Build a UX Dashboard

b. Use Journey Maps and Photo-ethnographies = Pictures of the User context, to

engage stakeholders (By means of Field visits, User interviews, … )
i. Each sticky note contains an observation
ii. Group observations (Darker note for group title)
c. Organize User experience in 5 sequential areas (Discover, investigate, prepare,
apply, use … )

22) The User Experience Debrief meeting

The goal: Determine “What do we do next”

Prerequisites for a successful meeting:

 - Members have observed a user experience study
- Distribute the information that will be presented well before the meeting and have
them read it
- Invite the main decision maker
- Participants know you
- People are in the room with you
- Enable giving ownership
- Provide Participant screening criteria
- You hold a kick-off meeting
- You’ve cultivated an eagerness for the team for achieving optimal usability
- ¼ van de tijd van de meeting = Presenteren van data en feiten
- ¾ van de tijd van de meeting besteden aan discussie
- Focus op verkrijgen van consensus over het probleem
-

- Create

i. Create a sticky note grid (helps to generate design ideas)

1. 5 columns ( User, Goals, Environment, Emotional intent, Design
pattern)
2. List 5 examples of each constraint
3. E.G. => User => 5 verschillende persona’s
Goals=> Various key user tasks/User needs
Environment => Relevant User elements

4.

5.

23) Putting research into action

d. Information radiator
i. Hallway envangelism:
1. Briefly summarises the research results
 - Five facts about the users
- Five findings from Usability
- Five ways people use your products in the wilde
e. Visualize research “Success” or “Unsuccesfull” behavior results in a bar such as the
one below

f. Visualize the time spent on tasks

g. Rating of difficulty

h. Information to add
- summary details about the study
- Representative participant comments
- Statistical comparison to the competitor
- Number of participants
- Task description
 i. EXAMPLE

j.

Guidelines

Idea generation and concept validation

 Apply these methodologies from the perspective “what of this design, lay-out, or expression
of functionality can I …..

 The process
o UX researcher explains the research done uptill that point (concerns: User, goals,
environment)
 Needs
 Constraints
o Sketch potential design solutions (Alone or with 2)
o Explain how the design solves the problem
o Let group critique / Ground critique
o Collaboratively design solutions
 Create a sticky note grid (helps to generate design ideas)
 5 columns ( User, Goals, Environment, Emotional intent, Design
pattern)
 List 5 examples of each constraint
 E.G. => User => 5 verschillende persona’s
Goals=> Various key user tasks/User needs
Environment => Relevant User elements



  Take these constraint
o User Scamper
o Choose most valid ideas
o Perform new iteration (sketch, explain, critique, scamper, choose)
o

Research

 User researcher fallacy: Your job is to understand the user: No, it’s your job to help your
team understand the users!
 Define a single Research question you’re looking to try and answer
 Inadequate research checklist
o Merely gathers data and regurgitates it
o Deals in generalities and superficial surveys, avoiding depth and analysis
o Asks no analytical questions
o Does not advance knowledge, but is happy to summarize what’s already known
o Is boring
 Why Scientific research is not always reliable
o The experimenter effect (70% influences in favor of researcher)
o Researcher effects
 Standardize everything
o The scripts for interviewing
o The same protocol
o Present task scenarios for participants to read aloud
 Have a monitor to watch the interviewer
 Stay out of the participants line of sight
 Run control studies that focus on monitoring biasing behavior
 Strong data (Valid Data measures things such as Completion Rate and efficiency)


Workshop

 Give each person a set of sticky notes. Tell them to imagine that we have an all-knowing,
insightful user outside the room who will answer truthfully any question we throw at them.

Usability testing

 Create a Usability testing checklist that speaks of

o Features that enable User control “Nielsons principles” e.g. : Users need the
freedom and control to turn back and/or exit at wish
o Make sure your checklist is also technology specific
o Example of checklist
 A clear route back to Home
 Check that back button isn’t disabled
o Let testers (Or yourself) gain experience by
 going to a users environment and observing them there
 Let Testers gain experience through observing User testing and the simple
challenges that they commonly face
  Letting them listen in on calls
o Engage one Usability Expert reviwer. (One with plenty of experience that is capable
of identifying showstoppers)
o
o

Observations and research of in context