USABILITY TESTING

Mirela Gutica
Comp 4952
Technical Programming
Computer Science Technology
BCIT
 Studies done with real users
 The study should be planed carefully
 A usability report should be produced

USABILITY TESTS/STUDIES
 Formative usability
 Usability studies done along with the project
design and implementation
 Measure effectiveness, productivity, satisfaction
with the purpose of updating the design
 Questions to answer:
 What issues were identified by users?
 What worked well? Not so well?
 What improvements were suggested by users?

USABILITY TESTS/STUDIES
GOALS (1)
 Summative usability
 Final usability study before launching the product
 Measure to what extent the final product meets the
requirements and users’ needs
 Questions to answer:
 Does the product meet the goals of the project?
 What is the overall usability of the product?
 How is the product’s usability in comparison to similar
current products?
 What improvement can be done in the future?
 Improvements can be done in new releases of the product

USABILITY TESTS/STUDIES
GOALS (2)
 Traditional moderated usability tests
 In a lab
 One or more moderators
 One-on-one between a participant and a moderator
 Online usability tests
 Could be moderated, but usually are unmoderated
 Online surveys
 Focus groups

EVALUATION METHODS
 Completion
 Time to complete a task
 Effort to complete a task
 Comparison
 Similar product(s)
 Alternative design
 Evaluation: e.g., give a task (e.g., find a certain information) and observe how
the user finds that information)
 Frequency of use
 Productivity (e.g., number of clicks, easiness to identify the next step, etc.)
 Navigation
 Impact of changes
 Measurement
 Noticeability
 Usability of a critical point

USABILITY METRICS
 The methodology that you will employ for data collection and
analysis will inform your approach, process and methods used to
analyze your data
 There are three methodologies:
 Quantitative
 Data is collected from a large number of participants (30+) via surveys
or focus groups
 Qualitative
 Data is collected from fewer participants (6) via interviews and/or
observations
 Mixed-methods
 Mixed methods are effective and can combine two or more sources of
data

DATA COLLECTION AND

ANALYSIS
  Executive summary
 Purpose of the study
 Participants
 Method
 Evaluation methods
 Usability metrics that were collected
 Location and settings
 Duration
 Etc.
 Findings and recommendations
 Discussion
 Appendices
USABILITY REPORT
 Include the questionnaire, survey, data collected, etc.
 The purpose of the study (an explanation of why the study is being done).
 The procedure being used for the study. This section should also include a time
expectation for the participant and the protocol for requesting a break.
 If there will be any type of recording, who will see the recordings, and what happens
to the recording material when the testing is completed (not all studies involve
recordings).
 A statement of confidentiality and how the anonymity of the participant is
preserved.
 Any risks to the participant (in most usability studies there is minimal risk).
 The fact that participation is voluntary and that the participant can withdraw at any
time with no penalty.
 Whom to contact with questions and for any further information after the study and
a statement that initial questions about the testing have been answered satisfactorily.
 (Dumas and Loring, 2008)

INFORMED CONSENT
GUIDELINES
 Tri-Council Policy Statement: Ethical Conduct for
Research Involving Humans – TCPS 2 (2018)
 https://
ethique.gc.ca/eng/policy-politique_tcps2-eptc2_2018.html
 Research ethics policies at BCIT
 https://www.bcit.ca/applied-research/research-support/resear
ch-ethics
/

RESEARCH ETHICS