[Name of Web Application/Site] Test

Taylor Whitmire

[September 23, 2023]

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 1
 Introduction
Williams-sonoma.com is a site that serves to drive traffic to their products
and brand. They provide their consumers with cookware, furniture, bakeware,
home essentials, and many items of the like.

A usability test is intended to determine the extent an interface facilitates a

user’s ability to complete routine tasks. Typically, the test is conducted with a
group of potential users either in a usability lab, remotely (using e-meeting
software and telephone connection), or on-site with portable equipment.
Users are asked to complete a series of routine tasks. Sessions are recorded
and analyzed to identify potential areas for improvement to the web site.

A digital marketing student conducted a usability test using the site Williams-
sonoma.com on their laptop. The same laptop was used to record the
participant’s navigation choices. Furthermore, a phone was used to capture
the participant’s face and comments. The test administrator was present in
the testing room, and took note of the participant’s navigational choices, time
the test took, comments, overall satisfaction ratings, as well as questions and
feedback.

Executive Summary
The digital marketing student conducted a usability test in a quiet room in
their home on September 23rd, 2023. The purpose of this test was to assess
the usability of the web interface design, and to test how easy it was to
navigate the site as a first-time user.

For this test, one person participated, and it took approximately 40 minutes.

The test identified a couple problems, including the following:

• Too many menus option
• Too much copy and photos on the homepage, no clear CTA
• Too many ads in the checkout process

This document contains the participant feedback, satisfactions ratings, task

completion rates, ease or difficulty of completion ratings, time on task, errors,
and recommendations for improvements. A copy of the scenarios and
questionnaires are included in the Attachments’ section.

Methodology
The test administrator was me, and I recruited the participant, my husband,
by asking them in person. We confirmed a time and place that the test would
be conducted.

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 2
 The usability test took 40 minutes. Before the test began, I gave an
explanation about how the test would go and the overall objective.

After each task, I asked the participant to rate the interface on a 5-point
Likert Scale with measures ranging from Strongly Disagree to Strongly Agree,
which included:
• How easy it was to find the information from the home page.
• Ability to keep track of their location on the website.
• Accurateness of predicting which section of the website contained the
information.

After the last task was completed, the test administrator asked the participant
to rate the website overall by using a 5-point Likert scale (Strongly Disagree
to Strongly Agree) for eight subjective measures including:
• Ease of use
• Frequency of use
• Difficulty to keep track of location in website.
• Learn ability - how easy it would be for most users to learn to use
the website.
• Information facilitation – how quickly participant could find
information.
• Look & feel appeal – homepage’s content makes me want to explore
the site further.
• Site content – site’s content would keep me coming back.
• Site organization.

In addition, the test administrator asked the participants the following overall
website questions:
• What the participant liked most.
• What the participant liked least.
• Recommendations for improvement.

Participants

There was one test participant, on September 23rd, 2023. The participant was
a 26-year-old, working male (not a student).

Role
Federal State / Public Federal Medical Research * Other
Staff/Agency Health Department Grantee Institution Institution Organization

0 0 0 0 0 1

Evaluation Tasks/Scenarios
The test participant attempted the following tasks:

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 3
 • Your friend is hosting a pizza party and you want to purchase a
pizza stone to bring to the party. Go through the process of
selecting and purchasing a stone under $70. Please stop right
before you enter your credit card/payment information.
• You want to buy a new espresso machine. Use the filters for the
brand and select “Nespresso Espresso” and choose one under
$300 to purchase. Like last time, please stop right before you
enter your credit card/payment information.

Results
Task Completion Success Rate

The participant did not take long and completed task 1 (finding a pizza stone)
successfully. He found it quickly.
The participant completed the task but did not understand what I meant by
“use the filters to find the brand.” He figured it out quickly though.
Did he complete task 1, 2, and 3?? Explain and summarize each task.

Task Completion Rates

Participant Task 1 Task 2

1 √ √

Success 1 1

Completion
100% 100%
Rates

Task Ratings
After the completion of each task, participants rated the ease or difficult of
completing the task for three factors:
• It was easy to find my way to this information from the homepage.
• As I was searching for this information, I was able to keep track of
where I was in the website.
• I was able to accurately predict which section of the website
contained this information.

The 5-point rating scale ranged from 1 (Strongly disagree) to 5 (Strongly

agree). Agree ratings are the agree and strongly agree ratings combined with
a mean agreement ratings of > 4.0 considered as the user agrees that the
information was easy to find, that they could keep track of their location and
predict the section to find the information.

Ease in Finding Information

The participant found it easy to find the pizza stone that I told him to find, but
it took a little longer for him to navigate to the espresso machine.

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 4
 Keeping Track of Location in Site
While finding the pizza stone and the espresso machine, the participant found
it easy to keep track of his location in the site.

Predicting Information Section

The participant agreed it was easy to predict where to find information. At the
top of the site, they have a menu that shows what they sell, making it easy to
find where different things may be located.

Test 1 – Mean Task Ratings & Percent Agree

Ease – Location in Predict
Task Overall
Finding Info Site Section
1 – Find pizza stone 4 5 4 4.33
2 – Buy new espresso
4 3 3 3.33
machine
*Percent Agree (%) = Agree & Strongly Agree Responses combined

Time on Task
The testing software recorded the time on task for each participant. Some
tasks were inherently more difficult to complete than others and is reflected
by the average time on task.
Task 1, finding the pizza stone under $70, took 62 seconds. Moreover, task 2,
finding a new espresso machine to buy, took a bit longer, at 75 seconds.

For example:
Time on Task
P1 Avg. TOT*
Task 1 62 62

Task 2 75 75

Overall Metrics
Overall Ratings
After task session completion, participants rated the site for eight overall
measures. These measures include:
• Ease of use
• Frequency of use
• Difficulty of keeping track of where they were in the site
• How quickly most people would learn to use the site
• Getting information quickly
• Homepage’s content facilities exploration
• Relevancy of site content
• Site organization

The participant though the site was easy to use. The user had no use to go
back to the site in the future. The participant thinks it is easy to keep track of
where they are. The participant thinks that most people would not learn very

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 5
 quickly how to use the site because there is a lot going on and it is very
simulating. They think that it takes a minute to navigate where you are
wanting to go on the site, so you can’t get information very quickly. The
participant thinks there is too much going on so there is no encouragement of
exploration – the viewer might get too overwhelmed. They do think that most
of the content is relevant.

Post-Task Overall Questionnaire

Strongly Strongly Mean

Disagree Neutral Agree
Disagree Agree Rating

Thought Website
4 4
was easy to use
Would use website
1 1
frequently
Found it difficult to
keep track of
2 2
where they were in
website
Thought most
people would learn
2 2
to use website
quickly
Can get
3 3
information quickly
Homepage’s
content makes me
2 2
want to explore
site
Site’s content
would keep me 2 2
coming back
Website is well
3 3
organized
*Percent Agree (%) = Agree & Strongly Agree Responses combined

4.6.2 Likes, Dislikes, Participant Recommendations

Upon completion of the tasks, participants provided feedback for what they
liked most and least about the website, and recommendations for improving
the website.

Liked Most
The following comments capture what the participants liked most:
The participant likes that the menu bar at the top of the page has all the
different kinds of things they sell, making it easy to navigate.

Liked Least
The following comments capture what the participants liked the least:
The participant does not like that there were so many things going on in the
homepage. It made it difficult for him to enjoy the experience.

Recommendations for Improvement

The participant has these recommendations for improvement: Less stuff on
the front page/a more focused CTA. Less options in the header.

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 6
 Recommendations
The recommendations section provides recommended changes and
justifications driven by the participant success rate, behaviors, and
comments. Each recommendation includes a severity rating. The following
recommendations will improve the overall ease of use and address the areas
where participants experienced problems or found the interface/information
architecture unclear.

For example:
Find Organizational or Individual Funding Information (Task 2)
Task 2 required participants to find organization funding (Test 1) or individual funding (Test 2).

Change Justification Severity

• Less copy and photos on the • The participant found it to High

homepage be too overstimulating for
• Less options in the header consumers.
• The participant thinks
there’s too much to look at

Conclusion
Implementing the recommendations and continuing to work with users (i.e.,
real lay persons) will ensure a continued user-centered website.

He found Williams-sonoma.com to be comprehensive and easy to navigate

but would be worked on regarding how stimulating it was by removing copy
and photos. Overall, it was user friendly, but just had a lot going on from the
participant’s perspective.

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 7