Usability Test Plan

Introduction
SURF LOOK moderated remote usability test (Mobile)

Background
SURF LOOK is a new surf report app which provides users with surf reports and recommendations customized to their skill level ,
interests, and location. It also provides interesting and relevant content to the surf community and the ability to connect with friends
and the wider surf community.

Goals
The goal of this usability test is to determine if the app is easy to learn and navigate for new users. I plan to observe the user as they
navigate and complete basic tasks in the app to gain a better understanding of SURF LOOK’s usability.

Test Objectives
● To determine if users are able to quickly and easily understand what the app is about and how to navigate through it and use
it’s functions.
● To watch users go through the onboarding process of the app and successfully sign up for a new account.
● To see users find a surf spot and view the surf report for that spot.

Methodology
The tests will be conducted through both remote and in-person moderated tests. Remote tests will be performed via
screen-sharing software such as Skype, Zoom, or Google Meet. The test will consist of a brief introduction and explanation before I
run the participant through several tasks to complete on the prototype as I record the session.
 Usability Test Plan

Participants
There will be 6 participants recruited through my personal network and/or social media platforms.

Schedule
There will be a total of 6 sessions which I plan to have completed by Thanksgiving or earlier. One participant will be scheduled for
each session. Sessions will either be in-person or conducted remotely through screen-sharing software such as Skype, Zoom, or
Google Meet.

Sessions
Study participants will participate in usability testing sessions which will take around 10-15 minutes.

Equipment
Testing will be done on a laptop or desktop computer. For in-person tests, I will provide a laptop for them to engage with the
prototype. For remote participants, they will take the test on their personal computer using the prototype link provided.

Metrics
Errors will be measured using Jakob Nielsen’s scale:
0 = I don't agree that this is a usability problem at all
1 = Cosmetic problem only: need not be fixed unless extra time is available on project
2 = Minor usability problem: fixing this should be given low priority
3 = Major usability problem: important to fix and should be given high priority
4 = Usability catastrophe: imperative to fix before product can be released
 Usability Test Script

Introduction
Hello! Thanks for taking the time to participate in this user testing session. My name is Heather Hopkins and I am the designer who is
working on this project. During this session, I will be observing and taking notes.

Before we get started, I would like to explain a little bit about the project and what we will be doing today to help you better
understand what to expect and give you a little context on the project.

Today, you will be trying out a new surf report app called SURF LOOK and we will do a few activities using the app. The goal is to
understand what works well and what does not work well in the app design. That is where you come in—I would like to hear your
honest thoughts and opinions. Please remember that you are not the one being tested…there are no right or wrong answers to
anything, and please do not worry about hurting my feelings. Your candid feedback will help me to make improvements to the app.

We will start out by asking a few general questions about demographics and your usage of apps in general before moving on to
completing a few activities using the SURF LOOK app. It would be helpful if you could “think out loud” when going through the app
so I can gain insight into what you are thinking and feeling as you explore the app. For example, you might let me know if you click
on a button in the app and encounter something different than what you had expected to find.

If at any time during the session you happen to find yourself confused or unsure as to what to do next, that is perfectly fine. Just let
me know and I will assist you in getting back on track. Feel free to let me know at any point if you need a break, or if you would like to
end the session early.

Finally, before we get started with the session, I would like to ask if it would be okay to record the session? This recording will only be
used to go back and reference your insights and make improvements to the app and will not be shared anywhere. Is that ok?
Let me know if you have any questions at all.
 Usability Test Script

Are you ready to continue?

Ok, let us get started. I will begin recording now.

Demographic Questions
What is your age group?
● 18-29
● 30-40
● 41-50
● 50-60
● 61+

Where do you currently live?

What is your current occupation?

Background Questions
● What device do you currently use the most, mobile or desktop?
● Do you ever use weather apps or surf report apps, or visit weather or surf report websites? If so, which ones do you frequent?
● Have you ever looked at a surf forecast? What was your experience with it?
● Do you surf, or have you ever tried surfing? If so, tell me about that experience.
 Usability Test Script

Open-Ended Questions
Thank you for answering those questions. Now, we are going to move on to the SURF LOOK app. I will start out by getting your initial
impressions of the app.

Take some time to look over the home screen of the app. Without clicking on anything, just let me know your initial impressions of
the app. Let me know what you like and what you do not like about it, how the information is displayed and organized on the page,
and any other general thoughts you may have. Think out loud if possible.

Can you tell me what you think the purpose of this app is by just looking at the home screen?

Ok, let us move on to a few activities…

Tasks
I will now give you some scenarios and a corresponding activity to complete for each scenario. As you go through each activity,
remember to think out loud and tell me what you are thinking and feeling as you go through the app.

● Scenario 1: You just heard about this new app, SURF LOOK, and you decided to download it and try it out. You just finished the
download. Please sign up for the app and complete the onboarding process.
● Scenario 2: You are on vacation in Encinitas, California. You heard that there are some good surf breaks there and you would
like to check one or two of them out while you are in town.
○ Please search for a good local surf spot in Encinitas, CA
○ Go to that surf spot’s page
○ View today’s surf report
 Usability Test Script

● Scenario 3: You are lounging at home on the couch and you have a little bit of time on your hands, so you decided to check
out the latest news in the surfing world. Please read a news article on the app.

Please spend a little bit of time just clicking around the app wherever you would like and let me know if you have any additional
impressions.

Final Questions
● While you were going through the test, did you find anything to be difficult?
● Do you have any general thoughts and feelings about the app or things you liked or disliked?
● Would you change anything about the app?

Wrap-Up
That is all! Thank you for all your valuable insights. Your participation will really help me to improve SURF LOOK. Before you go, do
you have any other questions or feedback that you would like to give?
 Participants
Location: Seal Beach, California
Laura (P1)
Demographic: Female, Native
Californian
Surfing experience: Used to surf
occasionally when she was
younger, still goes boogie
boarding occasionally. Loves
kayaking.
Testing method: In-person
Norwood (P3) Location: Oceanside, California
Demographic: Male, Native
Californian
Surfing experience: Occasional
surfer currently, used to surf
every single day when he was
younger. Also goes kayaking,
boogie boarding, snorkeling, and
just swimming in the ocean.
Testing method: In-person
Gregory (P2)
Location: Seal Beach, California
Demographic: Male, Originally from
Pennsylvania but has lived in California
for a long time
Surfing experience: Occasional surfer.
Also some experience windsurfing
and loves to go kayaking.
Testing method: In-person
Geraldine (P4)
Location: Rancho Palos Verdes, California
Demographic: Female, Native Californian
Surfing experience: Has surfed
occasionally. Grew up by the ocean with a
surf-obsessed older brother. Still lives by
the ocean, but mostly just does whale
watching these days.
Testing method: Video recording via
Zoom.
 Participants

Kerry (P5) Tiffany (P6)

Location: Philadelphia, Location: San Diego, California
Pennsylvania Demographic: Female
Demographic: Female Surfing experience: Not a surfer
Surfing experience: Not a surfer Testing method: Video recording
Testing method: Video recording via Skype
via Skype
 Affinity Map
Sign up and onboarding
process went smoothly
Easily found surf report
from the surf spot page
Should the logo have a
wave in it?
Liked all the icons on the
onboarding screens
Thought “fun board” should be a
favorite type of surf option
Maybe a slight shadow
on search icon?
Observations
Why would people want
to skip location services?
Onboarding was
comfortable and familiar
Was very easy to find the
news section
No problems getting
through profile setup
Easily found the news
section and articles
Why does it say ‘favorite
surf spots’ at ‘today’s surf’?
Thought community and
webcams were good
additions
Went to ‘today’s surf’ on home
page when asked to search
for a surf spot
Should have surf near you and
then your favorites below.
No problems finding the
news section.
She was drawn to the
travel news.
Loved how visual it all was
with the photos
I liked knowing about
parking and other
information
I need to add a filter on
‘today’s search
both of the news buttons
should go to the main news
page
Maybe a search bar
would be better.
It could say ‘search for a
surf spot’.
 Affinity Map
What is the setting that
determines what you see when
you click ‘today’s surf?
Maybe it shows closest by
location and your top spots
below?
Does it matter what your
level is set to?
How do the filters work? Is it the
‘excellent for you’ on the surf
report?
Maybe have a little question mark or
‘learn more’ button you can click that tells
you why it’s good for you or bad for you
today.
Maybe there can be something
that lets you change the location
you are searching on homepage.
Observations
Thought the search bar could be
improved by making it more like
it is on the actual search modal
On surf report page, I
immediately see the current
conditions are excellent for me.
I like the at a glance page for
the surf report and how you
can click to see more info.
When I get down to the details
section, my instinct is to click on
the photos to get even more.
(Surf Report) Could be something
that pops up like modal, or expands
to give you more.
It might be nice to have a marker on the
wave height graph that you can click the
times and see a little bubble pop up and
give you more info.
Wondering if I’m a new user
and I click today’s surf, does it
take my current location?
Good way to do it would be ‘near you’
and then maybe below shows your
favorites, or if you don’t have any yet, a
prompt to go heart some places.
Maybe on the news section you
could possibly have less categories.
Maybe some sub-categories?
 Affinity Map

Positive Quotes

Onboarding is easy and Home page makes sense Liked the layout of news The setup at the beginning
intuitive page was straightforward

Liked ‘future plans’ on Onboarding process was Profile, news, community It was simple and not too
home page good pages look good much going on

Surf journal could be called The community page Liked parking info b/c it’s
Home page is very
‘surf and travel journal’ layout is good hard to park at the beach.
visually appealing

Enjoyed the travel-related The sliders on the search Liked the ‘excellent for
I love the logo!
parts of the app page is very clear you’ icons

Liked the way search had Search page is not It was good to show surf
I like how you can sign
the sliders overwhelming for your skill level
up with social or email

Grid pages you don’t Liked how you can click

The profile page looks The onboarding graphics
need to scroll are ok surf report for more details
good and photos look great
 Affinity Map
A red exclamation point
is extremely clear
The font was big enough...and
I’m a senior!
There was a good variety
of things to do in the app
Loved the logo
Onboarding process was
good
I like that I can go through a
Facebook or Google account
that I have.
Positive Quotes
Really loved the visuals,
color scheme, and images.
She said it was
aesthetically beautiful.
First impression is that
everything looks clear and I
know that this is a surf app.
Login and sign up entry
points is really clear to me.
Wondered what the surf
webcams were
the quick overview was good
to give me an introduction
without being too much.
I love the icons that show if
the surf is good for me or not.
I like the simple, initial
overview of the weather
report.
I love that you can
connect to google maps.
I like the look of the
profile page.
Notification labels on the
community page are nice.
I’m feeling inspired looking
at the beautiful pictures!
the look is incredible.
I don’t feel like there are too
many clicks or anything.
Even for me as a non-surfer, I’m
able to look at this app and intuit
what everything means.
It’s very welcoming, branding
is clear and looks nice.
Calls to action are clear.
First impressions of the
app are good.
 Affinity Map

Positive Quotes

I like that I can sign up via

social or email. I would
choose email.

Completing profile went

smoothly and easily
navigated through it.

Had no problems finding and

using the search function

It’s cool it gives me the icons

for when it’s good and when
it’s not.

Completing profile went

smoothly and easily
navigated through it.
 Affinity Map
“What’s your surfing level’
better way to phrase profile
setup question
Wants to see more travel
stuff on the app
‘Today’s Surf’ should
show surf report at top
Search could say, “Where
do you want to surf?”
Today’s surf report should
have bigger report at the top
Think about hierarchy on
surf report page
Negative Quotes
Make full width tiles on Search bar is probably the most
important thing that needs to be
pages you have to scroll fixed. Needs to be more clear.
Says the word ‘search’ too On ‘today’s surf’ it should
much on search page have option to change the
location.
Found search easily but What if the location I want
thought could be better to look at isn’t on my list?
Need to indicate search Forecast shouldn’t just show favorite
surf spots, should also show near
is a touch target me with option to change location.
One thing should have On the onboarding screens,
prominence on surf report the body copy is a little too
long for me.
X on search should be more
similar to back button
 Affinity Map
Some issues with
non-clickable aspects of
prototype
Had trouble finding the
search function
Search icon wasn’t
immediately intuitive
First instinct was to go to
‘today’s surf instead of search
Errors
Make sure to make all aspects
of a news article clickable.
There’s a typo on the
community page…’friendsl”
Easily navigated through the news
section. It’s intuitive and I knew
where to go exactly. No cons about
that.
 Conclusions, next steps

● The search bar needs to be made more intuitive and easier to find. All 6 participants had feedback on
this, or had trouble finding it. I think it needs to be made into an actual search bar with prompt text in it
rather than just an icon as it is now.
● Make the descriptions in the onboarding flow a bit shorter.
● Surf reports and surf spot page should have main surf report for today as main prominent item and other
items about the spot less prominent. After all, we’re here to get the surf reports first and foremost.
● For any pages that you have to scroll through a lot of tiles on, might be better to make them look more
like the main news page with the full width tiles.
● Today’s surf should have today’s surf near you, with option to change location at the top, and then show
maybe a few favorite surf spots below.
● Fix a typo on the community page.
● News tab on home page should go to main news page and not just the latest news.
● Some helper text for why surf is ‘excellent for you’ or ‘bad conditions for you’ would be helpful.
● On surf report ‘more details’ make something happen when you click on the photos to show more
details.
● If possible, marker on the wave height graph to give more info at different times of day.
 Session Notes

● Overall, it went well. Three of my sessions were in-person done over Thanksgiving, which was nice. I was
able to take a video with my phone of the session and that worked well.
● The other three sessions were done via Zoom or Skype. No technical difficulties with two of the sessions
and got good videos, but my third remote session with Tiffany had some technical difficulties. She
shared her screen with me but for some reason the prototype wouldn’t show. I saw her mouse clicking
on a desktop but no prototype. We decided to continue with the session and I would just record her face
and audio. She did a great job describing to me exactly where she was in the app, and I followed along
on the prototype on my end to see what she was seeing, so it worked out well. I have a video where I
can see her talking and walking me through it.
● Overall, it was a good experience and I definitely got some opinions that were different from various
people, but also uncovered some themes as I went along. The biggest issue was the search
function….everybody either had a comment on it being a little tricky, or in the case of some participants,
didn’t actually end up clicking on it, so that needs to be fixed for sure.
● I got a few other good insights as well that made me think about how I might want to improve the app.
 Usability Test Report: Issue #1 - Search functionality is not intuitive

Search function was not intuitive for Before After

users. Severity: HIGH
The search function needs to be made
more intuitive and easier to find. All 6
participants had feedback on this, and
some of them had trouble finding it at all.

Suggested Change
It needs to be made into an actual search
bar with prompt text in it rather than just an
icon as it is now so it is easier for users to
find.

Evidence
The search icon was not intuitive enough
and some commented that it wasn’t
immediately clear it was clickable. I noticed
also there was a generational divide...older
users had more trouble figuring out how to
search than younger users. This needs to
be rectified so surfers of all ages can use
the app. A search bar is a familiar solution
more people will already understand. Changed the search icon into a search bar to make it more clear - did this on all
pages in the app.
 Usability Test Report: Issue #2 - Add Location Setting Option to Today’s Surf

Today’s Surf should have a location Before After

setting option. Severity: HIGH
Today’s surf should have today’s surf
near you, with option to change location
at the top, and then show a few favorite
surf spots below.

Suggested Change
Add an option at the top to set your
location, or choose from your favorites
below.

Evidence
It became clear during the test that we
were only allowing people to see the
surf at their favorite spots on the main
surf report page, but many people may
want to see the surf for a number of Added location setting for surf report to the main tile, and changed clicking
locations at any time. It isn’t good ‘today’s surf’ to more location-based surf report page with favorites below rather
usability to make them go search first. than just sending to favorite surf spots for user to then see the surf report. Users
can change location on the page if they want to.
 Usability Test Report: Issue #3 - Surf Report more visible on some pages

Surf Reports and Surf Spot page Before After

need more visible surf report.
Severity: MEDIUM
Surf reports and surf spot page should
have main surf report for today as main
prominent item and other items about the
spot less prominent. After all, we’re here
to get the surf reports first and foremost.

Suggested Change
On the surf reports and surf spot pages,
add a prominent surf report tile at the
top with basic surf report details.
Evidence
It was pointed out during testing that
since this app’s primary function is to be
a surf report app, the surf report needs
to be front and center and easy to Added bigger, more prominent surf report tile to the top of the surf spot page so
access at all times. users can see the report at a glance without the extra click - though if they click
they get to see all the details, this just gives them a quick glance
 Usability Test Report: Issue #4 - Helper text...why is surf good or bad for me?

Surf Ranking needs some helper Before After

text. Severity: MEDIUM
Some helper text for why surf is
‘excellent for you’ or ‘bad conditions for
you’ would be helpful.

Suggested Change
Add a place for a blurb describing why
the surf is good or bad for the user on
the surf report, next to the icon.
Evidence
Testers pointed out that it was cool that
the app pointed out if the surf matched
their abilities and interests as reported in
the app, but that it would be more
useful if it also explained why the
ranking was given. Are the waves too big
and dangerous? Are the waves a bad Added some helper text to ‘excellent for you’ or ‘poor conditions’ type tags so
match for their board type? users can have a better understanding of our ratings system
 Usability Test Report: Issue #5 - Make the descriptions in the onboarding shorter.

Make the descriptions in the Before After

onboarding flow a bit shorter.
Severity: LOW
The descriptions in the onboarding flow
are too long, according to several
testers.
Suggested Change
Shorten them up to make them more
succinct.

Evidence
Although this is a less severe usability
issue, it was pointed out to me that if
there is too much text, people might not
read the instructions and might not
know what the app can do or what they
are supposed to do next because they
Shortened these longer descriptions so that users don’t have to read so much as
will skip these steps in onboarding.
they scroll through the onboarding process
TOOLS
Usability Hub was used for the main test of the splash screen/start page. However, I have a large
network of friends who are artists and designers (and surfers), so I ran an additional, more
informal poll on my Facebook page to test this screen plus changes to two more and
received some great feedback as well, so I will include those here in addition to the
Usability Hub results.

SCREENS TESTED
Screen 1: Splash screen/starting page
Screen 2: Complete Profile? (from onboarding)
Screen 3: Community page

NUMBER OF PARTICIPANTS
Usability Hub Test of Splash/Starting screen: 19 participants
Facebook poll of all three screens: 17 participants
 The Usability Hub Test

For the Usability Hub test, I chose to focus on the

main screen that is the first thing you see when
you open the app, the splash/starting screen. First
impressions are important, so this is the one I
focused on the most. I got 19 participants to take
the Usability Hub test, who I recruited on both the
CareerFoundry Slack forums as well as in a
UX-specific online group I am in. It did not take
long to get 19 results, so I can see how Usability
Hub would be a great way to gain fast opinions on
a design in a very short amount of time. At first
glance, The more complex design with the van
image is the more popular, but when I dive into the
comments expressing why people chose the one
they did, it seems that people tend to like the
image of the water better, but they like the layout
of the elements on the page better in the winning
image. This gives me a lot to think about and leads
me to believe that I should combine the best of
both worlds and set up a version that includes the
things people liked from both of the options. This
was an interesting experience and quite valuable
for me as I design this app.
 The Facebook Poll

In addition to running a test on the splash/start screen on Usability Hub, I also ran a more informal poll with my friends and family on
Facebook for the splash/start screen as well as two other screens in the app to take advantage of the insights of my network, which
includes a lot of creative and tech professionals as well as many people who are into surf and watersports. While I loved the feedback
and clear interpretation of the data I got from the Usability Hub study, I also found quite a bit of value in this more informal poll as I feel
the format encouraged people to express more of their opinion and in some cases leave some very detailed and insightful comments. In
all, I had 17 participants leave me feedback before I compiled and calculated the data, and they gave me quite a bit of insightful
feedback that will help me to improve these three screens on the app.
 Test #1: Splash Screen/Starting Page - Usability Hub and Facebook Poll (combined)

Version 1 Preferred by 56% Version 2 Preferred by 44%

I averaged the percentages of the As a result of running these
two tests for this one. All in all, tests, I have learned that
there was a bit of a split between the best course of action
these two versions with some moving forward with this
people strongly preferring one or page will be to use the
the other, and some feeling torn version 1 layout but replace
about which one they liked best. the image with the one
The most interesting thing about seen here in version 2,
running this test was, in fact, the since that combines the
comments I received on both the elements that people
Usability Hub test and the reacted most strongly to
Facebook poll. I started to notice across both of these tests.
a trend in the responses I saw,
with most people preferring the
water image in version 2 to the
van image in version 1, but also
preferring the layout of the text,
buttons, and logo on version 1.
 Test #2: Complete Profile? Facebook Poll
Version 1 Preferred by 59%
This one had a fairly clear
winner, with 59% preferring
version 1. However, when I
dove into the comments,
there was a clear trend.
People chose version 1
more often because they
liked the image. However,
people also preferred the
messaging of version 2.
Version 2 Preferred by 41%
Changes I will make will be
to change the text on
Version 1 to be more like
the text in version 2. For the
‘skip’ link, I will have it say,
“I’ll skip this for now” as the
feedback was that the
messaging was better, but
that the text was a bit long
and could be shortened.
 Test #3: Community Page. Facebook Poll

Version 1 Preferred by 18% Version 2 Preferred by 82%

This one was crystal clear. I will be changing the
People strongly preferred layout in the prototype for
version 2 over version 1 for this page to version 2.
this screen.
 Summary and Changes Made

I combined both versions and kept the Kept original image but changed button There was a strong preference for this
text and buttons of the original with the and skip link messaging since people version, so I updated the prototype to
image from the new version. liked that better. use this version.