User Test Report

Monica Gomez
Madeline Franchock
Andy Almquist
Kyle Winkler

Table of Contents
 1

Introduction.2

Executive Summary...
..3

Methodology...3

Participants

Evaluation Tasks

Results

Task Complete Success Rate

Errors

Test Reader Comments

Recommendations

Conclusion

Likert- Scale Test Questions

Observation Sheet for User Test

Follow-Up Interview Questions for Readers

Introduction
The best part about legos is the ability to build anything. However, it can get annoying
real fast if people just cannot seem to get the final product. The instruction set we created was to
 2

the formation of a seaplane. There were a fair amount of small pieces that have to be attached
just the right way or else the seaplane would not be correctly put together. We split the building
into different sections. First we described how to build the bottom section of the plane and then
ended with how to correctly attach all the different parts.
A test was provided to ensure our instructions were easily readable and people could
build the plane in a timely manner. To conduct our test, a test subject was needed. A group of two
were given the instructions to building the seaplane and were asked to follow along with them.
We then took notice of how long it took them to build the plane and what parts of the instructions
they got tripped up on. We then asked the group to provide feedback as to what we could
improve upon to make sure there were little to no flaws in the instruction set.
 3

Executive Summary

Our user test day took place March 8, 2017 at 11:45 am. The test was
conducted in Bachelor hall, room 252. The purpose of the user test was to see if the
instruction set our group created was efficient enough to build the desired structure
we had chosen. Our group took a couple weeks working on creating a set of
instructions that were going to be used by a test group. The user test was also
created to survey if how real users were going to interact with the our product. The
benefit to a usability test is to possibly make any changes to the product to improve
it. You get direct feedback on what works with your product and what might need
to be changed to make it better.
On test day, there were two participants working with our product and
provided set of instructions. This was a smaller group than what we had expected
to be working with. We were given the duration of the class (one hour and 20
minutes) to complete our user test reports. The user test began roughly at 11:45am.
The user test group completed the structure in about 20 minutes (12:05pm). As a
group we were satisfied with the time it took our test group to complete the
product.
Overall, our candidates experience for our user test was positive. They were
able to complete the specific task that was asked of them. They seemed to be able
to follow our written directions somewhat easily. Our instructions did not include
any pictures. We were worried that might inhibit our testers ability to construct the
figure. The outcome proved not including the pictures did not have a huge affect
on the outcome. We do plan to include pictures in our final draft of the instruction
set.
Problems that occurred during the user test:
Which direction certain pieces had to be placed
Not including pictures to help user complete the product more efficiently
Wording of our instruction set was a bit difficult to understand at times, may need
to simplify instructions.
The document that we provided was a step by step set of instruction on how
to create a LEGO seaplane. We broke the instructions into different parts of the
plane (i.e. body, wings, landing skis). Once each of the individual parts were built,
 4

we had the user attach each portion together to create the final product. We thought
this was the best process for constructing the product. There are a lot of small
intricate pieces involved with building this product. Breaking the piece into
smaller portions resulted in a less overwhelming task.

Methodology
Participants were randomly selected from our classroom. We asked
participants to create the lego sea plane using only our instruction set--which, at
the time, did not include any pictures. Thus, the test session to nearly 20 minutes as
participants struggled to understand some of the wording in our instruction set.
Specifically, it was particularly difficult to distinguish between each lego piece
without pictures; thus, our instruction set was very wordy, and at times hard to
follow. However, the participants did end up finishing the sea plane. We knew
there needed to be changes made to the instruction set; pictures were added and the
wordiness of the set was cleaned up. I have included out post-test questionnaire
and likert scale below:

Problem Interpretation Solution

(What difficulty (What might (What might
did the reader have caused the prevent this
have?) difficulty?) difficulty?)
Knowing exactly Not including We went back
where to place pictures along and made our set
the pieces. with our written more efficient by
instruction set. deleting
unneeded words
and including
pictures.
Distinguishing The directions Adjusting some
between each were unclear. wording and
unique piece. making the
directions more
understandable.

1. Observation sheet:
2. Likert-Scale Questions:
a. I am confident I can build the seaplane.
Agree
b. The instructions provided were clear and easy to understand
 5

Somewhat agree
c.I had no problems building the seaplane
Agree
d.I am satisfied with the outcome of the seaplane.
Agree
3.Follow up questions:
a.Did you find the instructions provided to be clear and useful?
The instructions were not very clear, they were wordy and confusing at times.
b.Would you recommend this product to someone?
I would recommend this product, it was fun to build and looked unique.
c.Would you purchase this item again?
I would purchase the item again.
d.Did you enjoy building the seaplane?
I enjoyed building the seaplane; although, it could have been a lot easier had there been pictures.
What is something our company can improve on?
Making the instructions more clear.

Participants
Our user test day was Wednesday March 8, 2017. On user test day, our test
group we were working with had two members in it. There were originally suppose
to be three but one dropped out. Our two user test participants included one female
and one male. The two team members seemed to work well together. One read the
instructions, while the other partner built the object. They sometimes took turns
alternating between who read and who built. Their strategy seemed to be efficient.

Evaluation Tasks

Results

Task Completion Success Rate

Tasks Easy to Somewhat Neutral Somewhat Very difficult

comprehend easy to difficult to to
comprehend comprehend comprehend

Wings X
 6

Skiis X
Body X

Time on Task
 7

Errors
One of our team members was responsible for observing the participants
create the seaplane. Whenever participants became stuck or confused, Andy would
make a note. Our reason for doing this was so that we could focus on places in our
instruction set where we could improve participant efficiency.
The most errors were made while creating the bottom of the plane. This is
because there are a lot of unique pieces used and the configurations of pieces can
be easily mixed up. While our group was creating the plane ourselves for the first
time, this is where we struggled the most as well. At one point in the trial, I thought
the participants would not complete the plane at all. One participant even started
looking at the packaging that the lego set came in to guide his building process--
rather than following our instructions.
There were not any critical errors that were made during the building
process. However, although the plane was completed, it was not put together in the
completely correct manner. For instance, the wings of the plane were put on
backwards, along with other minor, miscellaneous errors. Ultimately, the plane was
created and closely resembled the correct seaplane, but there was adequate room
for improvement.

Test Reader Comments

What was liked most about our instructions was that the instructions went into great
detail as to which piece went where. It went through step by step and described what each piece
being used looked like.
What the test group liked least was that there were no pictures to confirm the pieces they
were using were accurate. The pictures would make the instruction set hit all targets of audience:
the people that need to visually see the pieces in action and the people that need the worded
directions to follow along to.
One recommendation the test group provided was to add pictures to the final copy. The
pictures are an easy way to visually see all the pieces acting together. Also, when people get
tripped up at finding how the small pieces fit in, they can look at a picture for extra guidance.
Pictures were added to the final draft of instructions to please all kinds of learners that would be
building the seaplane.