ELON

DINING
USER TEST
REPORT
Julia Kocsis & Zachary Fertig
December 2, 2021
 1

EXECUTIVE SUMMARY
We selected Elon University’s Dining website to test, and we aimed to investigate the usefulness of
the website. We did this by seeing if the current Elon Dining website offers an efficient and effective
user experience for current Elon students (both undergraduate and graduate) looking to access
dining hours, menu information, mobile ordering, and dining locations. These insights are important
because the better the overall experience on the Elon Dining website, the higher potential for
students to form positive attitudes about Elon University’s dining options.

For this study, we conducted six separate user tests. There was one participant per undergraduate
year (first-year, sophomore, junior, and senior) and two to represent graduate students (from the
iMedia program). Participants came to the private editing bays located inside of Elon’s Long building
when they were available, and participated in the test without moderation. Our methods were to
utilize the Apple Quicktime Player to record the screen and audio of participants using the Macs in
the editing bays, and to use Google Forms as a way to instruct participants and capture responses.

Overall, the participants found the Elon Dining website easy to navigate. In fact, on a scale of 1-5,
with 5 being very easy, four out of the six participants ranked it a 4. The average time taken to
complete each task was 67 seconds and 3 clicks, averaged across all tasks. Task 4, the mobile
ordering task, was a noticeable outlier here, with an average of 104 seconds and 5 clicks. The mobile
ordering task was a problem throughout. Only 50% of the participants completed task 4 successfully.
All other tasks had a 100% task completion rate. The only tasks with errors were this mobile order
task and task 5, which involved finding Village Juice’s location. Every participant had an error on task
4, and 33% had an error on task 5.

Suggestions:
Our results uncovered a real need for a more defined and useful main top navigational header.
We suggest a simple header with the items: Locations, Menu and Hours, Meal Plans, and Mobile
Ordering. This would allow users to always be able to refer to these pages with one click from
any page of the site and cause less reliance on the homepage.
For the menus, we determined that there was a need for an accordion list embedded into the site
as opposed to the current pdf menus that open in the same tab. The new menu format would
need to not take you to another page and be sure to include Elon-specific and meal plan
information.
Lastly, the ability to mobile order food on the website needs to be enabled. If there is a section
devoted to mobile order, it should work, and allow users to order food quickly and easily.

USER TEST REPORT| KOCSIS & FERTIG 2021

 2

TABLE OF
CONTENTS

Purpose 04-
03 Statement
Method
05

06- Results
10- Findings and
Recommendations
09 12

14- Appendix
13 Reflections
23

USER TEST REPORT| KOCSIS & FERTIG 2021

 3

PURPOSE STATEMENT
This usability test focuses on the Elon University Dining website. As current Elon
graduate students, and former Elon undergraduate students, we know how
important dining information can be in students’ daily lives. The Elon Dining
website is full of valuable information and includes, but is not limited to, menus,
hours, locations, mobile ordering, and events. We aim to investigate the
usefulness of the Elon Dining website. We will do this by seeing if the current Elon
Dining website offers an efficient and effective user experience for current Elon
students (both undergraduate and graduate) looking to access dining hours,
menu information, mobile ordering, and dining locations. These insights are
important because the better the overall experience on the Elon Dining website,
the higher potential for students to form positive attitudes about Elon
University’s dining options.

USER TEST REPORT| KOCSIS & FERTIG 2021

 4

METHOD
For this study, we conducted six separate user tests. These tests were conducted in the
private editing bays of Long Building, the Interactive Media building on Elon University’s
campus. There was one participant per undergraduate year (first-year, sophomore,
junior, and senior) and two to represent graduate students (from the iMedia program).
Each participant was from a convenient sample and we chose participants from each
year in order to create an intentionally diverse study makeup. There were two males and
four females represented in this study, similar to the overall makeup of Elon University.

Each participant was invited to participate via text message, where the study was
explained in detail to them and they could agree or deny to participate. Participants came
to the editing bays when they were available, and participated in the test without
moderation. We chose to make this study unmoderated to preserve the natural
experience of users without the potential influence of peer pressure. The session was set
up for them, with the screen and audio already being recorded and the Google Forms
with the instructions pulled up. We welcomed all participants, then read an orientation
script (see appendix). After there was time for any preliminary questions, Kocsis and
Fertig left the participants in the bays to complete the tasks on their own time, without in-
person guidance.

USER TEST REPORT| KOCSIS & FERTIG 2021

 5

METHOD
Our methods were to utilize the Apple Quicktime Player to record the screen and audio of
participants using the Macs in the editing bays, and to use Google Forms as a way to
instruct participants and capture responses. The Google Forms (see appendix) was
staggered, so participants only saw one task at a time. Each participant completed five
tasks in the same order.

The tasks, in order, included free exploration of the site, viewing menus and hours,
ordering food online, and locating dining areas. These tasks were chosen because they
involved the main daily functionalities of the site. There was a space under each task for
participants to type their thoughts. Additionally, the beginning of the forms collected
demographic/background information and the end had general post-test/evaluation
questions. The post-test questions analyzed effectiveness and usefulness, as well as
weaknesses, specific to the tasks they completed, as well as the overall experience during
their session. Each session lasted an average of 10-20 minutes.

To analyze these sessions and collect data, Kocsis and Fertig reviewed the information
inputed into the Google Forms for all six participants, as well as watched the videos of
each of the six sessions. The work was split evenly, with Kocsis reviewing three
participants and Fertig reviewing three participants. By watching the videos, we were able
to identify and log task completion time, errors, and clicks. Errors were identified as an
action done by the user that took them to an unintended stage in the process, or a stage
where they had to go backward in the task. All information was compiled into qualitative
notes and then quantitative data, as you will see below, in the results section.

USER TEST REPORT| KOCSIS & FERTIG 2021

 6

RESULTS
Demographics:
In terms of the demographics of the participant group, we had two males and four females,
similar to the makeup of the Elon University community. Similarly, there were two graduate
students and four undergraduate students. Five out of the six participants had used the Elon
Dining website before, while only one had not before the user testing process. Four of the six
participants had used Elon’s mobile ordering in the past, while two had not. Three out of the
six participants have a meal plan currently, and the other half do not.

Overall:
Overall, the participants found the Elon Dining website easy to navigate. In fact, on a scale of
1-5, with 5 being very easy, four out of the six participants ranked it a 4, 1 said a 5, and one
said a 3. This is a great place to start.

USER TEST REPORT| KOCSIS & FERTIG 2021

 7

RESULTS
Task Completion:
All tasks were completed by all participants, excluding task 4. Only 50% of the participants
completed task 4 successfully. All other tasks had a 100% task completion rate.

Average Time:
The average time taken to complete each task was 67 seconds and 3 clicks, averaged across
all tasks. Task 4 was a noticeable outlier here, with an average of 104 seconds and 5 clicks.

USER TEST REPORT| KOCSIS & FERTIG 2021

 8

RESULTS
Error Percentage:
The only tasks with errors were tasks 4 and 5. Every participant had an error on task 4, and
33% had an error on task 5. These errors were with failing to reach the mobile order page at
all or clicking on the menu and being sent to a pdf menu with no way to order food.

Helpfulness:
Some information found in the tasks was more helpful for the participants than others.
Helpfulness of the information was envisioned as to how likely the information presented on
the site was to further the user's dining experience. Tasks like task 3 were indicated to be
mostly helpful, on average. Task 4 had very spread results, with helpfulness ranking spread
across every rank number, and not concentrated in any area (helpful or not helpful).

USER TEST REPORT| KOCSIS & FERTIG 2021

 9

RESULTS

USER TEST REPORT| KOCSIS & FERTIG 2021

 10

FINDINGS AND
RECOMMENDATIONS
Overall:
Initial thoughts on the website were positive from most participants. During the free
exploration first task, there were several compliments about the overall look of the website
and its professional feel. This is a great start, and a great first impression. Overall, our results
uncovered a real need for a more defined and useful main top navigational header. Right
now, there is no real top navigational header. There are elements in this area, like the
hamburger menu, logo, and Menu and Hours button, but no clear header. This needs to be
fixed for better clarity and usability. We suggest a simple header with the items: Locations,
Menu and Hours, Meal Plans, and Mobile Ordering. This would allow users to always be able to
refer to these pages with one click from any page of the site, and cause less reliance on the
homepage. We also determined that there was a need for an alternative to the current pdf
menus, like an embedded accordion list. Additionally, the ability to mobile order food on the
website needs to be enabled. If there is a section devoted to mobile order, it should work, and
allow users to order food quickly and easily.

Mobile Ordering:
Several problems emerged as the users began to complete actual everyday tasks. The main
issue that stalled the user test was the inability for many participants to find mobile ordering,
and then the inability to actually order on the website. The mobile order page is relatively
hard to find. It is a tab near the bottom of the very long hamburger menu at the top right of
the site. Once you click there, however, you are not able to mobile order and are informed
that you must download the app. This frustrated the users who were able to find this page.
We suggest making mobile ordering more visible and making it possible to place a mobile
order without the phone app.

USER TEST REPORT| KOCSIS & FERTIG 2021

 11

FINDINGS AND
RECOMMENDATIONS
Hamburger Menu:
Another trend we noticed was that our participants did not gravitate towards the hamburger
menu to navigate the site. Most participants avoided the hamburger menu, and many actually
never used it at all. We believe that it is because it is not very prominent and the dropdown
options are too extensive and overwhelming. To see all options, you have to scroll, which is
generally not ideal. Also, it far exceeds the magic number 7 rule. We suggest narrowing the
options and making the hamburger dropdown part of a more defined navigation header.

PDF Menus:
Multiple participants noted that the pdf menus frustrated them, specifically because they
opened in the same tab. These pdfs were static and included no meal plan information or Elon-
specific information, which was not ideal for these participants. Also, they worried about closing
the tab by accident. We suggest making these menus embedded into the site in an accordion
format so that it does not take the user to a different page and so that there is more room to
list Elon-specific and meal plan information.

Popular Pages:
Overall, the most frequented pages seemed to be the Location page and the Menu and Hours
page. This is why we see these being beneficial as main navigational header items. There is a
button for Menu and Hours at the top right, and the participants did not tend to use this. We
believe this is because it did not fit in as a part of a compelling main navigational header, that is
much needed in this site. Overall, this indicates that Location and Menu and Hours need to be in
a main top navigational header, above all else, for consistency and reachability.

USER TEST REPORT| KOCSIS & FERTIG 2021

 12

FINDINGS AND
RECOMMENDATIONS
Homepage:
The participants did not tend to linger on the homepage. For almost every task, participants
immediately clicked on Locations or Menu and Hours. Only one participant, the only participant
who did not go to Elon for undergraduate studies, utilized the homepage more frequently to
navigate. We did not see this as something that needs to be fixed, but instead, it provides useful
context when updating the homepage over time. Less detail here is better for users.

Sizing:
There were several “above the fold” issues indicated. Several users were frustrated by the size
of the map and of carousel pictures because they took up almost all of the screen before
scrolling. To see any other information, you had to scroll on several important pages, and this
took more time for users. All of the most important information should typically be located
“above the fold” or in that first screen of the site without scrolling downwards. We suggest
downsizing the map and carousel element’s width.

USER TEST REPORT| KOCSIS & FERTIG 2021

 13

REFLECTIONS

While we did our best to create the best user testing experience possible, we know that it had
imperfections. Two of the six sessions were conducted over break, so dining halls were not
open, which affected the participant’s ability to view what locations were open in task 2.
Additionally, because task 4 tripped the participants up, we worry that our wording may not
have been clear enough or not emphasized that portion of the task enough. To improve upon
these things, we would ensure that all sessions occur during normal dining operations and that
our wording is more carefully reviewed.

Some alternate paths we could have taken in this process would be to pursue eye-tracking. If
we had more time, we think that this would have been a great pursuit. Additionally, we could
have recruited more participants who had not used the website before (instead of just one)
because we found the results from this participant to differ slightly from the others in ways that
were interesting to potentially explore further.

USER TEST REPORT| KOCSIS & FERTIG 2021

 14

APPENDIX

APPENDIX A
Test Plan

PURPOSE LOCATION
We aim to investigate the usefulness of the Elon Dining These tests will be
website. We will do this by seeing if the current Elon conducted remotely via
Dining website offers an efficient user experience for Zoom, in the comfort of
current Elon students looking to access dining hours, each individual’s
menu information, mobile ordering, and dining preferred location. We will
locations. hold six sessions, for six
different participants.

PROCEDURE

The ideal length of our sessions, after the introduction, should last between 15–20
minutes. This is because Elon Dining’s website is not very in-depth and we feel like
the insights of our test would not be any more informative if it exceeded 20
minutes.

In each session, the participants will join the Zoom session, we will read our
orientation script, then we will send them the Google Form with screening
questions and the actual session tasks. We will allow the participants to read all
questions/instructions and leave a moment open for preliminary questions. Then,
we will click record and turn our cameras and mics off while allowing the
participant to independently explore the website with the instructions as their form
of guidance. The participants are instructed to keep their camera and mic on the
entire time, and to speak their thoughts and challenges aloud.

As each participant reaches the end of the instructions, they will say aloud that
they are done, and we will return to ask our concluding questions (subjective
metrics). Before ending, we will save the recording and inform the participant that
they may leave, and thank them for participating.

USER TEST REPORT| KOCSIS & FERTIG 2021

 15

APPENDIX

APPENDIX A cont.

EQUIPMENT

We will only use Zoom and Google Forms, no extra equipment feels necessary. We will
record and review the session’s audio and screen capture.

PARTICIPANTS
Participant 1: Participant 2: Participant 3:
Elon First-year Elon Sophomore Elon Junior
Female Male Female
Julia’s friend’s sister Zack's friend Julia’s roommate
Frequently uses the Has used the website Has used the website
website Will take a pre-session Will take a pre-session
Will take a pre-session survey survey
survey
Participant 4: Participant 5: Participant 6:
Elon Senior Elon IME student Elon IME student
Female Female Male
Zack’s sister May need to use the May need to use the
Has used the website website website
Will take a pre-session Will take a pre-session Will take a pre-session
survey survey survey

USER TEST REPORT| KOCSIS & FERTIG 2021

 16

APPENDIX

APPENDIX A cont.

TASKS & SCENARIOS

Scenario: You are an Elon University student and are looking for information on
the on-campus dining services offered.
Free exploration warm-up: You should explore the website freely for 3 minutes.
Task: Navigate to the sections you feel are most useful to you and speak aloud
about what you are looking at and why.
Dining hours: You want to have lunch at Lakeside Dining Hall.
Task 1: When is Lakeside open today?
Task 2: What are they serving for lunch today?
Meal plans: You are considering purchasing a meal plan at Elon.
Task: You want to explore your various meal plan options.
Online ordering: You are in a hurry and need to order food for pickup.
Task: You want to order food online for pickup on campus.
Locations: You want to try Village Juice for the first time, but don't know where it
is located.
Task: Find Village Juice's location on the Elon Dining map.

USER TEST REPORT| KOCSIS & FERTIG 2021

 17

APPENDIX

APPENDIX A cont.

MEASUREMENT METRICS
Objective Metrics (measured by us after session):
Time to complete each task in seconds (if completed)
Number of clicks per task
Time spent on each page they interact with in seconds
How many errors were made

Subjective Metrics (asked live at end of session):

Rate the usefulness of each task component (mobile ordering, the map, etc.)
What do you think is the most useful page of the website? Why? How easy was
this page to access?
Who is the most targeted user of this website, in your opinion? How could you
tell?
What, if any, were the weaknesses of this website? What would you like to see
differently?

ROLE OF MODERATORS

We will both attend the remote Zoom sessions to introduce ourselves and we will
alternate who reads the orientation script and who asks the concluding questions.
The instructions will be given all at once by sharing a Google Doc with the
participant during the introduction section of the session. They will be asked to
read the instructions before we leave, and given time to ask any clarifying
questions before they begin. They can technically ask us questions at any time
during the session, because we can hear them, but we will strongly discourage any
content-related or leading questions.

USER TEST REPORT| KOCSIS & FERTIG 2021

 18

APPENDIX

APPENDIX B
Orientation Script

Hi,

We are Julia and Zack, and we will be helping you through this session today.

Today, we would like you to check out Elon Dining’s website so that we can get a better idea of
how real users are interacting with this interface.

Keep in mind that we just want to see how you use the website and aren’t evaluating you at all.
We also have no connection to Elon Dining’s website, so please be completely candid about
your experience with the website today.

As you go along during this session, we ask you to think out loud and articulate each thought
you have, no matter how important. This is why we will be recording audio during this session,
to review what you have had to say.

We also ask you to keep your screen shared at all times, so we can see where you are on the
website.

We will not be present during this session, we will just ask you to record the session and email us
the file when you are finished (around 15 minutes total).

Your input will only be used for the purposes of our class.

Thank you for agreeing to participate today, we really appreciate it.

Do you have any questions before we begin?

USER TEST REPORT| KOCSIS & FERTIG 2021

 19

APPENDIX

APPENDIX C
Pre-test questionnaire

1. Full Name

2. Gender
a. M
b. F
c. Other

3. Year
a. First Year
b. Sophomore
c. Junior
d. Senior
e. Graduate

4. Are you a current member or were you previously a member of Elon's undergraduate
program?
a. Yes
b. No

5. Have you used the Elon Dining site before?

a. Yes
b. No

6. What is your experience with Mobile Ordering?

1 - Not Experienced
5 - Very Experienced

USER TEST REPORT| KOCSIS & FERTIG 2021

 20

APPENDIX

APPENDIX C cont.

7. Do you have an Elon Meal Plan?

a. Yes
b. No
c. No longer have one

8. How often do you eat at Elon restaurants?

1 - Never
5 - Very Often

USER TEST REPORT| KOCSIS & FERTIG 2021

 21

APPENDIX

APPENDIX D
Scenario and Tasks

Scenario
You are a student of Elon University and are looking for information on the dining services.

Task 1
Explore the website for 2-3 minutes. Speak or type any observations you make. Afterwards,
continue to the next section.

Task 2
You want to go to Lakeside for lunch this afternoon. Speak or type any observations you make.
Afterwards, continue to the next section.
Find what times Lakeside is open today.
Find what food is being offered for lunch.

Task 3
You want to try Village Juice because you have heard it is good. Speak or type any
observations you make. Afterwards, continue to the next section.
Learn what you can about the prices and offerings of different meal plans.

Task 4
You want to try order food from the Acorn. Speak or type any observations you make.
Afterwards, continue to the next section.
Find how you can complete a remote order

Task 5
You want to try Village Juice for the first time, but don't know where it is located. Speak or type
any observations you make. Afterwards, continue to the next section.
Navigate to the embedded dining map.
Locate Village Juice on that map.

USER TEST REPORT| KOCSIS & FERTIG 2021

 22

APPENDIX

APPENDIX E
Post-test questionnaire

1. What sections of the website do you feel are the most useful or effective?

2. Who do you see the primary audience of the website being?

3. Is this website easily navigated?

1 - Not Easy
5 - Very Easy

4. In task three about the meal plans, is the information provided helpful?
1 - Not Helpful
5 - Very Helpful

5. In task four about mobile ordering, is the information provided helpful?

1 - Not Helpful
5 - Very Helpful

6. What was the weakest area of the website?

USER TEST REPORT| KOCSIS & FERTIG 2021

 23

APPENDIX

APPENDIX F
Google Form

https://drive.google.com/file/d/1SybdxvW759Kc4etmmWUxuayliGuyjb3s/view?usp=sharing

APPENDIX G
Observations and Data

https://drive.google.com/file/d/1upgJTJ9_PP1FDAsKR-3DF9_JgME2vZYR/view?usp=sharing

USER TEST REPORT| KOCSIS & FERTIG 2021