December 11, 2018 
 
To the Amtrak Research Team:  
 
Attached to this email is the usability report of the Amtrak website that was publicly available 
between the dates of November 11, 2018 and November 16, 2018. It was conducted by our team 
at the University of Washington as part of the HCDE 417 Usability Research Techniques course.  
 
The study objective was to evaluate the seamlessness of the site’s common task flows and 
discover any difficulties or areas of confusion for the users. We conducted usability tests with 
eight participants to uncover the following key findings:  
● There are semantic and organizational problems with how customers are able to 
purchase tickets for passengers with disabilities. 
● The ticketing page has confusing elements such as unclear icons and layout. 
● Users encounter frustration regarding Julie, the search assistant. 
 
Thank you for your time and for the opportunity to investigate these usability issues. If you have 
any questions regarding the contents of this report, please do not hesitate to contact our team at 
any of the following emails:  
● Connor Garrett, ​x2@uw.edu 
● Reine Abubakar, ​reineka@uw.edu  
● Sierramatice Karras, ​karras@uw.edu 
● Ruby Davis, ​rkdavis@uw.edu 
 
Best regards, 
Usability Research Team Group #3  

   

1 
 
 
 
 
 
 
 
 
 
 
 
 

 
USABILITY REPORT 
December 11th, 2018 
 
 
 
 
 
 
 
 
 
 
 
Connor Garrett 
Reine Abubakar 
Sierramatice Karras 
Ruby Davis 
 

2 
Table of Contents
Executive Summary 4 

Usability Report 5 
Methods 8 
Metrics 10 
Results 10 
Severity Rating Definitions 11 
Key Findings 11 
All Findings 12 
Purchasing Tickets 12 
Add to Cart 12 
Seating Chart and Other Information 13 
Page Layout/Lack of Comparisons 13 
Buses 14 
Multi-City Pricing 14 
Style Inconsistencies 15 
Accessible Tickets 15 
Passenger with Disability ‘Discount” 15 
Muli-City + Passenger with Disability was not an option 17 
Search Engine AKA Julie 18 
Tracking 19 
Train Status 19 
Map Tracker 20 
Informational Pop Ups 21 
Stations 22 
Conclusion 22 

Appendixes 24 
Appendix I: Screener Questionnaire (Final Test Kit) 24 
Appendix II: Screener Results 27 
Appendix III: Facilitation Scripts and Verbal Questionnaires (Final Test Kit) 30 
Appendix IV: Moderator/Observer Task Scenarios (Final Test Kit) 32 
Appendix V: Participant Packet 34 
Appendix VI: Notetaker Data 38 
Appendix VII: Data from UserZoom 47 
Appendix VIII: Affinity Diagrams 51 
Final Reflection 52 

3 
Executive Summary 
As part of the Usability Research Techniques (HCDE 417) course at the University of Washington, 
a group of four students conducted a usability test of the Amtrak website (www.amtrak.com) at 
the the Seattle campus in the Sieg building conference rooms. Using a screener designed for this 
study, eight participants were recruited in order to assess the processes of ticket purchasing and 
researching relevant trip information. The goal was to learn how to improve these common task 
flows. The research questions the study aimed to answer were: 

1. Are users able to successfully purchase tickets for their specific needs?  
2. Can users find the the most efficient ticket for their itinerary at the best price? 
3. Can users get through common tasks without reaching out for assistance?  
4. How should the site be navigationally organized to improve efficiency for inexperienced 
Amtrak customers? 
5. Can users fully comprehend the train tracking information without difficulty? How quickly, 
easily, and successfully can users navigate between the different tracking systems to find 
the information they need? 
6. Are users able to successfully use the search feature? 
 
A series of eight usability tests were conducted between November 11 and November 16. After 
each usability session, the notes were combined into a single spreadsheet. On November 20, the 
group analyzed the results in the spreadsheet data to determine the findings of this study.  
 
The central findings of this study are in three main categories and are described as follows: 
Ticket Purchasing Page  Users found the ticket purchasing page difficult to navigate. They did 
not understand how the information was organized and were 
confused about how to proceed with ticket checkout. 
 
Users wanted to know how to find out more information about their 
tickets and whether or not the routes would include bus rides in 
addition to train rides. Style inconsistencies across different pages 
made finding specific information difficult. 

Accessible Tickets  Users had difficulty purchasing tickets for passengers with 
disabilities. They also wanted more politically correct terminology 
when purchasing tickets for passengers with disabilities.   

Julie: Search Assistant  Users grew frustrated with the website’s search feature, Julie. They 
felt that the feature was intrusive, finding it annoying and 
unnecessarily complicated.   
 
This report will outline the objectives of the usability study, detail the methods employed and the 
participants who engaged in the tests, and describe the findings. 

4 
Usability Report 
Amtrak is the major consumer rail service in the United States, serving approximately 500 
destinations and 85,000 passengers daily1. For this project, a usability study was conducted on 
Amtrak’s flagship website, amtrak.com. While this study did not begin with an initial goal to focus 
resources on a specific feature, exploration of the current version of the Amtrak website led to 
the following overarching goals/research questions: 
 
● How can the overall user experience of the Amtrak website be improved to make it more 
usable? 
● What areas of the current Amtrak website are the biggest problem points? 
● Is the current version of the website accessible to all users? 
● Are users able to purchase tickets without assistance? 
● Are users able to find important information that may be hidden a few pages deep on the 
website? 
 
The study was conducted by four seniors at the University of Washington as a project for HCDE 
417. These four members were Connor Garrett, Reine Abubakar, Sierramatice Karras, and Ruby 
Davis. Contact information for the team is available on ​page 1​ of this report. 
 
This report contains some terms that may not be clear on first glance. These terms are listed and 
defined below. 
 
Home Screen 

 
The main page of amtrak.com. This is the first page that the user sees upon arriving at the 
website. 

1
"Company Profile." Amtrak. 2017. 
https://www.amtrak.com/content/dam/projects/dotcom/english/public/documents/corporate/nationalfactsh
eets/Amtrak-Company-Profile-FY2017-0318.pdf. 
5 
Main Ticketing System 

 
The default ticketing process, used for one-way and round-trip bookings. 
 
Multi-City Ticket System 

 
The alternate ticketing system used to book itineraries that touch more than two cities. 
 
   

6 
Julie Chatbot / Search Engine 

 
The main search feature of Amtrak’s website employs Julie, a chatbot that attempts to assist 
users with their searches. 
 
Map Train Tracker 

 
A variant of the main train status database that displays that info for all trains on a map. 
 
   

7 
Train Status Tracker 

 
A second method of tracking trains that allows users to search for a specific train and see only its 
info. 

Methods  
The following sections detail how the usability study was conducted and the participants 
involved.  
 
Participants 
Using a screener questionnaire (​Appendix I​) the research team reached out to potential 
participants. Out of the twenty-seven respondents (​Appendix II​), eight people qualified to be 
participants in the usability study. 
  
Our qualifications for study entry are as follows:  
● Participants should be between 18 and 70 years old. 
● We would like to recruit a proportion of 50% men, 50% women, but this can be adjusted 
based on available participants.  
● Participants should have traveled at least twice in the last year. 
● Participants should not use a travel agent, work in the travel industry, or work in UX/UI. 
● Participants should be Novice users, defined as having used the Amtrak site less than 5 
times in the past year. 
● Participants need to be able to come to campus for the study and provide contact 
information. 
   

8 
Our Participants 
Participant  Age  Gender  Bias  Familiarity   

1  22  Female  No  Novice   

2  20  Genderqueer  No  Novice   

3  21  Male  No  Novice   

4  21  Non Binary  No  Novice   

5  21  Female  No  Novice   

6  22  Genderqueer  No  Novice   

7  20  Female  No  Novice   

8  20  Non Binary  No  Novice  Figure 1. Pie chart of participant genders. 
 
Bias represents respondents who use a travel agent, work in the travel industry, or work in UX/UI. 
 
Study Environment 
The study sessions were conducted in HCDE conference rooms or lab spaces in Sieg Hall on the 
University of Washington campus.  
 
Study Procedure 
Prior to the participant arriving, the Moderator and the Notetaker would set up two laptops as the 
moderator and participant in UserZoom. When the participant arrived, they were welcomed. After 
they were introduced to the study and how the session would proceed, they were given a 
consent agreement and asked to sign it. Some instructions given with regards to the procedure 
of the study were “Do tasks as you would do anywhere”, “You are testing the product, you are 
not being tested”, “Provide both positive feedback and constructive criticism”, “Read tasks out 
loud before you begin”, and “Think aloud while you complete the tasks.” After they were 
prepared to begin the study, the participant was verbally asked some initial background 
questions (​Appendix III​) to gauge their familiarity with the site and travel more generally. At this 
point in the study, the participants were asked to complete the given tasks (listed below and in 
Appendix IV​) and then respond to post-task questions within the task list. After the tasks were 
complete, the participants were verbally asked post-study questions about how they felt about 
the site and their experience of it (​Appendix III​).  
 
Study Roles 
Each member of the research team both ran study sessions as a Moderator and observed as a 
Notetaker. This setup was decided upon to maximise the detail of the notes and the attention 
given to the participant as they completed tasks.  
 
   
9 
Tasks 
See ​Appendix IV​ for full Tasks 
Task 1:  Find a ticket between two cities for a given passenger specification. These 
One Way Ticket  specifications covered factors such as passengers with a disability, 
w/ Passenger  passengers with a military discount, and passengers traveling with a bike.  
Specifications   
Intended to test the main ticketing system and the different options available 
with it.  

Task 2:  Plan a trip between the set of given cities on the given days.  
Multi City Ticket   
Intended to test the difference in usability between the main ticketing system 
and the multi-city ticketing system.  

Task 3:   Find specific information that someone might research prior to traveling with 
Search Engine /  Amtrak.  
Julie Chatbot   
Intended to test users opinion on the search engine chatbot and the 
information architecture of the site.  

Task 4:  Track a specific train and determine if it will arrive at a station as scheduled. 
Train Tracking   
Intended to determine the preference of users between the two train 
tracking systems, as well as both systems ease of use and discoverability.  

Metrics 
Several different types of data and metrics were collected during the actual conducting of the 
study. Before heading to the website, each participant was asked a series of pre-test questions to 
help further establish their level of experience and to keep an eye out for any variables that may 
be confounding. This pre-test questionnaire was conducted verbally and is available in ​Appendix 
III​. A similar line of questioning was conducted at the end of the tests and is also available in 
Appendix III​. The post-test questions were used to learn more about the user’s overall opinions of 
the site and to catch feelings that might not be brought out by the mid-test conversations and 
post-task surveys. At the end of each task, users were asked a consistent series of questions, 
which are available in ​Appendix V​. The responses to these questions are available in ​Appendix 
VI​. 

Results 
To process the data that was gathered during our studies, several methods were used. We 
began by creating affinity diagrams using issues that we noted during the sessions, grouping 
those issues by section and overall root cause. Photographs of these diagrams are available in 
Appendix VIII​. These diagrams were used to help focus our resources for the next steps on the 

10 
most important and solvable issues. These results were also used to help classify the severity of 
the issues. 
 
Below are the results of the usability tests and the severity rankings of the issues discovered. 
First, an overview of the key findings will be shared, ordered by severity. Then the severity 
criteria and the issues discovered will be presented with evidence and recommendation for 
improvement. These issues are presented in the following categories: purchasing tickets, 
accessible tickets, search / chat bot issues, tracking system issues, and miscellaneous. 

Severity Rating Definitions2  

Severity 1: An issue that blocks a substantial number of users from using the application. E.g., 
Users are unable to start a game. Nearly impossible task during the initial experience. 
 
Severity 2: An issue that blocks a substantial number of users from making use of an important 
feature or accomplishing an important task. E.g., Unclear solution. Undiscoverable feature. 
 
Severity 3: An issue that hinders some users from accomplishing a task or goal. E.g., Inconsistent 
or misleading text. Users don't understand a non-essential aspect of the application or feature. 
Delays. 
 
Severity 4: An issue that causes some users to become mildly frustrated or express minor 
complaints about a feature element. E.g., Suggested Feature. Minor or infrequent complaint. 

Key Findings 
Severity 1:  
● No options for passengers with disabilities on the multi-city tickets 
Severity 2:  
● Passengers with disabilities as a ‘Discount’ passenger option 
● Comparing ticket prices and information while purchasing tickets 
● Determining multi-city trip prices while purchasing tickets  
● Multiple Train Tracking Systems 
Severity 3:  
● Finding seating charts and other information while purchasing tickets 
● Adding tickets to cart while purchasing tickets 
● The Julie chatbot and search engine 
● Labeling of Stations 
Severity 4:  
● Determining whether a route is a bus/train route while purchasing multi-city tickets 

2
Liz Sanocki 
11 
All Findings 
The following are the issues found during this study as well as the evidence from our participants 
and recommendations for Amtrak to make to their site. 

Purchasing Tickets 
Many areas of ticket selection are unclear or challenging to process. This issue applies to both 
one-way/round trips and multi-city itineraries. Every participant had at least one issue with the 
ticket selection process. Some of the issues we found were as follows: 

Add to Cart 

 
Figure 2. Screenshot of ticket selection page with the green “Add to Cart” button displayed on 
the left. 
Severity Ranking:​​ 3 
 
Evidence 
Three of our participants encountered this issue, and while they were able to find the button 
eventually, it caused significant delays. Participants continued to scroll up and down the screen 
while clicking around to search for ways to add the given tickets to their cart. In one instance, a 
participant took four minutes and forty-eight seconds to find the add-to-cart button after selecting 
a train. 
 
Recommendation 
Place the ‘Add to Cart’ button at a traditional location for eCommerce websites: on the right side 
of the item (in this case a ticket) and in a section separated from the ticket information. This 

12 
increases scannability for the overall layout of the page and increases the visibility of the ‘Add to 
Cart’ button. The current checkout process also only allows for the purchase of tickets from one 
route at a time; users cannot add multiple items to their cart. We recommend changing the name 
of this button to a phrase that reflects this fact, such as “select” to indicate that you may only 
select ticket(s) for one train at a time.  

Seating Chart and Other Information  

The ticket selections did not provide specific information that the participants were curious about 
such as seating charts. In the case for multi-way trips, the cost of the ticket was not displayed and 
participants wanted to learn more about their trip segments. 
 
Severity Ranking:​​ 3 
 
Evidence 
Five of our participants were searching for further information including train seating charts and 
ticket costs. This applied to both one-way and multi-way ticket purchases, and participants were 
unsure of where to search for further information. The term “Reserved” for each ticket led our 
participants to believe that there would be assigned seating, and they scrolled down the screen 
and clicked on icons to determine whether or not they would reveal further information about the 
train ride/route.  
 
Recommendation 
We recommend adding a seating chart or more detail about assigned seats. Tickets should list 
whether seating charts are available, and there should be visible links in case the user wants to 
learn more information about the type of train that they are looking at.  

Page Layout/Lack of Comparisons 

The ticket selections are listed with little visual informational hierarchy, making it difficult to 
compare prices between tickets. While browsing the list of available tickets for a given route, 
tickets are listed in a horizontal manner that makes it challenging to compare them while 
scrolling. 
 
Severity Ranking: ​2 
 
Evidence 
As users attempt to find the most appropriate ticket for their given scenario and route, they 
encounter a page where tickets are contained within a window within the larger page. The 
smaller window is scrollable, and two of our eight participants complained of this double scrolling, 
claiming that it was “annoying” and hindered their ability to compare tickets with one another. 
Additionally, the application of filters did not help as presentation of information on the page itself 
was still confusing after being filtered.  
 

13 
Recommendation 
We recommend re-structuring the layout of the ticket purchasing page. The tickets should take 
up the full span of the page (to prevent the double scrolling) and should have clear filters at the 
top of the page distinct from the list of tickets. The information could be organized vertically per 
ticket such that similar information between tickets are on the same row and are more easily 
comparable.  

Buses 
Multi-way train routes suggest the times and routes that best fit the users’ time and location 
preferences. However, with transfers between trains, participants did not find it clear that a few of 
the transfers would actually be taken on a bus as opposed to train.  
 
Severity Ranking: ​4 
 
Evidence 
All of our participants, while asked to plan a multi-way trip, were able to complete the task and 
select appropriate routes for a multi-city trip. However, when asked whether they were aware of 
the bus transfers, some of our users said that they were not aware that the whole trip would not 
be conducted via train.  
 
Recommendation 
We recommend listing each trip segment with a larger icon or a clear indicator/word that 
indicates the mode of transportation. As it currently stands, there is a small icon that is nearly 
indistinguishable between trains and buses. Whether it is by bus or train, users should be able to 
clearly differentiate between the two available modes of transport on the site. 

Multi-City Pricing 
The ticket selections did not provide specific information that the participants were curious about 
such as ticket pricing. While constructing the various segments of the larger trip, there are no 
updates as to how much the trip would cost depending on the method of transportation or time 
the user chooses.  
 
Severity Ranking: ​2 
 
Evidence 
When asked to purchase tickets for a multi-city trip, five of our participants were searching for the 
pricing of each trip segment and of the trip overall. For each segment/section where the user 
travels from one city to another, the overall route can be updated as they select a ticket, but they 
are not able to see how each choice influences the overall cost of the trip. Two users noted that 
they would like to see the overall cost.  
 
Recommendation 

14 
We recommend that the page have a running cost of how much the overall trip would cost. With 
each selection of a segment, the cost would update to reflect the price. Optionally, as users scroll 
through the list of possible times and methods of travel, there could be a column for costs that 
display the amount it would add to the overall trip.  

Style Inconsistencies 
The multi-city planning and booking tool looks different from the one-way selector such that it 
feels like a different website. This sentiment was echoed by four of the eight participants. 
 
Severity Ranking: ​4 
 
Evidence 
The disparity between the multi-city and single-city booking pages were stark, and it may have 
delayed our users’ tasks for booking their trip. Four out of eight of our users expressed that the 
page layout was unexpected, and all of our users spent additional time learning how to book the 
segments of their trip as it was not the same as booking a single-city trip.  
 
Recommendation 
A consistent layout for single-city and multi-city ticket purchasing will reduce the learning curve 
required for users. If a user is familiar with the task of booking a single-city ticket, they should be 
able to intuitively and easily book a multi-city ticket based on the knowledge of booking a simpler 
trip. This will prevent delays in completing the task, and it will prevent any additional confusion 
regarding how to plan trips.  

Accessible Tickets 

Passenger with Disability ‘Discount” 

Participants had difficulty accessing Amtrak’s robust accessible ticket options because they could 
not locate the “Passenger with Disability” selection, which is nested in the “Discounts” drop-down 
menu. The categorization of accessibility options as a “discount” was also deemed not politically 
correct. 

15 
  
Figure 3. Screenshot of the Amtrak homepage. On the right, users may indicate the number of 
travelers and apply any discounts to their purchase. For passengers with disabilities, it is not 
immediately obvious how to indicate their needs. 
 
Severity Ranking:​​ 2 
 
Evidence 
Two out of five who booked tickets for passengers with disabilities could not locate the 
“passenger with disability” option at all, which had major consequences later on in the ticket 
purchasing process. Accessibility options are only available when this “discount” is selected, and 
thus users were prevented from viewing these necessary options. The three users who were able 
to find the “passenger with disability” selection still had difficulty locating this option. In addition, 
one participant, who was a person with a disability, was unhappy with their accessibility needs 
being trivialized as a “discount”. 
 

 
Figure 4. Screenshot of the discounts dropdown menu, where the “Passenger with Disability” 
option is nested. 

16 
 
Recommendation 
Consider separating accessibility options from discounts in the popup menu. As the selection of 
“passenger with disability” triggers more ticket options than just a discounted price, it may be 
more appropriate to nest these options elsewhere. One possible solution exists on the 
Greyhound ticket purchasing interface, where a separate checkbox has been added to allow 
wheelchair-accessible ticket purchasing separately from the discount drop-down menu.  
 

 
Figure 5. Screenshot of a portion of the ticket purchasing interface on Greyhound.com. Instead 
of nesting accessibility options under “discounts”, a checkbox allows users to indicate if they will 
be travelling in a wheelchair. 

Muli-City + Passenger with Disability was not an option 

Participants were unable to book tickets for passengers with disabilities when using the Multi-City 
ticket booking option. 
 
Severity Ranking:​​ 1 
 
Evidence: ​We asked participants to complete the tasks as they would if they were at home. As 
one of our participants uses a motorized wheelchair, they attempted to book accessible tickets 
for multi-city travel. At no point in the process were they able to locate a place to indicate their 
needs. The “Passenger with Disability” option was no longer listed in the discount drop-down 
menu, and thus none of the necessary selections were able to be made. Our participant did not 
feel comfortable with multi-city ticket booking because of this issue and indicated that if this 
problem were not resolved, they would not be inclined to use Amtrak’s services at all. 
 
 

17 
Recommendation 
Accessibility options must be integrated into multi-city ticket booking. Consider including a 
selection option early on, such as a check box, which allows passengers to indicate at the 
beginning of the process whether they will need special accommodations. This will provide 
necessary accessibility options and reassure passengers that their needs will be met before they 
venture too far into the lengthy booking process. 

Search Engine AKA Julie 

 

 
Figure 6. Screenshot of the Julie chat box that appears when users attempt to use the search 
feature. 
Severity Ranking: 3 
 
Evidence 
All six users who came into contact with Julie expressed dissatisfaction and/or frustration with her 
presence. No users expressed any satisfaction with Julie, other than the fact that the search 
engine itself often brought them to the correct page. Some of the main complaints against Julie 
involve the surprise of her appearing (participants often didn’t expect a chat window popup), the 
fact that the popup can hide important information on the page, and Julie’s tendency to drag 
users to the bottom of the page that they’re viewing.  
 
Recommendation 
Consider removing the “virtual assistant”/Julie feature of the search engine. Place more of a 
focus on the search engine’s ability to bring users to the correct page, rather than Julie’s ability to 
explain the page to them. 

18 
Tracking 
There are two different systems for tracking train status on the Amtrak website. The second 
tracker is the Train Tracker Map which is accessed through a button at the bottom of the home 
page. This system is opened in a new tab and can track trains by searching for stations/train 
numbers or by clicking on stations and trains on the map.  

Train Status 
The first tracking system is the Train Status system, it is accessed through the header bar of the 
Amtrak site. It can track either between stations or with a train number and it provides the 
expected time of arrival. 

 
Figure 7. Screenshot of the Schedule Train Tracker. 
Severity Ranking:​​ 3 
 
Evidence 
Three out of eight participants had to ask for assistance and be given a hint as to where the train 
status button was.  
 
Recommendation 
This research team suggests making the button stand out more from the surrounding bar. This 
would draw more attention to the button and make it easier for users to find. 

19 
Map Tracker 
The second tracker is the Train Tracker Map which is accessed through a button at the bottom of 
the home page. This system is opened in a new tab and can track trains by searching for 
stations/train numbers or by clicking on stations and trains on the map.  
 

 
Figure 8. Screenshot of the Map Train Tracker. 
 
Severity Ranking:​​ 2 
 
Evidence 
Five out of eight participants were confused by how to use the map tracking system and not 
confident in the results they got from it. One particularly confusing part of the map tracker to 
participants was which station you were supposed to enter into the search bar. In addition, two 
out of eight participants expressed that they did not expect to find map when they clicked this 
train tracking method. One participant also had difficulty navigating back to the main Amtrak site 
after opening the map tracker.  
 

20 
Recommendation 
Consider either removing the map feature entirely or incorporating it into the main site and 
redesigning it. This could possibly be done by combining it with with the Train Status feature.  

Informational Pop Ups 

Throughout the site there are a number of pop ups that provide information about a wide variety 
of different things on the Amtrak site. There is not a clear consistency in the style of pop up, or in 
when pop ups are expected. 

 
Figure 9. Screenshots of informational sections for different ticket options and dining. 
 
Severity Ranking:​​ 3 
 
Evidence:  
Four out of eight participants found the informational pop-ups confusing or annoying. Many of 
those who did not comment on them did not click on the links that trigger them.  

21 
 
Recommendations:  
This team suggests that the site incorporate more of the information into the main page instead 
of in full screen pop ups, this can be done by replacing the single full screen pop up with 
individual drop downs. Additionally the style of pop-up should be made consistent throughout the 
site 
 

Stations 
The Amtrak site gives users station suggestions as they type out city names.  

 
Figure 10. Screenshot of auto-complete menu when “Portland” is typed. 
 
Severity Rating: ​3 
 
Evidence 
Every one of the eight participants were confused about which station was the correct one for 
Portland, a city with multiple stations, and four had to be given assistance in determining the 
correct station. Those who had difficulty finding the correct stations frequently received error 
messages.  
 
Recommendation 
This team recommends clearer demarcation of the locations / types of stations. Providing 
information about what types of services are available from a station would help users narrow 
their choices. A user might not already know that greyhounds are a bus system or the different 
locations of stations within a single city.  
 

Conclusion 
Next steps for improving Amtrak’s usability would be to begin rolling out some of the new 
changes suggested in this report. Additional usability testing could help determine if these 
changes successfully address the issues we discovered. Preferably these tests would include a 
wider age range than was available for this user test. Including more participants with disabilities 

22 
would also be a great way to ensure that future iterations of Amtrak’s website are accessible both 
in design and in function. 
 
Overall, most participants were able to complete most tasks in this study, despite the issues they 
were confronted with. If Amtrak implements some key changes, such as more intuitive ticket 
checkout processes, more clear accessibility options, and less intrusive search features, the flow 
of user activities will be improved significantly. Amtrak can be an inclusive service that 
accommodates all users. 
 

   

23 
Appendixes 

Appendix I: Screener Questionnaire (Final Test Kit) 

Thank you for your interest in our research study! Please answer the following questions as 
honestly as possible. The study will be between 30 and 60 minutes long and will take place on 
the University of Washington Seattle Campus between November 11th and 17th 
 
Your answers personal information will not be shared outside of the researchers conducting this 
study. Any answers you provide to our questions will be anonymized before it is presented in our 
report. 
 
1] How old are you?  
______ 
 
2] What gender do you identify as? 
______ 
 
3] Given your best estimate, how many times did you travel in the last year?  
______ 
 
4] Do you typically use a travel agent to plan your trips?  
- Yes (exit survey) 
- No 
 
5] Have you ever or do you currently work in the travel industry? 
- Yes (exit survey) 
- No 
 
6] Have you ever or do you currently work in UX/UI?  
- Yes (exit survey) 
- No 
 
7] How frequently do you travel by train?  
- Often 
- Sometimes 
- Rarely 
- Never (skip question 7a) 
 
7a] What was the purpose of your train rides?  
- Commuting 

24 
 - Domestic Vacation  
- International Vacation 
- Other 
 
8] Which of the following train ticket sites have you used in the past? (Select all that apply)  
[randomize order in survey] 
- Soundtransit.org (skip to question 9) 
- Wanderu.com (skip to question 9) 
- Amtrak.com (continue to questions 8a - 8d) 
- Raileurope.com (skip to question 9) 
- Other 
 
8a] Approximately how many times have you visited Amtrak.com in the last year?  
_____ 
 
8b] Did you use the site to book a ticket?  
- Yes 
- No 
 
 
9] Would you be available to meet for a one hour study at the University of Washington Seattle 
Campus on any of the following days? (select all that might work for you) 
- Sunday November 11th 
- Monday, November 12th 
- Tuesday, November 13th 
- Thursday, November 15th 
- None (exit) 
 
10] Please leave your name, phone, and email address. 
Name _________________ 
Phone ________________ 
Email _________________ 
 
Thank you for filling out our survey! We really appreciate you taking your time to help us with our 
research. 
 
We’ll take a look at your answers to this survey, and if it looks like you’ll be a good fit for our 
study, we’ll be in touch soon to schedule a session.  
 
Thanks again, and have a great day! 

25 
 
Qualifications for Study Entry 
Question 1: Participants should be between 18 and 70. 
 
Question 2: We would like to recruit in a proportion of 50% men, 50% women but this can be 
adjusted based on available participants.  
 
Question 3: Participants should have traveled at least twice in the last year. 
 
Questions 4-6: Participants should not use a travel agent, work in the travel agency or in UX/UI. 
 
Question 9: Participants should have used the Amtrak site less than 5 times in the past year. 
 
Question 10-11: Participants need to be able to come to campus for the study and provide contact 
information. 

   

26 
Appendix II: Screener Results 
Questions 
A: Timestamp 
B: How old are you? 
C: What gender do you identify as? 
D: About how many times did you travel in the last year? 
E: Do you use a travel agent? 
F: Have you ever traveled by train/rail? 
G: What was the purpose of your train rides? 
H: Have you visited Amtrak.com in the past? 
I: Have you visited Wanderu.com in the past? 
J: Have you visited Raileurope.com in the past? 
K: Have you visited Soundtransit.org in the past? 
L: If any, what other train ticket sites have you visited? 
M: About how many times have you visited Amtrak.com in the last year? 
N: Did you use Amtrak.com to book a ticket? 
O: Have you ever/do you currently work in UX/UI? 
P: Have you ever/do you currently work in the travel industry? 
Q: Would you be able to meet for a one hour study at the University of Washington Seattle 
Campus on any of the following days? (select all that work for you!) 
 
Some questions and answers removed to respect respondent's privacy 
 
Key 
Unavailable / No contact information 

Qualified 

Scheduled 

Disqualified 
 
 
 
 
 
 
 
 
 
 
 

27 
 Results 
A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q 
Commuting, 
11/7/2018  Domestic 
12:08:04  49  Female  4  No  Yes  Vacation  Yes  No  No  Yes    1  No  No  No  Tuesday, Nov 13th 

11/7/2018  Cis  International  Sunday, Nov 11th, 

15:07:04  21  female  6  No  Yes  Vacation  Yes  No  Yes  Yes    1  Yes  No  No  Monday, Nov 12th 
11/7/2018  Domestic 
16:52:50  18  Male  3  No  Yes  Vacation  Yes  No  No  Yes    0  No  No  No  Sunday, Nov 11th 
11/7/2018 
17:30:55  22  Female  4  No  No    Yes  No  No  No    1  No  No  No   
11/7/2018  Domestic 
17:32:50  21  Woman  4  No  Yes  Vacation  No  No  No  Yes        No  No   
androg
ynous/ Sunday, Nov 11th, 
11/7/2018  nonbin Thursday, Nov 
17:34:48  21  ary  4  No  No    No  No  No  Yes        No  No  15th 
11/7/2018  International 
18:22:45  19  female  2  No  Yes  Vacation  No  No  Yes  Yes        No  No   
11/7/2018 
18:30:47  22  Male  2  No  No    No  No  No  No        No  No   
Commuting, 
Domestic 
Vacation,  thetrain
11/7/2018  Gender  International  line.co Sunday, Nov 11th, 
18:35:48  20  queer  5  No  Yes  Vacation  Yes  No  Yes  Yes  m  5  Yes  No  No  Tuesday, Nov 13th 
Tuesday, Nov 13th, 
11/7/2018  Domestic  Thursday, Nov 
19:21:01  21  female  10  No  Yes  Vacation  Yes  No  No  Yes    12  Yes  No  No  15th 
11/8/2018 
0:06:04  21  Male  1  Yes                         
Domestic 
Vacation, 
11/8/2018  International 
2:04:07  19  Male  4  No  Yes  Vacation  Yes  No  No  Yes    1  No  No  No   
Commuting, 
Domestic 
Vacation, 
11/8/2018  Nonbin International 
5:54:54  20  ary  5  No  Yes  Vacation  No  No  No  Yes        No  No   

28 
11/8/2018  Metrara
7:36:22  20  Female  3  No  No    Yes  No  No  No  il.com  1  No  No  No  Sunday, Nov 11th 
11/8/2018  Domestic 
9:39:31  19  Female  3  No  Yes  Vacation  Yes  No  No  Yes    3  No  No  No   

11/8/2018  Sunday, Nov 11th, 

9:42:36  21  Male  5  No  Yes  Commuting  No  No  No  Yes        No  No  Monday, Nov 12th 
11/8/2018  International 
11:21:26  22  female  2  No  Yes  Vacation  No  No  Yes  No        No  No  Sunday, Nov 11th 
11/8/2018  Domestic 
11:22:03  20  Female  0  No  Yes  Vacation  Yes  No  No  Yes    4  No  No  No   
11/8/2018 
11:26:35  19  Female  0  No  No    Yes  No  No  Yes    6  No  No  No  Sunday, Nov 11th 
Commuting,  Tuesday, Nov 13th, 
11/8/2018  Domestic  Thursday, Nov 
11:33:07  20  Female  5  No  Yes  Vacation  No  No  No  Yes        No  No  15th 

Gender Monday, Nov 12th, 

queer/ Tuesday, Nov 13th, 
11/8/2018  Non-bi Thursday, Nov 
11:38:43  22  nary  2  No  Yes  Fun day trip  Yes  No  No  Yes    2  No  No  No  15th 
Commuting, 
11/8/2018  Domestic 
11:41:16  21  Male  4  No  Yes  Vacation  Yes  No  No  Yes    4  Yes  No  No   
Commuting, 
11/8/2018  Domestic 
12:24:17  21  female  3  No  Yes  Vacation  Yes  No  No  Yes    1  No  No  No   
11/8/2018 
12:25:05  18  Female  2  No  Yes  Commuting  No  No  No  No        No  No  Tuesday, Nov 13th 
11/8/2018 
13:27:45  24  M  1  No  No    Yes  No  Yes  Yes    2  No  No  No   
11/8/2018  Domestic 
20:31:45  22  Male  10  No  Yes  Vacation  No  No  No  No        No  No   
 

   

29 
Appendix III: Facilitation Scripts and Verbal 
Questionnaires (Final Test Kit) 
Introductory Task Before Participant Enters 
❏ Have UserZoom open and ready to go on participant and researcher laptops 
❏ Start and then pause recording before welcoming participant into the room 
 
“Hello! Thank you for coming. Today we will be conducting a usability study of the Amtrak 
website. I will be walking you through the session today. My research team member is here to 
make sure we are accurately recording the session and they will be taking notes along the way. I 
will be reading from a script to ensure my instructions to every participant are the same.”  
 
“We've asked you to come here to try using the Amtrak.com website so we can see whether or 
not it works as intended. I want to make it clear that we're testing the site, not you, the user. 
There are no wrong or right answers.” 
 
“None of us had any part on the design process for this site and our objective is to improve it so 
we need to know what you honestly think. You won’t hurt any of our feelings, and it would be 
very helpful for us if you just speak your thoughts and gave your full opinion.” 
 
“As we go along, I'm going to ask you to think out loud, to tell me what's going through your mind 
as you complete tasks. If you have any questions as we go along, please ask them. I may not be 
able to answer them right away, but I will take note of it and see if I can answer it at the end of the 
session. If you still have any questions when we’re done I’ll try to answer them then.”  
 
“If you need to take a break at any point, just let me know. With your permission, we're going to 
record your computer screen and audio.”  
 
“We have a consent form for you to sign today. This will allow us to record the session for later 
review. The video will be used only to help us figure out how to improve the site, and it will only 
be seen by people working on the project and in result presentations.” 
 
“Now I will explain how the session will work, but do you have any questions so far?”  

30 
 
“We will start the session by reading a few scenarios and asking you to complete some tasks. 
After each task, we will ask you a couple of questions about your experience performing the task. 
Are you ready to begin?” 
 
Post-Script Tasks 
❏ Have the participant sign the consent form 
❏ Start the UserZoom Study (audio, video recording) 
❏ Thinking aloud warm up (practice speaking as they click around) 

 
Pre-Test Questionnaire 
[Verbally ask the participant, Note-taker will record notes] 
 
1. How frequently do you travel?   
2. How often do you travel by rail?  
3. Could you tell us about your experience with booking transportation for a previous trip 
(including planes, cars, etc.)? 
4. Do you typically travel with luggage? 
5. When was the last time you used Amtrak.com? What was the experience like?  
6. Do you believe that traveling by rail is a difficult experience? 
7. Why do you choose your preferred mode of travel? 
 

Post-session Questionnaire 
[Verbally ask the participant, Note-taker will record notes] 
 
1. What do you like about the website? Why? 
2. What do you dislike about the website? Why? 
a. Is there anything you found particularly frustrating? 
3. Would you be confident booking a ticket alone? Are there any features you felt were 
missing from the website? 
4. If you could change one thing about this site what would it be? 
5. Is there anything that we could do to improve the participant experience for our study? 

31 
Appendix IV: Moderator/Observer Task Scenarios (Final 
Test Kit) 
Moderator will read scenarios out loud to the participant. The participant will also have a copy to 
reference that will include post task questions.  
 
Task 1: One way ticket 
Goal​​: Starting on the Home Page, search for train tickets that fit a given scenario covering 
different types of travelers and desires for the route (bike carry-on, price, length of trip). Compare 
the tickets to find either the the fastest, best deal, or least transfers as specified in the senario. 
Starting Condition:​​ Home Page 
Success​​: Find a ticket that meets the given criteria 
Fail​​: The user is unable to purchase a ticket with for the route with the specific constraints 
Needs​​: Train routes that we know have the necessary amenities for a given situation 
(dogs and bikes are not allowed on all routes)   
Scenario​​:   
You and a friend want to travel to Portland, Oregon on Friday, November 30th for a concert the 
following day. You will be leaving from Seattle, Washington. You would like to find the best ticket. 
(Half of users will be given A and half will be given B) 
Scenario A: ​Your friend uses a wheelchair that may be stowed during travel. This friend 
also has a small service dog that will assist them. 
Scenario B: ​Your friend is a member of the military. Both you and your friend will be 
transporting your bikes with you on your trip. 
 

Task 2: Multi Trip Tickets 

Goal​​: Find a multi-city route that fits a given scenario covering different types of travelers and 
desires for the route. 
Starting Condition​​: Home Page 
Success​​: Find a ticket that meets the given criteria 
Fail​​: The user is unable to purchase a ticket with for the route with the specific constraints 

32 
 Needs​​: Train routes that we know have the necessary amenities for a given situation 
(dogs and bikes are not allowed on all routes)   
Scenario​​:  
You live in Seattle, but need to attend a conference on November 20th in Portland (station PDX). 
After the conference, you need to visit a friend in Vancouver, BC (traveling on the 21st). After your 
visit, you need to return to Seattle on the 22nd. Book this trip as one journey. 
 

Task 3: Search Feature 

Goal​​: Use the search feature to answer a given question. 
Starting Condition​​: Home Page 
Success​​: Find the information that answers the user’s question 
Fail​​: The user does not discover the answer to their question 
Needs​​: Search terms we know will highlight the issues in the search page 
Scenario​​:  
You are thinking about traveling overnight with a friend on the Coast Starlight train later this 
month. Your friend is gluten free and wants to know if there are meals available for their dietary 
restrictions.  
Mid Task Prompt: “​​Now see if you can find the information another way”  
 

Task 4: Train Tracking 

Goal​​: Ask a user to track the status of a train and then navigate from the schedule tracking 
system to the map train tracking system 
Starting Condition​​: Home Page 
Success​​: Get between the two trackers and track the same train 
Fail​​: The user is unable to locate the status of the train or they are unable to navigate 
from the schedule to the map  
Needs​​: A train/bus number to give users 
Scenario​​:  
Your friend from Portland is currently en route to visit you today on a train traveling from the 
Portland Union Station to the Seattle King Street Station. Track the train to see if it will be arriving 
on time.  
Mid Task Prompt​​: “Now see if you can find an alternate method to track a train”  

33 
Appendix V: Participant Packet 
Consent Form 

Participant: ________ 
\ 
I agree to participate in the study conducted by the Amtrak.com Usability Test Group at the 
University of Washington, Human Centered Design and Engineering Department. 
 
During this study: 
- I will be asked to perform certain tasks on a computer 
- I will be interviewed regarding the tasks I’ve performed 
- I will be recorded through audio and video during the session 
 
I understand and consent to the use and release of the recording by the Amtrak.com Usability 
Test Group at the University of Washington, Human Centered Design and Engineering 
Department. I understand that the information and recording is for research purposes only and 
that my name and image will not be used for any other purpose. I relinquish any rights to the 
recording used by Amtrak.com Usability Test Group at the University of Washington, Human 
Centered Design and Engineering Department without further permission. 
 
Participation in this usability study is voluntary. All information and recordings will remain strictly 
confidential. The descriptions and findings may be used to help improve the Amtrak.com website 
design. At no time will my name or any other identification be used. I have the ability to withdraw 
consent to the experiment and stop participation at any time. 
 
The recordings are for research purposes only and will only be used by the involved students in 
class and portfolio presentations.  
 
Below is my signature indicating that I have read and understood the information on this form and 
that any questions I might have about the session have been answered. 
 
Date:_____________ 
 
 
Participant’s printed name: __________________________________________________ 
 
 
Participant’s signature : ____________________________________________________ 
 
Thank you, we appreciate your participation and your time. 

34 
 Participant ____ 
Task 1  
You and a friend want to travel to Portland, Oregon on Friday, November 30th for a concert the 
following day. You will be leaving from Seattle, Washington. You would like to find the best ticket. 
 
Scenario A: ​Your friend uses a wheelchair that may be stowed during travel. This friend 
also has a service dog that will assist them during travel. 
 
Scenario B: ​Your friend is a member of the military. Both you and your friend will be 
transporting your bikes with you on your trip. 
 
How difficult was it to complete the task?  
Easy Difficult 
1 ------------- 2 --------------- 3 --------------- 4 ---------------- 5 
How understandable was the process of completing this task? 
Understandable Confusing  
1 ------------- 2 --------------- 3 --------------- 4 ---------------- 5 
Anything else you want to tell us? 
 
  
 
   

35 
 Participant ____ 
Task 2 
You live in Seattle, but need to attend a conference on November 20th in Portland (station PDX). 
After the conference, you need to visit a friend in Vancouver, BC (traveling on the 21st). After your 
visit, you need to return to Seattle on the 22nd. Book this trip as one journey. 
 
How difficult was it to complete the task?  
Easy Difficult 
1 ------------- 2 --------------- 3 --------------- 4 ---------------- 5 
How understandable was the process of completing this task? 
Understandable Confusing  
1 ------------- 2 --------------- 3 --------------- 4 ---------------- 5 
Anything else you want to tell us?  
 
 
 
 
 
Task 3 
You are thinking about traveling overnight with a friend on the Coast Starlight train later this 
month. Your friend is gluten-free and wants to know if there are meals available for their dietary 
restrictions.   
 
How difficult was it to complete the task?  
Easy Difficult 
1 ------------- 2 --------------- 3 --------------- 4 ---------------- 5 
How understandable was the process of completing this task? 
Understandable Confusing  
1 ------------- 2 --------------- 3 --------------- 4 ---------------- 5 
Anything else you want to tell us?  
 
 
 

36 
 Participant ____ 
Task 4 
Your friend from Portland is currently en route to visit you today on a train traveling from the 
Portland Union Station to the Seattle King Street Station. Track the train to see if it will be arriving 
on time.  
 
How difficult was it to complete the task?  
Easy Difficult 
1 ------------- 2 --------------- 3 --------------- 4 ---------------- 5 
 
How understandable was the process of completing this task? 
Understandable Confusing  
1 ------------- 2 --------------- 3 --------------- 4 ---------------- 5 
 
Anything else you want to tell us?  
 
 

   

37 
 Appendix VI: Notetaker Data 
Task 1 
Partic Successes  Errors  Non Verbals  Assists  + Comment  - Comment  Other 
ipant 

1    went to tickets  Distressed by  Asked if bike    "Where do I  Looked under 

w/o selecting  popup  route counts  select the  more options 
a passenger  for wheelchair  price?"  for passenger 
with  storage  w/ disability. 
disabilities     
HAD TO GIVE  Had to assume 
LOCATION  which station is 
OF PRICES  the right one 
 
 

2    Clicking the    PDX station,    "Why are  Filter menu, 

wrong  needed help  there so  difficult to sort. 
location  finding/clickin many  Finding cheap 
  g prices,  popups?"  tickets 
Failed to find  "On 
Mil discount  principal I 
would leave 
this site. 
Maybe try 
an app or 
different 
service." 

3  Got to routes  Did not see  Seemed        Looked 

screen easily  disability flag.  confused by  through 
  popup  popups for all 
Completely  classes 
missed   
service  Opened the 
animal/wheelc baggage info 
hair page  menu for "what 
I can stow" 
 
Using a lot of 
popups 
 
Delved through 
a ton of pages 
in the popups 

4  Found the    "Oh, that's a      "I don't like  "And now I wait 
discounts  cute dog"  that I the  for it to load" 
dropdown  price is  ... 
(first one to)  separated  "What is NARP" 

38 
   from the 
Found the  seat" 
bicycle thing   
easily.  Didn't like 
add to cart 
button 
... 
Wasn't clear 
if it was 
return trip 

5    Didn't know    Told the       

which  participant 
stations to  which 
pick  stations to 
use.  

6  Got stations  "How do I just  Typing seems  Had to tell  "Here's the  "Strange  "I wonder if 
and  say I want it?"  difficult  that there  full list of  that  they'll actually 
discounts  [After seat  was no  options"  companion  let us to sit 
correct  popup]  specific seat  option  together."; "Is 
easily    selection.  exists"  there a seating 
Clicked      chart?"; 
baggage info  Had to point  [Ticket 
looking for  out add to  page] 
add to cart  cart button  "That's 
weird, this 
part scrolls 
separately" 

7  Found  Next button           

person with  not obvious 
disability  for "add to 
option  cart" 

8  Easiest time            "These prices 

of any  fluctuate so 
participant.  much." 
Found 
military 
discount, 
found "add 
to cart", and 
easily found 
bike option 
 
   

39 
 Task 2 
Partici Successes  Errors  Non Verbals  Assists  + Comment  - Comment  Other 
pant 

1  Found Multi  Took a    Asked about    Would be  Focusing a lot 

City Quick.  moment to  dates;  nice to see  on the times of 
  find select  inconsequen everything  things 
Successful  button  tial  on one 
city entry  screen 
process. 

2    Typed in  Scrolling up    Smart to  Dislike the  Reasonable 

wrong cities  and down a lot.   have border  layout. "If I  time, not tired 
crossing info  was  upon arrival. "A 
at top.  red-green  lot of the 
Noticed  color blind it  busses don't 
which  might not  allow baggage. 
options  have stood  " 
were  out." 
transfers. 

3  "Multi City            Reading a lot 

Sounds  more than the 
Relevant"  others did 
   
Got through  "Shows me the 
info input  address; thanks 
easily.  I guess?" 
   
Noticed tiny  "I wouldn't buy 
sold out text.  food on a train" 
   
Completed  Picked a bus 
task with no 
errors 

4  Got through        "Multi-city,  "Oh that's  "This is a bus, 

itinerary  that sounds  weird, this is  not a train, 
page super  like what I  so different  that's not 
easy.  wanna do...  from [Task  allowed!" 
  oh, complex  1]".  ... 
rail?"    Surprised by 
"I'm gonna  rooms options 
click the 
first one, 
then I have 
to scroll 
allllll the 
way down" 

5  found multi  doesn't know    had to point    Wanted it to   

city fast  how to add  out the  give her 
tickets and  select button  prices 

40 
 thinks there  for each   
might not be  segment  none are at 
any available  good times 
 
can't 
un-select a 
segment 
once 
selected 

6  Found Multi  Didn't notice        "Oh this is a   

City pretty  that one route  completely 
easily  was a bus  different 
    format, not 
Got through  Didn't see  confusing at 
multi-city  accessibility  all!" 
setup just  options.   
fine  "Haven't 
given me an 
option to 
say I'm 
disabled" 
 
"I guess it 
could be 
worse" 

7  Entered in          "I'm scared"  Shorter travel 

info  ...  time, earlier for 
successfully,  Loading  conference 
completed  problems, 
task  too 
expensive, 
would 
probably do 
single trip 
travel (like 
google 
flights) 

8  Easy once  Picked  After  Hint to get    "These are  Looked for a 
multi-city  multiple  "multi-city" and  them off of  so  student 
found  normal trips  trip description,  the individual  expensive.  discount, but 
and tried to  made face like  trip method  That's not  there was 
use cart to  😬  what I like."  none. 
purchase  Leaned in for  ...  ... 
them all  small text.  (upon  "Amtrak...what 
together  finding the  secrets are you 
...  multi-city  hiding?" 
Once on  option) 
correct track,  "That's 
didn't realize  stupid" 
that there  ... 

41 
 were things to  "Doesn't 
select initially  say price. 
Don't like 
that." 
 
Task 3 
Partici Successes  Errors  Non Verbals  Assists  + Comment  - Comment  Other 
pant 

1  Found Coast    Getting in to    "It's obvious  "I don't like  FUCKIN HATES 
Starlight  view very  that there  Julie"  JULIE 
fast!!!!  small text at  are things     
bottom of the  that are free  No GF icons  Found coast 
menu  of glute"  on the  starlight, then 
  menu  menu and 
Found  searched the 
additional  menu for GF 
GF info on  options.  
special diet 
reqs 

2  Used Julie to  Types    "It will not    Dislikes  Julie go away, I 
go to Coast  Starlight into  show up in a  julie. "Who's  don't need you 
Starlight. ...  destination  normal train  Julie? I  anymore. ... 
Special  bar.   booking  don't like  Looking to 
Dietary  search."  that." ... "I  book a train?  
Requirement don't want 
s page, says  to read all 
no options.  of this" (re: 
  menu) From 
this there's 
nothing that 
says there's 
a gluten 
free option, 
you'd have 
to go 
through 
every 
option. ... "I 
don't trust 
Julie. I don't 
like the 
image of 
the woman 
that pops 
up." 

3  Used Julie to    Leaning in to  Hint about  I didn't  "That was..."  Opened all 
get to CS  view small text  alternate  expect  with Julie  three menus at 
train  on menu  method.  search to  popup  once 

42 
     work, but it   
Used ctrl F to  Made face  did.  using a ton of 
find word  when forcibly  keyboard 
Gluten in  dragged to  shortcuts 
menu.  bottom by 
Julie 

4  Got to meals          Doesn't like  "I would've 

and dining far  GF policy.  searched first 
quicker than    outside of this 
anyone else.  "WHO'S  study" 
  JULIE" 
Found data 
via search 
super quick. 

5  Went to        Pleasantly  Didn’t like   

experience,  surprised by  72 hr 
on board  kosher  advance 
meals  notice 

6  Found meals    Went to      "Nothing  "Curious about 

page  bottom of  peanut-free,  the disabilities 
relatively  page looking  guess we'll  one but I'll stick 
quickly  for options  kill  to the task" 
someone?"   
  julie seems to 
"Oh there's  be mildly 
Julie eh-  entertaining 
Julie's 
weird... you 
make it 
seem like 
you have 
the thing, 
but you just 
say fuck 
you" 
 
"Julie tells 
lies" 

7  Success...so Goes to        Re: Julie  Used Ctrl+F 

mehow.  "Book a Trip"  "Nope.  significantly 
Through a  section first  Don't like 
Seattle to  this." (even 
Chicago train  though was 
booking page  taken to 
...  correct 
Also  page, went 
searched  back) 
through  ... 
google and  "Now I want 

43 
 found answer  to think 
immediately  about 
throwing 
this out a 
window" 
... 
(while 
scrolling 
down main 
page 
looking for 
gluten free 
options) "I 
feel like 
there 
should be 
an option 
here." 

8  Did find info          Expected  "Oh my gosh, I 

menu info  don't have my 
to be on  glasses on" 
main page  (leans in closer 
area,  to screen) (too 
checked  small text on 
drop-downs  menu) 
second 
 
Task 4 
Partici Successes  Errors  Non Verbals  Assists  + Comment  - Comment  Other 
pant 

1  Found TS  Went to  Made face  Had to point    "Confused   

Immedately,  schedules.  while saying  out train  that train 
but then    "I'm gonna ask  status menu  says from 
switched to  Julie for help"  Eugene" 
find   
schedules.  Was surprised 
(Due to  by the map 
autofiller?)  (not what she 
  was 
Found large  expecting) 
map tracker 
via search, 
then found 
train! 

2  Uses julie to  Navigates    Hint: Train    Not   

search track  away from  Status  confident in 
train. Goes to  map view.  button. "I  the results 
map view.  The arrows  keep not  of the 

44 
 Clicks on the  didn't show  wanting to  second one, 
correct route,  up first time.   look over  because it 
finds that  there  doesn't 
train is late  because  show other 
(20 minutes)  that's where  stops along 
Ruby's face  the way. 
is." 

3    Looked up    Had to tell    Station  "Saw a track 

wrong station  him to use  input is  button so I'm 
with track  train status a  unclear  gonna go 
train  different way    looking for 
  "How can I  that. 
Went to  find the   
schedule  train on the 
while looking  map" 
for second 
method, then 
used julie 
unsuccessfull
y. 

4  Found train    Went straight      "Looks like  "That wasn't 

status very  to zooming in  they're  very helpful 
quickly  on PDX-SEA  halfway  Julie" 
  area on map,  there   
then found  because  "Interesting 
search box  the thing is  that there was 
in the  such a 
middle"  difference in 
the two 
methods." 

5  Map first  Didn't know if    Where train    Was  Thought it 
to put start or  status is  confused by  would be easy 
end station  it listing the 
  final station 
Clicked   
around  Wanted 
bottom to find  more info 
other tracking 
 
Went to 
schedules 

6  Found the      Had to    "I can't use  "Go away Julie, 
518  mention that  control  why are you 
successfully,  it was train  zoom  still here?" 
and was able  518  though, I 
to identify if it  don't have 
was late or  enough 
not.  appendage
  s!" 

45 
 Used the   
Train status  "HOW DO I 
successfully  LEAVE 
THIS" 
[trying to 
exit train 
map] 
 
"Why is 
there a 
service alert 
if my train 
isn't on that 
track?" 

7  Found both          "What do   

options  you mean?" 
quickly, used  (@error 
search bar on  message) 
map page to 
find correct 
train 

8    Attempted to        (upon  "Portland 

follow train  seeing map  Question 
route lines  tracking  Mark?" 
before  thing)   
realizing they  "Baah.  Train status 
could input  Baah.  differed from 
the train  Escusi?  map 
number  What the 
instead  fuck." 
...   
Attempts  "There's 
looking at  literally a 
train  train status 
schedules,  button, I'm 
"error" ->  an idiot" 
"Nani"  ... 
...  (after 
For train  seeing that 
status, did  their train # 
not enter the  info and 
correct  destination 
station the  info wasn't 
first time  saved after 
because they  returning to 
weren't sure if  home page) 
it was arrival  "Amtrak 
station or  you ho" 
departure 
station 

46 
 

Appendix VII: Data from UserZoom 

Task 1  
Time on Task 
Mean  Median  Min  Max  StDev 

7m 13s  7m 46s  3m 5s  9m 54s  2m 32s 

 
Unique Pageviews 
Mean  Median  Min  Max  StDev 

3.7  3  2  8  2.1 
 
Clicks 
Mean  Median  Min  Max  StDev 

53.3  47  23  103  24.1 

 
How hard was it to complete this task?  

 
 
How confusing was the process of completing this task? 

 
 
Other comments typed into user zoom: 
- :( amtrak why 
- Not much information on screen at once when choosing tickets. 
- add to cart button is stupidly placed 
- I saw the one next button and hit it bc it was rlly obvious 

47 
 
Task 2 
 
Time on Task 
Mean  Median  Min  Max  StDev 

6m 7s  5m 48s  4m 25s  9m 4s  1m 28s 

 
Unique Pageviews 
Mean  Median  Min  Max  StDev 

3.6  3  3  5  0.7 
 
Clicks 
Mean  Median  Min  Max  StDev 

52.1  42  31  101  21.6 

 
How difficult was it to complete this task? 

 
 
How confusing was the process of completing this task? 

 
 
Other comments typed into user zoom: 
- Need more info about crossing the border 
- I’m a cheap ho they need to tell me the prices of the tickets 
   

48 
Task 3 

Time on Task 
Mean  Median  Min  Max  StDev 

4m 53s  4m 12s  2m 56s  7m 12s  1m 37s 

 
Unique Pageviews 
Mean  Median  Min  Max  StDev 

5.4  5  3  10  2.1 

 
Clicks 
Mean  Median  Min  Max  StDev 

52.1  21.7  14  44  14 

 
How difficult was it to complete this task? 

 
 
How confusing was the process of completing this task? 

 
 
Other comments typed into user zoom: 
- Amtrak your buttons are bad and Julie is inherently not trustworthy 
- Amtrak needs gluten free options 
 
   

49 
Task 4 
 
Time on Task 
Mean  Median  Min  Max  StDev 

4m 28s  4m 23s  2m 26s  6m 14s  1m 12s 

 
Unique Pageviews 
Mean  Median  Min  Max  StDev 

3.4  3  3  4  0.5 
 
Clicks 
Mean  Median  Min  Max  StDev 

38.1  33  23  54  11 

 
How difficult was it to complete this task?

 
 
How confusing was the process of completing this task? 

 
Other comments typed into user zoom: 
- Amtrak why do all of your links go to the same map 
- The map was overwhelming and the box to enter the train number should have been 
bigger. On the "train status" screen they should specify station of departure or arrival 
because honestly what the fuck 

50 
Appendix VIII: Affinity Diagrams 

 
 

   

51 
Final Reflection 
The study provided valuable insights about the Amtrak website’s usability problems related to its 
common task flows regarding ticket purchasing and information searching. Our findings indicated 
glaring issues in the way disabled passenger tickets are purchased, the ticketing page, and the 
search feature of the site. However, these findings were concluded through testing with our eight 
users in a particular age range.  
 
If more time was permitted for this study, our group would have tested with more participants. 
Conducting tests with a larger sample size would produce more accurate results. A total of 15-20 
usability tests would be a sufficient number of participants to provide analytical data. With a larger 
pool of users, we would test with participants over a larger age range that is more representative 
of Amtrak users since their customers are of a wider age range than the ones included in this 
study. People of different age ranges may have varying preconceptions of web standards and 
may interact with the website differently from one another. These are influential factors in how 
people interact with the website, so a larger age range would account for these differences.  

52