Amtrak.com Usability Study Plan 

Group 3: ​Ruby Davis, Sierramatice Karras, Reine Abubakar, Connor Garrett 
October 28th, 2018 

Purpose 

Amtrak is a railroad service that provides medium and long-distance travel for people in the 
United States and Canada. Most of Amtrak’s operating costs are covered by revenue generated 
by passenger fares, as well as from government funding [1].  
 
The website aims to promote their business by providing information about their services, as well 
as facilitating the ticket-booking and trip-planning process. However, key features of of the site 
are lacking in their ease of use and efficiency. The goals of this usability study are to assess the 
process of purchasing tickets and researching relevant trip information and find ways to improve 
these common task flows. Usability tests can areas of confusion, speed bumps, and common 
errors. Improving the usability of these features will make for a more pleasant and efficient 
experience, encouraging users to purchase tickets and return for future trips, thus increasing 
Amtrak’s revenue and consumer loyalty.  
 
[1] ​National Railroad Passenger Corporation. "Basic Amtrak Facts". Amtrak National Facts, FY 2011. National Railroad 
Passenger Corporation. Archived from the original on August 7, 2012. Retrieved October 18, 2018. 
 

Research Questions 

1. Are the users able to successfully purchase tickets for their specific needs?  
2. Can users find the the most efficient ticket for their itinerary at the best price? 
3. Can users get through common tasks without reaching out for assistance?  
4. How should the site be navigationally organized to improve efficiency for inexperienced 
Amtrak customers?  
5. Can users fully comprehend the train tracking information without difficulty? How quickly, 
easily, and successfully can users navigate between the different tracking systems to find 
the information they need?  
6. Are users able to successfully use the search feature?  

 
1 
 

Participant Characteristics 

We will aim to recruit 9 people in the age range of 18 to 70 and they will be half women and half 
men. We will be focusing on infrequent and new users who we define as people who have used 
the site less than four times in the last year. They will travel at least one to two times a year and 
plan their own trips without the help of a travel agent. We will exclude people who work in the 
travel industry. 

Method 

1. Welcome 
a. Greeting  
i. “Hello! Thank you for coming. Today we will be conducting a usability 
study of the Amtrak website. [Summary of schedule for the session]” 
b. Complete consent agreement 
c. Instructions on procedure 
i. Do tasks as you would do anywhere 
ii. You are testing the product, you are not being tested 
iii. Provide both positive feedback and constructive criticism 
iv. Read tasks out loud before you begin 
v. Think aloud while you complete the tasks 
2. Initial background questions 
3. Tasks 
4. Post Interview 

Task Lists 

1. Starting on the Home Page, search for train tickets that fit a given scenario covering 
different types of travelers and desires for the route (bike carry-on, price, length of trip). 
Compare the tickets to find either the the fastest, best deal, or least transfers as specified 
in the senario. 
a. Starting Condition: Home Page 
b. Success: Find a ticket that meets the given criteria 

 
2 
 

c. Fail: The user is unable to purchase a ticket with for the route with the specific 
constraints 
d. Needs: Train routes that we know have the necessary amenities for a given 
situation (dogs and bikes are not allowed on all routes)   
2. Find a multi-city route that fits a given scenario covering different types of travelers and 
desires for the route (bike carry-on, price, length of trip).  
a. Starting Condition: Home Page 
b. Success: Find a ticket that meets the given criteria 
c. Fail: The user is unable to purchase a ticket with for the route with the specific 
constraints 
d. Needs: Train routes that we know have the necessary amenities for a given 
situation (dogs and bikes are not allowed on all routes)   
3. Use the search feature to answer a given question. 
a. Starting Condition: Home Page 
b. Success: Find the information that answers the user’s question 
c. Fail: The user does not discover the answer to their question 
d. Needs: Search terms we know will highlight the issues in the search page 
4. Ask a user to track the status of a train and then navigate from the schedule train tracking 
system to the map train tracking system 
a. Starting Condition: Home Page 
b. Success: Get between the trackers and track the same train 
c. Fail: The user is unable to locate the status of the train or they are unable to 
navigate from the schedule to the map  
d. Needs: A train/bus number to give users 
 
Study Design:  
We will ask all of our participants to do all tasks in the same order and with the same details of 
the provided goals (same constraints, etc.) The usability study session should last approximately 
45 minutes. The first 5-10 minutes of the session will be dedicated to participant-researcher 
introductions and questions about the user’s travel habits and experience with Amtrak. The 
following 25 - 30 minutes of the study will be devoted the participant’s attempt at the given tasks. 
The final 10 minutes of the session will be spent conducting the post-interview questions.  
 

 
3 
 

Initial Questions:  
1. How frequently do you travel?   
2. How often do you travel by rail?  
3. Could you tell us about your experience with booking transportation for a previous trip 
(including planes, cars, etc.)? 
4. Do you typically travel with luggage? 
5. When was the last time you used Amtrak.com?  
 
Between Tasks:  
1. Ease of use rating 
2. How understandable was the process of completing that task? 
3. Probe for more specifics on feelings of task 
 
Post Interview:  
1. Is there anything you found particularly frustrating? 
2. What do you like about the website? Why? 
3. What do you dislike about the website? Why? 
4. If you could change one thing about this site what would it be?  

Logistics 
Test Environment 
● Test Dates: In the week of November 11th to 17th Exact dates currently TBD. 
● Test Location: Neutral space on UW campus, reservations have been made in Sieg. 
 
Equipment 
● Testing will be done using the UserZoom software and two laptops belonging to the 
researchers; participants do not need to bring their own laptops. 
● Notepad for note taking. 
● Timer to sync notes and footage. 
 
 

 
4 
 

Moderator Role 
● 2 researchers present, one functioning as moderator and one as notetaker. 
● The moderator will guide the participant through the tasks, answering any questions they 
may have. 
● The Note taker will not take an active role in administering the tests, however they may 
interject with questions if they feel they may provide useful insights. 
● Additional note takers may participate however they are not required. 
 
Related Logistics 
● It may be interesting to try multiple types of venues as these tasks could be conducted 
anywhere. 
● Consider bringing a small stipend for participants, such as chocolate or a $5 gift card. 
● If a user wants to, they may bring their own laptop. However, this may complicate the 
situation and isn’t explicitly recommended. 
● We may occasionally go to location of the user if it makes sense for the study and is 
reasonably nearby. We would ask the participant about which location they would be 
more comfortable with and elect to conduct the study in the environment that they prefer. 
 

Facilitation Approach 

Usability sessions will be conducted in a controlled environment with one participant and two 
researchers present. It will take place at a relatively quiet room on the University of Washington 
campus (such as an empty classroom or a secluded area of a larger communal room). This will 
simulate a quiet, comfortable environment in which a user may typically plan trips and itineraries.   
 
Participants will use either a Windows PC or a MacBook of one of the researchers. Through 
UserZoom, and with permission from the user, the participant will be recorded aurally and visually 
(by screen recording and recording the participant’s facial reactions).  
 
During the session, one researcher will act as a moderator, guiding the user through tasks and 
asking questions. The second researcher will act as a notetaker, writing down significant 
observations about the user and their tasks. 
 

 
5 
 

Data Collection 
1. Sessions Recording 
a. Participant’s face  
b. A view of their screen 
c. Audio of session 
2. Survey Data  
3. Quantitative Data 
a. Time on task 
b. Error rate within tasks 
c. Task completion rate of success 
d. Number of assists 
4. Qualitative Data 
a. Quotes from participants 
b. Responses to initial and post interviews 
c. Ease of use ratings 

Data Evaluation 

We will share the following information with clients: 

● Average task time 
● Average error rates 
● Rate of success 
● Insights from survey data 
● A sample of insightful quotes from participants to show user frustration or satisfaction 
 

Reporting 

The results of this study will be delivered via a written report and an oral presentation. We will 
also share preliminary results in a short written report. 
Preliminary Results:​​ November 20th 
This report will describe the research methods used thus far, the participants, and the 
most significant preliminary findings. 
 
 

 
6 
 

Results Presentation: ​December 6th 

This presentation will provide an overview of the study, a description of methods and 
procedures, the major findings of the study (including video, quantitative, and qualitative 
evidence), and recommendations for possible changes. 
Final Report:​​ December 11th 
This will be the most thorough reporting method. The formal report will include an 
overview of Amtrak, the study goals, and research questions, as well as a detailed 
description of the methods used to complete the study, including the metrics used. The 
results and an analysis of the results will also be included. The report will present detailed 
findings and recommendations for possible changes and will conclude with a proposal of 
future steps. 

 
7