Usability Test Steam

store.steampowered.com An online store to purchase and play

computer games.

Nathan Park
ENGL 317 4/16/17
Steam User Experience Analysis Park | 1

Contents
Introduction ................................................................................................................................................ 2
Methodology .............................................................................................................................................. 3
Tasks ....................................................................................................................................................... 3
Test Environment .................................................................................................................................... 4
Evaluation Criteria............................................................................................................................... 4
Results ........................................................................................................................................................ 5
Task 1 Creating an Account .................................................................................................................. 5
Task 2 Search for a Game ..................................................................................................................... 5
Task 3 Purchase a Game ...................................................................................................................... 6
Task 4 Download a Game ..................................................................................................................... 6
Task 5 Use the Discovery Queue to Find a Game ................................................................................. 6
Task 6 Follow a Steam Curator ............................................................................................................. 7
Task 7 Find a game through a Curator Review ..................................................................................... 7
Task 8 Add a Friend .............................................................................................................................. 7
Task 9 Change your Profile Picture ....................................................................................................... 8
Task 10 Change your Profile Description .............................................................................................. 8
Task 11 Join a Community .................................................................................................................... 8
Task 12 Create a Community ............................................................................................................... 9
Task 13 Change your Community Profile Picture ................................................................................. 9
Task 14 Change your Community Description ..................................................................................... 9
Task 15 Invite Someone to your Community ..................................................................................... 10
Task 16 Remove Someone from Your Community ............................................................................. 10
Task 17 Add an Officer to your Community ....................................................................................... 10
Task 18 Add a Game to your Community Curation List ...................................................................... 11
Task 19 Download a Mod from the Steam Workshop........................................................................ 11
Task 20 Rate a game on Steam Greenlight ......................................................................................... 11
Recommendations .................................................................................................................................... 12
References ................................................................................................................................................ 13
Appendix: Test Data .................................................................................................................................. 14
Steam User Experience Analysis Park | 2

Introduction
This technical document is an analysis of usability for the website and gaming platform Steam. Steam is
an online service that provides a storefront to purchase games as well as a platform for multiplayer
gaming, socializing, and critiquing of games. As of February 2015, Steam has over 125 million active
users this number is rising steadily every month. Steams unique position as the largest PC gaming
platform makes it even more important to analyze due to its widespread influence throughout the
industry. The tools it provides to discover and play games are being improved often this is very
important, as there are about 8 games being released per day (Lahiti). it is becoming increasingly
important to have powerful discovery tools to filter content for each user.
Steam User Experience Analysis Park | 3

Methodology
Tasks
20 tasks have been chosen to rigorously test Steams user experience design. Each of these tasks are
things that I would consider integral features of the Steam user experience.

1. Create an account
2. Search for a game
3. Purchase a game
4. Download a game
5. Use the Discovery Queue to find a game
6. Follow a game curator
7. Find a game through a curator review
8. Add a friend
9. Change your profile picture
10. Change your profile description
11. Join a community
12. Create a community
13. Change your community profile picture
14. Change your community description
15. Invite someone to your community
16. Remove someone from your community
17. Make someone an officer of your community
18. Add a game to your community curation list
19. Download a game modification from the Steam Workshop
20. Rate a game on Steam Greenlight
Steam User Experience Analysis Park | 4

Test Environment
Information in this document is valid and up-to-date for the current version of the Steam website and
client. The Steam client used was built March 22, 2017. Barring any major client and website changes,
the basic experience recorded in this document should remain accurate.

The tests were conducted on a Dell XPS 13 laptop running Windows 10 Pro x64 Version 1607 Build
14393.953 with an Intel Core i7-4500U CPU and 8 GB of RAM. The browser used was Google Chrome
57.0.2987.133 (64-bit). Hardware differences should be insignificant for this platform, but operating
system differences (i.e. Windows vs Mac OS vs Linux) could see different Steam client behavior. The web
experience should be consistent across all operating systems.

Evaluation Criteria
The criteria used to evaluate the tasks listed above are as follows:

Effective: This task was completed without using support resources.

Efficient: This task was completed in a reasonable amount of time/clicks.

Engaging: The style of this tool/process enhanced your experience.

Error Tolerant: This task highlighted errors and offered solutions to the errors.

Easy to Learn: This task was easy to learn how to execute.

These criteria were rated on a 1-4 Likert scale, where 1 is Strongly Disagree, 2 is Mildly Disagree, 3 is
Mildly Agree, and 4 is Strongly Agree. I used a 4-point system to avoid the midpoint problem, where a
responder may feel unmotivated to pick a response in one direction or the other. This point scale leads
to a total of 20 points per task, and a total of 400 points across all tasks.
Steam User Experience Analysis Park | 5

Results
Task 1 Creating an Account
Creating an account is the first
impression of any online service, so it is
important that it goes as smoothly as
possible.

Fortunately, this account creation page

is well-made. The most important
feature of an account creation page is
the error tolerance features. Each field
except for the captcha has some form of
error tolerance the first field checks
for username availability without
submitting the form, the 2nd and 3rd
fields tell the user if the passwords are
valid and the same, and the email fields
tell the user if the email being used has
already been registered. Figure 1. Creating an Account

The only thing that drags this task down is that finding the account creation page was not immediately
obvious from the front page.

Effective: 4; Efficient: 3; Engaging: 2; Error Tolerant: 4; Easy to Learn: 3; Total: 16/20

Task 2 Search for a Game

A quality search feature is the best
way for a user to locate something
that they are looking for. If a user is
looking for a specific product and
they know what its called, this is
the best way to find it. The search
feature includes some
autocomplete features and lists top
results before the user submits it.
This helps to reduce the amount of
clicks for the average user since
popular games that are commonly
searched for will be on this list. The
search function on this website
does everything right. Figure 2. Searching for a Game.

Effective: 4; Efficient: 4; Engaging: 4; Error Tolerant: 4; Easy to Learn: 3; Total: 20/20

Steam User Experience Analysis Park | 6

Task 3 Purchase a Game

The process of purchasing any product online is
filled with tedium, and this is no different. It is
hard to make spending money online pleasant
considering the number of fields that need to be
filled out. The error tolerance is okay, but not
great. If the user makes a mistake, some sensitive
fields reset and must be reinput before the user
can proceed. The upside is that payment
information can be saved, but this is a security
risk that many users may not wish to take.
However, the process is straightforward, so no
help was required to complete it.

Effective: 4; Efficient: 2; Engaging: 2; Error Figure 3. Purchasing a Game.

Tolerant: 2; Easy to Learn: 3; Total: 13/20

Task 4 Download a Game

Downloading a game is very simple. Once a game is
added to the library, it can be double-clicked on to be
downloaded. After deciding where the game will be
installed, the Steam client handles the rest of the
process. There is no error to be made and it is a solid
solution to the problem.

Effective: 4; Efficient: 3; Engaging: 4; Error Tolerant: 4;

Easy to Learn: 4; Total: 19/20

Figure 4. Downloading a Game.

Task 5 Use the Discovery Queue to Find

a Game
The Discovery Queue is a feature that shows the
user a variety of games to try to show them
something they might like. This is not very
effective initially, but as the user plays and rates
games, the queue will start to show personalized
recommendations. Often the queue can show
undesirable games but it can also find hidden
gems that the user might not seek out
themselves.

Effective: 3; Efficient: 3; Engaging: 4; Error

Tolerant: 4; Easy to Learn: 4; Total: 18/20
Figure 5. Using the Discovery Queue
Steam User Experience Analysis Park | 7

Task 6 Follow a Steam Curator

The Steam Curators program allows users to review
and recommend games to other users. Following a
curator gives the user access to a feed of games that
they have reviewed. This is another method of finding
good games more effectively by finding curators with
similar taste. It was a little harder to navigate to the
Steam Curator page (a common theme with many
features of the website), but once arriving there it was
easy to follow a curator.

Effective: 3; Efficient: 3; Engaging: 3; Error Tolerant:

3; Easy to Learn: 3; Total: 15/20
Figure 6. Steam Curators Listing

Task 7 Find a game through a Curator Review

By navigating to a specific curator
page, a list of game reviews can be
seen. Through the review, there is a
direct link to the store page to
purchase the game. If the user can
navigate to the Steam Curator page,
its easy to select a curator and find a
game through one of their reviews. A
review from another user can be much
more engaging and relatable than a
recommendation by an automated
system, so this method of finding a Figure 7. Steam Curator Review
game is very effective for many users.

Effective: 4; Efficient: 4; Engaging: 4; Error

Tolerant: 4; Easy to Learn: 4; Total: 20/20

Task 8 Add a Friend

Besides selling games, Steams primary focus is
on playing games together. The most basic part
of social interaction on Steam is the Friends list.
Adding a friend lets the user chat with them
over text and voice, and join multiplayer games
with them.

There is one big problem with adding friends on

Steam. Users can have overlapping screen
names, so when a name is searched it retrieves
every user with the same name, leaving the user
to sift through names and pictures until they
Figure 8. Finding a Friend
find the person they are looking for. This can
Steam User Experience Analysis Park | 8

also lead to problems with error tolerance with typos, having to re-search names and dig through
multiple pages again.

Effective: 3; Efficient: 2; Engaging: 2; Error Tolerant: 2; Easy to Learn: 4; Total: 13/20

Task 9 Change your Profile Picture

The ability to customize the profile allows for a
user to express themselves and make an identity
for people to recognize. It can be easier for
someone to be recognized based on their profile
picture than their name. The process for changing
the profile picture is simple, but again, figuring
out how to navigate to the profile page is not very
easy.

Effective: 4; Efficient: 3; Engaging: 3; Error Figure 9. Setting a Profile Picture

Tolerant: 3; Easy to Learn: 3; Total: 16/20

Task 10 Change your Profile Description

This task is similar to changing the profile picture, and is
done in the same place. It is an easy process but is hard to
locate the configuration tool. A profile description can be
more helpful if you are a public figure, but it can be nice for
anyone to write something about themselves on their
profile.

Effective: 4; Efficient: 3; Engaging: 3; Error Tolerant: 3;

Easy to Learn: 3; Total: 16/20

Task 11 Join a Community

Communities are a large part of the social aspect of Steam.
They let users group together to discuss topics, play Figure 10. Setting a Profile Description
games, and curate games. There are many large
communities on steam for affluent curators, but there are also small communities for groups of friends
to get together. Joining a community is very intuitive and easy compared to other tasks, and there is no
real way to make an error other than selecting the wrong
community, which is easily reversible. This is a very solid user
experience.

Effective: 4; Efficient: 4; Engaging: 4; Error Tolerant: 4; Easy to

Learn: 4; Total: 20/20

Figure 11. Joining a Community

Steam User Experience Analysis Park | 9

Task 12 Create a Community

Some users may wish to create a
community group for their friends.
Unfortunately, this is the least
effective task that was analyzed.
The main problem is that the page
to create a community isnt in a
spot that makes sense, forcing me
to search their support page to
figure out how to create a
community. After locating this page,
the creation process is intuitive. The
issue of hard-to-find pages is
something that is pervasive
throughout this platform and is
something that will be discussed
later in detail. Figure 12. Creating a Community

Effective: 1; Efficient: 3; Engaging: 3; Error Tolerant: 3; Easy to Learn: 3; Total: 13/20

Task 13 Change your Community Profile Picture

The configuration page for community profiles is very
similar to the page for user profiles. As such, the process
is very much the same. The only difference is that there
are no stock icons to use. The user must provide their
own image to use as a profile picture. It is possible to
have issues with picture file formats which can cause
errors, but assuming the user knows how to change the
file format of an image this isnt much of a hurdle to
overcome. This configuration page is easier to find than
the user profile one because it forces the user to go to it
after creating the community. Figure 13. Adding a Community Picture

Effective: 4; Efficient: 4; Engaging: 3; Error Tolerant: 3; Easy to Learn: 4; Total: 18/20

Task 14 Change your Community

Description
Using the same configuration page from the
previous task, the community description
can be changed. For larger communities, this
can be useful to describe to people who find
the community page what it is about. Its a
simple task, so there isnt much to say here.

Effective: 4; Efficient: 4; Engaging: 3; Error

Figure 14. Changing the Community Description
Tolerant: 4; Easy to Learn: 3; Total: 19/20
Steam User Experience Analysis Park | 10

Task 15 Invite Someone to your

Community
A community isnt a community without
people. Fortunately, inviting other users to
your community is very easy. If a
community is public, anyone can search for
it and join it on their own, but if it is not,
moderators and the creator can directly
invite someone. This is easiest if the person
being invited is on the users friends list, as Figure 15. Invited a person to the community
friends are listed on the invitation page.

Effective: 4; Efficient: 4; Engaging: 3; Error Tolerant: 4; Easy to Learn: 4; Total: 19/20

Task 16 Remove Someone from

Your Community
Moderation of a group is important to
make sure people arent being harassed.
Part of this is being able to remove
problem people from the community.
Luckily, this is very easy and intuitive to do.
There is a page specifically for moderators
to manage members of the community,
and each person has a button that can be
clicked to remove them from the
community quickly and easily. There are
possibilities to miss-click and remove the Figure 16. Removing a person from the community
wrong member, which lowers the error
tolerance on this task slightly. Users that are accidentally removed have to be re-invited via the page
from the previous task.

Effective: 4; Efficient: 4; Engaging: 4; Error Tolerant: 3; Easy to Learn: 4; Total: 19/20

Task 17 Add an Officer to your Community

Adding an officer is done through the same page as
removing a member. Adding officers is helpful
because moderation can then be done by multiple
people in order to make sure the community is able
to be moderated when the creator is gone. Adding
and removing officers is even easier because it is just
a toggle on this page, so the error tolerance is high.

Effective: 4; Efficient: 4; Engaging: 4; Error Tolerant:

4; Easy to Learn: 4; Total: 20/20
Figure 17. Adding an Officer
Steam User Experience Analysis Park | 11

Task 18 Add a Game to your Community

Curation List
One of the uses of a community is to create a curation
list. On the community page, any officer can navigate
to the Curation tab and create a review for any game
on Steam. There are a variety of different options to
put your review in different categories, and the
interface is very simple to use. Reviews can be edited
at any time, so this task is very error tolerant.

Effective: 4; Efficient: 3; Engaging: 4; Error Tolerant:

4; Easy to Learn: 4; Total: 19/20

Task 19 Download a Mod from the Steam Figure 18. Creating a Curator Review

Workshop
The Steam Workshop is a place where users can
upload various game modifications that change
gameplay, art, sound, etc. in a variety of games.
Uploading mods to the Workshop is a more
advanced topic, but downloading mods is one of
the easiest things to do on Steam. It is quite
literally a click of a button to download or delete
a mod on the Workshop. Everything about the
process is easy, and it is one of the best user
experiences on the website.

Effective: 4; Efficient: 4; Engaging: 4; Error

Tolerant: 4; Easy to Learn: 4; Total: 20/20

Task 20 Rate a game on Steam

Figure 19. Downloading a Mod
Greenlight
Steam Greenlight is another page that allows
community participation. The Greenlight program
allows independent game developers to post their
pitches for games on to the page, and the community
can vote on what games they want to see. This allows
small developers to post games on Steam without
having to seek out a publisher. The process of rating a
game is simple navigate to the Greenlight page,
click on a game, and click the rating button. This task
is also quite error tolerant because the rating can be
changed without needing to refresh the page.

Effective: 4; Efficient: 3; Engaging: 3; Error Tolerant: Figure 20. Rating a game on Greenlight
4; Easy to Learn: 4; Total: 18/20
Steam User Experience Analysis Park | 12

Recommendations
Overall, Steam is a solid platform with a decent user experience. The strongest UX tasks had to do with
game discovery whether through search, the Discovery queue, or through community curators. This
makes sense though. They profit the most when they are selling games, so helping users to find
appealing games is something that the developers would want to emphasize. The weakest UX tasks
mostly surrounded social features finding a friend can be frustrating in the sea of other people with
similar names, and creating a community proves difficult due to relevant pages being tucked away in
less obvious areas of the interface. There is still a lot of good user experience to be found in features like
the Workshop and Greenlight for the community to influence the platform. The best change that could
be made would be to improve searching for friends to make it less frustrating to find people you know
on Steam, but this may be a hard problem to solve. However, something that would go a long way to
improve the user experience would be to make some of the more important pages like account creation,
curation, and other pages easier to find. I can see the average new visitor not even making use of many
features because they are hidden behind menus.

Overall Likert score: 350/400.

Steam User Experience Analysis Park | 13

References

Lahti, Evan. "There Aren't 'too Many' Games on Steam." PC Gamer. PC Gamer, 22 July 2015.

Web. 15 Apr. 2017.

Steam User Experience Analysis Park | 14

Appendix: Test Data

Task 1: Create an Slightly difficult to find the Strongly Mildly Mildly Strongly
Account account creation page but Disagree Disagre Agree Agree
otherwise easy e
Effective 4
Efficient 3
Engaging 2
Error Tolerant 4
Easy to Learn 3
Task 2: Search for a Searched for "Portal 2" using the
Game store search bar
Effective 4
Efficient 4
Engaging 4
Error Tolerant 4
Easy to Learn 4
Task 3: Purchase a The purchase process is as simple as it can be but inputting payment
Game information is a subpar experience on any website
Effective 4
Efficient 2
Engaging 2
Error Tolerant 2
Easy to Learn 3
Task 4: Download a Downloading a purchased game is extremely simple and
Game requires very little human interaction
Effective 4
Efficient 4
Engaging 3
Error Tolerant 4
Easy to Learn 4
5: Use the The discovery queue often returns undesirable games, but it can find rare gems
Discovery Queue to that might not be found otherwise.
Find a Game
Effective 3
Efficient 3
Engaging 4
Error Tolerant 4
Easy to Learn 4
6: Follow a Game Finding the Steam Curators page was slightly difficult but once the user arrives
Curator there it is easy to follow a curator.
Effective 3
Efficient 3
Engaging 3
Steam User Experience Analysis Park | 15

Error Tolerant 3
Easy to Learn 3
7: Find a game Using the Curators page or the front page of the store, games
through a curator recommended by curators can be found easily
review
Effective 4
Efficient 4
Engaging 4
Error Tolerant 4
Easy to Learn 4
8: Add a Friend Adding a friend can be difficult without a direct link to their community page.
Due to overlap of screen names, multiple people share names and the user has
to sift through search results
Effective 3
Efficient 2
Engaging 2
Error Tolerant 2
Easy to Learn 3
9: Change your Navigating to the user's profile is slightly obtuse, but the profile
Profile Picture configuration tools make this task easy.
Effective 4
Efficient 3
Engaging 3
Error Tolerant 3
Easy to Learn 3
10: Change your Changing the profile description is done through the same tools as the profile
Profile Description picture. It is easy to change and apply.
Effective 4
Efficient 3
Engaging 3
Error Tolerant 3
Easy to Learn 3
11: Join a Searched for "Extra Credits" community group. Once navigated to the
Community community hub, searching for and joining a community is an easy process.
Effective 4
Efficient 4
Engaging 4
Error Tolerant 4
Easy to Learn 4
12: Create a Finding the page to create a community was very difficult, but the
Community process for creation is simple and quick.
Effective 1
Efficient 3
Steam User Experience Analysis Park | 16

Engaging 3
Error Tolerant 3
Easy to Learn 3
13: Change your Changing the community profile settings is
Community Profile similar to changing user settings.
Picture
Effective 4
Efficient 4
Engaging 3
Error Tolerant 3
Easy to Learn 4
14: Change your Again, changing community profile settings is nearly
Community identical to user settings. Easy process
Description
Effective 4
Efficient 4
Engaging 3
Error Tolerant 4
Easy to Learn 4
15: Invite Someone If you already have someone on your friends list, it is very easy to
to your Community add them to your community group
Effective 4
Efficient 4
Engaging 3
Error Tolerant 4
Easy to Learn 4
16: Remove The administration tools make it easy to moderate your
someone from your community by removing members
Community
Effective 4
Efficient 4
Engaging 4
Error Tolerant 3
Easy to Learn 4
17: Make someone Additionally, adding an officer to help
an Officer of your moderate your community is very easy
community
Effective 4
Efficient 4
Engaging 4
Error Tolerant 4
Easy to Learn 4
Steam User Experience Analysis Park | 17

18: Add a game to Adding a game to the curation list is simple and includes a variety of options to
your community provide information about the game being reviewed.
curation list
Effective 4
Efficient 3
Engaging 4
Error Tolerant 4
Easy to Learn 4
19: Download a Downloading mods for purchased games is very easy to do. Once the user finds
Mod from the one, it is literally a click of a button to download it.
Steam Workshop
Effective 4
Efficient 4
Engaging 4
Error Tolerant 4
Easy to Learn 4
20: Rate a game on Easy to find a game and rate it.
Steam Greenlight Simple process.
Effective 4
Efficient 3
Engaging 3
Error Tolerant 4
Easy to Learn 4