Professional Documents
Culture Documents
Ball State University Achievements App Usability and Game Design Study
Ball State University Achievements App Usability and Game Design Study
Table of Contents
!"#$%&'(#)*%++,-.)////////////////////////////////////////////////////////////////////////////////////////////////////////////////)0! 12,3'4'&.)5)6,+#)7#2'89)*&%:.)//////////////////////////////////////////////////////////////////////////////////////////);! <9&-=:%$&'=9)5)>#&?=:=4=8.)////////////////////////////////////////////////////////////////////////////////////////////////////////);! "#$%&'&(#)%*!+!,-./0$#(1&'*!22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!3! 4/'5*!6$/5(!7-**&/)*!22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!8! 75$9-:!222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!;! @'9:'982) ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////)A! <*#=&>&%:!22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!?! 6#.-!,-*&0)!22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!@! B#$=++#9:,&'=92)C=-)@%&%-#)7#(#4=D+#9&)///////////////////////////////////////////////////////////////)EE! <9&#-C,$#)7#2'89)F?,98#2)C=-)12,3'4'&.)///////////////////////////////////////////////////////////////////////////////////)E0! A#&)!"#0-!222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!BC! D-#E-$!F/#$E!22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!BC! G'/)*!H/$!I'1&-9-.-)%*!#)E!J#%-0/$&-*!2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!BC! "$/H&>-!+!J/.(>-%-E!I'1&-9-.-)%*!222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!BC! K5%/$&#>!+!L->(!7-'%&/)!222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!B3! I'1&-9-.-)%!J->-=$#%&/)!I)&.#%&/)!222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!BM! 7/5)E!NHH-'%*!22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!BM! 6,+#)7#2'89)F?,98#2) ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////)EG! I'1&-9-.-)%*!2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!B8! D-#E-$=/#$E!+!J/.(-%&%&/)!22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!B;! O-P#$E*!+!J5$$-)':!22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!B;! 6#.-!A#)#0-.-)%! 222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222!B?!
Executive Summary
The purpose of this study was to create a functioning prototype of an interactive, achievement-based, mobile application to give to a group of 25 Ball State University students, and get feedback from them on how the application could be improved both in terms of usability and game design. Participants were given access to the application on their personal iOS devices, or were loaned an iOS device for the study. In this application, participants were instructed to complete achievements that were listed in the application with the understanding that they would be rewarded for their participation. Toward the end of the semester, the students participated in a focus group to discuss their findings and observations while using the application. The following is a summary of the findings from the focus groups: Usability: ! Categories & Achievements should go on the front page ! Profile summary should go on a secondary page ! Completed achievements page needed ! Certain areas need helper text or icons to explain their function ! An introduction tutorial/video would be beneficial to new users ! Push notifications wanted ! More colors/pictures to discern categories and achievements Game Design ! Categories of achievements/More achievements/Different achievements ! Leaderboards (weekly, monthly, semester, career, department specific, etc.) ! Bonus points for doing achievements with friends ! Major/Department specific categories ! Improved rewards ! Social media interactions ! Time based achievements It is recommended that many, if not all, of these recommendations be created and implemented into the app before full release of the game to the Ball State University student population. Creating a large amount of good, fun achievements should be first priority as they form the heart of the game. Categorization and sorting of achievements will greatly aid the usability of the application. Leaderboards, progression goals, and daily achievements will help create habitual usage of the application and completion of achievements in a competitive atmosphere. Finally, well-balanced material and non-material rewards will keep students interested and create a lot of free word of mouth advertisement for the game.
!&?9'$'&.)
C8! H,-&'$'D,9&2) CQ! B8! BQ! 8! Q! J#5'#*&/)! N%1)&'&%:! CQ! IH$&'#)! I.-$&'#)! C! I*&#)! B! L&*(#)&'! C!
F4,22)
BQ! @! S! ?! ;! 8! M! 3! C! B! Q! J>#**!
H,-&'$'D,9&2)
4$-*1.#)! M!
7/(1/./$-! @!
R5)&/$! M!
7-)&/$! S!
I8#)
BQ! H,-&'$'D,9&2) S! ;! M! C! Q! I0-! BS:$! ;! B@:$! 8! CQ:$! M! CB:$! S! CM:$! B! 3C:$! B!
All focus groups were recorded via Scott Reinkes audio recorder on his iPhone and Samuel Noble and Gloria Pavlik took down notes for all focus groups.
Survey
Participants that were unable to participate in the focus groups due to scheduling complications were asked to answer the question script used for the focus group on their own time and email the responses back to Scott Reinke. Participants who participated in the focus groups were not asked to complete this task.
Findings
As the focus groups were held in a very open format, the findings for both usability and game design will be paraphrased and briefly expanded upon. Interestingly, there was little to no conflict between ideas raised between the groups. All three groups came to very similar conclusions across the board for both usability and game design. It should be noted that this section only seeks to reveal the findings of the focus group, but not discuss implementation. Implementation will be addressed in the next major section of this report: Recommendation for Future Development.
Usability
Drawing Exercise
The drawing exercise revealed several pieces of valuable information on the effectiveness of the main interface. Participants drawings from the drawing exercise can be found in the Appendix. The most effective aspect of the design was by far the use of colored shapes in the bottom off the page. All participants drew shapes of some kind at the bottom of the page, and one participant even drew the shapes in the correct order. Most of the participants had the same or similar shapes to what was in the actual application. The second most remembered feature was the experience bar, and finally, the third most remembered feature was the navigation button (three small lines) in the top right corner. Features such as the users name, credits earned, rank, specific numbers, and section titles were only included sparingly on most drawings and often out of place. It is very clear that shapes and color use was very effective in the design of the interface, however; the use of copy was not.
Discussion
These are key areas of interest that all groups unanimously agreed were good ideas and should be implemented into the application. These areas were not only mentioned, but also discussed and defended heavily by participants. ! Categories should go on the front page ! More color/pictures to discern categories achievements One area of confusion for many participants was trying to figure out what the symbols on the main interface were and what they did. Participants explained that they first tried to tap the symbols because they thought they were buttons. This lead the participants to suggest that the achievements themselves be on the main interface page instead of under another category for easier access. The participants also
expressed that it would be beneficial to color code or create symbols for achievements that were in the same category or that could be completed in a similar fashion. ! Profile summary on a secondary page ! Completed achievements page needed These two points are different, but closely related in terms of reorganization of information. The participants expressed that they would like to be able to see a list or some sort of graphical acknowledgment that an achievement has been completed. They said that this could be done in the form of a different category for completed achievements or simply checking off or some other graphical cue on the achievement itself. Along with a possible completed achievements secondary page, the participants also said that a profile summary would be better suited in a secondary page instead of on the main page. ! Helper text or icons to explain buttons or areas ! An introduction tutorial/video would be beneficial to new users The participants all thought it would be very beneficial to new users to have a sort of introduction tutorial video or walkthrough the first time the app is opened. They explained that this would be good to explain exactly what the app is, what it does, and how the user can interact with it. Along those same lines, the participants also suggested for continued support to offer a help section or help buttons in certain critical areas such as the profile summary or achievements sections. ! Push Notifications This is fairly self explanatory and anticipated, but everyone in the focus groups agreed that push notifications would be a good addition to the application for completed achievements or when the user is in close proximity to an achievement that could be done. In reality, this was always an intended feature, but just was not implemented in the short development cycle for this round of testing.
Minor Areas of Interest
These areas of interest were mentioned by most focus groups, but not necessarily with as much interest as the major areas of interest. Most of these are smaller observations and more detail oriented in the functionality of the application. As that is the case, they are fairly self-explanatory to just be listed without further discussion. ! ! ! ! Categorization of achievements Sorting methods for achievements More colors in the interface Email and Phone numbers should be tappable throughout the application
Game Design
Discussion
These are areas of interest in the game design of the achievements app that the participants absolutely want to see in the app in the future. All of the groups agreed that these would drive more students to use the app and enjoy using the app in the process. It should also be noted that the participants were not only excited about the about the rewards to be had, but also the achievements to be completed in the application. Most participants spoke of how the application could improve both students experiences as well as benefit the University. ! Categories of achievements ! More achievements ! Different achievements The core idea here that was expressed was that the participants want more and varied achievements throughout the application and easy methods of sorting and categorizing those achievements. This is obviously something that is intended for future development, but none-the-less good to hear from the participants. The following are some ideas the students developed for broad categories of achievements: Exploration, Health, Greek Life, Future Planning, Department/Major Specific, Daily Achievements (special achievements that could only be done or done for bonus points on a given day), Time & Location Based (accumulating points per hour of library time, for instance), Sporting Events, Cultural Events, and Academic Progression (passing the writing comp, becoming a senior, finishing core classes, etc.). ! Leader boards Many participants were very excited about the idea of creating leader boards to encourage competition between students. They even suggested that there could be subsets of leader boards such as a Telecommunications leader board specifically for Telecommunications students or a fraternity or sorority specific leader board for a Greek organization, and so on. Leader boards could be reset at certain points throughout the year or semester to encourage fresh starts and new competition weekly or monthly. The participants also suggested that this could be a good place to create very special rewards for students that get the most achievements in the ladder. These rewards could be less material and more experience based such as leading the football team onto the field for homecoming, a personal lunch with the president of the University or chair of their department, or even a trip up inside the bell tower. It was noted that these types of achievements could be very enticing and greatly appreciated by students while creating minimal cost to the University. ! Bonus points for doing achievements with friends Many participants suggested that creating a way to invite friends to do achievements with them would be very fun and possibly a way to make new friends. The participants went further to say that it
10
would be nice to incentivize this behavior in some way by rewarding bonus points for doing achievements together or in groups. This is a particularly useful finding as the previous focus group conducted for the BMI project had a very heavy emphasis on students claiming that a peer who was a good influence on them was a huge driving factor in their success in college. This implementation would directly address that notion and encourage peer interaction with goals centered on values of the University. ! Rewards Needless to say, all of the students agreed that rewards would incentivize students to use the application and complete achievements, however; most students were adamant on leveraging the Universitys already established products and services. Many participants suggested using Ball State branded products such as clothing, lanyards, glass wear, and other goods as purchasable awards as well as discounts on tuition or at the Ball State Book Store. Other more unique rewards suggested were free parking passes and oops vouchers, credit on meal cards, and access to early class registration. Students also unanimously agreed that gift cards of almost any sort would always be greatly appreciated.
Minor Areas of Interest
As with the previous Minor Areas of Interest section, this section is composed of ideas that were expressed, but not discussed with as much vigor as other ideas. Also like the previous section, they are self-explanatory and do not warrant much discussion. ! ! ! ! ! Achievements for Muncie as well as on campus Use of achievements to teach students about Ball State resources Tiered achievements (achievements that must be completed in order) Social media interaction (posting achievements to social media or achievements for doing so) Breaking down larger achievements into smaller progressions of achievements
11
12
Leader Board
A secondary page should be added to include a leader board. This should include the option to look at separate leaderboards that the user is a member of if any exist and the overall leaderboard for the given time frame. The user should be able to automatically jump to their position on the leader board, sort by name and score, and search for other users by name to view their score.
13
14
Sound Effects
Sound should be used for two purposes in this application: User feedback and affirmation of desired behavior. For user feedback, small ticking or popping sounds could be used when selecting an achievement, opening a category, or switching to a different screen. For affirmation of desired behavior, each achievement animation should also have a small jingle, sound bite of a crowd cheering, or something else that is fitting for the specific achievement category. This may sound silly, but this will go a long way toward psychological satisfaction of the user when completing achievements and interacting with the application.
15
16
available as dailies, but the player would only be allowed to complete one. The player would then have to make the decision to complete the achievement that awards the most points, is the least difficult, takes the least time, or whatever other factors affect their day. One important note on daily achievements: Whichever departments or facilities that are being highlighted with a daily achievements should be notified in advance. A daily achievement to visit one of the Chefs, for example, could turn out very poorly with disappointed players if the Chef runs out of food not being prepared for the surge of business. Likewise an understaffed counseling facility, etc.
17
Finally, rewards for Leaderboard competition should be entirely unique and be newsworthy. The leaderboard competitions will be the best way to get word of mouth traveling like wildfire about the app and get students excited to participate. For the overall leaderboards, make the grand prize something that would be very impactful to the student such as waived tuition for the following semester or year, a car, a trip to the U.S. city of their choice, and so on. For smaller competitions, create smaller unique awards for the top performers such as dinner with the President, box seats at a BSU football game, or backstage passes for a concert.
Game Management
This section is a general recommendation of how to handle an on-going management of this game. As there will be thousands of players at any given time, there will be a constant need to add content, support users, and balance the economy. For those reasons, it is recommended to have three core management positions for the game: Lead Game Designer, Economist, and a Ball State University Board Member. The Game Designers duty would be to very simply to make the game fun. They would be in charge of making sure that the achievements, rewards, and progression encourage gameplay and strategy. The Game Designer would be considered the person who has the last word on any decision about BSU Achievements. The Economist would be an advisor to the Game Designer and would balance the currency and rewards systems in the game. The BSU Board Member would also serve as an advisor to the Game Designer by ensuring that achievements and rewards are in line with the values and goals of the University. These three positions would serve as a check and balance system to make sure the game remains fun, beneficial to the student, and financially under control. The general process would be for departments to send achievements to the Game Designer who would then rewrite the achievements if needed to fit gameplay and assign currency values. The Economist would then check those currency values and balance if needed, and finally the Board Member would read through the achievements and approve or disapprove of the concept. The achievements would then go back to the Game Designer and be implemented or changed accordingly.
Student Participants
Brittany Means Sarah Horn Trevor Nantais Jenny Nguyen David Jewell
Jamie Hahn
Marie Prevost
Taylor Jessup
Dan Edwards
Kenneth Winner
Joi Fortson
Caroline Everidge
Paige Schmitt
Rachel Thomas
Geoff Kimball
Brice Brenneman
Mitch Hannon
Luke Jones
Alexandria Smart
Tyler Varnau
Heather Ashby
Alea Bowling
Sarah Swingley
Christy Carpenter
What parts of the interface were easy for you to remember? What parts of the interface were more difficult to remember? Does the interface feel intuitive for you? How would you make the interface more intuitive for new users? If you could add, take away, or change any part of the application, what would it be and why? What are your overall thoughts on the design and usability of the application?
Summary of Future Features [5 minutes] I (Scott Reinke) will briefly discuss some of the future features that would be a part of the application to set the frame for the following discussion. Concept Discussion [15 minutes]
What are achievements that you think would be fun to see in the application? What rewards would be enticing to you as a player? What would drive you to regularly interact with this application? How do you feel that social media interactions would affect this application? How might the BSU achievements application influence how you interact with the University? If you could add, take away, or change any concept of the BSU Achievements App, what would it be and why? What are your overall thoughts on the BSU Achievements App?
F oc usGr oup Dr a wi ng E x e r c i s e
F oc usGr oup Dr a wi ng E x e r c i s e