How Persuasive Is A Phishing Email? A Phishing Game For Phishing Awareness

Journal of Computer Security 27 (2019) 581–612 581
DOI 10.3233/JCS-181253
IOS Press
How persuasive is a phishing email?

A phishing game for phishing awareness
Rubia Fatima, Affan Yasin, Lin Liu ∗ and Jianmin Wang
School of Software, Tsinghua University, Beijing, P.R.China
E-mails: rubia.fatima@qq.com, affan.yasin@qq.com, linliu@tsinghua.edu.cn,
jimwang@tsinghua.edu.cn
Abstract.
CONTEXT: In the current era of digital technology, social engineers are using various tactics to undermine human weaknesses.
Social Engineers target human psychology to achieve their target(s) which are in the form of data, account details, or IT devices
etc. According to our research, one of the first methods social engineers used to target victims is Phishing/Spear Phishing.
OBJECTIVE: The objective of this study is to utilize serious game to: i) educate players regarding phishing and spear-phishing
attacks; ii) make aware and educate players regarding dangers associated with excessive online information disclosure.
METHOD: In order to address the objectives we have: i) performed an in-depth literature review to extract insights related
to social engineering, phishing, game design, learning functions, human interaction, and game-based learning etc; ii) proposed
and aligned the game design with social engineering ontology concepts; iii) performed an empirical evaluation to evaluate the
effectiveness of the designed board game.
CONCLUSION: From this research study, we conclude that: i) PhishI game is useful in educating players regarding excessive
online information disclosure and phishing awareness; ii) game-based learning is an effective method for inculcating and general
cyber-related awareness in players.
Keywords: Security and privacy, human and social aspects, information assurance, social engineering, serious game,
collaborative learning, empirical evaluation
1. Introduction
Information security plays a subtle yet significant role in our daily lives. It is of great importance for
organizations around the globe to provide effective security protection. From a hacker’s perspective, in
any organization, the employees are the critical junction through which information breach can occur.
The information system security becomes more important in smart cities where all devices will (poten-
tially) be connected using Internet of Things (IoT) [75], making societies more vulnerable to attacks
[36]. Applications and Information included in IoT are more sensitive and need stringent security mea-
sures. Various studies show that human are the weakest link [27,34,50,61,63], who are vulnerable to
attack, e.g. hospitals I.T systems, smart cars, and smart phones [36,75]. Imagine a situation where an
attacker controls a house hold network system by guessing the weak password of the system. He can first
lock the house and then increase its temperature or create other pressure-building situations so that the
victim necessarily pays the ransom amount which is dreadful to its own right. Today’s technology makes
a lot of good things happen such as, mobile payment, social networks, remotely controlling of house and
* Corresponding author. E-mail: linliu@tsinghua.edu.cn.
0926-227X/19/$35.00 © 2019 – IOS Press and the authors. All rights reserved
582 R. Fatima et al. / Phishing a silent attack
vehicles, etc but, it also empowers malicious attackers which may lead to great harmful consequences
without systematic protection. Human is often considered a weakest link in Information security chain
and its necessary to aware and educate them.
Social engineering can be defined as “the art of influencing people to divulge sensitive information,
and the process of doing so is known as a social engineering attack” [50]. “In information security
terms, social engineering refers to an incident in which an information system is penetrated through the
use of social methods” [68]. One of the social engineering methods is to use phishing attacks to get the
desired information. Phishing [17] is one of the most dangerous threats to the world of information and
technology. Phishing can be explained as a fraudulent activity in which attacker gets (secret) information
such as user’s identity, account information (debit or credit card), pass-codes, etc. All of these activities
are performed through spoofed emails, messages, and websites that seem original to the target victim.
Phishing attacks follow deception theory which can be explained as “a message knowingly transmitted
by a sender to foster a false belief or conclusion by the receiver” [11,13,44,47]. Generally, spoofed
websites and emails are used to deceive and breach valuable information [11,44].
In cyberspace, spear phishing is one of the most severe threats to Internet security around the globe.
Attackers aim to get users’ identities such as passwords and financial details by using spoofed emails and
messages. The primary concern for companies today is to protect data from cyber-attacks. Recently1,2
many events have taken place where the causes of breach were linked to human. Whether the breach is
accidental or intentional, the victim bears the financial and reputational loss. Phishers mostly use fake
emails or messages to persuade users to Phish. According to recent studies, the frequency and intensity
of phishing attacks will increase in future, and hence, there is a strong need for public awareness and
training [5,41,79]. Research studies showed that security awareness has a noticeable positive effect on
users’ detection of deceptive attempts [35]. Social engineering attacks such as phishing attacks have
severely impacted the world. As reported by Microsoft in 2014, annual world impact is US $5 billion.
Spear phishing accounts for 91% of all attacks, being the most successful.
Many researchers across the world are working on mitigating the effects of cyber-attacks. Various
studies have been performed where digital games are developed to train people regarding phishing emails
[6,70]. Recent3 intense attacks show that there is a growing need to educate the public, as in digital
era, nearly every individual own a smartphone connected to the Internet. In particular, there is a need
to educate people regarding phishing [5,12,27,79]. Researchers conducted a training session for users
using different education materials such as lectures, videos, and games. After conducting the training
session, the results showed that participants who were playing the game could easily recognise a fake
website [62]. Similarly, Junger et al. [40] concluded that due to lack of knowledge and defence strategies
humans are vulnerable to phishing attacks. To mitigate this vulnerability, one have to receive training
regarding phishing and spam emails.
Games provides scenario that are embedded in our daily lives and a learning experience in which play-
ers understand the rules, constraints, and things one should and shouldn’t do. Further, the motivational
and game design elements help the participants to take the critical role and make bold decisions without
worrying about the consequences [72]. Serious Games have been used as a possible way to engage par-
ticipants/learners, and to transfer knowledge in a unique and easy way [22]. The effect of game-based
learning can be seen in various fields, in various levels of education [72], e.g. maths [37], programming
1 https://info.phishlabs.com/blog/2019-phishing-trends-intelligence-report-the-evolving-threat
2 https://www.businessinsider.com.au/scammers-have-dominos-australia-customer-data-and-are-sending-scarily-
customised-spam-emails-2017-10
3 https://www.hackmageddon.com/category/security/cyber-attacks-timeline/
R. Fatima et al. / Phishing a silent attack 583
[31], collaborative airport management [28], and cyber security awareness [48,77], etc. Qian et al and
Chang et al. from different fields performed experiments and analyzed the effects of game-based learn-
ing on students’ performances, they concluded that there are significant positive impact of game-based
learning on players’ learning [14,56].
The motivation to design a game for cyber security is taken from the recent portal4 and from [10,57,
77]. In a comparative study on game-based learning [52,53], the possible limitations were explored and
how non-digital games can overcome those shortcomings were analysed. Moreover, research studies
verify that non-digital game-based learning engenders a positive effect on student’s comprehension.
Some of the limitations of the aforementioned digital games for education are that students and teachers
need some prior experience of the game and need some investment to build software and a setup for
education. Besides this, some of the benefits of non-digital game-based learning are that no technical
prerequisite skills are needed for teachers and students; it is cost-effective, and is helpful in enhancing
social interactions. The aforementioned discussion is the motivation for us to design a non-digital game
for phishing awareness education.
Motivation and contribution: In order to protect individuals and organizations from phishing attacks,
there are a few ways which can help us achieve that:
(1) Building a phishing attack repository that accepts crowd reports on phishing emails and phone calls
which can be found by searching for Fraudwatch and Millersmiles.
(2) Understanding a phishing security test which would find out whether a given user is phishing prone
or not. This would include phishing attempts to test how employees react on receiving suspicious
emails and requests which would solicit confidential information. In our study, we have performed
a pre-survey to check the initial knowledge level of the participants.
(3) Security education by games that are carefully designed with in-depth knowledge about phishing,
to explain the reasons for its effectiveness as an attack technique and to explain the methods used
to stop phishing attacks.
The objective of the study is to design a game-based solution to thwart spear-phishing attacks, to cre-
ate a know-how about the spear-phishing process, to educate people regarding the methods to identify
the (signs of) phishing emails, to generate security requirements using phishing emails, and to further
educate people regarding online information disclosure. In particular, the PhishI game is designed by us-
ing the detailed design findings from the research literature. Also, formulated a game design framework
which can be used to extend the game design. Lastly, an empirical evaluation is performed to analyze
the effectiveness of the learning activity.
The paper is divided into six sections. The first section introduces the topic and motivates the research
problem. Section 2 further enlightens the game design rationale, the designed and proposed framework
for the PhishI game, the various elements used in it, and its game process. Section 3 explains the em-
pirical evaluation conducted to analyze the game effectiveness. Section 4 discusses the observation(s),
the case study, and our methodology to evaluate the participants and their discussion. Section 5 explains
the related work on phishing awareness and the gaps filled by our game. Lastly, Section 6 concludes our
paper. The details can be seen in Fig. 1.
4 http://www.gamification.co/2016/03/02/teaching-kids-cybersecurity-game-based-training/
Fig. 1. Research protocol of the study.
2. PhishI game assets and process
This section explains the game context and game elements as well as the game process for the PhishI
game. Furthermore, this section shed light on the win/lose conditions, challenging part of the game from
the participants perspective, and important design rationales.
2.1. Game elements
2.1.1. Scenario based learning – map/floor plan & assets

Scenario Based Learning is one of the ways where participants create scenarios in an hypothetical
(game) scenario and further learn by brainstorming, interaction, and discussion. The same concept is
used in building an hypothetical environment using storyline and game map.
Storyline of the game: The Yeovil district hospital recently announced a prize for anyone who can
hack into their system and further tell them possible weaknesses in their security system. You, as a
specialized team of hackers, are given the task to get that system down. So far, by initial analysis, it is
seen that they have used one of the best security systems and software which makes it near impossible to
hack by usual ways. Furthermore, the only possibility is through phishing or spear phishing attacks. You
people are provided with important information extracted from social media (profiles) of the employees.
Focus on one employee and try to gather as many information from the social media (Open Source
intelligence). Then, draft a phishing email so that victim can install malware by this method and finally,
by this way, you can undermine the system. Hospitals are and will be the target of ransomware attacks
in future [1]. This is the main reason our first design of the game depicts hospital as an organization.
Furthermore, the motivation of creating a controlled environment is taken from [57].
Map/Floor plan of the Hospital system is shown in Fig. 2. This map represents one of the site of the
hospital where all the offices and important rooms are located. We can further see that in each department
one of the human assets is located. A human asset can be a doctor, an IT person, Nurse, patient or intern.
Each of the human asset is connected to the virtual world by his/her IT device(s). The blue rectangular
box on the top right shows the virtual world which the hospital people use for connecting to the web.
Fig. 2. Yeovil district hospital map adapted for game settings.
2.1.2. Role based learning – players play the role of attacker

An Attacker is a person who uses sociological and physiological principles on people to perform
actions or to expose their confidential information or use deception via obfuscated image(s) or spoofed
GUI component(s) of a computer program. In the game, the player plays the role of an attacker. Badge
or Player role represents the character roles of the players in the game. Players role/badge is further
motivated from the study [45,66]. As, in our game, we are only focusing on phishing attacks, we have
only designed three identity card/badge for players. The motivation for designing this card is so that
players can wear the hat of a social engineering attacker and can take actions in the game environment
which they can use in real life. Figure 3 represents the role card/badge of the players.
2.1.3. Social engineering body of knowledge – attack type/techniques cards
Behind any game, there is a body of knowledge and skills that is being practised. We have used Mou-
ton’s social engineering ontology [50] as the design know-how of the game. As the focus of the game is
on Phishing awareness we have only designed the attacks related to phishing. Figure 4a represents the
attack cards used in the game, and Table 1 lists the complete set of attacks and possible explanation of
each attack.
Fig. 3. Attacker role card.
Fig. 4. Attack cards and social media cards.
2.1.4. Psychology needs of target victim – compliance principle /psychology cards

Social relations is the reasons why a target responds to the attackers’ requests. The reasons include:
friendship, scarcity, authority and so on. Attackers used these psychology needs to attack the target
victim. Figure 5 shows the compliance principles cards. We have used various compliance principles in
our game which are shown in Table 2. One motivation to design this type of cards is that players may
learn the psychological technique employed by social engineers.
Table 1
Attack type used in PhishI from literature
Attack type Explanation Reference
Phishing Phishing is a process in which attackers send a malicious email or SMS [10]
to the mass of victim, to get the financial or confidential information.
Attackers pretend to be from a reputable organization.
Spear phishing Spear phishing is a type of attack in which attackers target specific [65]
victim, i.e., person or organization(s). As this kind of attack is specific
to the target, it is most successful in obtaining the information.
Trojan horse attack Trojan horse is a type of malware which is used to trick the victim into [29,32]
installing it on his/her personal computers and further infect the system
by using that malware or by providing a backdoor to the attacker.
Need & greed attack The need of a person makes him/her vulnerable. In this type of attack, [10]
attackers target the victim using his/her vulnerability or need. This
attack aims to use victims need and further make him/her greedy.
Suggest your attack In this type of attack, players suggest their attack type. The motivation Nil
for providing this option is to help the players to think out of the box
and can relate, and share their knowledge and experience.
Fig. 5. Compliance principle or psychology to target adapted from [77].
2.1.5. Point of contact selection – social media cards

In the game PhishI, we have proposed social media cards which represent social media information
(Open Source intelligence) present on various social media channels, e.g. Facebook, Twitter, etc. The
motivation of using cards which are not extracted from the database is that the players can enjoy the game
irrespective of their ability to use Internet, their access to a social media account, etc. The social media
cards represent the information regarding targeted assets. Social media cards can be seen in Fig. 4b.
Table 2
Psychology/compliance principles/human behavioral patterns
Psychology to target Explanation Reference
Fear By using different fear tricks, company’s employee gives access and share sensitive [10,38]
information. For example, new hiring employees may be the target.
To obey authority Studies show that high level of obedience leads to social engineering attack. For [10,24,38,64]
example, attacker can target the obedient person to gain access the critical information.
Trust Research shows that people may be easily target by excessive trust. Definitely, it is the [10,38]
first priority in business dealing but too much trust is harmful for companies data and
leads to social engineering attack.
Need & greed As studies shows that when an attacker knows the needs of a person, he/she wants to [10,24]
manipulate that person by his need and that make him/her greedy.
Guilt If a person induces to someone may feel guilty. In this case, an attacker said him/her, if [10]
you want out from this guilt and should complete the task given by attacker.
Curiosity The eagerness that wants to learn and get more and more about something is known as [10]
curiosity. For example attacker may use different promotional codes to win a cash or gift
prize to get sensitive information.
Panic/time pressure Research shows that attacker can create a panic situation to the victim. For example, an [10,24]
attacker may tell the victim that employee’s company in the loss and will fire soon,
accept attacker’s offer, it will be beneficial for both the company and the person. The
person may go in a panic situation and accept the offer of an attacker.
Emotional An attacker may target his/her victim emotionally by getting critical information easily. [10,38]
These emotions may be positive or negative.
Social proof People usually mimic what majority of the people do as in this way they are least likely [24,64]
to held responsible of unseen situation.
Deception Usually, people abide to whom they feel attracted and similar, etc. So in this case [24,38]
deception principle become the strongest tool for attackers.
Suggest new way This card gives an opportunity to players to suggest some new and innovative Nil
psychology to target victim.
2.2. Game process
The PhishI game has four phases, each phase takes 15–20 minutes.
• Phase 1: Victim selection: Game players select a target victim from the organization map shown in
the Fig. 2. The first phase of attack preparation has four steps: selection of communication media,
target devices/victim, attacking techniques, and attack material preparation. In PhishI, players have
to choose the target, medium, technique, and compliance principle.
• Phase 2: Gather information about target: In this phase, game players have to gather the infor-
mation regarding the target victim from various social media sources (Open Source Intelligence)
provided in the game. This concept is adapted from [4,23,42,54], which explains how the attackers
use social media information of victim in planning the social engineering attack. In Phishi, we have
designed social media cards which disclose sensitive important information of the victims. Using
the information from the social media cards can help the player to learn the vulnerability associ-
ated with excessive online information disclosure. Additionally, players can learn how attackers can
easily design and execute a spear-phishing attack.
• Phase 3: Draft phishing email: Here, game players have to draft a phishing email to the target
victim using the information given above. The players can see the phishing message design cards
(learning cards) in order to design a successful phishing email. Phishing message design cards
shown in Fig. 6 are incorporated in the game PhishI as part of the learning strategy. The players will
Fig. 6. Phishing message design cards used by players.
read the cards and try to incorporate the important steps needed to identify phishing emails. The
steps are adapted from studies [15,25,62,78].
While constructing phishing emails, players refer to the phishing message design cards as discussed
below:
∗ URL: Try not to use URL with numbers, e.g. http:147.46.226.55/Paypal.html. As the victim can
easily identify the phishing link.
∗ Subdomain URL: Try to use famous company names and logos for communication, e.g. http://
secure-sinin.ebay.com.ttps.us/. This link belongs to “.ttps.us” website instead of “ebay.com”.
∗ Similar and deceptive domains: Try to use hyphen “-” with famous company name to make it
more realistic, e.g. http://www.pay-pal.com/login/php.
∗ Email content linked with need and greed: Make best use of the information gathered from
social media or online media. Try to use victims needs to convince the victim.
∗ Official Logo and email template: Try to use official logo and email template to make the email
looks more realistic.
∗ Time and date: Try to send email on data and time when there is minimum support available,
e.g. during Christmas holidays, and at night, etc.
∗ Customer service number/email: Use fake contact numbers so that when victim contacts by
phone, he/she may use the number provided in the email.
∗ Psychology to target: Make the best use of compliance principles. Exploit psychology weakness
to target victim.
∗ Send fake attachment: Send attachment containing malware or virus. Upon downloading and
installing, the attacker may get access to victims’ computer.
∗ Ask for personal details: Ask the victim to provide financial or personal information by clicking
the fake link.
• Phase 4: Spread the email: In the last phase, two player teams swaps the phishing email drafted for
review. The phishing email reviewed by using the phishing message design cards (learning cards)
given to the teams.
The step by step procedure of the game process is described below:
(1) Select a target victim on the map whom you want to attack using phishing email.
(2) Note the description of the victim in information gathering for future record.
(3) On the virtual space area of the game, select five cards from the social media deck representing
about the victim players.
(4) Record all the information on the sheet given to the team.
(5) Draft an email using the compliance principles/psychology cards.
(6) Revise an email and review for possible improvements.
(7) Review by the opponent teams.
(8) Discussion Session.
2.3. Other game design considerations
2.3.1. Challenges
• The players have to collect maximum information from various sources provided in the game.
• The players have to draft the email which relates to the needs of the target victim and by using
various compliance principles.
• The players have to draft a phishing email with minimum deficiencies.
2.3.2. Win/lose conditions
The scoring system is divided into three areas, such as email structure (2 points), idea/scenario (2
points), and efficient usage of available information in drafting phishing email (2 points). The team
which scores the most points wins the game. The decision will be done after the discussion session
which is the last phase of the game.
2.3.3. What and where to teach in game:
In Table 3, we have explained which important learning aspects that we have embedded in different
phases of the game.
2.3.4. Game design framework:
A game design is a function of several areas which together create an environment in which users en-
joy and learn. For example, game patterns, user experience, psychological aspects, knowledge base, and
learning functions are some factors influencing the learning experience. To design an effective serious
game, we have adapted game design patterns and psychological needs [26,58] which are relevant to our
game. We have focused on four main areas, which are social engineering domain knowledge, psycho-
logical needs of players, learning functions, and human interaction. Table 4 represents the mapping of
domains in our game.
Table 3
What and where to teach in PhishI
What to teach Where to teach
Too much sharing of personal information can Phase 2: Information gathering
be damaging (information disclosure).
Identify fake/phishing email Phase 3: Drafting phishing email using hypothetical scenario making
Identify fake URL’s & subdomains Phase 3: Drafting phishing email using hypothetical scenario making
Phase 4: Using phishing message design cards
Identify similar and deceptive domains Phase 3: Drafting phishing email using hypothetical scenario making
Emails – need and greed (psychology) Phase 3: Drafting phishing email using hypothetical scenario making
Identify psychology targeting in phishing email Phase 3: Drafting phishing email using hypothetical scenario making
Table 4
Main concepts and embedding in PhishI game
Learning Mapped game design Incorporating in PhishI game Psychological needs satisfaction
functions pattern’s (GDP)
Interpreting Pre-defined goals, Social media cards, persona centered Need for autonomy, need for gaining
gain information information, goal defined information and understanding.
Analyzing Randomness, Randomness of social media cards Need for confidence, need for
strategic knowledge selection, attack type cards selection. surprise/interest
Feedback Score, points Review session Need for companionship need for social
relatedness, need for competence
Explaining Direct information Game rules, discussion between team Need for competence, need for achievement
members, discussion in review session
Applying Game elements Phishing sheet, phishing message Need for understanding
design cards, points, avatars of
attackers, story of game
3. Empirical evaluation
Phishing attacks generally result in billions of dollars annual losses. To address the issue of phishing
and spear phishing, researchers around the globe have been working on various solutions. Two types of
techniques are used to test the awareness of participants. One is the test based technique, and the other
is the wild technique. For our study, we have used a combination of the techniques adapted from the
cyber-phishing platform [33]. We have conducted a controlled test activity and used survey to assess
the learning of the participants. In parallel, we have used the observation methodology to evaluate the
learning and collect feedback.
Section 3 explains the recruitment process of the participants and describes how we have designed the
evaluation and our goals for the study. It also explains the results of the study.
3.1. Recruitment process
The game sessions were conducted with multiple sessions on campus. Author of the paper, lab mates,
colleagues were invited to take part in the pilot experiment to understand and perform the game. In order
to come up with the comparison chart we have used Delphi approach for the conclusion, in case of dis-
agreement. To perform our game activity, we advertised it by posting flyers and by dispatching messages
to (different) departmental groups. A total of 63 participants (Masters and PhD students) participated in
the activity. With in 63, 40 were male and 23 were females. 63 participants were further divided into
thirty teams (three sessions). The participants belong to various department. Some are from school of
software, others are from Department of Energy, Department of Computer Science, and Material Sci-
ence. English language is used as an activity language.
3.2. Empirical evaluation design
According to recent studies published in Nature [59] and Springer [4], a same threat can be interpreted
differently. In phishing attacks, an identical phishing email sent by an attacker made some of the targets
click on the infected link while others ignore it. In this scenario, where different people have different
perceptions regarding the same threat, a possible way is to provide an environment where players can
learn in a friendly environment. Moreover, they can discuss and share their knowledge and views. Our
PhishI game provides this environment to the participants where they design and develop the phishing
email and discuss with peers to understand the essence of a phishing email.
Figure 7 shows the detail of experiment timeline. In Table 5, we have shown the responses of the
participants. The participants before the activity tried to identify the URL’s given on the paper sheet.
From Table 5, we can see that participants correctly identified the famous URLs such as paypal, ebay
but got confused while identifying URLs with numbers or combinations of famous websites. This pre-
survey helped us identify the area(s) which need special attention in our game design.
Fig. 7. Empirical evaluation flow and time division.
Table 5
Pre-game participants identification of URL
Phishing URL Not sure Legitimate URL Correct answer Percentage
http://www.paypal.com/accept 7 11 45 Legitimate URL 71.4%
http://www.ebay.com 9 5 49 Legitimate URL 77.7%
http://147.91.75.1/ebay/ 15 15 33 Phishing URL 23.8%
http://secure-signin.ebay.com.ttps.us/ 19 27 17 Phishing URL 30.1%
http://www.msn-verify.com/ 5 5 53 Phishing URL 7.9%
Below are the goals for our study:

Goal 1: Game based learning by PhishI game and feedback collection
After the controlled activity, the players were asked to fill a survey questionnaire. Players were asked
regarding the phishing attacks and their possible impacts on the individuals’ lives. For example, we
asked: “It is extremely likely that my computer will be infected by a phishing attack in future.” The
majority of the respondents’ agreed with the statement. Only one of the respondents disagreed. The
diverse responses from the players depict that there is a room for improvement in the game which can
be bridged by discussing the latest phishing attacks and their effects. This will, in turn, help the players
know the importance of saving themselves from Phishing attacks. In the second question, the players
were asked: “My chances of being targeted by phishing attacks are great.” The majority of the players
agreed; however, some participants disagreed. This might be because the players trusted their anti-virus
software to keep looking for such kind of emails. To account for this, players’ understanding of phishing
can be further improved by adding limitations of Anti-viruses in the game. Finally, we asked the players:
“I feel phishing attack will infect my computer in the future.” To this statement, players mainly selected
the answers lied in the “agree” and “neutral” option.
Goal 2: Phishing knowledge and avoidance behavior
The URL survey page which was taken before the game playing session was then again used to check
post-game learning. Table 6 represents the responses from the participants. We can see that the majority
of the participants have correctly identified the answers which represent the overall positive learning
of the players. If we compare Table 5 and 6, we can see how the identification of URL improved after
playing the game session.
Goal 3: Educating regarding online information disclosure
Social media provides an opportunity for people and companies to share ideas and experiences. How-
ever, on the other side, they divulge personal as well as public information as well as other’s information.
So, information disclosure on social media allows an attacker to access sensitive information from the
personal and organizational systems. Currently, companies are using social networking sites for educa-
tional, industrial, and marketing purposes. But there is a need to educate general public about online
information disclosure as well [49,80]. In PhishI game, using the social media information available to
design the phishing email will probably make players think of how their information can be used against
them. Furthermore, this will also make them consider disclosure of information online as something
that can be dangerous. This insight was also observed during discussion session at the end of the game
session.
Table 6
Post-game participants identification of URL
Phishing URL Not sure Legitimate URL Correct answer Percentage
http://www.paypal.com/accept 7 4 52 Legitimate URL 82.5%
http://www.ebay.com 4 3 56 Legitimate URL 88.8%
http://147.91.75.1/ebay/ 51 2 10 Phishing URL 80.9%
http://secure-signin.ebay.com.ttps.us/ 44 7 12 Phishing URL 69.8%
http://www.msn-verify.com/ 52 1 10 Phishing URL 82.5%
http://www.amazoon.com/ 47 5 11 Phishing URL 74.6%
Fig. 8. Research model.
3.3. Research model and analysis
We have used quantitative approach (survey) to calculate the feedback regarding the game and the
already existing phishing knowledge of the participants. The questions of the survey were adapted from
the already published literature, as we believe that this will increase the credibility of the obtained results.
The post survey was divided into two broad sections: the demographic questions and independent and
dependent questions which helped to support the research model. Furthermore, we have adopted 5 points
Likert scale to gather the behavioral responses of the participants. In our survey, value five represents
(Strongly Agree) and value one expresses (Strongly Disagree). For our research model, we have adapted
Technology Threat Avoidance Model (TTAT) from [7] and Technology Acceptance Model from [3]. The
post survey questionnaire can be seen in the Table 18. In total, we got 63 responses which, after pre-
processing,5 were reduced to 47. The research model shown in Fig. 8 was used to perform the analysis.
The hypotheses of our study are mentioned below.
H1: Fun to Play has a positive effect on Intention to play.
H2: Ease to Play has a positive effect on Intention to play.
H12: Ease to Play and Fun to Play have a positive effect on Intention to play.
H3: Intention to Play has a positive effect on Game Based Learning.
H4: Phishing Knowledge has a positive effect on Game Based Learning.
H34: Intention to Play and Phishing Knowledge have a positive effect on Game Based Learning.
H5: Game Based Learning has a positive effect on Avoidance Behavior.
Accordingly, we have come up with four linear regression equations:
Intention = β0 + β1 ∗ Fun + β2 ∗ Ease + μ1 (1)

Game-Based Learning = α0 + α1 ∗ Intention + α2 ∗ Phishing Knowledge + μ2 (2)
Avoidance Behavior = ρ0 + ρ1 ∗ Game-Based Learning + μ3 (3)
Avoidance Behavior = f (Fun, Ease) (4)
5 Theresponses which had missing data, or outliers (the participants selected same scale for all the survey questions), etc.
were removed.
Fig. 9. Graph matrix.
where: μ are the disturbance terms with E(μ) = 0 and a finite variance. The first three equations
are what we simply get from our model proposed above. The fourth equation can be derived simply
by substituting second and first equation in the third one. Thus, in total, we have four equations to
test. Before regressing variables, we can have a look at the graph matrix6 (Fig. 9) which shows the
general trends for our variables. Observing from the last row, we can see that the relation between our
independent and dependent variables are generally positive – indicating towards acceptability of our
model.
The regression results for our equations are shown in Table 7. From looking at the table, we can see
that all7 the coefficients values are positive, i.e. a positive increase in independent variable is associated
with a positive increase in y-variable. For some of the coefficients in the equations, the individual p-
values come out to be statistically insignificant at a 5% confidence level. This insignificance is due to
multicollinearity between variables.
We had earlier noted that (initially) it was generally hard for people to understand and apply phishing
concepts. Although, from the perspective of our model, the level of Phishing knowledge of individuals is
exogenously given,8 the difficulty to understand phishing concepts is partially explained by a relatively
lower coefficient value for phishing knowledge, i.e. 0.17.
The correlation matrix for variables is shown in Table 8. For most of the variable combinations, there is
a high correlation i.e. when one variable moves in one direction, the other one follows. So, for example,
6 Graph matrix plots all the scatter plots between the specified variables. This enables us to get a “feel” for the data before
any regression analysis.
7 The coefficient value for Fun to Play in the first equation comes out to be −0.047 in our multivariate model. Later on, we
will show that this negative value is due to multicollinearity and not a negative trend in the population.
8 Lower as compared to other coefficient values in the table.
Table 7
Regression analysis
Equation Independent variable Dependent variable p-value Coefficients R-squared Overall F-stat*
1 Fun to play Intention to play 0.816 −0.047 58.71% 0.0000
Ease to play 0.000 0.801**
2 Intention to play Game based learning 0.016 0.48 36.71% 0.0034
Phishing knowledge 0.167 0.17
3 Game based learning Avoidance behavior 0.014 0.47 24.26% 0.0137

4 Fun to play Avoidance behavior 0.243 0.27 46.74% 0.0000
Ease to play 0.003 0.47
* Robust standard errors are used wherever the residuals were heteroscedastic.
** For example, as an interpretation: a one unit increase in participants’ perception of Ease to Play is associated with a 0.8 unit
increase in their Intention to Play, on average.
Table 8
Correlation matrix
Fun to Ease to Intention Game based Phishing Avoidance
play play to play learning knowledge behavior
Fun to play 1.00
Ease to play 0.86* 1.00
Intention to play 0.65 0.77 1.00
Game based learning 0.34 0.48 0.59 1.00
Phishing knowledge 0.66 0.67 0.80 0.54 1.00
Avoidance behavior 0.64 0.67 0.90 0.49 0.73 1.00
* All the variables are highly correlated i.e. participants’ perceptions move together across a range of variables. This will generate
a tendency for our regression to suffer from multicollinearity.
if a participant perceives a game as one that is fun to play, he/she is much likely to perceive it as an
easy one too. Hence, when we add both of the variables together in one equation, one of the two turns
insignificant once the other is controlled for. Although there is sufficient multicollinearity between our
variables, the equations proposed above all are statistically significant at 5% confidence level (overall),
which means that our model and the associated trends hold in the larger population.
4. Discussion
4.1. Observation
Observation is an important research method it is used when we need to study human behavior, hu-
man behavior, body language, facial expressions, etc [43,60,77]. During the game session, two of the
researchers acted as the observers. One researcher sat within one of the teams and observed their argu-
ments and discussion points. The other walked around the room for a general and more holistic apprecia-
tion of the game sessions. Researchers noted interesting discussion points, such as “Lets phish the nurse
as she may not know about phishing,” “Normally doctors have little time in their schedules. Hence, if we
try deceiving them via situations built around the persuasion principle of “urgency”, it might work,” etc.
From the discussion points, we can conclude that the participants were thinking from the perspective of
Table 9
Primary observations and areas for improvement
Some of the important Interpretation Category
observations
“What is social engineering? and This may be due to the reason that these terminologies are not Game Pre-Session (PS)
phishing? Did you understand?” known by participants and they need time to understand. Once
they got attack examples they can easily relate that with their real
life examples. Other possible solution is to not use
terminologies/jargon’s, so that understanding can be much easy.
“How to play the game?” As in the initial phase of the game, some of the participants faced Game process (GP)
issue in understanding the process. But once they understand the
process/steps, they suggested reasonable phishing emails.
“Lets target that lady?” On inquiring about this comment at the end of the game, the Game design (GD)
participants told that the reason for this is that “Lady may be an
easy target and it seems easy to deceive them”.
“We are winning ! Yo !” One of the emotional sentence made by the participant whose Game elements (GE)
team scores more than the opponent.
Table 10
Players responses during game progression || an instance of game
Attributes/dimensions Game instance for Team 1
1. Social media information (from social Very much in need of new job
media cards)
2. Vulnerability of human asset Looking for new job
3. Psychology to target (from psychology Need and greed, curiosity
cards)
4. Phishing email drafted by Team 1 With reference to your friend, we came to know that you are
(Table 17 in Appendix) searching for a new job opportunity. We are contacting you from
National Hospital. We are looking for experienced resources for
our hospital. If you are interested kindly read the attached file
related to our hospital and reply with your resume.
5. Score given by reviewers team 4 out of 6
6. Suggestion given by reviewer’s team i) You must mention the name of a friend who is too social, so that
the reader may trust this completely; ii) You can talk about some
good salary package and position; iii) For further information you
need to give him/her some cell number; iv) You can use female
name and information as coordinator so that the chances of victim
to respond and contact will be more.
7. Reply by Team No. 1 members For suggestion i) The target victim may contact the friend
mentioned and the attack may fail.
8. Suggestion given by an expert/teacher i) Composed phishing email is reasonable. ii) You can also use
social media channel (linked In, etc) to contact the victim in-order
to first build TRUST and then launch phishing attack.
9. Reply by the Team No. 1 members Yes, good suggestion.
an attacker, and we can believe that this will help to learn various techniques and strategies used by the
social engineers. Some of primary observations on which all the researchers agreed upon are given in
Table 9.
Case study – Game instance: One particular instance of the game is shown in Table 10 for informa-
tion. We believe that the case instance will help in better understanding on how the activity carries out.
4.2. How we have evaluated the participants knowledge?
4.2.1. Developing and reviewing PhishI emails

In this research study, knowledge of the participants were evaluated by various methods and tech-
niques mentioned in research literature. Firstly, we observed the participants knowledge by analyzing
the quality of phishing email generated by them. As discussed in game design section, the players have to
use the given psychological principles, human weaknesses, attack techniques, and other valuable infor-
mation to generate deceptive or phishing emails. The players, when generate a phishing email, possess
working-knowledge on how any potential social engineer can use various techniques to target humans.
Furthermore, discussion session (among different team members at the end of scenario generation ses-
sion) helps to understand the context and other viable ways of attacks. The quality of phishing emails
and discussion and feedback session reflects the learning outcomes of the game session. Some of the
phished emails designed by the players during the activity session are shown in Appendix A. Scenario
based learning is the technique used in industry for training and education for the past years [20,71].
This method of evaluation is adapted from [10,77].
4.2.2. Game feedback & survey [43,60]
To receive feedback related to the game activity, we asked players to fill the post-game survey. The
results of the game, as shown in the results section, are overall positive. There is still much room for
improvement. The feedback obtained by the researchers will be used to improve the game with the next
version.
4.3. Limitations and validity threats
Some of the known limitations and validity threats are discussed below:
• a) One of the limitation of this study is the (sample size) number of participants for the evaluation
activity. As this is the preliminary evaluation, only 63 participants’ result is shown in the study.
We realize this limitation and are planning to verify the results with more empirical evaluations in
future. The motivation to report the results with 63 participants is: i) to report preliminary results
to the community; ii) that many research studies have used less than 50 participants for initial or
preliminary evaluation [34,67,77], supporting our rationale of using 63 participants to represent the
research results; b) Another limitation is that we have taken participants solely from the university
environment. We believe that, in future, the effectiveness of the game-based activity must be verified
by inviting general public or industry participants; c) One of the limitations of experiment-based
activity is that a layman can easily turn uninterested and bored. This effect “looking out of the
window” is mentioned in [10] and first introduced by Ketil Stólen. To minimize the window effect
during our study, we have implemented a two-step methodology. In the first step, the participants
were filtered out using intrinsic motivation (if participants are interested by themselves, they would
respond to the email or advertisement). Then, extrinsic motivators were embedded in the game
using game elements as well to maintain the fun-element throughout the game.
• a) Some threats, and limitations are associated directly with the type of research methodology se-
lected. If we take qualitative data as the data collection method in the study, two of the most impor-
tant possible shortcomings will be selective memory (partly remembering the event) and positive
attribution (only reporting the positive results). In order to minimize these factors in our study, be-
fore the empirical evaluation session, all the researchers were assigned observational duties. During
the activity session, the researchers noted important points and topics, which were further discussed
with other fellow researchers at the end of the activity. Conducting the meeting soon after the eval-
uation activity helped minimize the effect of selective memory, and the participation of all the
researchers helped to cross-report points which made the points reported authentic and true repre-
sentative of what happened during the sessions; b) A comparative analysis of our study with respect
to other phishing games can be seen in Table 11 and 12. The tables illustrate the places where our
game has bridged the gaps existing in other games and the places where it is not comparable to
other games. In this study, we have used limited number of phishing attacks, persuasion techniques,
and human weaknesses for the proposed game. In future, our plan is to update these lists and further
give participants more options to select, brainstorm, and learn from.
• Threats to internal and construct validity [55,73]: Internal validity deals with researcher’s bias-
ness and interpretation of the results, whereas the construct validity deals with the possible threats
associated with the experiment or activity design. In order to minimize the internal validity of the
study, we have controlled factors such as activity time distribution, learning context and material,
teaching method, and the language of activity. Furthermore, to minimize the construct validity,
survey questionnaire, URLs for identification and scenario-based analysis method are taken from
published research literature (to minimize the biasness in designing or any effect on the results or
result’s interpretation).
• Threats to external validity [55,73] deals with the generalizability of the results. As this is a
preliminary evaluation for the proposed activity, we cannot claim the results to be general (at this
time), but still we believe that the results may not vary much (by analyzing the results from the
empirical evaluation). Furthermore, external validity of the study is still to be verified by further
empirical evaluations in future.
• Others: a) One of the main limitations of our work, which was also earlier touched upon, is the
level of effort needed by users to play this game. In our experiment, since the teams belonged to
different departments and academic disciplines, the ‘effort-factor’ might have minimized. In future,
we are planning to address this challenge by using Design Science principles which will help partic-
ipants understand concepts easily; b) Other possible limitation can be biasness in filling the survey
responses by the respondents. In our case, after the game activity, we categorically announced to re-
main fair in filling the survey as this will help in improving the game. Furthermore, the validity can
be tested by performing more empirical evaluations in future; c) In this preliminary evaluation of
the game, where sharing knowledge and healthy discussion is the main essence, we are not sure how
this game will behave in case of single players (individuals playing the game). This can be a future
task to first verify and further see how it is effective as compared to played by multiple players. d) In
order to analyze how psychology factors affect victim, first we need to understand the pattern or de-
ciphering of social engineering attacks. We have not evaluated this aspect in this particular version
of the study. On the other hand, our research group is working on the above dimension. The study
particularly focuses on emotional journey of victim during a phone scam (another kind of social
engineering attack). The journey of the victim goes from E1 to E8. Starting from the Trust factor to
becoming Curious and passing through Fear, Anger, Urgency and lastly Relaxed (temporarily).
5. Related work on phishing awareness techniques
After an in-depth literature review, it can be argued that a significant number of cyber-security studies
have been published on the topic of phishing techniques and awareness. If we further categorize these
studies, we can see that phishing literature can be categorized into technical and non-technical solutions
to counter phishing. Furthermore, many studies showed that computer users are vulnerable to phishing
due to many reasons. Some of them are: i) “look and feel” of the fake website is greatly comparable to
the original website and hence, could effect user’s ability to identify the difference; [19] ii) end users are
not adequately aware of the phishing attacks, techniques, and process; [21] iii) end users don’t observe
and pay heed to indications of browser security [74].
Taxonomy and theoretical model for phishing: Ahmed et al. [5], developed a taxonomy by per-
forming an in-depth literature review. The proposed taxonomy covered four essential dimensions which
include i) communication media; ii) attacking techniques; iii) targeted devices for an attack; iv) counter-
measures for phishing attacks. The researchers then, categorized the phishing studies into five areas i)
machine learning; ii) text mining; iii) human users; iv) profile matching and v) others (honeypot coun-
termeasures) to give a clear idea of the research studies. The researchers of the study believe that the
mentioned taxonomy will be helpful in identifying phishing attacks and can be used as countermeasures
to such attacks.
Phishing awareness evaluation: Due to an increase in the number of phishing attacks Jansson et al.
[39] inquired how vulnerable their students are to such attacks. They designed an experiment which was
divided into two phases. In the first phase, the target victims were categorized into different groups. Four
different types of phishing emails were then generated, and each group received a unique type of phishing
emails. To make phishing attacks more context-oriented, one phishing email explained that the university
database has crashed and to retrieve their account information, they had to click on the provided link.
In short, the results of the training with simulating phishing attacks has shown that this can be used to
enhance user’s phishing resistance [39]. Kumaraguru et al. [44] designed an experiment and, as the first
step, sent an email for volunteer participation in an activity. The participants were first given training
using the game “Phish Guru” which helped them learn and understand the legitimate and phished URLs
using an interactive fish-themed activity. After the training, the participants received emails at random
hours (for seven days) concerning winning a game and/or other cash prizes via filling a simple form.
The form asked for university credentials as a pre-requisite for participating in the lottery. The study
concluded that the participants who took “Phish Guru” training were more effective in identifying phish
emails and that they retained phishing knowledge for 28 days after the training. Furthermore, the author
concluded that the participants who were between 18 to 25 years of age were more vulnerable to phishing
attacks than the elder users.
Anti-phishing games: Sheng et al. [62], designed and evaluated a game “Anti-phishing Phil” for
educating and making participants aware of phished URL’s. The digital game created a swimming pool
or a water tank in which fish were placed at positions. Clicking over each fish revealed either a legitimate
or a phished URL. The player, then, is a small fish which is new to this environment and by the help of its
mentor identifies the legitimate and the phished URLs. For example, participant (the small fish) and its
mentor swim to another fish at a position in the tank. Upon clicking on that fish, a URL pops up, which
may be legitimate or phished. By using the knowledge from the tutorial at the start of this activity, the
participant has to correctly identify the type of the email, which would earn him/her points. The study
concluded that the participants who played the game were better able to identify phishing emails and that
“Anti-phishing Phil” makes people more aware of phishing attacks viz-a-viz other educational methods
based on reading, etc. M. Baslyman et al. [9], developed a board game named “Smells Phishy” that
claimed to educate users about online phishing scams. The experiment was performed on 21 participants.
In the start of the game, each player received “X” amount of money for the gameplay. The idea of the
game revolves around purchasing items from a place on a map. The player, upon buying, may either
successfully get the item or be phished. After it’s evaluation, it was established that users (after playing
the game) could defend themselves better from phishing attacks. The results of the study confirmed
that after playing the game, the user depicted better awareness of phishing scams and also learned the
techniques to protect themselves.
5.1. Commercial tools and table comparison
PhishMe9 is a commercial company which aims to utilize human asset to minimize phishing attacks.
It works on the concept of crowd sourcing. Studies have been published recently [28] where researchers
have used crowd as source of information to counter phishing attacks. The goal of the PhishMe is to
educate people via contextual knowledge. For example, if one of your close friends texts you in an
unusually formal tone, you can take a safe bet that something has gone wrong. So once a client identifies
phished stuff, he/she can report it to PhishMe, where an expert analyzes the threat and add the phishing
email and its template to the database and also update all other users regarding the email. If we compare
PhishMe with our proposed game, it turns apparent that our version covers many other dimensions of an
attack scenario. For example, we have used game-based learning techniques (discussed in game design
and rationale section). Moreover, we have not only focused on the context but also understood human
needs as one of the means via which attackers manipulate people into installing malware or providing
important information. Players by wearing the hat of an attacker can use various persuasion technique(s),
attack vector and human needs to develop phishing emails. Phishing attacks by using emails can never
have a zero success rate because of; i) new employees joining and transferring between companies; ii)
new emotional triggers discovered; iii) new deception techniques, etc. In order to train our players, we
gave them a list of URLs (phished vs legitimate) so that they can learn and identify the phished one by
observation.
Wombat security10 is also a commercial organization that focuses on preventing people from falling
for phished emails. Wombat developed a game-based solution known as Anti-Phishing phyllis. In the
game, phyllis character teaches various important sections in emails that can be used to identify fraud-
ulent emails. In the exercise, email is shown on a screen and a bubble appears at various places on the
email. When the player hovers the pointer over that bubble, a box pops up and displays the options of
“ignore” and “disharm.” Depending upon the option selected and the original correct answer, a tick or
cross sign appears, signifying a correct or wrong attempt. Furthermore, it explains why the option se-
lected is correct or wrong. In one single round, the player has to review three emails and, after the round,
a summary is presented for player’s recap. If we compare Anti-Phishing phyllis with our proposed game,
the main difference is that of the way in which the problem is dealt with. In wombat game, the player
tackles the situation as a user − as a passive subject, but in our game, the player has to live the role of
an attacker (active interaction) and use all the important lessons to draft a phished email. Definitely and
additionally, our game includes a team-based learning component which is not present in these games.
Comparison of awareness games: All of the tools which are developed to date have failed to reduce
phishing attacks as the strategy, process, and techniques of phishing attacks are evolving with the ad-
vancement in technology. Moreover, it has been discovered that the best anti-phishing tools lose more
than 20% of the phishing sites [16,19,62]. The deficiency is reinforced by the fact that most systems are
dependent on individuals who make confidential decisions when they perform activities on the internet.
The researchers of this study played the games mentioned in Table 11 & 12 and further summarized the
9 https://cofense.com
10 https://www.wombatsecurity.com
602
Table 11
PhishI comparison with other similar games
Aspects Characteristics PhishI Anti-phishing PhishGuru Smells
phil [62] [44] phishy [9]
Role Playing (Attack) Characters in game x x
Security context Story line of the game
Dynamic Nature of Map (Changeability) some+ x x some+
Map Used in the game for reference some+ x
R. Fatima et al. / Phishing a silent attack

Players play by moving on the Map x x x
Game element Different type of Attack Cards x x
Dice for Randomness x x x
Security knowledge area Game address social engineering issues some+ some+ some+ some+
Game educate network security related issues x x x x
Game educate physical security related issues x x x x
Security mechanism Making scenario x
Attack mechanics x x
Defence mechanics x x x some+
Game design Multi-player x x x

Digital game x some+ x
Game design base from research literature x x x
Targeted learning areas Educating regarding spear phishing x some+ x
Educating regarding information disclosure x x x
Eliciting phishing security requirements x x x
Online social media information can be misused x x x
Team-based learning Discussion session (knowledge/experience sharing) x x x
Table 12
PhishI comparison with other games adapted from [76,77]
Aspects Characteristics PhishI CSRAG Ctrl-ALt-Hack Social Dox3d
[77] [18] engineering [10]
Role Playing (Attack) Characters in game x
Security context Story line of the game x
Dynamic nature of map (changeability) some+ some+ x x
Map used in the game for reference x
R. Fatima et al. / Phishing a silent attack

Players play by moving on the map x x x
Security mechanism Attack mechanics
Defence mechanics x x x x
Making scenario x x
Game element Different type of attack cards some+ x

Dice for randomness x x x
Security knowledge area Social engineering issues some+ some+
Network security related issues x some+ x
Physical security related issues x some+ x some+
Security protection target Mission for the team and player x

Team-based learning Discussion session some+ some+
Evaluation design Different methods for evaluation x x x
Targeted learning areas Educating regarding spear phishing x x some+ x
Educating regarding information disclosure x x x x
Eliciting phishing security requirements x x x x
Online social media information can be misused x x x x
603
characteristics of all the games based on various aspects such as security context, security mechanism,
targeted learning areas, etc. Table 11 gives an in-depth comparison of the similar phishing awareness
games, and on the other hand, a comprehensive comparison of cyber security (general) is shown in Ta-
ble 12, which focus on generic cyber security awareness games. For all the non-digital games such as
Dox3d, and Ctrl-Alt-Hack etc the games were ordered online. On the other hand, digital games men-
tioned in Table 11 were played online. Researchers used Delphi approach, it is used where there is
a disagreement between the individual subjective analyses. If we look closely, we can further analyze
that overall PhishI provides good coverage of the characteristics as compared to other mentioned games.
Overall, we believe that PhishI is a game which not only trains players with spear phishing related
concepts but also focuses on information disclosure awareness. In Table 11 & 12, ‘Some+’ identifies
that this phase is present to a good extent or present in some scenarios of the game. ‘’ represents the
complete presence of that phase in the game, and ‘X’ represents the absence of that phase from the
game.
R. Zhao et al. [79], designed a toolkit that could automatically create fraudulent websites which
seemed much like their original counterparts. Zhao et al. experimented on 194 users to check the useful-
ness of this toolkit. After the experiment, the results showed that more than 90% users became a target of
illegitimate websites. Zhao et al believes that phishing attacks in future will increase and there is strong
need to defend these attacks collectively. In order to minimize the phishing attacks our game offers many
aspects as compared to other similar games discussed in Table 11 & 12: i) players adopt the role of at-
tacker in the game environment and think from the view of an attacker, which helps them understand the
perspective of the attacker first-hand; ii) team-based learning and discussion helps to better understand
the situation and other concepts; iii) our game provides a know-how about persuasion techniques, attack
vectors, and human weaknesses which are useful in deciphering phishing emails in real life; iv) the use
of social media (cards) to collect relevant information helps players understand how excessive online
information disclosure can be utilized by the attackers.
6. Conclusion
We are living in a world where physical conflicts have now moved to the cyber space, which is be-
coming an ever more insecure place [46]. In this situation, security awareness is of vital importance.
Thus, this makes it more necessary than ever to devise a mechanism for better security awareness [51].
To draw the attention of all user groups, we need a method that is more engaging and easy to use [30].
We know that awareness training methods are only effective if used properly [51]. In the past decades,
more and more effort is spent on making the operating system and network environment secure. Due to
this, attackers have changed their targets from information systems to human elements to break into or-
ganizations. The number of attacks in the recent past include attackers who have used social engineering
as their method of invasion in target organizations [2].
This paper introduced PhishI as a systematic approach to design serious games for security education.
We define a game design framework that integrates the body of knowledge on social engineering, the
psychological needs of organizational players, and the candidate game pattern that serve different needs.
We use spear phishing as a key example to show how the proposed approach works, and then evaluated
the learning effects of the generated game based on empirical data collected from student activity. In
PhishI game participants are required to swap phishing emails and be able to comment on the effective-
ness of the attack scenario, the effectiveness of this method can be justified as this method of evaluation
is used in recent designed games such as [10,77]. Our results showed that students’ awareness of spear
phishing risks is improved and that the resistance to potential first attack contact is enhanced. Further-
more, the game showed positive effect on participants’ understanding of excessive online information
disclosure.
In future, we are planning to design and develop an automated tool supporting the game composition
process, with knowledge repository of known phishing attack scenarios. Thus, some functionalities as
shown by of the game PhishI will be shifted online.
Acknowledgments
Financial support from the Natural Science Foundation of China Project no. 61432020 is gratefully
acknowledged. We thank Awaid Yasin for reviewing the paper.
Appendix A. Phishing email generated by players
The players were given a context of a company and according to the situation and various variables
such as Human Asset (target asset) responsibility in organization, his/her interest (social media informa-
tion), compliance principle, players need to generate an email. Below mentioned are some of the phished
emails generated by the players during game play. These emails generated by the players are shown in
Table 13, 14, 15, 16. 17 for understanding.
Appendix B. Post activity survey
The post survey questionnaire can be seen in the Table 18.

Table 13
Spear – phishing email generated by players – Team A
Human asset: Dr. Rubia (Gynecologist)
Social media information 1. He/she mentioned in his/her WeChat moments that his/her Alipay account is not
working. (You can use this information to prepare a spear phishing attack.)
2. He/she is paying income-tax from several years. (You can email them as an
income tax officer and get the financial details.)
Psychology 1. To obey Authority
card/compliance 2. Fear
principles 3. Curiosity
Email scenario Hi Dear Dr. Rubia,

We are emailing you from national security department and recently we have
surveillance some unusual transactions from your account. Kindly follow the link
(www.revenue-bureau.org/assetdeclare.php) below to fill the form so that necessary
actions can be taken soon. Remember that your one step will help us in making our
country a safe place to live.
Mr. Aslam,
Intelligence Bureau
Contact No: 1111-222-111
Table 14
Spear – phishing email generated by players – Team B
Human asset: Dr. Ali
Social media information 1. He/she mentioned in his/her WeChat moments that he/she loves to travel by
Emirates.
2. He/she mentioned in his/her Facebook wall that he/she will travel to China soon.
Psychology 1. Curiosity
card/compliance 2. Need and Greed
principles
Email scenario Hi Dr. ALi,
Your booking number BPS123 has been confirmed on your request. Please find the
attached ticket in the email. Your flight number BHS55 from England to China has
been confirmed. Kindly acknowledge the email by latest.
Mr. Chris,
Qatar Airs
Contact No: 1000-777-001
Table 15
Spear – phishing email generated by players – Team C
Human asset: Miss Yanan
Social media information 1. He/she mentioned in his/her Facebook moments that someone is trying to update
his/her account password.
Psychology card/compliance principles 1. Curiosity
Email scenario Hi Miss Yanan,

This is the confirmation email from Facebook. Your Facebook account information
has been updated successfully. If this is not done by you, kindly login by using this
link (www.faceb00k.com/login.php) and update the information.
Mr. Zukerberg,
Faceook Accounts Team,
Contact No: 888-777-000
Table 16
Spear – phishing email generated by players – Team D
Human asset: Miss Jia Yidi
Social media information 1. He/she mentioned in his/her Facebook moments that someone is trying to
change his/her paypal account password.
Psychology card/compliance principles 1. Curiosity
Email scenario Hi Jia Yidi,
This email message is confirming that your request to delete the paypal account has
been recieved. If you have not requested for deletion kindly click the following
link (www.paypaal.com/confirmation.php) and fill the form as soon as possible.
Otherwise your paypal account will be deleted in a weeks time.
Mr. Tony,
Paypal Accounts Team,
Contact No: 123-999-132
Table 17
Spear – phishing email generated by players – Team E
Human asset: Mr Wang Jue
Social media information 1. He/she mentioned in his/her Facebook time-line that he/she is looking for job
opportunity.
Psychology 1. Curiosity
card/compliance 2. Need and Greed
principles
Email scenario Hi Wang Jue,
With reference to your friend, we came to know that you are searching for a new
job opportunity. We are contacting you from National Hospital. We are looking for
experienced resources for our hospital. If you are interested kindly read the
attached file (Containing Malware/virus) related to our hospital and reply with
your resume.
Mr. Baig,
National Hospital,
Human Resource Management,
Contact No: 123-123-322
Table 18
Post survey questionnaire
Fun to play [3] Strongly disagree Disagree Neutral Agree Strongly agree
I find the PhishI game fun/enjoyable.
The actual process of using PhishI game is pleasant.
I have fun playing the PhsihI game.
Ease to play [3] Strongly disagree Disagree Neutral Agree Strongly agree
Learning to use PhsihI game is easy for me.
I find it easy to learn security concept while playing
PhishI game.
My interaction with the PhishI game was easy and
understandable.
I find the PhsihI game to be flexible & easy to
interact with.
It was easy for me to learn basic concepts and
knowledge on security using PhsihI.
I find the PhishI game easy to use.
Similar native language of the all the game players
have positive impact on learning.
Intention to play [3] Strongly disagree Disagree Neutral Agree Strongly agree
I have access to PhsihI game, I intend to use it.
Given that I have access to PhsihI game, I predict
that I would use it.
I plan to use PhishI game in future.
Game based learning [7] Strongly disagree Disagree Neutral Agree Strongly agree
Game based anti-phishing education is useful for
detecting phishing attacks.
Game based anti-phishing education has increased
my performance in protecting my computer.
Game based anti-phishing education has enabled me
to detect phishing attack on my computer.
I feel game based anti-phishing education is useful
for protecting my computer from phishing attacks.
Phishing knowledge [69] Strongly disagree Disagree Neutral Agree Strongly agree
It is extremely likely that my computer will be
infected by a phishing attack in future.
My chances of getting phishing attacks are great.
I feel phishing attack will infect my computer in
future.
Avoidance behavior [8] Strongly disagree Disagree Neutral Agree Strongly agree
I gain anti-phishing knowledge to avoid phishing
attacks.
I update my anti-phishing knowledge frequently.
Update anti-phishing knowledge is very important to
avoid phishing attacks.
References
[1] Hospitals become major target for ransomware, Network Security 2016(4) (2016), 1–2, http://www.sciencedirect.com/
science/article/pii/S1353485816300319. doi:10.1016/S1353-4858(16)30031-9.
[2] J. Abawajy, User preference of cyber security awareness delivery methods, Behaviour & Information Technology 33(3)
(2014), 237–248. doi:10.1080/0144929X.2012.708787.
[3] F. Abdullah, R. Ward and E. Ahmed, Investigating the influence of the most commonly used external variables of {TAM}
on students’ perceived ease of use (PEOU) and perceived usefulness (PU) of e-portfolios, Computers in Human Behavior
63 (2016), 75–90, http://www.sciencedirect.com/science/article/pii/S0747563216303387. doi:10.1016/j.chb.2016.05.014.
[4] S.M. Albladi and G.R.S. Weir, A semi-automated security advisory system to resist cyber-attack in social networks, in:
Computational Collective Intelligence, N.T. Nguyen, E. Pimenidis, Z. Khan and B. Trawiński, eds, Springer International
Publishing, Cham, 2018, pp. 146–156. ISBN 978-3-319-98443-8. doi:10.1007/978-3-319-98443-8_14.
[5] A. Aleroud and L. Zhou, Phishing environments, techniques, and countermeasures: A survey, Comput-
ers & Security 68 (2017), 160–196, http://www.sciencedirect.com/science/article/pii/S0167404817300810.
doi:10.1016/j.cose.2017.04.006.
[6] N.A.G. Arachchilage and M. Cole, Design a mobile game for home computer users to prevent from phishing attacks, in:
International Conference on Information Society (i-Society 2011), 2011, pp. 485–489.
[7] N.A.G. Arachchilage and S. Love, A game design framework for avoiding phishing attacks, Computers
in Human Behavior 29(3) (2013), 706–714, http://www.sciencedirect.com/science/article/pii/S0747563212003585.
doi:10.1016/j.chb.2012.12.018.
[8] N.A.G. Arachchilage and S. Love, Security awareness of computer users: A phishing threat avoidance perspective,
Computers in Human Behavior 38(Supplement C) (2014), 304–312, http://www.sciencedirect.com/science/article/pii/
S0747563214003331. doi:10.1016/j.chb.2014.05.046.
[9] M. Baslyman and S. Chiasson, “Smells phishy?”: An educational game about online phishing scams, in: 2016 APWG
Symposium on Electronic Crime Research (eCrime), 2016, pp. 1–11. doi:10.1109/ECRIME.2016.7487946.
[10] K. Beckers and S. Pape, A serious game for eliciting social engineering security requirements, in: 24th IEEE Interna-
tional Requirements Engineering Conference, RE 2016, Beijing, China, September 12–16, 2016, IEEE, 2016, pp. 16–25.
doi:10.1109/RE.2016.39.
[11] A. Bergholz, J. De Beer, S. Glahn, M. Moens, G. Paaß and S. Strobel, New filtering approaches for phishing email,
Journal of Computer Security 18(1) (2010), 7–35. doi:10.3233/JCS-2010-0371.
[12] J. Bullee, L. Montoya, M. Junger and P.H. Hartel, Spear phishing in organisations explained, Inf. & Comput. Security
25(5) (2017), 593–613. doi:10.1108/ICS-03-2017-0009.
[13] D.B. Buller and J.K. Burgoon, Interpersonal deception theory, Communication Theory 6(3) (1996), 203–242.
doi:10.1111/j.1468-2885.1996.tb00127.x.
[14] C.-C. Chang, C. Liang, P.-N. Chou and G.-Y. Lin, Is game-based learning better in flow experience and various types of
cognitive load than non-game-based learning? Perspective from multimedia and media richness, Computers in Human
Behavior 71(Supplement C) (2017), 218–227, http://www.sciencedirect.com/science/article/pii/S0747563217300377.
doi:10.1016/j.chb.2017.01.031.
[15] K.L. Chiew, K.S.C. Yong and C.L. Tan, A survey of phishing attacks: Their types, vectors and technical approaches, Ex-
pert Systems with Applications 106 (2018), 1–20, http://www.sciencedirect.com/science/article/pii/S0957417418302070.
doi:10.1016/j.eswa.2018.03.050.
[16] L.F. Cranor, S. Egelman, J.I. Hong and Y. Zhang, Phinding phish: An evaluation of anti-phishing toolbars, in: Proceedings
of the Network and Distributed System Security Symposium, NDSS 2007, San Diego, California, USA, 28th February–2nd
March 2007, The Internet Society, 2007, http://www.isoc.org/isoc/conferences/ndss/07/papers/phinding_phish.pdf.
[17] L. De Kimpe, M. Walrave, W. Hardyns, L. Pauwels and K. Ponnet, You’ve got mail! Explaining individual differences in
becoming a phishing target, Telematics and Informatics 35(5) (2018), 1277–1287, http://www.sciencedirect.com/science/
article/pii/S0736585317304677. doi:10.1016/j.tele.2018.02.009.
[18] T. Denning, A. Lerner, A. Shostack and T. Kohno, Control-Alt-Hack: The design and evaluation of a card game
for computer security awareness and education, in: Proceedings of the 2013 ACM SIGSAC Conference on Computer
& Communications Security, CCS ’13, ACM, New York, NY, USA, 2013, pp. 915–928. ISBN 978-1-4503-2477-9.
doi:10.1145/2508859.2516753.
[19] R. Dhamija, J.D. Tygar and M.A. Hearst, Why phishing works, in: Proceedings of the 2006 Conference on Human Factors
in Computing Systems, CHI 2006, Montréal, Québec, Canada, April 22–27, 2006, R.E. Grinter, T. Rodden, P.M. Aoki,
E. Cutrell, R. Jeffries and G.M. Olson, eds, ACM, 2006, pp. 581–590. doi:10.1145/1124772.1124861.
[20] A. Dix, J.E. Finlay, G.D. Abowd and R. Beale, Human–Computer Interaction, 3rd edn, Pearson. ISBN 978-0-13-046109-
4.
[21] J.S. Downs, M.B. Holbrook and L.F. Cranor, Decision strategies and susceptibility to phishing, in: Proceedings of the
Second Symposium on Usable Privacy and Security, SOUPS ’06, ACM, New York, NY, USA, 2006, pp. 79–90. ISBN
1-59593-448-0. doi:10.1145/1143120.1143131.
[22] I. Dunwell, P. Petridis, S. Arnab, A. Protopsaltis, M. Hendrix and S. de Freitas, Blended game-based learning environ-
ments: Extending a serious game into a learning content management system, in: 2011 Third International Conference on
Intelligent Networking and Collaborative Systems, IEEE, 2011, pp. 830–835. doi:10.1109/INCoS.2011.58.
[23] M. Edwards, R. Larson, B. Green, A. Rashid and A. Baron, Panning for gold: Automatically analysing online social
engineering attack surfaces, Computers & Security 69 (2017), 18–34. doi:10.1016/j.cose.2016.12.013.
[24] A. Ferreira, L.M. Coventry and G. Lenzini, Principles of persuasion in social engineering and their use in phishing, in:
HCI (22), Lecture Notes in Computer Science, Vol. 9190, Springer, 2015, pp. 36–47.
[25] I. Fette, N. Sadeh and A. Tomasic, Learning to detect phishing emails, in: Proceedings of the 16th International Con-
ference on World Wide Web, WWW ’07, ACM, New York, NY, USA, 2007, pp. 649–656. ISBN 978-1-59593-654-7.
doi:10.1145/1242572.1242660.
[26] N.H. Flores, A.C.R. Paiva and P. Letra, Software engineering management education through game design patterns,
Procedia – Social and Behavioral Sciences 228(Supplement C) (2016), 436–442, 2nd International Conference on
Higher Education Advances, HEAd’16, 21–23 June 2016, València, Spain, http://www.sciencedirect.com/science/article/
pii/S1877042816309934. doi:10.1016/j.sbspro.2016.07.067.
[27] W.R. Flores and M. Ekstedt, Shaping intention to resist social engineering through transformational leadership, informa-
tion security culture and awareness, Computers & Security 59 (2016), 26–44. doi:10.1016/j.cose.2016.01.004.
[28] M. Freese, Game-based learning: An approach for improving collaborative airport management, in: European Conference
on Games Based Learning, Academic Conferences International Limited, 2016, p. 835.
[29] D. Fuentes, J.A. Álvarez, J.A. Ortega, L.G. Abril and F. Velasco, Trojan horses in mobile devices, Comput. Sci. Inf. Syst.
7(4) (2010), 813–822. doi:10.2298/CSIS090330027F.
[30] F. Giannakas, G. Kambourakis and S. Gritzalis, CyberAware: A mobile game-based app for cybersecurity education and
awareness, in: Interactive Mobile Communication Technologies and Learning (IMCL), 2015 International Conference on,
IEEE, 2015, pp. 54–58. doi:10.1109/IMCTL.2015.7359553.
[31] U. Güleç, M. Yilmaz and M.A. Gozcu, Bireylerin Programlama Yeteneklerini ve Bilgi Seviyelerini Arttirmak Amaciyla
Dusunulmus Ciddi Oyun Tabanli Ogrenme Catisi – CENGO(Serious Game-Based Learning Framework to Improve
Programming Skills and Knowledge Levels of Individuals – CENGO), in: Proceedings of the 11th Turkish National
Software Engineering Symposium, Alanya, Turkey, October 18–20, 2017, Ç. Turhan, A. Coskunçay, A. Yazici and
H. Oguztüzün, eds, CEUR Workshop Proceedings, Vol. 1980, CEUR-WS.org, 2017, pp. 171–183, http://ceur-ws.org/
Vol-1980/UYMS17_paper_8.pdf.
[32] S. Gupta, A. Singhal and A. Kapoor, A literature survey on social engineering attacks: Phishing attack, in:
2016 International Conference on Computing, Communication and Automation (ICCCA), 2016, pp. 537–540.
doi:10.1109/CCAA.2016.7813778.
[33] M.L. Hale, R.F. Gamble and P. Gamble, CyberPhishing: A game-based platform for phishing awareness test-
ing, in: 2015 48th Hawaii International Conference on System Sciences, 2015, pp. 5260–5269, ISSN 1530-1605.
doi:10.1109/HICSS.2015.670.
[34] R. Heartfield and G. Loukas, Detecting semantic social engineering attacks with the weakest link: Implementation and
empirical evaluation of a human-as-a-security-sensor framework, Computers & Security 76 (2018), 101–127, http://www.
sciencedirect.com/science/article/pii/S0167404818301780. doi:10.1016/j.cose.2018.02.020.
[35] R. Heartfield, G. Loukas and D. Gan, You are probably not the weakest link: Towards practical prediction of susceptibility
to semantic social engineering attacks, IEEE Access 4 (2016), 6910–6928. doi:10.1109/ACCESS.2016.2616285.
[36] H. Hellaoui, M. Koudil and A. Bouabdallah, Energy-efficient mechanisms in security of the Internet of things:
A survey, Computer Networks 127(Supplement C) (2017), 173–189, http://www.sciencedirect.com/science/article/pii/
S1389128617303146. doi:10.1016/j.comnet.2017.08.006.
[37] M. Host’oveckỳ and M. Novák, Game-based learning: How to make math more attractive by using of serious game, in:
Computer Science on-Line Conference, Springer, 2017, pp. 341–350.
[38] M. Jakobsson (ed.), Understanding Social Engineering Based Scams, Springer, 2016. ISBN 978-1-4939-6455-0.
doi:10.1007/978-1-4939-6457-4.
[39] K. Jansson and R. von Solms, Phishing for phishing awareness, Behaviour & Information Technology 32(6) (2013), 584–
593. doi:10.1080/0144929X.2011.632650.
[40] M. Junger, L. Montoya and F.-J. Overink, Priming and warnings are not effective to prevent social engineering attacks,
Computers in Human Behavior 66 (2017), 75–87, http://www.sciencedirect.com/science/article/pii/S0747563216306392.
doi:10.1016/j.chb.2016.09.012.
[41] D. Ki-Aries and S. Faily, Persona-centred information security awareness, Computers & Security 70 (2017), 663–674,
http://www.sciencedirect.com/science/article/pii/S0167404817301566. doi:10.1016/j.cose.2017.08.001.
[42] K. Krombholz, H. Hobel, M. Huber and E. Weippl, Advanced social engineering attacks, Journal of Information Security
and Applications 22 (2015), 113–122, Special Issue on Security of Information and Networks, http://www.sciencedirect.
com/science/article/pii/S2214212614001343. doi:10.1016/j.jisa.2014.09.005.
[43] R. Kumar, Research Methodology: A Step-by-Step Guide for Beginners, 3rd edn, SAGE Publications
Ltd, 2010, https://www.amazon.com/Research-Methodology-Step-Step-Beginners/dp/1446269973/ref=sr_1_2?
ie=UTF8&qid=1530868080&sr=8-2&keywords="Research+methodology. ISBN 1849203008, 9781849203005.
[44] P. Kumaraguru, J. Cranshaw, A. Acquisti, L. Cranor, J. Hong, M.A. Blair and T. Pham, School of phish: A real-world
evaluation of anti-phishing training, in: Proceedings of the 5th Symposium on Usable Privacy and Security, SOUPS ’09,
ACM, New York, NY, USA, 2009, pp. 3:1–3:12. ISBN 978-1-60558-736-3. doi:10.1145/1572532.1572536.
[45] E. Kyewski and N.C. Krämer, To gamify or not to gamify? An experimental field study of the influence of badges on
motivation, activity, and performance in an online learning course, Computers & Education 118(Supplement C) (2018),
25–37, http://www.sciencedirect.com/science/article/pii/S0360131517302506. doi:10.1016/j.compedu.2017.11.006.
[46] A. Le Compte, D. Elizondo and T. Watson, A renewed approach to serious games for cyber security, in: Cyber Conflict:
Architectures in Cyberspace (CyCon), 2015 7th International Conference on, IEEE, 2015, pp. 203–216.
[47] L.K. Marett and J.F. George, Deception in the case of one sender and multiple receivers, Group Decision and Negotiation
13(1) (2004), 29–44. doi:10.1023/B:GRUP.0000011943.73672.9b.
[48] N. Micallef and N.A.G. Arachchilage, Changing users’ security behaviour towards security questions: A game based
learning approach, in: 2017 Military Communications and Information Systems Conference (MilCIS), IEEE, 2017, pp. 1–
6.
[49] T. Morlok, Sharing is (not) caring – the role of external privacy in users’ information disclosure behaviors on social
network sites, in: 20th Pacific Asia Conference on Information Systems, PACIS 2016, Chiayi, Taiwan, June 27–July 1,
2016, T. Liang, S. Hung, P.Y.K. Chau and S.-I. Chang, eds, 2016, p. 75, http://aisel.aisnet.org/pacis2016/75.
[50] F. Mouton, L. Leenen and H.S. Venter, Social engineering attack examples, templates and scenarios, Computers & Secu-
rity 59 (2016), 186–209. doi:10.1016/j.cose.2016.03.004.
[51] A. Nagarajan, J.M. Allbeck, A. Sood and T.L. Janssen, Exploring game design for cybersecurity training, in: Cyber
Technology in Automation, Control, and Intelligent Systems (CYBER), 2012 IEEE International Conference on, IEEE,
2012, pp. 256–262. doi:10.1109/CYBER.2012.6392562.
[52] N. Naik, A comparative evaluation of game-based learning: Digital or non-digital games? in: European Conference on
Games Based Learning, Vol. 2, Academic Conferences International Limited, 2014, p. 437.
[53] N. Naik, Non-digital game-based learning in the teaching of mathematics in higher education, in: European Conference
on Games Based Learning, Vol. 2, Academic Conferences International Limited, 2014, p. 431.
[54] A. Paradise, A. Shabtai and R. Puzis, Detecting organization-targeted socialbots by monitoring social network profiles,
Networks and Spatial Economics (2018), 1–31.
[55] K. Petersen and C. Gencel, Worldviews, research methods, and their relationship to validity in empirical soft-
ware engineering research, in: 2013 Joint Conference of the 23rd International Workshop on Software Measure-
ment and the 8th International Conference on Software Process and Product Measurement, 2013, pp. 81–89.
doi:10.1109/IWSM-Mensura.2013.22.
[56] M. Qian and K.R. Clark, Game-based learning and 21st century skills: A review of recent research, Computers in Hu-
man Behavior 63(Supplement C) (2016), 50–58, http://www.sciencedirect.com/science/article/pii/S0747563216303491.
doi:10.1016/j.chb.2016.05.023.
[57] A. Robles, J. Norris, S. Watson and A.F. Browne, Survey of non-malicious user actions that introduce network and system
vulnerabilities and exploits, in: SoutheastCon 2018, 2018, pp. 1–5, ISSN 1558-058X. doi:10.1109/SECON.2018.8478938.
[58] M. Sailer, J.U. Hense, S.K. Mayr and H. Mandl, How gamification motivates: An experimental study of the effects of
specific game design elements on psychological need satisfaction, Computers in Human Behavior 69 (2017), 371–380.
doi:10.1016/j.chb.2016.12.033.
[59] L.D. Salay, N. Ishiko and A.D. Huberman, A midline thalamic circuit determines reactions to visual threat, Nature
557(7704) (2018), 183–189, http://www.nature.com/articles/s41586-018-0078-2. doi:10.1038/s41586-018-0078-2.
[60] M.N.K. Saunders, Research Methods for Business Students, Pearson Education Limited, Harlow, Essex, England, 2016.
ISBN 978-1292016627.
[61] P. Schaab, K. Beckers and S. Pape, Social engineering defence mechanisms and counteracting training strategies, Inf. &
Comput. Security 25(2) (2017), 206–222. doi:10.1108/ICS-04-2017-0022.
[62] S. Sheng, B. Magnien, P. Kumaraguru, A. Acquisti, L.F. Cranor, J.I. Hong and E. Nunge, Anti-Phishing Phil: The design
and evaluation of a game that teaches people not to fall for phish, in: Proceedings of the 3rd Symposium on Usable
Privacy and Security, SOUPS 2007, Pittsburgh, Pennsylvania, USA, July 18–20, 2007, L.F. Cranor, ed., ACM International
Conference Proceeding Series, Vol. 229, ACM, 2007, pp. 88–99. doi:10.1145/1280680.1280692.
[63] H. Siadati, T. Nguyên, P. Gupta, M. Jakobsson and N.D. Memon, Mind your SMSes: Mitigating social engineering in
second factor authentication, Computers & Security 65 (2017), 14–28. doi:10.1016/j.cose.2016.09.009.
[64] M. Silic and A. Back, The dark side of social networking sites: Understanding phishing risks, Com-
puters in Human Behavior 60(Supplement C) (2016), 35–43, http://eproxy2.lib.tsinghua.edu.cn:80/
rwt/33/http/P75YPLUUMNVXK5UDMWTGT6UFMN4C6Z5QNF/science/article/pii/S0747563216301029.
doi:10.1016/j.chb.2016.02.050.
[65] J. Steer, Defending against spear-phishing, Computer Fraud & Security 2017(8) (2017), 18–20, http://www.sciencedirect.
com/science/article/pii/S136137231730074X. doi:10.1016/S1361-3723(17)30074-X.
[66] R.B. Svensson and B. Regnell, Is role playing in requirements engineering education increasing learning outcome?, Re-
quirements Engineering 22(4) (2017), 475–489. doi:10.1007/s00766-016-0248-4.
[67] A. Tang, F. Bex, C. Schriek and J.M.E.M. van der Werf, Improving software design reasoning – a reminder
card approach, Journal of Systems and Software 144 (2018), 22–40, http://www.sciencedirect.com/science/article/pii/
S0164121218301043. doi:10.1016/j.jss.2018.05.019.
[68] P. Tetri and J. Vuorinen, Dissecting social engineering, Behaviour & Information Technology 32(10) (2013), 1014–1023.
doi:10.1080/0144929X.2013.763860.
[69] H.S. Tsai, M. Jiang, S. Alhabash, R. LaRose, N.J. Rifon and S.R. Cotten, Understanding online safety behaviors: A pro-
tection motivation theory perspective, Computers & Security 59 (2016), 138–150, http://www.sciencedirect.com/science/
article/pii/S0167404816300190. doi:10.1016/j.cose.2016.02.009.
[70] S.S. Tseng, K.Y. Chen, T.J. Lee and J.F. Weng, Automatic content generation for anti-phishing educa-
tion game, in: 2011 International Conference on Electrical and Control Engineering, 2011, pp. 6390–6394.
doi:10.1109/ICECENG.2011.6056921.
[71] L. Van der Merwe, Scenario-based strategy in practice: A framework, Advances in Developing Human Resources 10(2)
(2008), 216–239. doi:10.1177/1523422307313321.
[72] C. Vogeler, Game-based learning with OER in higher education: Development and evaluation of a serious game, in:
European Conference on e-Learning, Academic Conferences International Limited, 2018, pp. 592–XX.
[73] C. Wohlin, P. Runeson, M. Höst, M.C. Ohlsson and B. Regnell, Experimentation in Software Engineering, Springer, 2012.
ISBN 978-3-642-29043-5. doi:10.1007/978-3-642-29044-2.
[74] M. Wu, R.C. Miller and S.L. Garfinkel, Do security toolbars actually prevent phishing attacks? in: Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems, CHI ’06, ACM, New York, NY, USA, 2006, pp. 601–610.
ISBN 1-59593-372-7. doi:10.1145/1124772.1124863.
[75] I. Yaqoob, E. Ahmed, M.H. Rehman, A.I.A. Ahmed, M.A. Al-garadi, M. Imran and M. Guizani, The rise of ransomware
and emerging security challenges in the Internet of things, Computer Networks (2017), http://eproxy2.lib.tsinghua.
edu.cn:80/rwt/33/http/P75YPLUUMNVXK5UDMWTGT6UFMN4C6Z5QNF/science/article/pii/S1389128617303468.
doi:10.1016/j.comnet.2017.09.003.
[76] A. Yasin, L. Liu, T. Li, R. Fatima and W. Jianmin, Improving software security awareness using a serious game, IET
Software (2018), http://digital-library.theiet.org/content/journals/10.1049/iet-sen.2018.5095.
[77] A. Yasin, L. Liu, T. Li, J. Wang and D. Zowghi, Design and preliminary evaluation of a cyber security requirements edu-
cation game (SREG), Information and Software Technology 95 (2018), 179–200, https://www.sciencedirect.com/science/
article/pii/S0950584917301921. doi:10.1016/j.infsof.2017.12.002.
[78] Y. Zhang, J.I. Hong and L.F. Cranor, Cantina: A content-based approach to detecting phishing web sites, in: Proceedings
of the 16th International Conference on World Wide Web, WWW ’07, ACM, New York, NY, USA, 2007, pp. 639–648.
ISBN 978-1-59593-654-7. doi:10.1145/1242572.1242659.
[79] R. Zhao, S. John, S. Karas, C. Bussell, J. Roberts, D. Six, B. Gavett and C. Yue, Design and evaluation of the
highly insidious extreme phishing attacks, Computers & Security 70 (2017), 634–647, http://eproxy2.lib.tsinghua.
edu.cn:80/rwt/33/http/P75YPLUUMNVXK5UDMWTGT6UFMN4C6Z5QNF/science/article/pii/S0167404817301633.
doi:10.1016/j.cose.2017.08.008.
[80] M. Zhitomirsky-Geffet and Y. Bratspiess, Professional information disclosure on social networks: The case of Face-
book and LinkedIn in Israel, Journal of the Association for Information Science and Technology 67(3) (2016), 493–504.
doi:10.1002/asi.23393.
Copyright of Journal of Computer Security is the property of IOS Press and its content may
not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's
express written permission. However, users may print, download, or email articles for
individual use.

How Persuasive Is A Phishing Email? A Phishing Game For Phishing Awareness

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

How Persuasive Is A Phishing Email? A Phishing Game For Phishing Awareness

Uploaded by

Copyright:

Available Formats

Journal of Computer Security 27 (2019) 581–612 581

How persuasive is a phishing email?

Fig. 1. Research protocol of the study.

2. PhishI game assets and process

2.1. Game elements

2.1.1. Scenario based learning – map/floor plan & assets

Fig. 2. Yeovil district hospital map adapted for game settings.

2.1.2. Role based learning – players play the role of attacker

Fig. 3. Attacker role card.

Fig. 4. Attack cards and social media cards.

2.1.4. Psychology needs of target victim – compliance principle /psychology cards

Fig. 5. Compliance principle or psychology to target adapted from [77].

2.1.5. Point of contact selection – social media cards

2.2. Game process

Fig. 6. Phishing message design cards used by players.

2.3. Other game design considerations

3.1. Recruitment process

3.2. Empirical evaluation design

Fig. 7. Empirical evaluation flow and time division.

Below are the goals for our study:

Fig. 8. Research model.

3.3. Research model and analysis

Intention = β0 + β1 ∗ Fun + β2 ∗ Ease + μ1 (1)

Fig. 9. Graph matrix.

3 Game based learning Avoidance behavior 0.014 0.47 24.26% 0.0137

4.2. How we have evaluated the participants knowledge?

4.2.1. Developing and reviewing PhishI emails

4.3. Limitations and validity threats

5. Related work on phishing awareness techniques

5.1. Commercial tools and table comparison

R. Fatima et al. / Phishing a silent attack

Game design Multi-player x x x

R. Fatima et al. / Phishing a silent attack

Game element Different type of attack cards some+ x

Security protection target Mission for the team and player x

Appendix A. Phishing email generated by players

Appendix B. Post activity survey

The post survey questionnaire can be seen in the Table 18.

Email scenario Hi Dear Dr. Rubia,

Email scenario Hi Miss Yanan,

You might also like