This article was downloaded by: [University of Central Florida]

On: 22 November 2014, At: 11:04

Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

International Journal of Human-Computer Interaction

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/hihc20

A Multimethod Evaluation of Online Trust and Its

Interaction with Metacognitive Awareness: An
Emotional Design Perspective
a b
Supavich Pengnate & Pavlo Antonenko
a
Oklahoma State University , Stillwater , Oklahoma , USA
b
University of Florida , Gainesville , Florida , USA
Accepted author version posted online: 02 Nov 2012.Published online: 14 Jun 2013.

To cite this article: Supavich Pengnate & Pavlo Antonenko (2013) A Multimethod Evaluation of Online Trust and Its Interaction
with Metacognitive Awareness: An Emotional Design Perspective, International Journal of Human-Computer Interaction, 29:9,
582-593, DOI: 10.1080/10447318.2012.735185

To link to this article: http://dx.doi.org/10.1080/10447318.2012.735185

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and
should be independently verified with primary sources of information. Taylor and Francis shall not be liable for
any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of
the Content.
This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
 Intl. Journal of Human–Computer Interaction, 29: 582–593, 2013
Copyright © Taylor & Francis Group, LLC
ISSN: 1044-7318 print / 1532-7590 online
DOI: 10.1080/10447318.2012.735185

A Multimethod Evaluation of Online Trust and Its Interaction

with Metacognitive Awareness: An Emotional Design Perspective
Supavich Pengnate1 and Pavlo Antonenko2
1
Oklahoma State University, Stillwater, Oklahoma, USA
2
University of Florida, Gainesville, Florida, USA

Meek, 2004; Garbarino & Strahilevitz, 2004; Rodgers & Harris,

Downloaded by [University of Central Florida] at 11:04 22 November 2014

Despite the rapidly increasing numbers of informational
websites and learners using the World Wide Web to research
topics, empirical evidence on the relationships between website
design elements and website trustworthiness is scarce. This study
used self-reported and behavioral screen-capture data to investi-
gate the impact of Don Norman’s (2003) emotional design levels
and metacognitive awareness on website trustworthiness during an
information search learning task. The results suggest that the inter-
action effects of website visual appeal (visceral level) and website
usability (behavioral level) can override the effects of the quality
or relevance of the information (reflective level) on website evalua-
tion. In addition, in the context of limited time to find the answers,
these effects on the evaluation of website trustworthiness are not
moderated by users’ metacognitive awareness.
1. INTRODUCTION
The World Wide Web has evolved and become an important
source of information for the global society. With the increasing
use of the World Wide Web as an information source, the need
to understand how online users develop information literacy and
evaluate the trustworthiness of websites has never been greater.
Multiple studies have provided suggestions on how to enhance
the trustworthiness and credibility of websites, pointing out that
the major determinant of website trust is the overall design of
the website (Cyr, 2008; Cyr, Head, & Larios, 2010; Fogg et al.,
2003; Jarvenpaa, Tractinsky, & Vitale, 2000; Schlosser, White,
& Lloyd, 2006).
Although the influence of various website design features on
user behavior has been explored extensively in the past decade,
most studies tend to focus on the website design manipula-
tions and do not take into account user characteristics such
as differences in navigation preferences, cognitive style, or
metacognitive awareness. Most studies of website design that do
investigate the moderating effects of user characteristics focus
on cultural differences (Cyr, 2008; Cyr et al., 2010; Gage, 1999;
Simon, 2000), gender (Cyr & Bonanni, 2005; Dittmer, Long, &
Address correspondence to Supavich Pengnate, 204 Hanner Hall,
Oklahoma State University, Stillwater, OK 74078. E-mail: fone.
pengnate@okstate.edu
582
2003), dispositions (Li, Sarathy, & Xu, 2011; McKnight &
Chervany, 2001), and familiarity (Casaló, Flavián, & Guinalíu,
2008; Gefen, 2000). One area that has received little attention
in the empirical literature is a systematic examination of how
website users’ emotional, cognitive, and metacognitive pro-
cesses influence the users’ evaluation and judgment of website
trustworthiness, which is the objective of this study.
Multiple human–computer interaction (HCI) frameworks
have been developed, tested, and employed in practice to
explain how user interface design improves user experience.
Most of these frameworks focus either on the affective aspects
of user experience or on the (meta)cognitive aspects. For exam-
ple, Jordan’s (2000) pleasure model focuses on the pleasurable
aspects of users’ experience with the system, and McCarthy
and Wright’s (2004) affective model describes how sense-
making takes place in the development of experience via four
core types of processes: compositional, sensual, emotional, and
spatio-temporal.
Don Norman’s (2003) emotional design model is another
framework that explains and predicts user experience from
an affective perspective, describing how emotion and cog-
nition are processed by different functional areas of the
human brain and should therefore both receive attention of
user interface designers. Norman’s emotional design frame-
work is comprehensive because encompasses visceral, behav-
ioral, and reflective aspects of users’ information processing
and overall experience. This emotional design framework fit
the goals of this study because it afforded categorization of
website components to match users’ emotional, cognitive, and
metacognitive processing—reflecting the full spectrum of user
characteristics that are crucial for determining website user’s
experience.
The primary objective of this study was to investigate the
impact of website design components at each of the three lev-
els of Norman’s (2003) emotional design framework on users’
evaluation of website trustworthiness as part of an information
search task that is typical for any formal or informal learning
context. Although Norman’s taxonomy of visceral, behavioral,
and reflective emotional design levels has been widely adopted
 EMOTIONAL DESIGN, ONLINE TRUST, AND METACOGNITION
in website design practice, the impact of each level on trust
development has not been validated empirically.
In addition, the second objective of this study was to examine
the extent to which online users’ evaluation of website trust-
worthiness depends on their metacognitive assessment of the
relevance of information presented on the website—in the con-
text of limited time to complete an information search task and
a range of websites to be evaluated. Metacognition refers to
knowledge about when and how individuals plan and monitor
their information processing and use such planning, monitor-
ing, and reflection strategies for learning or for solving prob-
lems (Metcalfe & Shimamura, 1994). Metacognition has been
explored extensively in educational contexts (Ally, 2004; Artino
& Jones Ii, 2012; Azevedo & Jacobson, 2008; Garrison, 2003;
Garrison & Cleveland-Innes, 2005; Liu, Chen, Sun, Wible,
Downloaded by [University of Central Florida] at 11:04 22 November 2014
& Kuo, 2010). As for user interface design, user characteris-
tics such as metacognitive awareness and metacognitive skills
are considered as a major predictor of users’ evaluations and
their actual web browsing behavior (Tractinsky & Lowengart,
2007). Thus, this study was concerned with exploring the medi-
ating effects of users’ metacognition on their perception of
trustworthiness of a number of websites with variable design
features. It is an initial attempt to determine the influence of
metacognition on website trust and provide new insights on how
user characteristics interact with website design features at the
visceral, behavioral, and reflective level, impacting the users’
evaluation of websites.
2. THEORETICAL FRAMEWORK AND HYPOTHESES
2.1. Emotional Design
The HCI literature provides a broad range of models explain-
ing how emotional responses impact users’ evaluation of inter-
active products (Cyr et al., 2010; Deng & Poole, 2010; Éthier,
Hadaya, Talbot, & Cadieux, 2006; Flavián, Guinalíu, & Gurrea,
2006; Li et al., 2011; van der Heijden, 2004). This study
adopted Norman’s (2003) emotional design model, which has
been well received in the HCI domain and widely adopted by
website designers. This model proposes that human emotion
and behavior when interacting with an object result from three
different levels of information processing—the visceral level,
the behavioral level, and the reflective level. This framework is
consistent with prior conceptual and empirical research in cog-
nitive science (e.g., Sweller, van Merrienboer, & Paas, 1998),
which typically distinguishes between three distinct levels of
information processing—affective (cf. visceral), cognitive (cf.
behavioral), and metacognitive (cf. reflective). According to
Norman’s model, the first level of emotional design is visceral.
At this level, visual design features of a system (e.g., graphics,
cleanliness, perceived attractiveness) dominate human percep-
tion and, in the website design context, this is when the user
makes rapid (millisecond) judgments of the website based on
the initial emotional responses to the color scheme and over-
all layout. The second level of processing is behavioral, which
583
involves active, task-driven use of the website and cognitive
evaluation of its usability. Finally, the reflective level is at the
top of Norman’s emotional design model. This level is associ-
ated with metacognitive, reflective processing of the website’s
usefulness and deals mostly with the evaluation of informa-
tion presented in the website. Norman’s emotional design
model is a useful framework for designing and evaluating
websites—particularly because it reflects the three recognized
levels of human information processing—affective, cognitive,
and metacognitive.
2.2. Initial Trust in the Online Context
An important predictor of website use is the user’s per-
ceived trust (Jarvenpaa et al., 2000). Information processing
at the visceral, behavioral, and reflective level contributes to
the development of online trust. McKnight Cumming, and
Chervany (1998) theorized that initial trust—that is, trust that
first-time website visitors have in a website without an estab-
lished reputation—has three antecedents: individual disposi-
tion to trust, institution-based trust, and cognitive-based trust.
Although individual disposition to trust is a personality char-
acteristic that develops during childhood, institution-based trust
refers to the security one feels about a situation because guar-
antees, safety nets, or other regulatory or legal structures are
constituted. Cognitive-based trust is developed based on rapid,
cognitive cues or first impressions. In accordance with this
conceptualization of initial trust, interface design of websites
should be fundamental to the development of cognitive-based
trust, impacting online users’ evaluation of website trustwor-
thiness. Several studies have suggested that website trust is
an important antecedent of user responses and a critical fac-
tor of success in online business impacting such variables as
user satisfaction (Flavián et al., 2006; Kim, Ferrin, & Rao,
2009), intention to purchase (Jarvenpaa et al., 2000; Lindgaard,
Dudek, Sen, Sumegi, & Noonan, 2011; Pennington, Wilcox, &
Grover, 2003; Wells, Valacich, & Hess, 2011), actual purchas-
ing behavior (Lim, Sia, Lee, & Benbasat, 2006), and e-loyalty
(Cyr, 2008).
2.3. Website Interface, Emotional Design, and Online
Trust
Conceptual and empirical research on online trust and
website design has been performed by scholars from sev-
eral disciplines, ranging from art to travel and tourism, with
topics focused on website aesthetics and gender differences
(Moss, Gunn, & Heller, 2006), effects of culture on percep-
tions of website design (Clemmensen, 2011; Cyr, 2008), effects
of cognitive style (imagers vs. verbalizers) users preferences
of usability and aesthetics (Lee & Koubek, 2011), or website
design characteristics and hotels’ marketing practices (Baloglu
& Pekcan, 2006). However, no empirical research exists differ-
entiating the impacts of Norman’s (2003) emotional levels of
 584 S. PENGNATE AND P. ANTONENKO
website design with the development of website users’ initial
online trust.
With regard to the visceral level, the first level of emo-
tional design, several studies have identified the important role
of visual design features on website users’ emotional states,
such as liking, enjoyment, and perception of the design as
visually pleasing (Éthier et al., 2006; Éthier, Hadaya, Talbot,
& Cadieux, 2008). Schwarz’s (1986) framework of affect-
as-information can be used to support the view that emo-
tions have the ability to impact user psychological reactions.
According to this framework, the halo effect of emotional
response toward an object influences the evaluation of object
characteristics (Deng & Poole, 2010; Dunn & Schweitzer, 2005;
Schwarz, 1986). In information systems context, Cyr and col-
leagues found that visual appeal evoked enjoyment, which in
Downloaded by [University of Central Florida] at 11:04 22 November 2014
turn increased website trust (Cyr, 2008). Van der Heijden’s
(2004) study revealed that perceived enjoyment influenced
online users’ intention to use websites. Porat and Tractinsky
(2008) suggested that visual appeal affected users’ attitudes
toward online stores. Tractinsky and Lowengart (2007) also
found the influence of visual appeal on website trustworthiness
in their study. Furthermore, based on the work of Hassanein
and Head (Hassanein & Milena, 2007), Shin and Shin (2011)
provided empirical evidence that perceived social presence can
influence user perception of security, which is a key antecedent
of trust. The findings of this line of research suggest that
website’s visceral design elements elicit emotional responses in
online users, which affect the users’ evaluation of websites and
online trust.
The second level of emotional design—that is, the behav-
ioral level—is reflected in website’s usability. Usability refers
to the features of website design that help users accomplish
their website browsing goals (Sharp, Rogers, & Preece, 2007).
Several studies have suggested that website usability is a deter-
minant of the initial establishment of trust (e.g. Kim & Moon,
1998). Based on the technology acceptance model (Davis,
1989), Gefen, Karahanna, and Straub (2003) provided empir-
ical evidence demonstrating that website usability influenced
both website trust and perceived usefulness in the online shop-
ping context. Schlosser et al. (2006) found that website usability
signaled the capability of website providers to provide prod-
ucts and services, which resulted in trust and intention to
use websites. Flavian et al. (2006) found that the level of
website trust increased when users perceived that the website
was usable. In addition, in diverse cultural groups (Canada,
Germany, and Japan), Cyr (2008) demonstrated that website
usability was a determinant of website trust and the level of
significance was determined by cultural effects.
With respect to the reflective level, the third level of
emotional design, information quality has been described as
an assessment of website providers’ trustworthiness (Kelton,
Fleischmann, & Wallace, 2008). In this study, information qual-
ity refers to users’ judgment of information relevance, which
is one of the four criteria of information quality proposed
by Bovee (2004). The other three dimensions of information
quality include accessibility, interpretability, and integrity
(Bovee, 2004), which are considered as control variables in this
study. The quality of information presented on the website is
an important predictor of website trustworthiness in such con-
texts as online shopping (Nicolaou & McKnight, 2006), mobile
commerce (Siau & Shen, 2003), and supply chain management
(Li & Lin, 2006). As suggested by Norman (2003), the effect
of the reflective level (i.e., information quality) on the over-
all evaluation of artifacts overrides the effects of the visceral
(visual appeal) and behavioral (usability) levels. This study was
designed to conduct an empirical investigation of the validity of
this proposition. Specifically, we hypothesized the following:
H1: The effects of relevant information (i.e., the reflective
level) will override the visceral and behavioral effects on
participants’ website trust.
2.4. Metacognition and Online Trust
Because Norman’s (2003) emotional design model empha-
sizes the role of the reflective level as the top level of this
framework, users’ metacognitive skills—that is, the ability to
monitor and regulate, or reflect on one’s cognitive processing
(Flavell, 1979)—should be paramount to determining online
trust. In this study, we followed the definition of metacognition
proposed by Metcalfe and Shimamura (1994) as “knowing
about knowing”, which refers to knowledge about when and
how individuals can use particular strategies for learning or
for solving problems. Generally, metacognition contains two
distinct components, knowledge of cognition and regulation
of cognition (Garner, 1987). Knowledge of cognition includes
three different types of metacognitive awareness: declarative,
procedural, and conditional knowledge (Schraw, 1998; Schraw
& Moshman, 1995). Regulation of cognition refers to knowing
when and why to use specific knowledge, which helps individ-
uals control their learning or solving problems (Garner, 1990;
Schraw, 1998).
Metacognitive awareness has been mostly studied in online
learning context, and it is suggested that it is a critical fac-
tor in determining success of online learners (Ally, 2004;
Garrison, 2003; Garrison & Cleveland-Innes, 2005; Salmon,
2002). However, it is also important to understand the roles of
metacognitive awareness on online users’ evaluation of website
trust. Thus, we proposed that metacognitive awareness would
influence online users’ evaluation of website trust. Specifically,
we expected that online users with relatively high metacognitive
awareness would rely on information quality (reflective level)
rather than visual appeal (visceral level) or usability (behav-
ioral level) in determining website trust. This is also consistent
with Norman’s (2003) model of emotional design, which pre-
dicts that the effects of the reflective level override the impacts
of the visceral and behavioral level. In addition, we also
anticipated that the level of metacognitive awareness would
influence online user performance in performing a learning task,
especially the time spent on searching for information.
 EMOTIONAL DESIGN, ONLINE TRUST, AND METACOGNITION
Specifically, we hypothesized the following:
H2: Participants selecting mostly relevant concepts on the test
of learning will have higher metacognitive awareness than
participants who use mostly irrelevant information in their
test responses.
H3: Participants with higher metacognitive awareness will
spend less time searching for answers to the learning task
than participants with low metacognitive awareness.
3. METHODOLOGY
3.1. Participants
A total of 151 undergraduate students from education and
business programs at a major midwestern university partici-
Downloaded by [University of Central Florida] at 11:04 22 November 2014
pated in a laboratory experiment (n = 60 of business majors
and n = 91 of education majors). Almost all students from both
majors were junior and senior students. Most participants from
the business program were male (n = 40), and most educa-
tion students were female (n = 69). Most of the participants
were White (n = 123). An independent-samples T test showed
that the participants’ differences in computer usage and the use
of web-based resources to study was not significant. A total
of 133 usable responses (88%) were produced by this study’s
sample.
3.2. Research Design
The laboratory experiment was conducted in a controlled
setting in a computer laboratory where participants worked indi-
vidually on completing a learning task. The study was designed
as a 2 × 2 × 2 factorial experiment with two levels (low vs.
high) for each of three factors (visual appeal – visceral level,
usability – behavioral level, and relevance of the information –
reflective level). Thus, eight website conditions were created
for the experiment as presented in Table 1. Each experimental
website was designed to closely mimic the design of legitimate
organizations’ websites to increase the fidelity of treatment. The
TABLE 1
Eight Experimental Website Conditions
Website Condition Website Code Visual Appeal
1 V1B0R1
2 V1B1R1
3 V0B1R0
4 V0B1R1
5 V0B0R0
6 V1B1R0
7 V0B0R1
8 V1B0R0
585
website brands were removed to avoid branding effects and to
ensure that the websites would be unfamiliar to participants.
Thumbnails of the eight experimental websites are shown in
Figure 1.
Experimental websites were created by a website design
expert. The independent variables were manipulated in the
following way. Visual appeal (visceral level) manipulations
included the color scheme, page layout, and quality of graph-
ics. For example, less visual appeal websites contained low-
resolution images, inconsistent color scheme, and asymmetry
layout. Usability (behavioral level) was primarily manipulated
by changing the readability of the text. For example, less usable
websites contained long paragraphs of text with distracting text
formatting including horizontal and vertical scroll bars. The last
factor, the relevance of the information (reflective level), was
manipulated by providing concepts that were mostly relevant
to solving the learning task on half of the websites and mostly
irrelevant concepts on the other half of the websites. All ver-
sions of the text presented on each website were reviewed and
edited by 10 proofreaders to ensure appropriate content and face
validity. The level of manipulation for visual appeal, usability,
and information relevance was consistent across the website
conditions.
3.3. Pilot Study and Manipulation Checks
A pilot study was conducted with 41 undergraduate busi-
ness majors. Manipulation checks were administered to ensure
that the eight website conditions were manipulated sufficiently
and appropriately. Participants were asked to make comments
on the differences between the website visual appeal, usability,
and quality of information across all website conditions. Data
from the pilot study were then used to revise the website design
manipulations for the main experiment.
3.4. Procedure
Data collection took place in a computer lab with one
computer per participant to control exogenous variance relating
Website Characteristic
Usability Information Relevance
High Low Relevant
High High Relevant
Low High Irrelevant
Low High Relevant
Low Low Irrelevant
High High Irrelevant
Low Low Relevant
High Low Irrelevant
 586 S. PENGNATE AND P. ANTONENKO
Downloaded by [University of Central Florida] at 11:04 22 November 2014
FIG. 1. Thumbnails of the eight experimental websites.
to hardware and software issues, such as display size, monitor
color configuration, network response time, web browser, and
operating systems. Participants individually arrived at the com-
puter lab during a regular class time and were given a brief
written introduction to the study including the consent form.
Screen recording software, CamStudioTM (2010), was installed
on all computers in the lab to record participants’ website
browsing behavior. Before the data were collected, a trial task
was administered to acquaint the participants with CamStudio’s
screen recording interface (the Record and Stop buttons) and
the nature of the information search task. The experimental task
instruction was then provided to participants as follows:
You will now be presented with eight websites, which can be
accessed via the WindowsTM task bar. You need to choose a website
to answer this question: “What are the 5 most promising technolo-
gies?” You have four minutes to browse information on the websites
and six more minutes to record your answers in your answer sheet.
The experimental websites were presented on the monitor as
different browser windows in the Microsoft Windows task bar
in random order to avoid an order effect. The participants had
4 min to find the answers to encourage rapid selection of the best
website for the purpose. Time duration for finding the answers
was limited to four minutes according to website evaluation
time suggested by Lindgaard, Fernandes, Dudek, and Brown
(2006) to enforce rapid selection of trustworthy websites. After
participants wrote down the answers, a paper-based survey was
administered, and then they were shown a screenshot of each
website and asked to rate the trustworthiness of each website.
At the conclusion of the experimental session, the researcher
administered an open-ended question on the strategies used
by the participants to find the answers to the learning task.
An average experimental session lasted approximately 45 min.
Participants received up to 10 points (2.5%) extra credit toward
their course grade as compensation for their participation. The
extra credits were divided into 5 points for participation and
5 points based on the correctness of participants’ answers. One
point was earned for each answer relevant to learning task.
3.5. A Multimethod Approach
In this study, all data came from two types of sources—
self-reports and screen capture. The two methods were com-
plementary in that they offered different insights on the users’
navigation and decision making. On one hand, self-reported
data afforded the ability to statistically test the proposed hypoth-
esis dealing with participants’ perception of trust and their
metacognitive awareness. On the other hand, the screen-capture
approach allowed collection of subconscious and behavioral
data (that might not be reported accurately by participants;
Clark, 2010) as to how the participants browsed the websites
to complete the experimental task, how they interacted with
the websites, which websites they viewed, in what sequence,
and for how long (in milliseconds). The screen-capture data
collection method is also beneficial because it does not inter-
fere with primary cognitive processing like, for instance, the
commonly used think-aloud protocols (e.g., Antonenko, Toy, &
Niederhauser, 2012).
Although eye-tracking can be used to capture useful data
(Murata, 2006) regarding the website elements that users
examine on a web page per the visceral level of Norman’s
 EMOTIONAL DESIGN, ONLINE TRUST, AND METACOGNITION
(2003) user interface design model, it is not applicable at the
behavioral level because eye tracking cannot capture the actual
on-screen interactions of users with website design elements,
such as using the mouse to control scroll bars or switching
between multiple websites. Therefore, screen capture was more
appropriate in this context in the sense that it afforded recording
of participants’ interactions with the experimental websites via
a computer mouse.
This dual methodology contributes to enhancing online
trust research, which so far has relied heavily on either self-
reported or behavioral data (Williams, Cote, & Buckley, 1989).
Furthermore, the qualitative, open-ended question included in
the survey was used to afford a better understanding of “why”
and “how” participants evaluate the websites as evidenced by
the screen capture data.
Downloaded by [University of Central Florida] at 11:04 22 November 2014
3.6. Measurement and Construct Validity
The paper-based survey administered contained ques-
tions regarding metacognitive awareness, learning perfor-
mance, trust, website browsing strategies, experience with
the Internet and technology, and demographic information.
Participants’ metacognitive awareness was measured by using
the Metacognitive Awareness Inventory (MAI) developed by
Schraw and Dennison (1994). The items in MAI reflect the
knowledge of cognition and regulation of cognition dimen-
sions of metacognitive awareness and divide them further
into eight distinct factors: procedural knowledge, declarative
knowledge, conditional knowledge, information management
strategies, debugging strategies, planning, comprehension mon-
itoring, and evaluation). All items of the MAI were rated on a
6-point Likert scale from 1 (strongly disagree) to 6 (strongly
agree). Construct validity of the MAI measures was assessed
using Cronbach’s alpha and resulted in the value of .95 indi-
cating excellent internal consistency (Rivard & Huff, 1988).
In addition to establishing the construct validity, the MAI scores
were examined for their relationship with academic achieve-
ment as suggested by several prior studies (Coutinho, 2007;
Nietfeld, Cao, & Jason, 2005; Schraw & Dennison, 1994;
Young & Fry, 2008). Pearson’s correlations were computed
for participants’ metacognitive awareness scores, grade point
average, and performance on the experimental task. The results
showed that there were significant positive correlations between
metacognitive awareness, grade point average (r = .190), and
task performance (r = .182). Even though the correlations
are relatively low, these results are consistent with the litera-
ture and demonstrate that metacognitive awareness is positively
correlated with academic achievement.
Learning performance was measured by computing the num-
ber of relevant responses to the question (maximum score was
10). After participants submitted their responses on the learn-
ing task, they were asked to view a brief presentation that
demonstrated screenshots of each of the eight websites and
then rate their perceptions of trustworthiness for each website
587
in a self-reported using Likert-scale responses from 1 (least
trustworthy) to 9 (most trustworthy). In addition, participants
were then asked to provide a response to the following open-
ended question: “What strategies did you use to evaluate the
websites?” On average, each participant provided approxi-
mately eight to 10 lines of the text responding to the open-ended
question. Finally, participants were asked to answer questions
regarding their experience with the Internet and technology and
demographic information.
4. RESULTS
H1: The effects of relevant information (i.e., the reflective
level) will override the visceral and behavioral effects on
participants’ website trust.
Participants’ answers to the question regarding the five most
promising technology used in the experimental task were ana-
lyzed for the effects of the three website design levels on trust
(Figure 2). Based on the participants’ answer sheet, the top three
websites conditions that were most trusted include Website
Condition 2 (V1B1R1), 4 (V0B1R1), and 6 (V1B1R0), which
represent 35.3%, 27.8%, and 23.3% of the total responses,
respectively, and account for 86.4% of the total participants.
A one-way analysis of variance (ANOVA) was conducted
to investigate the differences in website trustworthiness across
website conditions based on the self-reported rating of trust rat-
ing for each website. Tukey’s Honestly Significant Difference
post hoc comparison was then run to examine specific dif-
ferences between website conditions. The results of Tukey’s
Honestly Significant Difference post hoc comparison and the
mean scores of trust across websites are presented in Table 2.
Overall, Hypothesis 1 was not supported, because trustworthi-
ness scores for Website Condition 6 (low information relevance,
high visual appeal, and high usability) and Website Condition
8 (low information relevance, high visual appeal, and low
usability) were significantly higher than website conditions
containing relevant information, F(7, 1055) = 26.25, p < .001.
FIG. 2. Summary of website trustworthiness survey.
 588 S. PENGNATE AND P. ANTONENKO

TABLE 2
Tukey’s Honestly Significant Difference (HSD) Post Hoc Comparison Results
Website Condition N M SD Tukey HSD Group
#2 – V1B1R1 133 7.01 2.33 a
#6 – V1B1R0 133 6.69 2.55 a
#8 – V1B0R0 132 6.31 2.30 a
#4 – V0B1R1 133 6.18 2.54 a
#7 – V0B0R1 133 5.13 2.80 b
#1 – V1B0R1 133 5.03 2.40 b
#3 – V0B1R0 133 4.53 2.40 b, c
#5 – V0B0R0 133 4.02 2.08 c
Note. The Tukey HSD Group column represents Tukey’s HSD groupings of means. Different
letters represent significant differences of the means among website conditions.
Downloaded by [University of Central Florida] at 11:04 22 November 2014

H2: Participants reporting relevant concepts on the test of evaluation). A one-way ANOVA analysis showed that there
learning will have higher metacognitive awareness than were no significant differences in the subdimensions of
participants who report mostly irrelevant information in metacognitive awareness across the top three website condi-
their test answers. tions. The means and standard deviations for each subdimension
are presented in Table 4. Participants selecting Website
To assess the relationship between metacognitive awareness Condition 4 (V0B1R1) demonstrated the highest average rating
and website trust behavior, a one-way ANOVA was conducted of procedural knowledge, declarative knowledge, conditional
using the top three trusted website conditions (2, 4, and 6) as knowledge, information management strategies, debugging
an independent variable and metacognitive awareness score as
the dependent variable. The ANOVA results indicated that the
metacognitive awareness of individuals selecting each of the TABLE 3
three website conditions was not significantly different across Means and Standard Deviation of
website conditions. Thus, Hypothesis 2 was not supported. Metacognitive Awareness of the Top Three
The means and standard deviations of metacognitive awareness Websites
across website conditions are presented in Table 3.
Website Condition N M SD
We then further investigated the participants’ responses
related to trust and the eight subdimensions of metacognitive #2 – V1B1R1 47 37.53 10.10
awareness (procedural knowledge, declarative knowledge, con- #4 – V0B1R1 37 38.97 5.83
ditional knowledge, information management strategies, debug- #6 – V1B1R0 31 36.54 8.83
ging strategies, planning, comprehension monitoring, and

TABLE 4
Descriptive Statistics for Metacognitive Awareness Subdimensions
#2a #4b #6c
V1B1R1 V0B1R1 V1B1R0

M SD M SD M SD
Procedural knowledge 3.10 .98 3.51 .86 3.19 .98
Declarative knowledge 6.31 1.77 6.45 1.55 6.25 1.61
Conditional knowledge 3.89 1.14 4.27 .83 3.87 1.23
Information management strategies 7.53 2.07 7.86 1.63 7.22 2.04
Debugging strategies 4.00 1.26 4.45 .55 4.16 1.03
Planning 4.74 1.82 4.67 1.66 4.35 1.79
Comprehension monitoring 4.44 1.94 4.27 1.71 4.25 1.93
Evaluation 3.65 1.63 3.59 1.32 3.32 1.79
Note. The values in boldface represent the highest means among the three website conditions.
a
n = 47. a n = 37. a n = 31.
 EMOTIONAL DESIGN, ONLINE TRUST, AND METACOGNITION
strategies, whereas participants selecting Website Condition
2 (V1B1R1) showed the highest average rating of planning,
comprehension monitoring, and evaluation.
H3: Participants with high metacognitive awareness will spend
less time searching for answers to the learning task than
participants with low metacognitive awareness.
Screen-capture data were analyzed for the time spent search-
ing for the answers from the eight website conditions. Video
player software programs like Windows Media Player provide
an effective means to analyze screen-capture data in the video
file format because they display a time line and allow raters to
calculate the total time participants spent browsing the websites
until they located the website that they trusted. Three indepen-
Downloaded by [University of Central Florida] at 11:04 22 November 2014
dent raters reviewed screen-capture videos for each participant
and coded the data indicating the amount of time spent by
each participant on each of the eight experimental websites and
the sequence of accessing these websites. Interrater reliability
was computed using Pearson product–moment correlation coef-
ficient, and this analysis confirmed high interrater agreement
(r = .92) within a 2-s margin of error. Participants could switch
between websites several times. However, they had to specify
only one website to answer the question in the answer sheet.
Analysis of the screen-capture data confirmed that all the par-
ticipants spent most time reading information from the website
that they trusted as indicated in the answer sheet.
To test Hypothesis 3, we ranked the participants based on
metacognitive awareness and then divided the data set into
terciles by metacognitive awareness scores. The top, mid-
dle, and bottom terciles represent group of participants with
high, moderate, and low metacognitive awareness, respectively.
In this analysis, we were interested in only the low and high
metacognitive awareness groups, so participants in the middle
tercile were excluded from the analysis.
Cases with the highest and the lowest scores were then used
in a T test that compared means based on the average time
spent searching for answers as the independent variable. The
results suggested that the effect of metacognitive awareness on
time spent searching for the answers was in the correct direction
as expected in the hypothesis—that is, participants with higher
metacognitive awareness tended to find the answers faster (137 s
on average) than those with lower metacognitive awareness
(147 s). However, the difference in metacognitive awareness
for these two participant groups was not statistically significant.
Therefore, Hypothesis 3 was not supported.
4.1. Results of Qualitative Analysis
To better understand the influence of the three emotional
design levels highlighted in the experimental websites, open-
ended data were analyzed by coding and then categorizing par-
ticipants’ website browsing behavior, followed by the creation
of emerging themes about website design and trust evaluation
589
FIG. 3. Summary of coded data.
(Denzin & Lincoln, 2005). Figure 3 contains a summary of the
coded data.
According to the open-ended data, the results revealed that
48% of the participants started browsing the websites by consid-
ering website usability (i.e., the behavioral level), especially the
layout and readability of the content as the first factor to deter-
mine if the websites were trustworthy. Approximately 28% of
the participants used the visual appearance and attractiveness
of the websites as the main indicator of website trustworthi-
ness. Finally, approximately 17% of the participants evaluated
the content of the information to determine website trustworthi-
ness. Almost 7% of the participants could not identify or recall
strategies they used in the website evaluation.
In addition to the three emotional design levels highlighted
in the experimental websites, analysis of open-ended response
data revealed two prominent themes for website design ele-
ments that influence online trust:
Visual rhetoric. Several participants believe that images dis-
played on websites convey important messages. For exam-
ple, several participants made comments that the most
trustworthy website was selected because the website con-
tained technology-related images that were relevant to the
question asked in the experimental task, and based on that
they assumed that website content was relevant.
Design appropriateness. Participants made comments regard-
ing how appropriate website design matched purposes
of websites. A number of participants reported that sev-
eral of the experimental websites looked like forum-based
websites or had layouts resembling blogs so the informa-
tion on those websites was perceived as opinion rather
than fact. Therefore, online trust in these situations was
diminished.
In general, open-ended data supported results from the quan-
titative analysis and provided some insights into the screen
capture analysis. Several participants reported that the major
reason why they started browsing the websites by focusing on
usability first was because it could help them find the answers
faster within the limited time constraints of the experiment.
 590 S. PENGNATE AND P. ANTONENKO
In some cases, participants mentioned that they felt websites
with an abundance of visual design elements had negative affec-
tive influences (i.e., “too gaudy” or “extravagant”) on website
evaluation.
5. DISCUSSION AND CONCLUSIONS
This study clarifies the impacts of emotional levels of
website design and metacognitive awareness on website trust-
worthiness during a learning task. Our research suggests that
when individuals have limited time and a high number of
websites to choose from, website trustworthiness is influenced
by affective (visceral) and usability (behavioral) design levels
rather than the relevance of information (the reflective level).
Downloaded by [University of Central Florida] at 11:04 22 November 2014
Results reveal that the interaction effects of website visual
appeal and website usability can override the effects of the qual-
ity or relevance of the information. In addition, results also
suggest that in the context of limited time to find the answers,
these effects of the website evaluation on trust are not moder-
ated by users’ metacognitive awareness. However, even though
the results of statistical tests were not significant, our findings
suggest that individuals with higher metacognitive awareness
tend to perform better on an information search task by spend-
ing shorter time searching for relevant information. To our
knowledge, this is the first study conducted to compare the
effects of the levels of emotional website design and effects
of metacognitive awareness on website trustworthiness when
online users have limited time and are exposed to a range of
websites.
From a theoretical perspective, this study provides the fol-
lowing contributions. First, website design elements are con-
firmed as important predictors of online trust. Although these
relationships have been supported by previous studies (Cyr,
2008; Cyr, Head, Larios, & Pan, 2009; Flavián et al., 2006; Fogg
et al., 2003), this study validated these results by testing the
impacts of various website design elements using experimen-
tal manipulations of emotional design—visual appeal, usability,
and information quality. According to our results, the experi-
mental website condition that was trusted most by participants
was Website Condition 4, which contained low visual appeal
but high usability and provided high-quality information. The
number of participants trusted this website condition is higher
than Website Condition 2, which had been designed to be the
“best” website where all visual appeal, usability, and quality
information were high. One of the explanations for this unex-
pected finding may be that processing of the multiple elements
presented on some of the websites, such as those found in
Website Condition 2, is compromised due to a competition for
participants’ limited working memory resources, and such com-
petition hinders evaluation of website trust in the context of
limited time for completing the task.
Second, another significant finding of this study is that the
interaction effect of website visual appeal (visceral level) and
usability (behavioral level) can override the effect of informa-
tion quality (reflective level) on trust. This finding extends work
on website credibility (Fogg et al., 2003) and on website design
(Cyr, 2008; Flavián et al., 2006). This study is also the first study
to extend Norman’s (2003) emotional design framework into
the realm of website design with subsequent consequences on
online trust.
Third, this study theorized that the moderation effects
of individuals’ metacognitive awareness on their perceptions
regarding website trust would be significant. However, this
was not the case in the context of this study. Even though
we offered rewards for each correct answer in the experimen-
tal tasks, it seems that participants were not intensively using
their metacognitive awareness in evaluating the experimental
websites when pressed for time. Specifically, they did not likely
have the time or motivation to process the websites at a deeper
reflective level but judged website trustworthiness based on
their feelings instead. Consistent with findings in the cognitive
science literature (Nelson, 1996; Nelson, Kruglanski, & Jost,
1998), one factor that stimulates metacognition is motivation
for accuracy. The elaboration likelihood model (ELM; Petty &
Cacioppo, 1986) can explain why this occurred. ELM proposes
that there are two major factors that influence human informa-
tion processing: motivation and ability. Without deep motiva-
tion, even though individuals may have the ability to process
information, they are likely to rely on a peripheral route, such as
visual appearance, in their assessment (Fogg et al., 2003; Petty
& Cacioppo, 1986). Consequently, if this study’s participants
had high motivation to examine the experimental websites care-
fully, ELM would predict that the participants would use the
central route, such as more intensive use of metacognition, and,
therefore, the moderation effects of metacognitive awareness
would play a more significant role. In addition to the effects
on trust, metacognitive awareness did not impact time spent
searching for the answers. Although, as expected, the partici-
pants with high metacognitive awareness on average spent less
time searching for the answers, this effect was not statistically
significant.
The fourth theoretical contribution of this study is the intro-
duction of two themes that emerged from the analysis of
responses to the open-ended question. Both visual rhetoric and
design appropriateness have been investigated in the online
shopping context as impacting the overall perception of the
purpose of the website (Cyr et al., 2009) and relevance of infor-
mation presentation format (Deng & Poole, 2010; Hong, Thong,
& Tam, 2004; Nadkarni & Gupta, 2007), respectively. However,
the roles of both concepts are still not well understood and may
result in different user responses in the educational website con-
text as online shopping websites and educational websites have
different goals and result in different motivations and patterns
of HCI. Hence, it is important to determine the applicability of
these concepts in online learning, as well as other domains.
From a practical perspective, there are several contributions
that arise from this study that will be of interest to online users
 EMOTIONAL DESIGN, ONLINE TRUST, AND METACOGNITION
and website designers. In particular, website users are advised
to pay attention to information quality when evaluating informa-
tion presented on websites, as there is a potential for halo effect,
where the judgment of website’s visual appeal can override
the assessment of information quality, as found in the exper-
imental Website Condition 6, which was rated as high visual
appeal and usability, but it actually did not present high-quality
information.
Further, website designers and managers should consider
the impact of website elements related to trust. Websites that
carry high-quality information but are perceived as inferior to
those with higher visual appeal and usability can cause distrust
among users because they perceive such websites as unap-
pealing. Finally, the results of this study can provide guidance
as to which types of website elements should be used in the
Downloaded by [University of Central Florida] at 11:04 22 November 2014
website interface to suit particular website audiences’ prefer-
ences. For example, if target audiences of a website tend to
have relatively low metacognition, such as children, the website
designers should design the interfaces that help its users achieve
their goals with minimum reflective effort.
As with any study, the findings should be considered in light
of their limitations. First, the 10-point extra credit inducement
may not have been sufficient to encourage the participants to
engage in deep reflective processes performing the experimen-
tal task. Additional research can compare results obtained in a
real setting with a greater stake in the outcome of the search task
so that participants are more motivated to look for indicators of
website quality. Second, experimental websites were manipu-
lated for visual appearance, usability, and information quality.
However, as noted by some participants, the layout of several
websites appeared inappropriate because it reminded them of a
discussion forum or blog. Other studies can, therefore, explore
the effects of website layout perceptions either in this context or
in other contexts.
To conclude, it is hoped that this study will inform future
research to expand our understanding of the role of website
design features and learners’ individual differences in website
evaluation. This research also has the potential to provide
website users and learners with enhanced knowledge on how
to evaluate website trustworthiness and credibility, which is
critical to the development of information literacy in the 21st
century, especially in the educational domain.
REFERENCES
Ally, M. (2004). Foundations of educational theory for online learning. In R.
McGreal (Ed.), Designing effective learning objects for distance education
(pp. 87–97). London, UK: Routledge Falmer.
Antonenko, P., Toy, S., & Niederhauser, D. (2012). Using cluster analysis for
data mining in educational technology research. Educational Technology
Research and Development, 60, 363–398. doi:10.1007/s11423-012-9235-8
Artino, A. R., Jr., & Jones Ii, K. D. (2012). Exploring the complex rela-
tions between achievement emotions and self-regulated learning behaviors
in online learning. The Internet and Higher Education, 15, 170–175.
doi:10.1016/j.iheduc.2012.01.006
591
Azevedo, R., & Jacobson, M. J. (2008). Advances in scaffolding learning with
hypertext and hypermedia: A summary and critical analysis. Educational
Technology Research and Development, 56, 93–100.
Baloglu, S., & Pekcan, Y. A. (2006). The website design and Internet site
marketing practices of upscale and luxury hotels in Turkey. Tourism
Management, 27(1), 171–176. doi: 10.1016/j.tourman.2004.07.003
Bovee, M. W. (2004). A Conceptual Framework and Empirical Validation.
Ph.D. Dissertation, University of Kansas, Lawrence, KS.
Casaló, L., Flavián, C., & Guinalíu, M. (2008). The role of perceived
usability, reputation, satisfaction and consumer familiarity on the website
loyalty formation process. Computers in Human Behavior, 24, 325–345.
doi:10.1016/j.chb.2007.01.017
Clark, R. E. (2010). Cognitive and neuroscience research on learning and
instruction: recent insights about the impact of non-conscious knowledge
on problem solving, higher order thinking skills and inter- active cyber-
learning environments. Paper presented at the International Conference on
Education Research, October, Seoul, Korea.
Clemmensen, T. (2011). Templates for cross-cultural and culturally specific
usability testing: results from field studies and ethnographic interviewing
in three countries. International Journal of Human-Computer Interaction,
27, 634–669. doi:10.1080/10447318.2011.555303
Coutinho, S. A. (2007). The relationship between goals, metacognition, and
academic success. The Journal of Doctoral Research in Education, 7,
39–47.
Cyr, D. (2008). Modeling web site design across cultures: Relationships to trust,
satisfaction, and e-Loyalty. Journal of Management Information Systems,
24(4), 47–72. doi:http://dx.doi.org/10.2753/MIS0742-1222240402
Cyr, D., & Bonanni, C. (2005). Gender and website design in e-business.
International Journal of Electronic Business, 3, 565–582.
Cyr, D., Head, M., & Larios, H. (2010). Colour appeal in website design within
and across cultures: A multi-method evaluation. International Journal of
Human–Computer Studies, 68, 1–21. doi:10.1016/j.ijhcs.2009.08.005
Cyr, D., Head, M., Larios, H., & Pan, B. (2009). Exploring human images in
Website design: a multi-method approach. MIS Quarterly, 33, 539–566.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user
acceptance of information technology. MIS Quarterly, 13(3), 319.
Deng, L., & Poole, M. S. (2010). Affect in web interfaces: A study of the
impacts of web page visual complexity and order. MIS Quarterly, 34,
711–730.
Denzin, N. K., & Lincoln, Y. S. (2005). Introduction: The discipline and practice
of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), The sage
handbook of qualitative research (2 ed.). Thousand Oaks, CA: Sage.
Dittmer, H., Long, K., & Meek, R. (2004). Buying on the internet: Gender
difference in on-line and conventional buying motivations. Sex Roles, 50,
423–444.
Dunn, J. R., & Schweitzer, M. E. (2005). Feeling and Believing: The Influence
of Emotion on Trust. Journal of Personality and Social Psychology, 88(5),
736–748. doi: 10.1037/0022-3514.88.5.736
Éthier, J., Hadaya, P., Talbot, J., & Cadieux, J. (2006). B2C web site quality and
emotions during online shopping episodes: An empirical study. Information
& Management, 43, 627–639.
Éthier, J., Hadaya, P., Talbot, J., & Cadieux, J. (2008). Interface
design and emotions experienced on B2C Web sites: Empirical test-
ing of a research model. Computers in Human Behavior, 24(6),
2771–2791.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new
area of cognitive–developmental inquiry. American Psychologist, 34(10),
906–911.
Flavián, C., Guinalíu, M., & Gurrea, R. (2006). The role played by perceived
usability, satisfaction and consumer trust on website loyalty. Information &
Management, 43(1), 1–14. doi:10.1016/j.im.2005.01.002
Fogg, B. J., Soohoo, C., Danielson, D. R., Marable, L., Stanford, J., & Tauber, E.
R. (2003). How do users evaluate the credibility of Web sites?: A study with
over 2,500 participants. Proceedings of the 2003 Conference on Designing
for User Experiences, 1–15.
Gage, J. (1999). Color and culture: Practice and meaning from antiquity to
abstraction. Berkeley: The University of California Press.
 592 S. PENGNATE AND P. ANTONENKO
Garbarino, E., & Strahilevitz, M. (2004). Gender differences in the perceived
risk of buying online and the effects of receiving a site recommen-
dation. Journal of Business Research, 57, 768–775. doi:10.1016/s0148-
2963(02)00363-6
Garner, R. (1987). Metacognition and reading comprehension. Cognition and
literacy. Westport, CT: Ablex.
Garner, R. (1990). When children and adults do not use learning strategies:
Toward a theory of settings. Review of Educational Research, 60, 517–529.
Garrison, D. R. (2003). Cognitive presence for effective asynchronous online
learning: The role of reflective inquiry, self-direction and metacognition.
In J. Bourne & J. C. Moore (Eds.), Elements of quality online education:
Practice and direction (pp. 47–58). Needham, MA: Sloan-C.
Garrison, D. R., & Cleveland-Innes, M. (2005). Facilitating cognitive presence
in online learning: Interaction is not enough. American Journal of Distance
Education, 19, 133–148. doi:10.1207/s15389286ajde1903_2
Gefen, D. (2000). E-commerce: the role of familiarity and trust. Omega, 28,
725–737. doi:10.1016/s0305-0483(00)00021-9
Gefen, D., Karahanna, E., & Straub, D. W. (2003). Trust and TAM in Online
Shopping: An Integrated Model. MIS Quarterly, 27(1), 51–90.
Downloaded by [University of Central Florida] at 11:04 22 November 2014
Hassanein, K. S., & Milena, H. M. (2007). Manipulating perceived social pres-
ence through the web interface and its impact on attitude towards online
shopping. International Journal of Human–Computer Studies, 65, 689–708.
Hong, W., Thong, J. Y. L., & Tam, K. Y. (2004). The effects of informa-
tion format and shopping task on consumers’ online shopping behavior: A
cognitive fit perspective. Journal of Management Information Systems, 21,
149–184.
Jarvenpaa, S., Tractinsky, N., & Vitale, M. (2000). Consumer trust in an Internet
store. Information Technology and Management, 1, 45–71.
Jordan, W. P. (2000). Designing pleasurable products. New York, NY: Taylor
& Francis.
Kelton, K., Fleischmann, K. R., & Wallace, W. A. (2008). Trust in digital
information. Journal of the American Society for Information Science and
Technology, 59(3), 363–374.
Kim, J., & Moon, J. Y. (1998). Designing towards emotional usability in
customer interfaces–trustworthiness of cyber-banking system interfaces.
Interacting with Computers, 10(1), 1–29.
Kim, D. J., Ferrin, D. L., & Rao, H. R. (2009). Trust and satisfaction, two
stepping stones for successful e-commerce relationships: A longitudinal
exploration. Information Systems Research, 20(2), 237–257.
Lee, S., & Koubek, R. J. (2011). The impact of cognitive style on user
preference based on usability and aesthetics for computer-based systems.
International Journal of Human–Computer Interaction, 27, 1083–1114.
doi:10.1080/10447318.2011.555320
Li, S., & Lin, B. (2006). Accessing information sharing and information quality
in supply chain management. Decision Support Systems, 42(3), 1641–1656.
Li, H., Sarathy, R., & Xu, H. (2011). The role of affect and cogni-
tion on online consumers’ decision to disclose personal information
to unfamiliar online vendors. Decision Support Systems, 51, 434–445.
doi:10.1016/j.dss.2011.01.017
Lim, K., Sia, C., Lee, M., & Benbasat, I. (2006). Do I trust you online, and if
so, will I buy? An empirical study of two trust-building strategies. Journal
of Management Information Systems, 23(2), 233–266.
Lindgaard, G., Dudek, C., Sen, D., Sumegi, L., & Noonan, P. (2011). An
exploration of relations between visual appeal, trustworthiness and per-
ceived usability of homepages. ACM Transactions on Computer–Human
Interaction, 18, 1–30. doi:10.1145/1959022.1959023
Lindgaard, G., Fernandes, G., Dudek, C., & Brown, J. (2006). Attention web
designers: You have 50 milliseconds to make a good first impression!
Behaviour & Information Technology, 25, 115–126.
Liu, I. F., Chen, M. C., Sun, Y. S., Wible, D., & Kuo, C.-H. (2010).
Extending the TAM model to explore the factors that affect intention to
use an online learning community. Computers & Education, 54, 600–610.
doi:10.1016/j.compedu.2009.09.009
McCarthy, J., & Wright, P. (2004). Technology as experience. Cambridge, MA:
MIT Press.
McKnight, D. H., Cummings, L. L., & Chervany, N. L. (1998). Initial trust
formation in new organizational relationships. The Academy of Management
Review, 23(3), 473–490.
McKnight, D. H., & Chervany, N. L. (2001). What trust means in
e-commerce customer relationships: An interdisciplinary conceptual typol-
ogy. International Journal of Electronic Commerce, 6(2), 35–59.
Metcalfe, J., & Shimamura, A. P. (1994). Metacognition: Knowing about
knowing. Cambridge, MA: MIT Press.
Moss, G., Gunn, R., & Heller, J. (2006). Some men like it black, some
women like it pink: consumer implications of differences in male
and female website design. Journal of Consumer Behaviour, 5(4),
328–341.
Murata, A. (2006). Eye-gaze input versus mouse: Cursor control as a function
of age. International Journal of Human–Computer Interaction, 21, 1–14.
doi:10.1080/10447310609526168
Nadkarni, S., & Gupta, R. (2007). A task-based model of perceived website
complexity. MIS Quarterly, 31, 501–524.
Nelson, T. O. (1996). Consciousness and metacognition. American
Psychologist, 51, 102–116. doi:10.1037/0003-066x.51.2.102
Nelson, T. O., Kruglanski, A. W., & Jost, J. T. (1998). Knowing thyself and
others: Progress in metacognitive social psychology. In V. Y. Yzerbyt,
G. Lories, & B. Dardenne (Eds.), Metacognition: Cognitive and social
dimensions (pp. 69–89). London, UK: Sage.
Nicolaou, A. I., & McKnight, D. H. (2006). Perceived Information Quality in
Data Exchanges: Effects on Risk, Trust, and Intention to Use. [Article].
Information Systems Research, 17(4), 332–351.
Nietfeld, J. L., Cao, L., & Jason, W. O. (2005). Metacognitive monitoring accu-
racy and student performance in the postsecondary classroom. Journal of
Experimental Education, 74(1), 7–28.
Norman, D. A. (2003). Emotional design: Why we love (or hate) every things.
New York, NY: Basic Books.
Pennington, R., Wilcox, H. D., & Grover, V. (2003). The role of system trust
in business-to-consumer transactions. Journal of Management Information
Systems, 20, 197–226.
Petty, R. E., & Cacioppo, J. T. (1986). The Elaboration Likelihood
Model of persuasion. In B. Leonard (Ed.), Advances in experimental
social psychology (Vol. 19, pp. 123–205). New York, NY: Academic
Press.
Porat, T., & Tractinsky, N. (2008). Affect as a Mediator between Web-Store
Design and Consumers’ Attitudes toward the Store. In C. Peter & R. Beale
(Eds.), Affect and Emotion in Human-Computer Interaction (Vol. 4868,
pp. 142–153): Berlin, Germany: Springer Berlin / Heidelberg.
Rivard, S., & Huff, S. L. (1988). Factors of success for end-user computing.
Communication of ACM, 31, 552–561. doi:10.1145/42411.42418
Rodgers, S., & Harris, M. A. (2003). Gender and e-commerce: An exploratory
study. Journal of Advertising Research, 43, 322–329.
Salmon, G. (2002). E-tivities: The key to active online learning. London:
Routledge Falmer.
Schlosser, A. E., White, T. B., & Lloyd, S. M. (2006). Converting web site
visitors into buyers: how web site investment increases consumer trust-
ing beliefs and online purchase intentions. Journal of Marketing, 70,
133–148.
Schraw, G. (1998). Promoting general metacognitive awareness. Instructional
Science, 26(1), 113–125. doi:10.1023/a:1003044231033
Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive
awareness. Contemporary Educational Psychology, 19, 460–475.
doi:10.1006/ceps.1994.1033
Schraw, G., & Moshman, D. (1995). Metacognitive theories. Educational
Psychology Review, 7, 351.
Schwarz, N. (1986). Feelings as information: Informational and motivational
functions of affective states. In T. Higgins & R. M. Sorrentino (Eds.),
Handbook of Motivation and Cognition (Vol. 2, pp. 527–561). New York:
The Guilford Press.
Sharp, H., Rogers, Y., & Preece, J. (2007). Interaction design: Beyond human-
computer interaction. Hoboken, N.J.: Wiley.
Shin, D.-H., & Shin, Y.-J. (2011). Consumers’ trust in virtual mall
shopping: The role of social presence and perceived security.
International Journal of Human–Computer Interaction, 27, 450–475.
doi:10.1080/10447318.2011.552060
Siau, K., & Shen, Z. (2003). Building customer trust in mobile commerce.
Commun. ACM, 46(4), 91–94.
 EMOTIONAL DESIGN, ONLINE TRUST, AND METACOGNITION
Simon, S. J. (2000). The impact of culture and gender on web sites: An empiri-
cal study. SIGMIS Database, 32, 18–37. doi:10.1145/506740.506744
Sweller, J., van Merrienboer, J., & Paas, F. (1998). Cognitive architecture and
instructional design. Educational Psychology Review, 10(3), 251–296.
Tractinsky, N., & Lowengart, O. (2007). Web-Store aesthetics in e-retailing:
A conceptual framework and some theoretical implications. Academy of
Marketing Science Review, 11(1).
Van der Heijden, H. (2004). User acceptance of hedonic information systems.
MIS Quarterly, 28, 695–704.
Wells, J. D., Valacich, J. S., & Hess, T. J. (2011). What signal are you sending?
How website quality influences perceptions of product quality and purchase
intentions. MIS Quarterly, 35, 373–396.
Williams, L. J., Cote, J. A., & Buckley, M. R. (1989). Lack of method variance
in self-reported affect and perceptions at work: Reality or artifact? Journal
of Applied Psychology, 74(3), 462–468.
Young, A., & Fry, J. D. (2008). Metacognitive awareness and academic achieve-
ment in college students. Journal of the Scholarship of Teaching and
Learning, 8(2), 1–10.
Downloaded by [University of Central Florida] at 11:04 22 November 2014
593
ABOUT THE AUTHORS
Supavich (Fone) Pengnate is a doctoral candidate at Oklahoma
State University. He completed his Master of Science at
University of North Carolina-Greensboro. Fone’s research
focuses on the influence of user interface design on users’
perceptions and responses in various computer platforms.
Pavlo (Pasha) Antonenko is an Assistant Professor of
Educational Technology at the University of Florida. Pavlo
holds a PhD in Education and Human Computer Interaction
from Iowa State University. He teaches educational technology
graduate courses and conducts research in problem-based learn-
ing, psychophysiology and cognition, and design of learner- and
teacher-friendly interfaces.