The iPerceptions

Satisfaction Index
The only metric designed exclusively to measure
customer satisfaction with the online channel
an iPerceptions white paper
2
Know Satisfaction
TM
Diverse methods have been deployed to
measure, and thereby understand, website
user behavior. These methods, which to
date have included clickstream analytics,
multivariate testing, and usability lab results,
have collectively been termed web analytics.
The windfall of learnings from these silos of
data has been prodigious. The collective output
of insights from these various methods has,
however, been incomplete. This is because
the most critical component of any attempt to
understand the experience of website visitors
is to gauge their attitudinal response to a site
within the context of an actual online session. It
is on this premise that survey-driven attitudinal
analytics operates and it is only in this manner
that a holistic picture of online visitor satisfaction
can emerge.
But to draw out actionable insights from
attitudinal responses and to make sense of the
vast reams of visitor opinion and experiential
data that surveys output, decision makers need
access to a trustworthy metric. To meet this
need, iPerceptions developed the iPerceptions
Satisfaction Index (iPSI), which has been used
to measure the online experiences of over 4
million actual website visitors in a broad array of
industries, including e-commerce, automotive,
hospitality, government, and B2B.
This paper will explore the genesis of the iPSI and
elucidate the reasons why industry leaders have
increasingly adopted it as the gold standard
for measuring and understanding visitor
satisfaction. The iPSI is the rst satisfaction index
conceived exclusively for the online experience;
it does not suffer from the same portability issues
that have plagued ofine metrics that have been
twisted to accommodate online behavior.
Moreover, the iPSI is the rst index built on a core
set of Attributes that take into account the totality
of the online experience, from site usability all
the way through to the sites capacity to sustain
and grow brand loyalty. While these Attributes
are predictive of next steps behaviors, they were
also crafted with the knowledge that not every
site functions as an e-commerce platform and
not every successful online outcome can be
reduced to transactional conversion events.
Finally, the iPSI is the rst index to bridge the
well-documented gap between quantitative and
qualitative visitor feedback in order to provide
a holistic, 360 degree composite image of the
visitor experience. It does so using an innovative
heuristic that maps open-ended feedback to
each Attribute on the basis of relevancy, tone,
diction, and frequency.
Introduction
3
Know Satisfaction
TM
The iPSI is the end result of the webValidator
sampling methodology, a proven, site-level
approach to online surveying, which solicits
ratings for the 15 most critical Attributes of a
particular web presence. The webValidator
survey is deployed in two-stages: rst, a polite,
landing-page solicitation is presented on arrival
to a domain, which, if accepted, then triggers
a second, minimized questionnaire window
that unobtrusively waits in the background
while the visitor completes his/her session.
Experimentation has proven that this solicitation
method consistently yields response rates in
the range of 4% to 6% and completion rates of
upwards of 90%, both of which compare quite
favorably to the rates achieved in traditional
ofine market research.
In the course of the survey, visitors are asked
to provide scores for website Attributes relative
to their best online experience. Although a
standard stem question provides a uniform
casing for the Attribute questions, each Attribute
question speaks to a very specic element of
the online experience or is predictive of a very
specic outcome, such as a visitors likelihood
to recommend the site to a friend or colleague.
Deviating from other data collection methods
in the market place, the Attribute scoring is
done on a 0 to 10 point scale. Experimentation
has proven the 11-point scale to be optimal for
measuring online customer satisfaction, while
allowing for sufcient granularity. The scale uses
six offset qualitative descriptors, ranging from
Very Bad to Outstanding, which lend context
to the numbers. This results in a truncated normal
distribution curve, which is not observed in other
11 point scales that use terminal and midpoint
descriptors. Moreover, starting the scale at 0 is
of critical psychological importance. Unlike the
approach used in Likert scale response sets,
providing the option to select 0 allows the
respondent to express absolutely no satisfaction
or agreement, which is a critical part of the
process of democratizing the customers voice.
The iPSI score is the mean of the 15 Attribute
scores being employed to measure visitor
satisfaction on a particular site. It comprises,
therefore, a broad array of data points; in doing
so, it traverses the full spectrum of the website
experience. Attribute sets are standardized within
industry verticals, which enables benchmarking
and intra-industry comparisons.
Moreover, the iPSI score is the top-most layer of
a tripartite scoring structure that evolves from the
granular to the aggregate. Underlying the iPSI
score are ve website Dimension scores, which
themselves comprise three Attribute scores. The
website Dimension scores enable practitioners
Measuring Online
Satisfaction with the iPSI
4
Know Satisfaction
TM
to understand the core aspects of the website
experience and function as taxonomical buckets
to which Attributes are assigned. If the Attributes
are the individual organs of the website, then
the Dimensions are the systems that ensure
these organs operate in cohesion.
The Dimensions are based on a perceptual
framework that situates the online experience
at the intersection of two theoretical axes: time
and orientation. The temporal axis gauges
the creation of immediate versus long-term
value, while the orientation axis has terminal
points that speak to front-end user desirability
and back-end site usefulness. The Dimensions
cluster around the intersection of these two axes
and are encountered by the user in a clockwise
loop.
The four formative Dimensions are the elements
of the 2 x 2 matrix that is formed by the crossing
of the axes. These are:
Navigation The rst Dimension of the online
encounter, which crosses immediacy with user
desirability and measures whether visitors can
nd what they are looking for and whether
visitors can rapidly move around the site.
Content Like Navigation, Content is concerned
with the creation of immediate value, but
more in the sense of furnishing the user with
useful information. This Dimension measures
whether the onsite content is up-do-date,
detailed, accurate, and relevant.
Interactivity Measures the creation of the
long-term value that can be engendered
through interactive features. This Dimension
measures functionality, collaboration,
informational exchange, and the intuitiveness
of transactional tools.
Motivation Measures the extent to which
the website can sustain and build on brand
equity and goodwill. Measures whether the
brand messaging aligns with ofine content,
whether the site is trustworthy, and whether
the site experience leaves visitors feeling that
their time was well-spent.
n
a
v
i
g
a
t
i
o
n





c
o
n
t
e
n
t
m
o
t
i
v
a
t
io
n


in
te
r
a
c
t
i
v
i
t
y
immediate
long term
d
e
s
i
r
a
b
i
l
i
t
y
u
s
e
f
u
l
n
e
s
s
adoption
5
Know Satisfaction
TM
The one summative (or outcomes) Dimension of
the website experience is:
Adoption Encapsulates the online experience
by measuring loyalty and advocacy. Speaks
to issues such as: What is the likelihood that
visitors will make repeat visits? Are visitors
likely to evangelize or become advocates for
the site?
The perceptual framework underpinning the iPSI
builds on more than two decades of psychometric
research in corporate settings undertaken by
the late Dr. Max Garnkle, which has been
expanded on since Dr. Garnkles passing by
Dr. Antonio Ciampi, Associate Professor in the
Department of Epidemiology and Biostatistics
at McGill University. In propounding his vision
for creating higher-learning agile organizations
(HALO), Dr. Garnkle posited a common set of
indicators that could cut through the complexity
of a company and distill its essential elements
into questions to which employees could easily
respond. This inspiration for understanding the
global through the very specic lives on in the
iPSI.
The iPSIs research pedigree explains both its
stability and its propensity to yield reproducible
results. As mentioned above, iPerceptions has
proven that, contrary to certain currents of belief,
the 11 point scale is indeed the most viable
approach by which to conduct online market
research. To conrm its iPSI data, iPerceptions
also asks an overall site satisfaction question
that uses a 6 point ordinal qualitative scale.
Changes in respondents scoring patterns on
this six-point qualitative scale, which functions
as the dependent satisfaction variable, have
been shown to be strongly predictive of
changes in scoring on the 11 point, numerical
scale. Furthermore, regression analysis has
demonstrated that the Attributes used to predict
overall site satisfaction have an average R
squared value of greater than 0.7, with 1st
principal components explaining 82.44% of the
variation. Finally, the iPSI metric is robust enough
that, with a sample size of only 100 respondents,
deltas of + or 0.1 points are deemed to be
statistically signicant at 90% condence.
6
Know Satisfaction
TM
To date, two distinct modes of measuring
customer satisfaction have been used by
researchers to understand website satisfaction
patterns. The rst of these is the quantitative
mode, which comprises such metrics as: page
views, time on site, net promoter scores, as well
as aggregated satisfaction metrics like the iPSI.
The second is the qualitative mode, where the
output is not delimited to preset response tags
and the voice of the customer is allowed to ow
through more freely.
These two modes, or analytical approaches,
have traditionally been segregated. Increasingly,
however, the call has come from many quarters
for a 360-degree view of the online customer,
one that fuses the quantitative and qualitative
streams of insight and offers a holistic vista into
the opportunities and challenges endemic to
the online space.
At the forefront of this initiative was the
development project that made the iPSI the rst
index to speak for the voice of the customer in
both of its modalities. This initiative began with a
groundbreaking approach to the grouping and
categorizing of positive and negative verbatim
respondent commentary. Rather than simply
assigning open-ended comments to catch all
buckets on the basis of straightforward coding
practices, a sophisticated linguistic algorithm
mines both open-ended streams to unearth
concordances between concepts (denotative
of specic phenomena onsite) and concept
modiers (for example, whether a visitor liked
or disliked a particular site Attribute). Each
concordance line from the Top 20 concepts
in each feed (positive and negative) is then
assigned to a relevant Attribute on the basis of
tone, diction, and semantic analysis.
This way, analysts can compute the share of
concepts assigned to specic Attributes within
both feeds. Taking this one step further, the ratio
between an Attributes share in the positive
feed and its share in the negative feed can be
examined to answer two important questions.
1 Which Attributes are the loudest? i.e. which
Attributes of the web experience engender
the most sizable amount of visitor chatter?
2 What is the tone of the discourse surrounding
a particular Attribute? i.e. Are visitors speaking
about a particular element of the website in a
demonstrably positive or negative fashion?
An analogical comparison with a sports arena
can be drawn. The noise of the crowd will vary
Bridging the Gap
Between Quantitative
and Qualitative Feedback
7
Know Satisfaction
TM
in accordance with the action on the eld,
with important or game-altering occurrences
generating a louder crowd response. The tone
of the crowd, however, will vary based on which
team is performing well. There could be loud
cheering or loud booing; alternatively, there
could be subdued cheering and more scattered
expressions of disgruntlement.
But the key point is that the full spectrum of
open-ended feedback now has a quantitative
repository in which to rest. The nature of the
website visit will most certainly change as the
web evolves and interactive technologies, such
as Ajax, redene what can be done on a single
page. Increasingly, insights will be used-led and
integrating quantitative and qualitative feedback
will be a core requirement of any attitudinal
analytics package.
It is a truth almost universally acknowledged
that the web is the most measurable of media.
As any meta-analysis of vendors will reveal,
iPerceptions competes in a space alongside a
plethora of other SaaS providers, and the iPSI
is thus but one element in a basket of web
metrics of varying utility. In addition to a unique
perceptual framework and a groundbreaking
approach to qualitative insight, two other
factors make the iPSI unique and without peer.
First, it is the only metric designed exclusively to
evaluate the experience of the online customer
in the context of an actual site visit. Second,
unlike other satisfaction metrics that simply
borrow from ofine paradigms, the iPSIs
gestation, birth, and growth have all occurred
online. Thus, the iPSI is not plagued by the
ofine-to-online portability defects that mitigate
the usefulness of other customer experience
metrics.
For much of its history, web analytics has
been concerned with measuring behavioral
phenomena. Unique visitor counts, pages
viewed/visit, time on site, bounce rates, click-
through rates, ad impressions, and, more
recently, rich internet application events have
Born and Bred on
the Web
8
Know Satisfaction
TM
dominated the conversation about the online
customer. In an effort to broaden the scope
of behavioral measurement, more subtle
and nuanced approaches have emerged,
attempting to subsume these indicators into
more sophisticated models that measure all
aspects of visitor engagement.
But studying clicks alone can never tell more
than a part of the visitors story and will never
bring to light true engagement data. Behavioral
analytics is akin to paleontology, in the sense
that it presents a fossilized record of a visitors
site experience, at least as much as can be
captured through page tags with optimal cookie
deletion rates. However much scientists study
the material implements of the past, however,
they will never truly be able to gauge whether
people were happy and led meaningful lives.
This is the primary shortcoming of behavioral
research; the tracks a visitor leaves are
necessarily incomplete without knowing intent,
attitude, and satisfaction. Further, engagement
indices will always be hollow metrics unless
they can be made to account for loyalty, trust,
and advocacy.
In a different fashion, audience measurement
vendors will claim to be advocates for the
voice of the customer, afrming that their panel
measurement approaches are representative
of the human aspect behind clickstream data.
They certainly have their value; they are justly
recognized as the leading authorities on
aggregated metrics such as search engine
usage. However, claiming to speak on behalf
of an individual websites customers based on
the results of a haphazardly qualied panel is
problematic. Frequently, the vendors employ
browser toolbars to measure their audience,
which points to possible sources of bias, as not
all web users are willing or savvy enough to
install browser add-ons. Even if a representative
panel can somehow be assembled, these
research methods can only approximate the
voices of real customers.
Contrast this with the webValidator method on
which the iPSI is based: a transparent, point-of-
entry solicitation, that follows the best practices
of random sampling, and encapsulates the
totality of the visitors experience, including
open-ended commentary. The magic of the web
is that, for the rst time, companies can dialogue
directly with their customers. There is no longer
the need for a panel to act as a proxypeople
can speak for themselves.
Further, the iPSI is the only truly web-optimized
attitudinal metric. Survey vendors who employ
the American Customer Satisfaction Index (ACSI)
will argue for its superiority by citing studies that
point to a correlative (but speciously causal)
relationship between a companys ACSI scores
and its scal performance. What they fail to
mention is that no studies have ever found any
9
Know Satisfaction
TM
link between ACSI scores and salutary website
outcomes. The reason is simple: the ACSI was
designed to measure satisfaction in a brick-
and-mortar customer ecosystem. It was rst
published in 1994, when the Internet was in its
infancy and the web was just a dream, a hope
for the future. The ACSI was conceived and
designed to answer the following question: were
end-users satised with the quality of products
and services that companies were putting out?
But the nature of products and services on
the web is only marginally a derivative of the
nature of ofine products and services. In its 2.0
iteration, the web is a vast cornucopia of social
networking, user generated content, blogging,
hyper-contextualized ad serving, and widgets.
The demise of the website itself has been
variously predicted. As never before, the old,
ofine semantic distinctions governing product
and service have largely been rendered
obsolete. Even transactional conversion data
is no longer enough in itself to fully point to
website success, which is why website decision
makers are increasingly looking to more holistic
schemas to gain customer insights.
Happily, the one thing that remains constant
in this time of rapid change is the voice of the
website user, and the iPSI is the only metric
engineered exclusively fully to encapsulate it.
Measuring satisfaction in this climate is less
monolithic than it once was; it is now an organic,
iterative process of dialoguing with the user and
gauging the fulllment of his or her needs and
aspirations.

10
Know Satisfaction
TM
In this current Web 2.0 environment, an
unprecedented amount of attention has been
devoted to understanding the behavior of the
website user. User generated content and social
media have disrupted the traditional patron-
client relationship that previously governed
media consumption and have placed the user
at center stage. In a time of rapid technological
change and innovation, website decision makers
have come to see more clearly than ever that
satisfying visitors is the rst priority for their web
presence. Increasingly, they have come under
pressure to draw upon metrics that can educe
clear and actionable insights from an inherently
noisy medium.
It is no longer simply enough to quantify the
visitor base in terms of clickstream metrics
and assume that you have squeezed all the
insights possible from your users. The voice
of the customer, in the form of representative
survey data, must be on the table from the
outset and it must be monitored continuously.
As iPerceptions client list illustrates, industry
leaders know that satisfaction data from actual
website visitors is an invaluable component of
any online business intelligence strategy.
At the forefront of iPerceptions ongoing
initiative to democratize the voice of the online
customer is the iPerceptions Satisfaction
Index. Building on a robust research pedigree
and leveraging a best-of-breed approach
to quantifying qualitative data, the iPSI is the
rst satisfaction metric designed exclusively
for the web. With more and more progressive
companies making the iPSI a central part of
their reporting culture, a common language
for knowing visitor satisfaction is emerging in
the online marketplace. Customer-centricity,
therefore, has a healthy future indeed.

Conclusion
11
Know Satisfaction
TM
For more information please contact a
Voice of Customer expert at
info@iperceptions.com
Canada
4999 St-Catherine St. West, Suite 500
Montreal, Quebec
H3Z 1T3
Canada
Tel: (514) 488.3600
Fax: (514) 484.2600
USA
575 Madison Avenue, Suite 1006
New York, NY
10022-2511
USA
Tel: 877.796.3600
Fax: 866.484.2600
www.iperceptions.com
Know Satisfaction
TM