Development of a framework for UX KPIs in Industry - a case
study
ABSTRACT
Tina Øvad∗ Kashmiri Stec
Preely, Copenhagen, Denmark Bang and Olufsen, Struer, Denmark
tina@preely.com ksh@bang-olufsen.dk
Lucca Julie Nellemann†
Demant, Copenhagen, Denmark
LuccaJNellemann@outlook.dk
ABSTRACT
This work addresses the needs companies face when assessing and
tracking the UX quality of their products, which is necessary to en-
sure that the desired level of quality is met, clarify potential areas of
improvement, and compare competitor products via benchmarking.
We present a framework for UX KPI assessment at Bang & Olufsen,
a luxury audio product manufacturer. The framework is inspired
by well-known UX scales such as the UEQ and AttrakDif as well as
more business-oriented measures such as NPS and CES scores. The
resulting UX KPI framework comprises the test procedures and a
scale containing these and ten unique features such as "Enjoyable”,
“Exciting”, “Sound experience". The paper presents and discusses
the UX KPI framework and the rationales behind it. Results of more
than 200 user tests are presented and discussed. Factor analysis has
been employed to analyse the assumptions behind the UX KPI and
identify where improvements can be made.
CCS CONCEPTS
• Human-centered computing; • Human computer interac-
tion (HCI); • HCI design and evaluation methods; • Usability
testing;
KEYWORDS
UX, User experience evaluation framework, KPI, Key Performance
Indicator, UX benchmarking for industry
ACM Reference Format:
Tina Øvad, Kashmiri Stec, Lars Bo Larsen, Lucca Julie Nellemann, and Jędrzej
Czapla. 2020. Development of a framework for UX KPIs in Industry - a case
∗ At
Bang and Olufsen at the time the work was carried out
† At Bang and Olufsen at the time the work was carried out
‡ At Bang and Olufsen at the time the work was carried out
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for proft or commercial advantage and that copies bear this notice and the full citation
on the frst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specifc permission and/or a
fee. Request permissions from permissions@acm.org.
OzCHI ’20, December 02–04, 2020, Sydney, NSW, Australia
© 2020 Association for Computing Machinery.
ACM ISBN 978-1-4503-8975-4/20/12. . . $15.00
https://doi.org/10.1145/3441000.3441042
141
Lars Bo Larsen
Aalborg University, Aalborg,
Denmark and Bang & Olufsen, Struer,
Denmark
lbl@es.aau.dk
Jędrzej Czapla‡
Aalborg University, Denmark
czaplapapla@gmail.com
study: ABSTRACT. In 32nd Australian Conference on Human-Computer
Interaction (OzCHI ’20), December 02–04, 2020, Sydney, NSW, Australia. ACM,
New York, NY, USA, 7 pages. https://doi.org/10.1145/3441000.3441042
1 INTRODUCTION
Every company has a strong need to assess and track the quality
of their products. This is necessary to ensure that the desired level
of quality is met, to clarify potential areas of improvement, as well
as to compare products via competitor benchmarking. Quality is
a multi-dimensional concept: It can refer to materials, tolerances,
durability, aesthetics, taste, usability, sustainability, etc. Each di-
mension requires diferent measures and standards to be assessed.
Product quality may in some cases be subjective and qualitative, and
thus difcult to describe and measure both generally and quantita-
tively. Furthermore, it may consist of many individually dependent
components. A way to make the quality concept more manageable
and operational in day-to-day work is to break it down into sets
of individual, one-dimensional entities. These can in turn be de-
scribed and evaluated, and later combined into an overall estimate
of product quality. In the present study we address the problem of
assessing the quality of user experience (UX) from an industrial
point of view.
We have exemplifed this via a case study performed at Bang &
Olufsen (B&O) [1] were we developed our framework for assessing
the quality of the UX for B&O’s [1] audio products, such as head-
phones, earbuds and loudspeakers. The framework also supports
benchmarking the UX of competitors’ products, enabling us to put
tangible numbers on UX for comparisons. With this framework,
we can quantitatively measure on-product and in-app UX and map
strong and weak points of product experiences while also getting
qualitative information on how products can be improved. This
gives us information of where to focus our UX resources to bring
most value to the product. By benchmarking competitor products,
we have obtained a competitor overview including “hard numbers”,
which makes it easy for us to know how the products and concepts
perform in the competitor UX landscape.
During the development of the framework, we also sat out to
strengthen stakeholder communication regarding UX. We found it
desirable to be able to share information about UX with access to
diferent levels of granularity in UX, depending on project needs.
OzCHI ’20, December 02–04, 2020, Sydney, NSW, Australia
Our goal was that the framework should be truly transparent. We
aim to produce one overall tangible number for the UX of a product,
which is the UX KPI (Key Performance Indicator), comprised of
a multidimensional set of underlying features with more detailed
information.
The KPI framework is developed within a company producing audio
products. However, the applied principles and methods are context
independent and the framework can easily be adapted to other
product domains, as only a few items in the framework refect the
specifc product categories of the company. These can be replaced
or extended to refect other application domains. Thus, we believe
our work can be extended to any physical or digital product or a
combination thereof. We have especially felt a high interest from
companies working with physical experiences, where this kind of
tool seems to have been missing.
1.1 The Foundation
Leading up to this work, B&O had a testing scheme in place. This
focused on “Industry standards”, which only concern the technical
protocols a product must adhere to; utility aspects; access to help
when needed; and which feedback channels are used. The industry
standards form a product category dependent checklist, which
is run through by the test facilitator and not presented to test
participants or otherwise part of a test. This will not be of further
concern to this study. Consistent user test protocols were not in
place and exit interviews depended highly on the concept or product
being tested. The aim of the UX KPI is to make this streamlined
and systematic, making it easier to track improvements and make
comparisons between products.
2 THE COMPONENTS OF THE UX KPI &
BENCHMARKING FRAMEWORK
2.1 Concept and Product Experience
Components
During our initial discussions about our (then future) UX KPI and
benchmarking framework, we defned that a B&O concept and
product experience consists of four components: Industry Standard
Experience, Diferentiating (Hero) Experience, Comfortableness, and
Polish.
These components refect B&O’s brand and company values and
were therefore a subjective choice which is not based on empirical
data. Diferentiating (Hero) Experiences refers to features that make
a product unique. Comfortableness refers to how comfortable the
user is while using and/or wearing the product, and Polish refers to
a design that delights the sense and whether the product lives up to
expectations of the brand. In order to measure this, we needed to
be able to measure interaction with the product, and we therefore
include user testing as a basis for the UX KPI. Although these
components are particular to B&O, they can easily be adapted to ft
the needs of other companies.
2.1.1 User Tests & Post Task Qestions. Each user test comprises
a number of diferent tasks to be carried out by the participant.
These may vary slightly, depending on the nature and capabilities
of the product, but in most cases include all on-device interaction
142
Tina Øvad et al.
as well as the in-app interactions which are shared across prod-
uct apps. More specifcally: Connecting to a mobile phone using
Bluetooth or WIFI; Using a companion app to setup the product
(may include Google Voice Assistant [2] or Alexa Voice Assistant
[3], if supported); and basic player controls such as next, previous,
play, pause, volume up or down. These functions are carried out on
the product and via B&O’s smartphone apps [4, 5] where relevant.
For products supporting phone calls, answering and making calls
are also included. The user is asked to “think aloud” during these
tasks and is closely observed by the facilitator. The facilitator takes
note of any relevant observations and flls out a template with in-
teraction measures using a ten-point scale. The template contains
scores for the tasks mentioned above. The mean is computed and
included in UX KPI as the feature “Interaction”, see Figure 4 below.
The individual interaction measures are not reported directly as
part of the UX KPI but are used to inform the development and
product team of potential problems. For diferent product categories
these tasks and interaction measures may be changed to refect the
relevant aspects of the product being tested. However, they will al-
ways represent the usability of the products. For each user task the
participant is asked three post task questions: “How easy was the
task?”; “How confdent did you feel doing the task?”; and “How was
your experience?”. The user answers each question on a ten-point
scale, with 10 indicating a very good experience and 1 indicating
a very poor experience. Further, the facilitator notes down any
relevant observations the user makes while trying to complete a
task, and qualitative information about the issues encountered.
2.1.2 Post Test Qestions. After the user test, it is important to
capture the participant’s experiences and attitudes about the tested
product. We approached this in two ways:
We employ an exit test questionnaire to capture the user’s expe-
rience. For this we developed the B&O UX KPI scale, inspired by
three well-known and widely used scales in academia and indus-
try: The UEQ- [6–8], the SUS- (system Usability Scale) [9] and the
Attrakdif [10, 11] scales.
We frst considered the System Usability Scale (SUS). However,
it does not capture the UX attributes which were important to our
team, such as attractiveness. We therefore turned to the User Expe-
rience Questionnaire (UEQ) for inspiration. UEQ is a widely used
and reliable questionnaire for measuring UX of interactive products.
The UEQ captures the diferent areas of the user experience we de-
sire [8]. Attractiveness is measured as an overall value of the tested
item, containing pragmatic qualities and hedonic qualities. The
pragmatic qualities are perspicuity, efciency, and dependability.
The hedonic qualities are stimulation and novelty. UEQ, measures
attractiveness with six questions and the other fve qualities are
measured with four question each, thus 26 questions (items) in
total.
Some of the questions, such as how easy it is to navigate the
product or the users’ confdence using a product, are parts of the
interaction table and are therefore omitted. Due to the audio do-
main we include the item “Sound Experience”. Other items inspired
from the three scales were divided under the three previously men-
tioned components or subscales; Diferentiating (Hero) Experience,
Comfortableness, and Polish. Figure 1 below shows the aspects
identifed for each of the three components.
Development of a framework for UX KPIs in Industry - a case study
Figure 1: UX KPI categories and items
The choice of these specifc aspects was made based on internal
discussions in the UX team and other stakeholders at B&O and re-
fects the issues considered most relevant to the company’s product
portfolio and brand identity. The aspects were represented as items
on a semantic diferential scale with ten points, see Figure 2 below.
2.1.3 Market Measures. Some of the aspects we want to include in
the UX KPI are not covered by the UX scales mentioned above. As
we are employing the KPI in a business environment, we include the
following scales in the UX KPI framework: CES (Customer Efort
Score), asking: “How easy was it to do the task? [18] and CSAT
(Customer satisfaction), asking: “How would you rate your overall
satisfaction of doing the tasks?” [13, 19]. The CES and CSAT scores
corresponds closely to “ease of use” and user satisfaction”, which
are also part of the ISO 9241 defnition of product usability [12].
Finally, the NPS (Net promoter score), asking: “How likely are you
to recommend the product to a friend or colleague?” is included, as
this has been found to be an important predictor for in particular
net commerce [14]. A discussion of these metrics can be found in
[20].
A further argument for these tools is that CES, CSAT and NPS
questions were already used by B&O’s customer experience team
to measure customer satisfaction after-market release. Thus, the
inclusion makes it possible to bridge the gap from pre- to post
launch of a product and has the added beneft of already being
known through-out the organisation. Initially, we asked the NPS,
CSAT and CES questions is the traditional way, but we found it
was confusing for users to switch between 10 and 7 pt. scales, so
we converted everything to ten-point scales to make it easier for
users, as well as to make the responses easier to compare and
interpret. However, this comes with the cost of less compatibility
to the original scales.
3 PUTTING THE UX KPI & BENCHMARKING
SCALE TOGETHER
The UX and market measures are integrated into a single 10-point
semantic diferentials questionnaire. The anchor points were chosen
to be identical to the aspects from Table 1 preceded by “Not” and
“Very”. This was done because the terms were considered clear
and understandable and to avoid an extra step formulating and
validating keywords and phrases for the anchors.
This completes the scale. Data is captured simply by converting
the participant’s selection to a corresponding number between 1-10,
from left to right. The following section describes how the UX KPI
scale is used in connection with user testing.
143
OzCHI ’20, December 02–04, 2020, Sydney, NSW, Australia
3.1 Aggregating the scores
The scores are aggregated and inserted in a colour-coded template to
gain a quick overview and facilitate interpretation of the results. The
aggregation is done by calculating the average of the individual item
scores across the test participants for a given product test. These are
in turn combined within the three UX components (Hero, Polish and
Comfortableness) by calculating the mean for each. Finally, a colour
coding scheme is applied to make potential problems and areas for
improvement clearer. The colour scheme and interpretation are
shown in Figure 3, and is inspired by the NPS [14].
3.2 Using the KPI & benchmarking framework
The framework is used in connection with a user test, as mentioned
above, and the UX KPI is calculated and the scheme flled out after
the completion of a series of tests of a product. Typically, we con-
duct 5 user test sessions on naive users per test following [16, 17].
In addition to the UX KPI, a test report is made including the fa-
cilitator’s observations and the test users’ comments. A post-test
interview is also carried out and included in the test report, but
that is not the focus in the present study.
4 RESULTS
By April 2020 the UX KPI & benchmarking framework has been
used in 228 tests of 36 products. Typically, fve tests are carried
out for a product, which we assume on average will uncover about
80% of the usability problems for a product (although with a large
margin), as demonstrated in [16, 17]. However, some are tested
by up to 15 participants. The products fall into several categories,
which are all sound reproduction equipment: fexible (movable, but
still with a power cord and Wi-Fi; 10 products); mobile (rechargeable
Bluetooth loudspeakers; 10 products), and wearable (headphones
and earbuds; 16 products). Test participants were recruited outside
of the company, except in some cases, such as unreleased products,
where they may be internal. The age and gender distributions are
(Avg. = 28.6 St. Dev = 10.8) and (Male = 61%, Female = 39%); all
were living in Denmark at the time they participated in user testing.
Figure 4 shows an example of a UX KPI scoring template, where
the measures described in Figure 2 and the colour coding scheme
shown in Figure 3 are used. The template comprises a compact,
easy to overview presentation of the UX KPIs, which can be used
for quick impressions, benchmarking, comparisons, tracking of im-
provements, discussions, etc. among the UX team and stakeholders
such as product managers, concept managers and technical teams. It
is accompanied by a more in-depth test report as mentioned above.
The testing has been taking place continuously since the second
half of 2018 and certain adjustments have been made over time.
For example, an item “Meets expectations” was originally included
in the “Polish” category, but was removed as it is a relative term,
depending on the user’s (unknown) prior expectations to, e.g., the
brand, and are thus better captured in a post-test interview. The
item “comfortable wearing” is only relevant for wearable products,
such as headphones and earbuds.
The boxplot in Figure 5 shows that the items are scored quite
uniformly, and the ten-point scale is not fully utilized. Indeed, except
for “Creative” and “Materials”, the lower and upper quartiles for all
items lie between 6 and 9 and the median is 8 for 7 of the 10 items.
OzCHI ’20, December 02–04, 2020, Sydney, NSW, Australia
Figure 2: UX-KPI Semantic diferentials scale including CES, CSAT and NPS
Figure 3: Interpretation of the aggregated KPI scores. Inspired by NPS [14, 15]
4.1 Factor Analysis
It is relevant to investigate whether the assumed grouping of the
items into the Hero, Polish and Comfort themes can be identifed in
the data. Factor Analysis (FA) is used for this. Initially, a Confrma-
tory FA (CFA) is performed across all tests, using the Hero, Polish
and Comfort model. However, this model cannot be confrmed. The
recommended number of data points for a CFA is 300 [21], so the
failure may be due to either lack of data, that some parameters
are highly correlated, or simply that the model does not match
the data. To further investigate the relationships, an Exploratory
FA (EFA) is carried out using R [22]. A Scree plot indicates that 3
factors will capture about 70% of the shared variance and that a 4
or 5 factor model will not signifcantly improve this. The 3 and 4
factor models yield signifcant CHI2 values. Based on this a 3-factor
model is chosen. [21] recommends that an oblique rotation should
be used if factor correlations exceed 0.3. In the current case this is
the situation, so the Oblimin rotation is chosen. [21, 23].
144
Tina Øvad et al.
Figure 6 shows the resulting EFA 3-factor model with Oblimin
rotation when applied on all data. For clarity, a cut-of has been
applied for loadings below 0.3. The resulting model accounts for
71% of the common variance in the data, which is acceptable. Three
items are cross-loading (Creative, Convenient and Enjoyable).
F1 accounts for 51% of the common variance and includes Stylish,
Presentable, Inviting, Materials as well as Exciting and Creative
(shared with F3). A common heading for F1 could be “Attractive-
ness”. F2 accounts for 21% of the common variance and includes
Comfortable Control and Enjoyable (shared with F3) and could be
labelled “Control”. F3 accounts for 27% of the common variance
and includes Sound experience and Convenience, i.e. issues related
to “Performance”.
4.2 Correlations
The question about “comfortable with the amount of control” can
be expected to show some correlation with the “Interaction” feature,
Development of a framework for UX KPIs in Industry - a case study OzCHI ’20, December 02–04, 2020, Sydney, NSW, Australia

Figure 4: Example of a UX KPI scoring template for random product

Figure 5: Boxplot showing the KPI items

as they address similar control issues. A test shows that the corre-
lation is signifcant (0.58 at p<0.00001 with 95% confdence interval
at [0.48..0.66]). This is an indication of the two concepts overlap
and is indeed what could be expected. However, it also means
we should be careful when including both features in the UX KPI
calculation.
5 DISCUSSION
While the UX KPI has not been formally validated, it is built on
standardised, widely used scales and measures. Therefore, a cer-
tain level of construct validity can be assumed. Furthermore, the
application of the UX KPIs over time – on both products under
development and market releases by competitors – allows compar-
isons across numerous products, thus facilitating the application
of the UX KPIs as a relative measure instead of an absolute one.
For ease of comparison, all numerical scales (except the NPS) have
been set to ten points. This has the advantage making internal
comparisons between items and the Hero, Polish and Comfortable-
ness sub-scales very easy and removing any need for re-scaling.
However, it also makes external comparisons to the original scales
harder and may tempt unwary readers of the KPI test result reports
to make too direct 1:1 comparison between e.g. sound quality and

145
OzCHI ’20, December 02–04, 2020, Sydney, NSW, Australia
Figure 6: 3-Factor EFA with Oblimin Rotation for all data.
CES, just because the scales “look similar”. Indeed, the computation
of the “Overall KPI” as an average of the individual items is an
example of this and should be used with some caution.
The boxplots shown in Figure 5 above indicate, that the 10-point
scale seems to be only partly utilised, with 7 of 10 items having
the same median. This is a common issue with numerical scales,
where the granularity becomes less than intended and responses
tend to cluster towards one end of the scale. In the present case
we chose a 10-point scale, which produces a better resolution than
e.g. a 5- or 7-point scale. However, the issue may be due to the fact
that all tested products are high end audio products, and thus we
could expect the ratings to cluster towards the upper end of the
scale. Increasing the granularity to e.g. 20 points (or using a VAS
scale [24]) will most likely just only result in extra noise, unless
we stick to recruiting expert test participants, or change the test
scheme to direct A/B comparisons. A regular untrained person will
not be able to distinguish between 20 levels.
The EFA analysis indicates no statistical evidence for the Pol-
ish, Hero and Comfort categories we introduced at the starting
point of are framework. Instead the items groups themselves into
categories pertaining to Attractiveness, Control and Performance,
with Attractiveness representing the majority of the items. From a
purely statistical viewpoint the identifed categories should thus
replace the original ones. However, the categories must also be
viewed in the context of the organization using them. Thus, if it
e.g. makes more sense to B&O’s design goals and visions to retain
the Polish, Hero and Comfort categories, one should be careful
replacing them. The large body of tests already done using the
original categories must also be taken into consideration. If they
are replaced, backward combability is lost and all stakeholders must
learn and understand the new ones.
Currently, the overall UX KPI quotient is computed as an un-
weighted average of the categories, thus assuming an equal impor-
tance of all items. If an overall KPI quotient is still to be calculated
146
Tina Øvad et al.
in an updated model, it could be considered to introduce weights,
e.g. based on the proportional loadings of the factors as weights.
The foundations, methods and analyses applied in the development
of the KPI framework are not dependent of the particular domain
or organisation, where it is applied, but are universally accepted
business and UX measures and statistical methods. Thus, the frame-
work can easily be modifed or extended to domains beyond the
audio context of B&O’s products. Indeed, only the item “Sound
Experience” is directly related to audio products while “Materials”
and “Comfortable Wearing” are only relevant for physical products.
These can be omitted or replaced and new can be added, depending
on the particular domain in which the framework is to be applied.
Likewise, the three categories or subscales can be updated to refect
the needs of the product being tested, or of the larger organizational
context.
6 CONCLUSION
Through inspiration from well-known and widely recognized
scales from the UX and business domains, we have derived a UX
KPI framework tailored for industrial use within a specifc do-
main. The purpose has been to provide a consistent and easy-to-
interpret tool for the UX team and stakeholders within the com-
pany to assess and track the UX of products as well as competitor
products.
The framework has successfully been employed at B&O for close
to three years. In this period numerous user tests have been car-
ried out on a large variety of products by diferent test facilitators
and environments. By using the framework, the test results are all
highly comparable and are presented and interpreted in a similar
manner. Thus, it has provided the UX performance tracking and
benchmarking capabilities we set out to achieve.
The framework has not been validated in a formal manner and
the statistical analyses indicate that a more accurate interpretation
may be achieved by adjusting e.g. the underlying factors and the
weights (when calculating the overall KPI). However, when consid-
ering the intended usage, it can be argued that this is less important
than it would be in a purely academic setting. Internal consistency,
stakeholder support and acceptance of results are important issues
in a company setting, all of which should be carefully considered
when making decisions about changes.
We have demonstrated how the KPI framework can be modifed
or extended to suit other products than the present case of audio
equipment.
6.1 Future work
The next steps will be to consider the results and analyses pre-
sented here and update to the framework, where appropriate. This
could e.g. to replace the Hero/Polish/Comfortableness subscales
or to introduce a weighting of the items in the UX KPI calcula-
tion. As mentioned above, it is also important to retain backwards
compatibility for future comparisons, so it is not only a decision
depending on the analyses, but also on usage within the company -
to ensure the fndings remain relevant and valuable to cross-org
stakeholders.
Another consideration could be to introduce UX KPIs as design
requirements in the future.
Development of a framework for UX KPIs in Industry - a case study
ACKNOWLEDGMENTS
We thank Bang and Olufsen and Aalborg University for their sup-
port of this work and most importantly all our test participants for
their time.
REFERENCES
[1] Bang and Olufsen: https://www.bang-olufsen.com (retrieved August 2020)
[2] Google Voice Assistant https://assistant.google.com/ (retrieved August 2020)
[3] Alexa Voice Assistant https://developer.amazon.com/en-US/alexa/alexa-voice-
service (retrieved August 2020)
[4] B&O App. Apple IOS version: https://apps.apple.com/us/app/bang-olufsen/
id1203736217 (retrieved August 2020)
[5] B&O App. Android version: https://play.google.com/store/apps/details?id=com.
bang_olufsen.OneApp&hl=en (retrieved August 2020)
[6] UEQ: Website: https://www.ueq-online.org/ (retrieved August 2020)
[7] Laugwitz B., Held T., Schrepp M. (2008) Construction and Evaluation of a User
Experience Questionnaire. In: Holzinger A. (eds) HCI and Usability for Education
and Work. USAB 2008. Lecture Notes in Computer Science, vol 5298. Springer,
Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-89350-9_6
[8] Schrepp, M., Hinderks, A., and Thomaschewski, J., 2017. Construction of a Bench-
mark for the User Experience Questionnaire (UEQ). International Journal of
Interactive Multimedia and Artifcial Intelligence, 4(4), pp. 40–44. ISSN 1989-1660.
doi: 10.9781/ijimai.2017.445.
[9] SUS: Brooke, J., 1996. SUS - A quick and dirty usability scale. Usability evaluation
in industry, 09, 189(194), pp. 4–7.
[10] Attrakdif: Website: http://www.attrakdif.de/index-en.html (retrieved August
2020)
[11] Hassenzahl, M., Burmester, M., and Koller, F., 2008. On the track of user experi-
ence. Available at: <https://www.uid.com/en/publications/attrakdif>. (retrieved
August 2020)
[12] ISO 9241-11:2018. Ergonomics of human-system interaction — Part 11: Usability:
Defnitions and concepts. Available at: https://www.iso.org/standard/63500.html
(retrieved August 2020)
[13] Reichheld, Frederick F. "One Number You Need to Grow". Harvard Business
Review December 2003. Available at: https://hbr.org/2003/12/the-one-number-
147
OzCHI ’20, December 02–04, 2020, Sydney, NSW, Australia
you-need-to-grow (retrieved August 2020).
[14] Reichheld, Fred; Markey, Rob (2011). The Ultimate Question 2.0: How Net Pro-
moter Companies Thrive in a Customer-Driven World. Boston, Mass.: Har-
vard Business Review Press. p. 52- ISBN 978-1-4221-7335-0. Available at: https:
//archive.org/details/ultimatequestion00reic_0/page/52 (retrieved August 2020)
[15] Net Promoter Score: https://www.netpromoter.com (retrieved August 2020)
[16] Faulkner, L. Beyond the fve-user assumption: Benefts of increased sample sizes
in usability testing. Behavior Research Methods, Instruments, & Computers 35,
379–383 (2003). https://doi.org/10.3758/BF03195514
[17] Nielsen, J., Landauer, K. ”A mathematical model of the fnding of usability
problems” in CHI ’93: Proceedings of the INTERACT ’93 and CHI ’93 Con-
ference on Human Factors in Computing Systems. May 1993 Pages 206–213. DOI:
https://doi.org/10.1145/169059.169166
[18] Matthew Dixon, Karen Freeman and Nicholas Toman. Stop Trying to Delight
Your Customers, Harvard Business Review Press. July–August 2010. Available
from: https://hbr.org/2010/07/stop-trying-to-delight-your-customers (retrieved
August 2020).
[19] Farris, Paul W.; Neil T. Bendle; Phillip E. Pfeifer; David J. Reibstein (2010). Market-
ing Metrics: The Defnitive Guide to Measuring Marketing Performance. Upper
Saddle River, New Jersey: Pearson Education, Inc. ISBN 0-13-705829-2.
[20] Mittal, Vikas and Frennea, Carly, Customer Satisfaction: A Strategic Review and
Guidelines for Managers (2010). MSI Fast Forward Series, Marketing Science
Institute, Cambridge, MA, 2010, Available at SSRN: https://ssrn.com/abstract=
2345469 (retrieved August 2020)
[21] Tabachick, Barbara and Fidell Using Multivariate Statistics. 4 th editon. Allyn
and Bacon, 2001. ISBN 0-321-05677-9
[22] R Core Team (2014). R: A language and environment for statistical computing.
R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-
project.org/ (retrieved August 2020)
[23] James Dean Brown: “Choosing the Right Type of Rotation in PCA and EFA”, in
Shiken: JALT Testing & Evaluation SIG Newsletter. 13 (3) November 2009 (p. 20 -
25)
[24] Grant, S.; Aitchison, T.; Henderson, E.; Christie, J.; Zare, S.; McMurray, J.; Dargie,
H. (1999). "A comparison of the reproducibility and the sensitivity to change of
visual analogue scales, Borg scales, and Likert scales in normal subjects during
submaximal exercise". Chest. 116 (5): 1208–17. DOI: https://doi.org/10.1378/chest.
116.5.1208