Proc. of 2020 7th Int. Conf. on Information Tech.
, Computer, and Electrical Engineering (ICITACEE)
Functional Suitability Measurement Based on
ISO/IEC 25010 for e-Commerce Website
Novrindah Alvi Hasanah
Department of Informatics Department of Informatics
Institut Teknologi Sepuluh Nopember Institut Teknologi Sepuluh Nopember
Surabaya, Indonesia
alvinovrindah@gmail.com
Abstract— The intense competition between e-Commerce
makes every e-Commerce has a way to be able to compete in
the market, one of which is to improve the quality of e-
Commerce websites. Therefore, research is needed to find out
the quality of the web with the buyer's intention on the E-
commerce web by evaluating it. There are various standard
software evaluation models used such as ISO / IEC 25010. This
paper, the ISO/IEC 25010 to assess the functional suitability of
various Indonesian e-Commerce websites, namely Wadezig!,
Screamous, 347 Clothing, Thanksinsomnia and Bloods, which
were determined through Bukalapak customer reviews.
Functional suitability has three calculated attributes, namely
completeness, correctness, and appropriateness. From the
measurements that have been made, the functional suitability
can help e-Commerce websites to improve the quality of their
website, especially in terms of functional suitability so as to
provide comfort and good service to customers. This evaluation
can also be used as a reference for developing e-Commerce
websites to be better.
Keywords—e-Commerce website, ISO / IEC 25010, functional
suitability, Completeness, Correctness, Appropriateness
I. INTRODUCTION
The e-Commerce industry has developed rapidly in this
decade; various convenience services are also offered by
various e-Commerce sites in Indonesia. Visitor satisfaction
is one of the priorities in developing e-Commerce services.
User satisfaction in e-commerce requires better innovation
on e-Commerce Website [1]. Stefani [2] believes that e-
Commerce websites can be evaluated through users using a
software evaluation model. Various standard software
evaluation models in question include McCall, FURPS,
Dormey, Boehm, ISO / IEC [3].
The quality of software products can be calculated using
ISO / IEC 25010 standard, [4] defines a software product
quality model along with a system quality in use
model. The ISO/IEC 25010 software product quality model
is composed of eight characteristics viz, functional
suitability, performance efficiency, compability, usability,
reliability, security, maintainability and portability [5] that
can be measured internally or externally.
Software quality can be judged by the better user
interface (UI) and functional devices[6]. Various studies
have been conducted to evaluate the quality of software
such as Purwati [7] evaluate the user interface (UI) on the e-
Commerce web. In addition, research is also carried out on
functional suitability Puspitanigrum et al [8] undertook to
examine the quality of the academic information system
(SIA) of tertiary institutions on functional suitability
characteristics using ISO 25010 especially Goal Quality
Metric (GQM). Wattiheluw [9] assessed functional
Authorized978-1-7281-7226-2/20/$31.00 ©2020
licensed use limited to: Instituto Tecnologico
IEEERica (ITCR). Downloaded
de Costa
Luthfi Atikah Siti Rochimah
Department of Informatics
Institut Teknologi Sepuluh Nopember
Surabaya, Indonesia Surabaya, Indonesia
luthfiatikah@gmail.com siti@if.its.ac.id
suitability, performance efficiency, reliability and usability
on the e-Commerce web by applying the Fuzzy Mamdani
method and the Analytical Hierarchy Process method to
weight the importance of the characteristics.
In this paper, we proposed ISO/IEC 25010 to assess the
functional suitability of various Indonesian e-Commerce
websites. This research was conducted on six local web
fashion brands in Indonesia that we chose from the
bukareview which is a collection of bukalapak customer
reviews. The brands are Wadezig!, Screamous, 347
Clothing, Thanksinsomnia and Bloods. We selected these
six brands based on the level of similarity the menu has. The
study was conducted for 1.5 months starting from a
literature study, determining case studies, selecting methods
to conducting a survey of each selected case study. The
results of the study are evaluations and recommendations
that can be used as a reference for developing e-Commerce
websites to be better.
II. LITERATURE REVIEW
Tanjung et al. [1] conducted research on several e-
Commerce websites that have developed rapidly in Indonesia
using a multidimensional approach to evaluating web sites
from e-Commerce. The results of the study are used to
identify areas of website innovation that are used to increase
user satisfaction and user feedback and create competitive
value from e-Commerce companies.
Purwati [7] conducted a study to evaluate the user
interface (UI) on the e-Commerce web. UI elements studied
are site navigation, home links, search, view carts, get help,
manage accounts, catalogues, and personalization functions.
The analysis was carried out on 120 web sites from the US,
China, Germany and Indonesia. The results of the study
indicated that there is no web standard that meets the de facto
standard of Nielsen. Hypothesis results obtained from the
research show that there are different standards for features
and designs used in four countries.
Puspitanigrum et al.[8] assessed the quality of academic
information systems (SIA) from tertiary institutions on the
characteristics of functional conformity using ISO 25010
especially Goal Quality Metric (GQM). The results of the
study are in the form of a framework based on institutional
goals that can be used to evaluate and develop SIA.
Stefani [10] conducted a study on the eco-system of
software quality based on the ISO 25010 standard. These
metrics are mapped based on the characteristics of e-
Commerce quality and user-system interaction so that the
results of the study can be used to develop the quality of the
70
on August 31,2021 at 04:06:29 UTC from IEEE Xplore. Restrictions apply.
 web. The study also produced 29 new metrics for functional
suitability and usability characteristics.
Wattiheluw [9] applied the Fuzzy Mamdani method to
assess functional suitability, performance efficiency,
reliability and usability on the ISO / IEC 25010 e-Commerce
website and used the Analytical Hierarchy Process method to
weight the importance of characteristics. The research
resulted in an acceleration rate of 0.684 that could help
developers improve the quality of the e-Commerce web.
III. RESEARCH METHODOLOGY
of the software in completing the tasks and objectives
specified.
The first step to measuring functional suitability is to
determine the standard functions that every e-Commerce
Website must-have. In this study, we use the standard
functions that have been proposed by [11] in accordance
with various heuristic references made by previous
researchers such as Jakob Nielsen. To enable the design of
e-Commerce websites in order to serve the various needs of
users. These standard functions can be seen in the following
table.

A. Block Diagram TABLE I. STANDARD FUNCTION OF E-COMMERCE

This study evaluates the functional suitability of the six No Standard Function
e-Commerce websites that we have determined previously. 1. Catalog
To conduct this experiment, the steps we took are explained 2. Registration / Login box
3. Shopping Cart
in the experiment blog diagram as shown in Fig.1 below: 4. Search
5. Navigation
Studying of 6. Personalization
Start a. Automatic product recommendations
Literature
b. Featured product
c. Related products
Defining a set
of attributes d. Popular products
e. Product Zoom
related to
functional 7. Payment method
suitability 8. Social Media + contact person
based on 9. FAQ + Help
ISO/IEC
25010 Table I shows the standard functions that every e-
Commerce Website must have to assist users in achieving
Planning
all their online shopping needs that have been proposed by
measurements [11].
The next step is to set various attributes for each sub-
characteristic. We associate this attribute with each
requirement used to calculate each sub-characteristic, and
Defining Implementing
Finish we adapt it to the needs of the e-Commerce web. The
Conclusion measurements
attributes of each sub-characteristic can be seen in the
Fig. 1. Block diagram. following table.
Figure block diagram shows some of the steps we took TABLE II. FUNCTIONAL SUITABILITY
to achieve our goals in this study. The first step we took in
this research was to conduct a literature study related to e- Sub-Characteristic Attribute
Functional Availability
Commerce evaluation using ISO 25010. Then we defined a Completeness
Functional Completeness
set of attributes related to functional suitability. There are System compatibility with the required level of
three attributes related to functional conformity, namely Correctness
precision
completeness, correctness, and appropriateness. Then, test The suitability of the function with its purpose
the validity of a set of attributes that is performed to Appropriateness Conformity of usage function with usage
procedure
evaluate software based on functional suitability metrics and
to evaluate and calculate the attributes. From the evaluation
Table II shows the attribute of functional suitability.
results, the conclusions can be used as recommendations for
There are two attributes of completeness sub-characteristics,
e-Commerce websites.
namely functional availability, which aims to assess the
B. Standard of e-Commerce readiness of the functions needed by the user. The second
This study measures functional suitability. These attribute is functional completeness which aims to evaluate
characteristics represent the extent to which e-Commerce the complete function of two functions. Sub-characteristics
systems provide functions that meet needs when used under of truth have attribute, namely the suitability of the system
certain conditions. Functional conformity has three sub- with the level of precision required. Whereas for the sub-
characteristics, namely completeness, correctness, and characteristics of conformity has two attributes, namely the
conformity. Functional completeness assesses how complete suitability of the function with its purpose and the suitability
the functions of the software are based on their respective of the use of the function with the usage procedure. After
standard functions. Functional accuracy assesses how true that, measure each sub-characteristic on each e-Commerce
the functions of a software are in achieving its objectives. Website according to the specified standard attribute
Functional appropriateness assesses how well each function function.

71
Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on August 31,2021 at 04:06:29 UTC from IEEE Xplore. Restrictions apply.
 The first measurement is functional completeness
measures to assess how complete the functions of the
software are based on their respective standard functions.
The equation is shown in the following (1):
(1)

Where,
A = Number of functions missing i
B = Number of functions specified
The second measurement is functional correctness
measures to assess how true the functions of a software are
in achieving its objectives. The equation is shown in the
following (2): Fig. 2. Functional completeness measures.

The results of functional completeness measures of the

(2)
Thanksinsomnia website has the value of 0.5 for both
attributes. Then 347 Clothing for the functional
Where,
completeness availability attribute show the value of 0.75,
A = Number of functions that are incorrect i
while for the completeness functional suitability is 0.917.
B = Number of functions considered
Cosmic, Widezig!, and Bloods measurement results show
the same results, for the completeness functional availability
The third measurement is functional appropriateness
attribute show the value of 0.834, while for the
measures to assess how well each function of the software in
completeness functional suitability attribute indicates a
completing the tasks and objectives specified. The equation
value of 0.917. Screamous measurement results for the
is shown in the following (3):
completeness functional availability attribute showed a
value of 0.5, while for the completeness functional
(3) suitability attribute showed a value of 0.67.
The results of the testing of the completeness of
Where, functional availability and completeness of functional
A = Number of functions missing or incorrect among those suitability, each of which has 12 questions on each e-
required for achieving user objectives commerce website, can be seen in Table III.
B = Number of functions required for achieving specified
user objectives TABLE III. SUB-CHARACTERISTIC COMPLETENESS

IV. DATASET Functional

e-Commerce Question True False
Completeness
This study used e-Commerce websites engaged in
Wadezig! 10 2
fashion. The selected e-Commerce website is an e- Cosmic 10 2
Commerce website obtained from bukareview which is a Functional Bloods 10 2
collection of various customer reviews of bukalapak.com. Completeness 347 Clothing 12 9 3
Availability
There are 20 local fashion brands that are most in demand Screamous 6 6
and sought after by customers, but we chose six brands to be Thanksinsomnia 6 6
case studies based on the level of similarity of the menu, Wadezig! 11 1
Cosmic 11 1
there are Thanksinsomnia, 347 Clothing, Cosmic, Functional
Bloods 11 1
Screamous, Wadezig!, and Bloods. Our research was Completeness
347 Clothing
12
11 1
conducted for 1.5 months starting from the literature study, Suitability
Screamous 8 4
determining the case study to collecting data by conducting Thanksinsomnia 6 6
a survey on each case study. Functional suitability
measurements are carried out three times in accordance with The examples of functional completeness measures for
sub-characteristics possessed by functional suitability, the first attribute in Thanksinsomnia are as follows:
namely functional completeness measures, functional
correctness measures, and functional appropriateness.
V. RESULT = 0.5
In this section, the results of all experiments and
calculations that have been carried out will be shown Value of 6 indicates the number of missing functions on
through tables, graphs, and calculation calculations for each the Thanksinsomnia website. The value 12 indicates the
existing formula. number of functions specified or question on the
The first measurement is functional completeness to Thanksinsomnia website. And the result is 0.5.
carried out twice in accordance with the attributes possessed The results of the functional completeness measurements
by functional completeness. The results is shown by Fig. 2. show that there are three brands that have the same high
value on the availability attributes, namely Cosmic,

72
Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on August 31,2021 at 04:06:29 UTC from IEEE Xplore. Restrictions apply.
 Widezig! And Bloods with a value of 0.834. While the
lowest value is on the Thanksinsomnia and Screamous sites
with a value of 0.5. The highest functional completeness
attribute value was obtained by 4 web sites at once, namely
347 Clothing, Cosmic, Widezig! And Bloods with a value of
0.917. That is because almost all menus owned by four
websites fulfill the standard functions of e-Commerce
websites.
The second measurement is functional correctness.
Functional correctness measures only have one attribute,
namely the suitability of the system with the required level
of precision. The results of functional completeness
measurements can be seen in Fig. 3 below.
indicates the number of functions considered. The result of
the functional value of the availability attribute is 0.5.
The third measurement is functional appropriateness
measures, carried out twice in accordance with the attributes
possessed by functional appropriateness, namely the
appropriateness of the function to its purpose and the
appropriateness of the use of the function with its use
procedure. Functional error measures aim to assess the
extent to which the function facilitates the completion of the
task and the objectives specified. The results of functional
completeness measurements can be seen in Fig. 4 below.

Fig. 4. Functional appropriateness measures.

Fig. 3. Functional correctness measures. On the Fig. 4, functional appropriateness measures

Thanksinsomnia get a value of 0.834 for the appropriateness
On the Fig. 3, the results of measurement of functional attribute of the function to its purpose, while the
correctness of Thanksinsomnia shows the values of 0.84, appropriateness attribute of the use of the function with its
347 Clothing shows the value of 0.889, Cosmic shows the use procedure shows a value of 0.75. Website 347 Clothing
value of 0.86, Screamous and Widezig! shows the same gets a value of 0.834 for the appropriateness attribute of the
values of 0.94 and Bloods shows the value of 0.79. The function to its purpose, while the appropriateness attribute
highest score is obtained by the website from Screamous of the use of the function with the usage procedure shows a
and Widezig! with a value of 0.94. As for the lowest value perfect result of 1.00. The Cosmic website gets a value of
obtained by the Bloods website with a value of 0.79. 0.834 for the appropriateness attribute of the function for its
The results of testing the correctness sub-characteristics purpose, while the appropriateness attribute for the user of
have several aspects that are assessed on each e-commerce the function with its usage procedure shows a value of 0.75.
web. Each e-commerce web has a number of different The Screamous website gets a value of 0.917 for the
aspects to be tested according to the menu of the website. appropriateness attribute of the function for its purpose,
but the test still refers to the standard e-commerce function while the appropriateness attribute for the use of the
that we refer to. The test results can be seen in the following function with its usage procedure shows a value of 1.00.
Table IV. Widezig! and Bloods both get the value 0.917 for the
appropriateness attribute of the function to its purpose,
TABLE IV. SUB-CHARACTERISTIC OF CORRECTNESS while the appropriateness attribute for the use of the
Website e- Succesful Failed
function and the usage procedure are both showing a perfect
Aspect result of 1.00.
Commerce pages Pages
Wadezig! 45 42 3 Attributes of the suitability of the function with the aim
Bloods 29 23 6 show that the highest value obtained by 3 websites namely
Screamous 19 18 1 Screamous, Widezig!, and Bloods with a value of 0.917. As
Cosmic 14 12 2
347 Clothing 18 16 2
for the attribute of the suitability of the use of the function
Thankinsomnia 18 15 3 with the highest value usage procedure, 4 websites are
obtained namely 347 Clothing, Screamous, Widezig!, and
The examples of functional correctness measures for the Bloods with a perfect score of 1.00.
first attribute in Thanksinsomnia are as follows: The test results of the appropriateness sub-characteristics
of the suitability of the function with its purpose with 12
questions and the sub-characteristics of the appropriateness
= 0.84 of usage function with usage procedure with 4 questions on
each e-commerce web can be seen in Table V. The
Value of 2 indicates the number of functions that are examples of functional appropriateness measures for the
wrong on the Thanksinsomnia website. The value 18 first attribute in Thanksinsomnia are as follows:

73
Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on August 31,2021 at 04:06:29 UTC from IEEE Xplore. Restrictions apply.

= 0.834
TABLE V. SUB-CHARACTERISTIC APPROPRIATENESS
Functional
Appropriateness
e-Commerce Question True
Wadezig 11
Cosmic 10
Appropriateness
the suitability of Bloods 11
12
the function with 347 Clothing 10
its purpose Screamous 11
Thanksinsomnia 10
Wadezig 4 0
Appropriateness Cosmic 3 1
of usage function Bloods 4 0
4
with usage 347 Clothing 4 0
procedure Screamous 4 0
Thanksinsomnia 3 1
At the examples, value of 2 indicates the number of
functions lost / wrong among those needed to achieve user
goals on the Thanksinsomnia website. The value of 12
indicates the number of functions determined to achieve the
target user specified. The result of the measures is 0.834.
This research shows that every e-Commerce Website has
advantages and disadvantages of each in the quality of the
software, especially functional suitability. If the e-
Commerce website has a low value than the calculations
that have been done can affect the convenience of customers
when transacting online shopping because customers can
have difficulty finding the menu needed to achieve their
goals.
VI. DISCUSSION
The functional suitability measurement using ISO 25010
is used to assess the extent to which the e-Commerce system
provides functions that meet the needs when used under
certain conditions. Functional suitability fit has three sub-
characteristics, namely functional completeness, functional
correctness, and functional appropriateness. Each sub-
characteristic has a different attribute. Data taken from six
fashion brand websites shows that in the functional
suitability attribute on the functional completeness sub-
characteristics, the highest value is obtained in three brands
namely Cosmic, Widezig! And Bloods with a value of
0.834. While the lowest value is on the Thanksinsomnia and
Screamous sites with a value of 0.5. The functional
completeness attribute in the Completeness sub-
characteristic shows that the highest score was obtained by 4
websites, namely 347 Clothing, Cosmic, Widezig! And
Bloods with a value of 0.917. That's because almost all the
menus owned by the four websites fulfill the standard
functions of the e-Commerce website.
In the functional correctness sub-characteristics of the
system suitability attribute with the required level of
precision, the highest score is obtained by the website from
Screamous and Widezig! with a value of 0.94. As for the
lowest value obtained by the Bloods website with a value of
0.79.The attribute of the appropriateness of the function to
the purpose of the functional appropriateness sub-
characteristics shows that the highest value obtained by 3
websites is Screamous, Widezig!, and Bloods with a value
Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on August 31,2021 at 04:06:29 UTC from IEEE Xplore. Restrictions apply.
of 0.917. While for the attribute of the suitability of the use
of the function with the highest value usage procedure, 4
websites are obtained namely 347 Clothing, Screamous,
Widezig!, and Bloods with a perfect value of 1.00.
The results show several calculations that have different
results for each e-commerce web. Every website has
False advantages and disadvantages of each in each attribute. So it
1
needs to be done an analysis and giving recommendations
2 so that the website can be better. This is shown in Table VI.
1
TABLE VI. ANALYSIS OF MEASUREMENTS
2
1 Functional Suitability Recommendation
2 Functional Completeness Complete the menu in accordance with
the standard e-Commerce functions.
Functional Correctness Make improvements to items that do not
meet e-Commerce standards.
Functional Appropriateness Improve the quality of the website to fit
the purpose of the e-Commerce function.
The table shows the results of the analysis of the
measurement of functional suitability as well as suggestions
or recommendations if the calculation of functional
suitability has a low value. The order of e-commerce
websites is in accordance with the website that has the
highest functional suitability calculation value to the lowest.
In functional completeness, Widezig! has the highest value
while the lowest value is in Thanksinsomnia. In the
correctness attribute, the highest value is in Widezig! and
the lowest is Screamous. In the appropriateness attribute the
highest value is Widezig! and the lowest is Thanksinsomnia.
A software product has a low value because the function
of the website does not comply with technical standards
[11]. Functional completeness is calculated from how
complete the e-Commerce website has the menus needed for
an e-Commerce website according to the standard functions
that every e-Commerce website must have to assist users in
achieving all their online shopping needs that have been
proposed by [8]. From the results of functional suitability
calculations for each attribute, this research can help every
e-Commerce Website can improve the quality of their
website, especially in terms of functional suitability. This is
so that every web-e-Commerce can provide comfort and
good service for all their customers.
VII. CONCLUSION
From the explanation of the results and discussion, the
conclusion of this research is:
• The functional completeness attribute on the
Completeness sub-characteristic shows that the
highest score is obtained by 347 Clothing, Cosmic,
Widezig! And Bloods.
• In the functional correctness sub-characteristics of
the system suitability attributes with the required
level of accuracy, the highest scores were obtained
by the websites of Screamous and Widezig!
• Appropriateness the suitability of the function with
its purpose sub-characteristic shows that the
highest scores obtained by Screamous, Widezig!,
and Bloods.
• Appropriateness of usage function with usage
procedure was obtained by 347 Clothing,
Screamous, Widezig!, and Bloods.
74
 From the measurements that have been made, the
functional suitability can help e-Commerce web sites to
improve the quality of their website, especially in terms of
functional suitability so as to provide comfort and good
service to customers. This evaluation can also be used as a
reference for developing a better e-Commerce website.
For future work, we will continue the research with more
aspects and samples so that the research results can be better
than before.
REFERENCES
[1] F. A. Tanjung and W. Dhewanto, “Formulation of E-Commerce
Website Development Plan Using Multidimensional Approach for
Web Evaluation,” Procedia - Soc. Behav. Sci., vol. 115, no. Iicies
2013, pp. 361–372, 2014.
[2] A. Stefani and M. Xenos, “Weight-modeling of B2C system quality,”
Comput. Stand. Interfaces, vol. 33, no. 4, pp. 411–421, 2011.
[3] K. Musa and J. Alkhateeb, “Quality Model Based on Cots Quality
Attributes,” Int. J. Softw. Eng. Appl., vol. 4, no. 1, pp. 1–8, 2013.
[4] I. Kadi, A. Idri, and S. Ouhbi, “Quality evaluation of cardiac decision
support systems using ISO 25010 standard,” Proc. IEEE/ACS Int.
Conf. Comput. Syst. Appl. AICCSA, vol. 0, no. 1, 2016.
[5] ISO/IEC, “ISO/IEC 25010:2011 - Systems and software engineering -
- Systems and software Quality Requirements and Evaluation
(SQuaRE) -- System and software quality models,” 2011.
[6] A. Stefani and M. Xenos, “E-commerce system quality assessment
using a model based on ISO 9126 and Belief Networks,” Softw. Qual.
J., vol. 16, no. 1, pp. 107–129, 2008.
[7] Y. Purwati, “Standard Features of E-Commerce User Interface for the
Web,” Res. World, vol. 2, no. 3, pp. 77–87, 2011.
[8] A. S. Puspaningrum, S. Rochimah, and R. J. Akbar, “Functional
Suitability Measurement using Goal-Oriented Approach based on
ISO/IEC 25010 for Academics Information System,” J. Inf. Syst.
Eng. Bus. Intell., vol. 3, no. 2, p. 68, 2017.
[9] F. H. Wattiheluw, S. Rochimah, and C. Fatichah, “Klasifikasi
Kualitas Perangkat Lunak Berdasarkan Iso/Iec 25010 Menggunakan
Ahp Dan Fuzzy Mamdani Untuk Situs Web E-Commerce,” JUTI J.
Ilm. Teknol. Inf., vol. 17, no. 1, p. 73, 2019.
[10] A. Stefani, “A Metrics Ecosystem for Designing Quality E-
Commerce Systems,” Int. J. Comput. Sci. Inf. Technol., vol. 10, no. 2,
pp. 79–93, 2018.
[11] L. G. Shardow and R. O. Mensah, “A Proposed Harmonisation
Framework for E-Commerce Websites Across The Globe,” Int. Res.
J. Eng. Technol., vol. 5, no. 4, pp. 852–861, 2018.

75
Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on August 31,2021 at 04:06:29 UTC from IEEE Xplore. Restrictions apply.