Download as pdf or txt
Download as pdf or txt
You are on page 1of 6

Proc. of 2020 7th Int. Conf. on Information Tech.

, Computer, and Electrical Engineering (ICITACEE)

Functional Suitability Measurement Based on


ISO/IEC 25010 for e-Commerce Website
Novrindah Alvi Hasanah Luthfi Atikah Siti Rochimah
Department of Informatics Department of Informatics Department of Informatics
Institut Teknologi Sepuluh Nopember Institut Teknologi Sepuluh Nopember Institut Teknologi Sepuluh Nopember
Surabaya, Indonesia Surabaya, Indonesia Surabaya, Indonesia
alvinovrindah@gmail.com luthfiatikah@gmail.com siti@if.its.ac.id

Abstract— The intense competition between e-Commerce suitability, performance efficiency, reliability and usability
makes every e-Commerce has a way to be able to compete in on the e-Commerce web by applying the Fuzzy Mamdani
the market, one of which is to improve the quality of e- method and the Analytical Hierarchy Process method to
Commerce websites. Therefore, research is needed to find out weight the importance of the characteristics.
the quality of the web with the buyer's intention on the E-
In this paper, we proposed ISO/IEC 25010 to assess the
commerce web by evaluating it. There are various standard
software evaluation models used such as ISO / IEC 25010. This functional suitability of various Indonesian e-Commerce
paper, the ISO/IEC 25010 to assess the functional suitability of websites. This research was conducted on six local web
various Indonesian e-Commerce websites, namely Wadezig!, fashion brands in Indonesia that we chose from the
Screamous, 347 Clothing, Thanksinsomnia and Bloods, which bukareview which is a collection of bukalapak customer
were determined through Bukalapak customer reviews. reviews. The brands are Wadezig!, Screamous, 347
Functional suitability has three calculated attributes, namely Clothing, Thanksinsomnia and Bloods. We selected these
completeness, correctness, and appropriateness. From the six brands based on the level of similarity the menu has. The
measurements that have been made, the functional suitability study was conducted for 1.5 months starting from a
can help e-Commerce websites to improve the quality of their
literature study, determining case studies, selecting methods
website, especially in terms of functional suitability so as to
provide comfort and good service to customers. This evaluation to conducting a survey of each selected case study. The
can also be used as a reference for developing e-Commerce results of the study are evaluations and recommendations
websites to be better. that can be used as a reference for developing e-Commerce
websites to be better.
Keywords—e-Commerce website, ISO / IEC 25010, functional
suitability, Completeness, Correctness, Appropriateness II. LITERATURE REVIEW
Tanjung et al. [1] conducted research on several e-
Commerce websites that have developed rapidly in Indonesia
I. INTRODUCTION
using a multidimensional approach to evaluating web sites
The e-Commerce industry has developed rapidly in this from e-Commerce. The results of the study are used to
decade; various convenience services are also offered by identify areas of website innovation that are used to increase
various e-Commerce sites in Indonesia. Visitor satisfaction user satisfaction and user feedback and create competitive
is one of the priorities in developing e-Commerce services. value from e-Commerce companies.
User satisfaction in e-commerce requires better innovation
on e-Commerce Website [1]. Stefani [2] believes that e- Purwati [7] conducted a study to evaluate the user
Commerce websites can be evaluated through users using a interface (UI) on the e-Commerce web. UI elements studied
software evaluation model. Various standard software are site navigation, home links, search, view carts, get help,
evaluation models in question include McCall, FURPS, manage accounts, catalogues, and personalization functions.
Dormey, Boehm, ISO / IEC [3]. The analysis was carried out on 120 web sites from the US,
The quality of software products can be calculated using China, Germany and Indonesia. The results of the study
ISO / IEC 25010 standard, [4] defines a software product indicated that there is no web standard that meets the de facto
quality model along with a system quality in use standard of Nielsen. Hypothesis results obtained from the
model. The ISO/IEC 25010 software product quality model research show that there are different standards for features
is composed of eight characteristics viz, functional and designs used in four countries.
suitability, performance efficiency, compability, usability, Puspitanigrum et al.[8] assessed the quality of academic
reliability, security, maintainability and portability [5] that information systems (SIA) from tertiary institutions on the
can be measured internally or externally. characteristics of functional conformity using ISO 25010
Software quality can be judged by the better user especially Goal Quality Metric (GQM). The results of the
interface (UI) and functional devices[6]. Various studies study are in the form of a framework based on institutional
have been conducted to evaluate the quality of software goals that can be used to evaluate and develop SIA.
such as Purwati [7] evaluate the user interface (UI) on the e-
Commerce web. In addition, research is also carried out on Stefani [10] conducted a study on the eco-system of
functional suitability Puspitanigrum et al [8] undertook to software quality based on the ISO 25010 standard. These
examine the quality of the academic information system metrics are mapped based on the characteristics of e-
(SIA) of tertiary institutions on functional suitability Commerce quality and user-system interaction so that the
characteristics using ISO 25010 especially Goal Quality results of the study can be used to develop the quality of the
Metric (GQM). Wattiheluw [9] assessed functional

Authorized978-1-7281-7226-2/20/$31.00 ©2020
licensed use limited to: Instituto Tecnologico 70
IEEERica (ITCR). Downloaded
de Costa on August 31,2021 at 04:06:29 UTC from IEEE Xplore. Restrictions apply.
web. The study also produced 29 new metrics for functional of the software in completing the tasks and objectives
suitability and usability characteristics. specified.
The first step to measuring functional suitability is to
Wattiheluw [9] applied the Fuzzy Mamdani method to
determine the standard functions that every e-Commerce
assess functional suitability, performance efficiency,
Website must-have. In this study, we use the standard
reliability and usability on the ISO / IEC 25010 e-Commerce
functions that have been proposed by [11] in accordance
website and used the Analytical Hierarchy Process method to
with various heuristic references made by previous
weight the importance of characteristics. The research
researchers such as Jakob Nielsen. To enable the design of
resulted in an acceleration rate of 0.684 that could help
e-Commerce websites in order to serve the various needs of
developers improve the quality of the e-Commerce web.
users. These standard functions can be seen in the following
III. RESEARCH METHODOLOGY table.

A. Block Diagram TABLE I. STANDARD FUNCTION OF E-COMMERCE


This study evaluates the functional suitability of the six No Standard Function
e-Commerce websites that we have determined previously. 1. Catalog
To conduct this experiment, the steps we took are explained 2. Registration / Login box
3. Shopping Cart
in the experiment blog diagram as shown in Fig.1 below: 4. Search
5. Navigation
Studying of 6. Personalization
Start a. Automatic product recommendations
Literature
b. Featured product
c. Related products
Defining a set
of attributes d. Popular products
e. Product Zoom
related to
functional 7. Payment method
suitability 8. Social Media + contact person
based on 9. FAQ + Help
ISO/IEC
25010 Table I shows the standard functions that every e-
Commerce Website must have to assist users in achieving
Planning
all their online shopping needs that have been proposed by
measurements [11].
The next step is to set various attributes for each sub-
characteristic. We associate this attribute with each
requirement used to calculate each sub-characteristic, and
Defining Implementing
Finish we adapt it to the needs of the e-Commerce web. The
Conclusion measurements
attributes of each sub-characteristic can be seen in the
Fig. 1. Block diagram. following table.
Figure block diagram shows some of the steps we took TABLE II. FUNCTIONAL SUITABILITY
to achieve our goals in this study. The first step we took in
this research was to conduct a literature study related to e- Sub-Characteristic Attribute
Functional Availability
Commerce evaluation using ISO 25010. Then we defined a Completeness
Functional Completeness
set of attributes related to functional suitability. There are System compatibility with the required level of
three attributes related to functional conformity, namely Correctness
precision
completeness, correctness, and appropriateness. Then, test The suitability of the function with its purpose
the validity of a set of attributes that is performed to Appropriateness Conformity of usage function with usage
procedure
evaluate software based on functional suitability metrics and
to evaluate and calculate the attributes. From the evaluation
Table II shows the attribute of functional suitability.
results, the conclusions can be used as recommendations for
There are two attributes of completeness sub-characteristics,
e-Commerce websites.
namely functional availability, which aims to assess the
B. Standard of e-Commerce readiness of the functions needed by the user. The second
This study measures functional suitability. These attribute is functional completeness which aims to evaluate
characteristics represent the extent to which e-Commerce the complete function of two functions. Sub-characteristics
systems provide functions that meet needs when used under of truth have attribute, namely the suitability of the system
certain conditions. Functional conformity has three sub- with the level of precision required. Whereas for the sub-
characteristics, namely completeness, correctness, and characteristics of conformity has two attributes, namely the
conformity. Functional completeness assesses how complete suitability of the function with its purpose and the suitability
the functions of the software are based on their respective of the use of the function with the usage procedure. After
standard functions. Functional accuracy assesses how true that, measure each sub-characteristic on each e-Commerce
the functions of a software are in achieving its objectives. Website according to the specified standard attribute
Functional appropriateness assesses how well each function function.

71
Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on August 31,2021 at 04:06:29 UTC from IEEE Xplore. Restrictions apply.
The first measurement is functional completeness
measures to assess how complete the functions of the
software are based on their respective standard functions.
The equation is shown in the following (1):
(1)

Where,
A = Number of functions missing i
B = Number of functions specified
The second measurement is functional correctness
measures to assess how true the functions of a software are
in achieving its objectives. The equation is shown in the
following (2): Fig. 2. Functional completeness measures.

The results of functional completeness measures of the


(2)
Thanksinsomnia website has the value of 0.5 for both
attributes. Then 347 Clothing for the functional
Where,
completeness availability attribute show the value of 0.75,
A = Number of functions that are incorrect i
while for the completeness functional suitability is 0.917.
B = Number of functions considered
Cosmic, Widezig!, and Bloods measurement results show
the same results, for the completeness functional availability
The third measurement is functional appropriateness
attribute show the value of 0.834, while for the
measures to assess how well each function of the software in
completeness functional suitability attribute indicates a
completing the tasks and objectives specified. The equation
value of 0.917. Screamous measurement results for the
is shown in the following (3):
completeness functional availability attribute showed a
value of 0.5, while for the completeness functional
(3) suitability attribute showed a value of 0.67.
The results of the testing of the completeness of
Where, functional availability and completeness of functional
A = Number of functions missing or incorrect among those suitability, each of which has 12 questions on each e-
required for achieving user objectives commerce website, can be seen in Table III.
B = Number of functions required for achieving specified
user objectives TABLE III. SUB-CHARACTERISTIC COMPLETENESS

IV. DATASET Functional


e-Commerce Question True False
Completeness
This study used e-Commerce websites engaged in
Wadezig! 10 2
fashion. The selected e-Commerce website is an e- Cosmic 10 2
Commerce website obtained from bukareview which is a Functional Bloods 10 2
collection of various customer reviews of bukalapak.com. Completeness 347 Clothing 12 9 3
Availability
There are 20 local fashion brands that are most in demand Screamous 6 6
and sought after by customers, but we chose six brands to be Thanksinsomnia 6 6
case studies based on the level of similarity of the menu, Wadezig! 11 1
Cosmic 11 1
there are Thanksinsomnia, 347 Clothing, Cosmic, Functional
Bloods 11 1
Screamous, Wadezig!, and Bloods. Our research was Completeness
347 Clothing
12
11 1
conducted for 1.5 months starting from the literature study, Suitability
Screamous 8 4
determining the case study to collecting data by conducting Thanksinsomnia 6 6
a survey on each case study. Functional suitability
measurements are carried out three times in accordance with The examples of functional completeness measures for
sub-characteristics possessed by functional suitability, the first attribute in Thanksinsomnia are as follows:
namely functional completeness measures, functional
correctness measures, and functional appropriateness.
V. RESULT = 0.5
In this section, the results of all experiments and
calculations that have been carried out will be shown Value of 6 indicates the number of missing functions on
through tables, graphs, and calculation calculations for each the Thanksinsomnia website. The value 12 indicates the
existing formula. number of functions specified or question on the
The first measurement is functional completeness to Thanksinsomnia website. And the result is 0.5.
carried out twice in accordance with the attributes possessed The results of the functional completeness measurements
by functional completeness. The results is shown by Fig. 2. show that there are three brands that have the same high
value on the availability attributes, namely Cosmic,

72
Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on August 31,2021 at 04:06:29 UTC from IEEE Xplore. Restrictions apply.
Widezig! And Bloods with a value of 0.834. While the indicates the number of functions considered. The result of
lowest value is on the Thanksinsomnia and Screamous sites the functional value of the availability attribute is 0.5.
with a value of 0.5. The highest functional completeness The third measurement is functional appropriateness
attribute value was obtained by 4 web sites at once, namely measures, carried out twice in accordance with the attributes
347 Clothing, Cosmic, Widezig! And Bloods with a value of possessed by functional appropriateness, namely the
0.917. That is because almost all menus owned by four appropriateness of the function to its purpose and the
websites fulfill the standard functions of e-Commerce appropriateness of the use of the function with its use
websites. procedure. Functional error measures aim to assess the
The second measurement is functional correctness. extent to which the function facilitates the completion of the
Functional correctness measures only have one attribute, task and the objectives specified. The results of functional
namely the suitability of the system with the required level completeness measurements can be seen in Fig. 4 below.
of precision. The results of functional completeness
measurements can be seen in Fig. 3 below.

Fig. 4. Functional appropriateness measures.

Fig. 3. Functional correctness measures. On the Fig. 4, functional appropriateness measures


Thanksinsomnia get a value of 0.834 for the appropriateness
On the Fig. 3, the results of measurement of functional attribute of the function to its purpose, while the
correctness of Thanksinsomnia shows the values of 0.84, appropriateness attribute of the use of the function with its
347 Clothing shows the value of 0.889, Cosmic shows the use procedure shows a value of 0.75. Website 347 Clothing
value of 0.86, Screamous and Widezig! shows the same gets a value of 0.834 for the appropriateness attribute of the
values of 0.94 and Bloods shows the value of 0.79. The function to its purpose, while the appropriateness attribute
highest score is obtained by the website from Screamous of the use of the function with the usage procedure shows a
and Widezig! with a value of 0.94. As for the lowest value perfect result of 1.00. The Cosmic website gets a value of
obtained by the Bloods website with a value of 0.79. 0.834 for the appropriateness attribute of the function for its
The results of testing the correctness sub-characteristics purpose, while the appropriateness attribute for the user of
have several aspects that are assessed on each e-commerce the function with its usage procedure shows a value of 0.75.
web. Each e-commerce web has a number of different The Screamous website gets a value of 0.917 for the
aspects to be tested according to the menu of the website. appropriateness attribute of the function for its purpose,
but the test still refers to the standard e-commerce function while the appropriateness attribute for the use of the
that we refer to. The test results can be seen in the following function with its usage procedure shows a value of 1.00.
Table IV. Widezig! and Bloods both get the value 0.917 for the
appropriateness attribute of the function to its purpose,
TABLE IV. SUB-CHARACTERISTIC OF CORRECTNESS while the appropriateness attribute for the use of the
Website e- Succesful Failed
function and the usage procedure are both showing a perfect
Aspect result of 1.00.
Commerce pages Pages
Wadezig! 45 42 3 Attributes of the suitability of the function with the aim
Bloods 29 23 6 show that the highest value obtained by 3 websites namely
Screamous 19 18 1 Screamous, Widezig!, and Bloods with a value of 0.917. As
Cosmic 14 12 2
347 Clothing 18 16 2
for the attribute of the suitability of the use of the function
Thankinsomnia 18 15 3 with the highest value usage procedure, 4 websites are
obtained namely 347 Clothing, Screamous, Widezig!, and
The examples of functional correctness measures for the Bloods with a perfect score of 1.00.
first attribute in Thanksinsomnia are as follows: The test results of the appropriateness sub-characteristics
of the suitability of the function with its purpose with 12
questions and the sub-characteristics of the appropriateness
= 0.84 of usage function with usage procedure with 4 questions on
each e-commerce web can be seen in Table V. The
Value of 2 indicates the number of functions that are examples of functional appropriateness measures for the
wrong on the Thanksinsomnia website. The value 18 first attribute in Thanksinsomnia are as follows:

73
Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on August 31,2021 at 04:06:29 UTC from IEEE Xplore. Restrictions apply.
of 0.917. While for the attribute of the suitability of the use
of the function with the highest value usage procedure, 4
= 0.834 websites are obtained namely 347 Clothing, Screamous,
Widezig!, and Bloods with a perfect value of 1.00.
TABLE V. SUB-CHARACTERISTIC APPROPRIATENESS The results show several calculations that have different
results for each e-commerce web. Every website has
Functional
Appropriateness
e-Commerce Question True False advantages and disadvantages of each in each attribute. So it
Wadezig 11 1
needs to be done an analysis and giving recommendations
Cosmic 10 2 so that the website can be better. This is shown in Table VI.
Appropriateness
the suitability of Bloods 11 1
12 TABLE VI. ANALYSIS OF MEASUREMENTS
the function with 347 Clothing 10 2
its purpose Screamous 11 1 Functional Suitability Recommendation
Thanksinsomnia 10 2 Functional Completeness Complete the menu in accordance with
Wadezig 4 0 the standard e-Commerce functions.
Appropriateness Cosmic 3 1 Functional Correctness Make improvements to items that do not
of usage function Bloods 4 0 meet e-Commerce standards.
4
with usage 347 Clothing 4 0 Functional Appropriateness Improve the quality of the website to fit
procedure Screamous 4 0 the purpose of the e-Commerce function.
Thanksinsomnia 3 1
The table shows the results of the analysis of the
At the examples, value of 2 indicates the number of measurement of functional suitability as well as suggestions
functions lost / wrong among those needed to achieve user or recommendations if the calculation of functional
goals on the Thanksinsomnia website. The value of 12 suitability has a low value. The order of e-commerce
indicates the number of functions determined to achieve the websites is in accordance with the website that has the
target user specified. The result of the measures is 0.834. highest functional suitability calculation value to the lowest.
This research shows that every e-Commerce Website has In functional completeness, Widezig! has the highest value
advantages and disadvantages of each in the quality of the while the lowest value is in Thanksinsomnia. In the
software, especially functional suitability. If the e- correctness attribute, the highest value is in Widezig! and
Commerce website has a low value than the calculations the lowest is Screamous. In the appropriateness attribute the
that have been done can affect the convenience of customers highest value is Widezig! and the lowest is Thanksinsomnia.
when transacting online shopping because customers can A software product has a low value because the function
have difficulty finding the menu needed to achieve their of the website does not comply with technical standards
goals. [11]. Functional completeness is calculated from how
complete the e-Commerce website has the menus needed for
VI. DISCUSSION an e-Commerce website according to the standard functions
The functional suitability measurement using ISO 25010 that every e-Commerce website must have to assist users in
is used to assess the extent to which the e-Commerce system achieving all their online shopping needs that have been
provides functions that meet the needs when used under proposed by [8]. From the results of functional suitability
certain conditions. Functional suitability fit has three sub- calculations for each attribute, this research can help every
characteristics, namely functional completeness, functional e-Commerce Website can improve the quality of their
correctness, and functional appropriateness. Each sub- website, especially in terms of functional suitability. This is
characteristic has a different attribute. Data taken from six so that every web-e-Commerce can provide comfort and
fashion brand websites shows that in the functional good service for all their customers.
suitability attribute on the functional completeness sub-
characteristics, the highest value is obtained in three brands VII. CONCLUSION
namely Cosmic, Widezig! And Bloods with a value of From the explanation of the results and discussion, the
0.834. While the lowest value is on the Thanksinsomnia and conclusion of this research is:
Screamous sites with a value of 0.5. The functional • The functional completeness attribute on the
completeness attribute in the Completeness sub- Completeness sub-characteristic shows that the
characteristic shows that the highest score was obtained by 4 highest score is obtained by 347 Clothing, Cosmic,
websites, namely 347 Clothing, Cosmic, Widezig! And Widezig! And Bloods.
Bloods with a value of 0.917. That's because almost all the • In the functional correctness sub-characteristics of
menus owned by the four websites fulfill the standard the system suitability attributes with the required
functions of the e-Commerce website. level of accuracy, the highest scores were obtained
In the functional correctness sub-characteristics of the by the websites of Screamous and Widezig!
system suitability attribute with the required level of • Appropriateness the suitability of the function with
precision, the highest score is obtained by the website from its purpose sub-characteristic shows that the
Screamous and Widezig! with a value of 0.94. As for the highest scores obtained by Screamous, Widezig!,
lowest value obtained by the Bloods website with a value of and Bloods.
0.79.The attribute of the appropriateness of the function to • Appropriateness of usage function with usage
the purpose of the functional appropriateness sub- procedure was obtained by 347 Clothing,
characteristics shows that the highest value obtained by 3 Screamous, Widezig!, and Bloods.
websites is Screamous, Widezig!, and Bloods with a value

74
Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on August 31,2021 at 04:06:29 UTC from IEEE Xplore. Restrictions apply.
From the measurements that have been made, the
functional suitability can help e-Commerce web sites to
improve the quality of their website, especially in terms of
functional suitability so as to provide comfort and good
service to customers. This evaluation can also be used as a
reference for developing a better e-Commerce website.
For future work, we will continue the research with more
aspects and samples so that the research results can be better
than before.
REFERENCES
[1] F. A. Tanjung and W. Dhewanto, “Formulation of E-Commerce
Website Development Plan Using Multidimensional Approach for
Web Evaluation,” Procedia - Soc. Behav. Sci., vol. 115, no. Iicies
2013, pp. 361–372, 2014.
[2] A. Stefani and M. Xenos, “Weight-modeling of B2C system quality,”
Comput. Stand. Interfaces, vol. 33, no. 4, pp. 411–421, 2011.
[3] K. Musa and J. Alkhateeb, “Quality Model Based on Cots Quality
Attributes,” Int. J. Softw. Eng. Appl., vol. 4, no. 1, pp. 1–8, 2013.
[4] I. Kadi, A. Idri, and S. Ouhbi, “Quality evaluation of cardiac decision
support systems using ISO 25010 standard,” Proc. IEEE/ACS Int.
Conf. Comput. Syst. Appl. AICCSA, vol. 0, no. 1, 2016.
[5] ISO/IEC, “ISO/IEC 25010:2011 - Systems and software engineering -
- Systems and software Quality Requirements and Evaluation
(SQuaRE) -- System and software quality models,” 2011.
[6] A. Stefani and M. Xenos, “E-commerce system quality assessment
using a model based on ISO 9126 and Belief Networks,” Softw. Qual.
J., vol. 16, no. 1, pp. 107–129, 2008.
[7] Y. Purwati, “Standard Features of E-Commerce User Interface for the
Web,” Res. World, vol. 2, no. 3, pp. 77–87, 2011.
[8] A. S. Puspaningrum, S. Rochimah, and R. J. Akbar, “Functional
Suitability Measurement using Goal-Oriented Approach based on
ISO/IEC 25010 for Academics Information System,” J. Inf. Syst.
Eng. Bus. Intell., vol. 3, no. 2, p. 68, 2017.
[9] F. H. Wattiheluw, S. Rochimah, and C. Fatichah, “Klasifikasi
Kualitas Perangkat Lunak Berdasarkan Iso/Iec 25010 Menggunakan
Ahp Dan Fuzzy Mamdani Untuk Situs Web E-Commerce,” JUTI J.
Ilm. Teknol. Inf., vol. 17, no. 1, p. 73, 2019.
[10] A. Stefani, “A Metrics Ecosystem for Designing Quality E-
Commerce Systems,” Int. J. Comput. Sci. Inf. Technol., vol. 10, no. 2,
pp. 79–93, 2018.
[11] L. G. Shardow and R. O. Mensah, “A Proposed Harmonisation
Framework for E-Commerce Websites Across The Globe,” Int. Res.
J. Eng. Technol., vol. 5, no. 4, pp. 852–861, 2018.

75
Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on August 31,2021 at 04:06:29 UTC from IEEE Xplore. Restrictions apply.

You might also like