The usability evaluation of Mudah.

my on
mobile device
Cite as: AIP Conference Proceedings 1891, 020058 (2017); https://doi.org/10.1063/1.5005391
Published Online: 03 October 2017

Azham Hussain, Emmanuel O. C. Mkpojiogu, Hassan Abubakar, and Hassan Mohammad Hassan

ARTICLES YOU MAY BE INTERESTED IN

A usability evaluation of Lazada mobile application

AIP Conference Proceedings 1891, 020059 (2017); https://doi.org/10.1063/1.5005392

Usability evaluation techniques in mobile commerce applications: A systematic review

AIP Conference Proceedings 1761, 020049 (2016); https://doi.org/10.1063/1.4960889

A user experience evaluation of Amazon Kindle mobile application

AIP Conference Proceedings 1891, 020060 (2017); https://doi.org/10.1063/1.5005393

AIP Conference Proceedings 1891, 020058 (2017); https://doi.org/10.1063/1.5005391 1891, 020058

© 2017 Author(s).
 The Usability Evaluation of Mudah.my on Mobile Device
Azham Hussain1, a), Emmanuel O.C. Mkpojiogu1, b), Hassan Abubakar1, c) and
Hassan Mohammad Hassan1, d)
1
Human-Centered Computing Research Lab, School of Computing, Universiti Utara Malaysia, 06010 Sintok,
Kedah, Malaysia
a)
Corresponding author: azham.h@uum.edu.my
b)
emelnuel@hotmail.com
c)
hass4real2007@gmail.com
d)
xasanwali@gmail.com
Abstract. This study tests and measures the usability of mudah.my mobile application in terms of its efficiency,
effectiveness, and satisfaction. Ten participants conscripted for the study performed five tasks on the application’s interface.
A summative usability evaluation based on user performance and perception was carried out. The usability testing was
conducted at Universiti Utara Malaysia (UUM) and the results obtained show that the Mudah.my mobile application is
easy to use and consistent as seen from the analysis and observations made. However, the app is laden with some usability
issues that should necessitate improvement. The study’s outcome will benefit developers and usability engineers in
providing a user-friendly and usable application that meets users’ needs.

INTRODUCTION
Electronic commerce (e-commerce) is improving the standard of doing business among the business community
all over the world. E-commerce as part of the information technology revolution has become widely used in world
trade in general. E-commerce, which literally means business trading through the Internet, has been around the globe
since mid-1990s. However, in recent years, it is getting more and more attention from entrepreneurs and consumers,
both locally and internationally. One of the main reasons for this is because of the highly successful operations of
some well-known names on the Internet, such as eBay, Yahoo and Dell. The sales revenue that these companies show
in their annual reports are without doubt, one of the biggest factors why e-commerce is important and is gathering
momentum in the commercial market nowadays [1]. E-commerce companies will have to make sure that
organizations’ design keeps pace with the rapidly evolving business strategy, along with fluid governance, strong
leadership and management development [4]. Mobile commerce, a newer arrival and an offshoot of electronic
commerce, creates opportunities for mobile devices and services, and for users to transact business while on transit,
anywhere, anytime [5]. Mudah.my is an online marketplace where people buy and sell a wide variety of goods,
products, and services of various categories such as real estate, automotive, careers, business products and services,
and many more among Malaysian people, business groups and other interest groups all over the world [22]. It is the
product of a partnership between 701 Search of Singapore [7] and Telenor ASA of Norway. The service allows
anyone to buy and sell in his or her region conveniently, without the need for a physical shop or a registered company.
As at June 2015, Mudah.my is the second most visited online Market in Malaysia, with about 1,887,000 unique
individuals [22].
Usability testing can be described as the activity that focuses on observing users working with a product, and
performing tasks that are real and meaningful to them [10]. In addition, it has been viewed as similar to other types of
software quality assurance testing. Developers of ten apply the techniques late in the development cycle when major
usability problems are very costly, if not impossible, to fix [11]. The objectives of this usability test are to carry out a
usability test evaluation with real users to measure the effectiveness, efficiency and satisfaction [6][9][20] of
Mudah.my’s interface. This is achieved by giving several tasks that are related to the Mudah.my mobile application
to the users. Furthermore, solutions and recommendations are suggested to help improve the application.
The term usability testing describes the activity of performing usability tests in a laboratory-like setting, with a
group of users and recording the results for further analysis [11]. Usability testing can be done with mobile phones
especially if the application is a mobile application [14]. However, usability testing is different from the usual black-
box, white-box, or unit testing in the sense that the aim is not to find functional problems, but rather to test the ease of
use of the application’s interface [15]. With respect to this present study, there are limited literatures on Mudah.my
application. Even though Mudah.my mobile application has enjoyed great popularity, there is no documented evidence

The 2nd International Conference on Applied Science and Technology 2017 (ICAST’17)
AIP Conf. Proc. 1891, 020058-1–020058-7; https://doi.org/10.1063/1.5005391
Published by AIP Publishing. 978-0-7354-1573-7/$30.00

020058-1
of its usability. Therefore this study seeks to evaluate the application. The objectives of this usability test are firstly,
to carry out a usability test with real users to measure the effectiveness, efficiency and satisfaction [2-3][13] of the
product by giving several tasks that are related to the Mudah.my mobile application to the users to carry out. Secondly,
the study aims at suggesting solutions and recommendations that will help improve the application if usability issues
are found in the application’s interface.

RELATED LITERATURE
E-commerce or E-business is a way to do business on the Internet in certain markets and for selected customers,
which can increase sales volume, lower costs or provide more real-time information to customers. E-commerce has
advantage over in-store sales in that consumers often associate online shopping with "deals" or lower prices, due to
auction sites and easy cross-referencing via search engines. Furthermore, consumers can easily have access to
countless Web retailers within the same timeframe and have ease of access to many retailers and greater opportunities
for bargain-hunting. In addition, competition on the web is fierce and there is price transparency in e-commerce [17].
Mobile electronic commerce or m-commerce is the e-commerce offered via mobile devices [22]. E-commerce
players see mobile commerce as the most preferred route with mobile wallet as the preferred way of payment [4]. M-
commerce means exchange of goods over the internet by the use of mobile devices like the mobile phone [12][19].
M-commerce emanated from the explosion of applications and services that are accessible from Internet-enabled
mobile devices such as smart cellular telephones, tablets, and personal digital assistants (PDAs) [18][21-24]. Mobile
e-commerce or M-commerce describes online sales transaction that uses wireless or mobile electronic devices. These
wireless devices interact with computer networks and are used to conduct online merchandise and purchases. The
rapid growth of mobile commerce is being driven by a number of factors: increasing mobile user base, rapid adoption
of online commerce and technological advances. M-commerce is a way of exchanging products, ideas and services
between mobile users and the service providers. It involves applications, wireless devices, middlewares, and wireless
networks. Many of the existing applications of e-commerce could be modified to run in wireless environment, and m-
commerce involves many more new applications that have become possible only due to wireless infrastructure.
Furthermore, m-commerce is creating many new service opportunities such as payments, banking transactions, online
gaming and paid mobile applications [23][25]. Today mobile phones are being used for web browsing, location
tracking, and watching news, sports, videos, music through mobile Internet. M-payment transactions such as
performing wireless transaction for banking, mobile shopping, ticketing are done through mobile phones. M-
commerce market is booming, and was expected to reach $119 billion by the end of 2015. It has already generated
$1.3 trillion revenue worldwide and was predicted to reach $1.8 trillion in 2016 [26].
Mobile applications are available in the market, these include: mobile retailing, mobile ticketing, mobile banking,
and mobile payment apps [23], restaurant apps, mobile marketing apps, mobile coupons, and social and gaming apps
[24]. Mobile extends users ability to make transactions across time and creates new transaction opportunities in fields
of retail, ticket booking, and reservations etc., [23]. M-commerce needs some development in specific area for secure
transaction and better shopping experiences. It is likely to take over e-commerce as online shopping is increasing more
on mobile apps and mobile apps are becoming new way to target consumers [8][22][16].

METHODOLOGY

The authors invited and conscripted ten participants to test the Mudah.my mobile application. The test was
conducted at the Universiti Utara Malaysia (UUM). All participants have experience in the use of computer and mobile
applications. However, not all the participants have used Mudah.my mobile application in the past; therefore, these
set of users may have encountered some issues, errors or complexity while using the application during the testing
exercise. Out of the 10 participants, 6 (60%) were male while 4 (40%) were female. The following are their age
categories: 21-30 (60%), 31-40 (30%), and 41-50 (10%). Five of the participants were Bachelor’s degree students
(50%), 3 (30%) were Masters and 2 (20%) were PhD candidates.
The usability testing was carried out with 5 tasks given to each study participants. Each participant’s tasks activity
was recorded using onscreen recorder. Another camera was used for observing the attitude of the participants during
the task operation. For each participant, a record was kept on the information on task, questionnaires and observations.
The users were asked to carry out some specific tasks on the Mudah.my mobile application, such as advertising a

020058-2
product, finding a car they wish to buy, finding a suitable job opportunity, finding a freezer to buy, and finding a
laptop for sell within Universiti Utara Malaysia. The usability testing process comprised of five basics steps, namely:
1) plan, 2) choose participants, 3) perform tests, 4) analyze results, and 5) document recommendation. The study
questionnaires contain two parts. The first is for recording users’ demographic information and the second part is for
recording users’ response related to the usability of Mudah.my mobile application. Three usability attributes/ qualities
were of interest to this study: effectiveness, efficiency, and satisfaction. Their attributes were captured in the usability
evaluation.

RESULTS AND RECOMMENDATIONS

Effectiveness: Task completion metric was used to measure effectiveness. If the user makes successful attempts
in completing the task, each successful attempt will be marked as “Yes”. This mark is given the full credit of 100%.
For tasks that are uncompleted, such tasks are marked “No” for each unsuccessful attempt and credited as 0%.
Unsuccessful task can be identified when users give up completing the task or when the users complete the task
incorrectly. In addition, a 50% credit is given for tasks that are marked “Partial” if the facilitator observed that the
level of tasks completion deserve being marked as “Partial”. Furthermore, users who seek the help or assistance of
the facilitator in completing their tasks are marked as “Partial” for each attempt they make in seeking such help. The
effectiveness of each user can be depicted as shown in Figure 1.

Effectiveness

Yes Partial No

FIGURE 1. Effectiveness per user

The information shown in the above chart is further presented in a detailed table form as in Table 1 below:

TABLE 1. Data Analysis of Effectiveness Usability Metrics

User 1 User 2 User 3 User 4 User 5 User 6 User 7 User 8 User 9 User 10 Subtotal
Yes 8 12 14 10 10 13 20 15 20 20 142
Partial 9 5 4 6 5 3 0 5 0 0 37
No 3 3 2 4 5 4 0 0 0 0 21
TOTAL 20 20 20 20 20 20 20 20 20 20 200

In Table 1 above, there are 20 task criteria with 5 attempts per task, totaling 200 attempts. 142 attempts were
successful and 37 were partially successful. There were a total of 21 unsuccessful tasks which was ignored as 21 x
0% = 0. All the unsuccessful tasks were ignored in the effectiveness calculation as they amount to zero when multiplied
by 0%. To achieve the overall effectiveness for this set of tasks, the following equation is used:

Effectiveness (%) = (Yes + (Partial x 0.5)) / Total x 100%

= (142 + (37 x 0.5)) / 200 x 100%
= 80.25%
From the above computation, it can be seen that the effectiveness of Mudah.my Mobile app is approximately 80.25%.
This implies that the Muday.my mobile application interface highly enables users to achieve their goals and carry out
their tasks.
Efficiency: Efficiency is measured in terms of task time. That is, the time (in seconds and/or minutes) the
participant takes to successfully complete a task. Efficiency can be measured in one of two ways: (1) Time-Based
Efficiency, (2) Overall Relative Efficiency. In this study, efficiency was measured based on Overall Relative
Efficiency as shown in the equation below.

020058-3
 By using the above equation, the Overall Relative Efficiency of task 1 was calculated as follow:
N = the total number of tasks = 5
R = the number of users = 10
t = the time per task per user

Overall Relative Efficiency for task 1=

((0*30.70) + (1*43.26) + (1*17.58) + (1*36.68) + (1*60.13) + (1*27.03) + (1*40.56) + (0*23.30) + (1*25.30) + (1*16.59))*100
30.70 +43.26 + 17.58 + 36.68 + 60.13 + 27.03 + 40.56 + 23.30 + 25.30 + 16.59

= 83.25 %

The rest tasks were also calculated as task 1 was computed. The average was then calculated as follow:
Efficiency (%) = (83.25 + 100.00 + 100.00 + 73.28 +49.53) / 5
= 81.21%

From the above computation, the efficiency of Mudah.my mobile application is approximately 81%. Thus, this
implies that the Muday.my mobile application is efficient and supports users achieving the tasks and goals in minimal
time and with minimal mental efforts. Users expended relatively little resources in successfully carrying out their
goals on the mobile application. Furthermore, using the formula given above, the efficiency scores of each task are
shown in Figure 2.

Overall Relative Efficiency

Satisfaction

FIGURE 2. Time Based Efficiency based on each task

Satisfaction: The measurement for satisfaction was taken subjectively after users had used the mobile application.
The questions and answers were structured using a 3 point Likert scale. In the 3-point Likert scale, “agree” represented
3 points, “neutral” represented 2 points, while “disagree” represented 1 point.
Satisfaction (%) = Answer Point / Total Point x 100%
= 665 / 840 x 100%
= 79.17%

020058-4
 From the computation above, the perceived satisfaction for Mudah.my mobile application is approximately 79%,
implying that the mobile application interface provides users with the comfort they need. Users had a feeling of
comport and thus, had a positive attitude towards the application. Using the same formula, the satisfaction scores for
each user is shown in Figure 3
Usability Score: The above analysis provides three usability metrics (effectiveness, efficiency, and satisfaction),
with each expressed in a percentage. By averaging these three scores, the usability of Mudah.my mobile application
can be defined as a percentage with a score between 1 and 100 and can be derived from the following equation:

Usability (%) = (Effectiveness + Efficiency + Satisfaction) / 3

= (80.25+81.21 +79.17) / 3
= 80.21%
From the above computation, the degree of usability of the Mudah.my mobile application is approximately 80%.
Therefore, with a usability score of 80%, it implies that Muday.my mobile application is usable. However, the
observations and recommendations made from the study are indicated in Table 2.

TABLE 2. Observation and Recommendation

Task No. Observations Recommendations

Task 1 Some of the participants complained about how to go about carrying
out the task. The task seems to be complex and have lengthy processes
before execution. The task is one of most important task in the
Mudah.my mobile application.
Task 2 Searching for a Proton car in the Mudah.my mobile application is
difficult as there are many cars, not only Proton cars displayed.
Task 3 Several job opportunities were displayed. There is need to specify the
kind of job needed. The participants experienced problem in searching
for their suitable job.
Mudah.my mobile application community and
developers should evolve a simpler and more effective
process for implementing this task
The application should introduce a good filtering
method, so that, only the requested items can be
displayed when searched for.
The Mudah.my development team should implement a
search input that categorizes jobs. This will make the job
searching easier, efficient and effective.

Task 4 The participants observed that not only the requested items were
displayed during the operation. They observed that there are several
types of freezers.
Task 5 Most of the participants choose locations different from the one
requested in the task. Despite the fact that the requested location was in
the list of locations in Mudah.my mobile application.
The developers of Mudah.my mobile application should
provide a search mechanism that filters the exact
requested items. This will improve the users’
accessibility, efficiency and effectiveness in searching
items.
The developers should ensure and indicate that the list
was arranged alphabetically (A-Z). This will help users
to know where to get, say, Universiti Utara Malaysia,
and locate it easily from the listed locations.

CONCLUSION
The purpose of this research was to test and evaluate the usability of Mudah.my mobile application and to find
the level of its effectiveness, efficiency and user satisfaction. The study reveals that the application offers easier
way for users to find goods and services from remote marketplace as well as advertise and sell products without
owning or renting a shop. In this study, 10 participants were used in conducting the usability test with 5 different
tasks. The result of the study revealed that the application is easy to use. However, there were some observed
usability issues with the application. The paper recommends that Mudah.my developers should further improve on
the interface of the mobile application so as to deliver a more usable application that satisfies users’ needs.

020058-5
 REFERENCES

1. M. Prasanna, “E-FRPPHUFHௗ,VVXHVDQGFKDOOHQJHVLQ,QGLDQEDQNV´4 (5), 716–720, (2014).

2. A. Hussain, E.O.C. Mkpojiogu and Z. Hussain, “Usability evaluation of a web-based health awareness portal on
Smartphone devices using ISO 9241-11 model”, Jurnal Teknologi (Sciences & Engineering), 77 (4), 1-5 (2015).
3. A. Hussain and E.O.C. Mkpojiogu, “The effect of responsive web design on the user experience with laptop and
smartphone devices”, Jurnal Teknologi (Sciences & Engineering), 77 (4), 41-47 (2015).
4. B. Saini, “E-Commerce in India - ProQuest,” Int. J. Bus. Manag., 1–5 (2014).
5. S. H. Kim, “Impacts of Mobile Commerce- Benefits, Technological and Strategic Issues and Implementation” J.
Appl. Sci., 6 (12), 2523–2531, (2006).
6. A. Hussain, E.O.C. Mkpojiogu and F. Hassan, “Systematic review of mobile learning applications for children”,
Proceedings of the 2nd International Conference on Information and Communication Technology for
Transformation (IC-ICT4T’16), 5-7 April 2016, Kota Kinabalu, Sabah, Malaysia (5-7 April, 2016).
7. A. Report, “People with Passion,” (2015).
8. E.O.C. Mkpojiogu, N.L. Hashim and R. Adamu, “Observed demographic differentials in user perceived
satisfaction on the usability of mobile banking applications”, Proceedings of the 8th Knowledge Management
International Conference (KMICe’16), Chiang Mai, Thailand, (29-30 August, 2016).
9. A. Hussain, E.O.C. Mkpojiogu and F. Hassan, Usability dimensions and sub-dimensions for the evaluation of
m-learning apps for children: A systematic review. Jurnal Teknologi (Sciences & Engineering).
10. U. Experience, U. Consultant, and E. Limited, Praise for Usability Testing Essentials. (2011).
11. X. Ferré, N. Juristo, H. Windl and L. Constantine, “Usability engineering Usability Basics for,” IEEE Softw., 18
(1), 22–29, (2001).
12. A. Hussain and E.O.C. Mkpojiogu, “Usability evaluation techniques in mobile commerce applications: A
systematic review. Proceedings of the International Conference on Applied Science and Technology
(ICAST’16), Kedah, Malaysia. AIP Conf. Proc. 1761 (1), (2016), pp. 020049.
13. A. Hussain and E.O.C. Mkpojiogu, “An application of ISO/IEC 25010 standard in the quality-in-use assessment
of an online health awareness system”, Jurnal Teknologi (Sciences & Engineering), 77 (5), 9-13 (2015).
14. M. C. Trivedi, “Role Of Context In Usability Evaluations: A Review,” Adv. Comput. An Int. J., 3 (2), 69–78,
(2012).
15. H. Belson and J. Ho, Chapter 2. (2012).
16. E.O.C. Mkpojiogu and N.L. Hashim, “The impact of users’ age, gender, education and experience on their
satisfaction perception of m-banking apps’ usability”, ARPN Journal of Engineering and Applied Sciences.
17. A. Toomey and A. Wysocki, “Distinguishing between Traditional and Online Retailing: Evaluating E-commerce
with Respect to the Food System1,” Edis.Ifas.Ufl.Edu, 1–4 (2009).
18. P. Badar and A. Iqbal, “E-Commerce Vs Mobile Commerece,” 2, 1–24, (2013).
19. A. Hussain, E.O.C. Mkpojiogu and F.M. Kamal, “A systematic review on usability evaluation methods in m-
commerce apps”, Journal of Telecommunication, Electronic & Computer Engineering (JTEC), 8 (10), 29-34
(2016).
20. A. Hussain, E.O.C. Mkpojiogu and F.M. Kamal, “Mobile video streaming applications: A systematic review of
test metrics in usability evaluation”, Journal of Telecommunication, Electronic & Computer Engineering (JTEC),
8 (10), 35-39 (2016).
21. S. Barnes and B. Hunt, “E-commerce and v-business,” (2013).
22. P. P. Parameswari, “M-Commerce in Apps and its security issues,” 2 (5), 25–27 (2015).

020058-6
23. M. Giri and S. Singh, “Issues in Mobile e-FRPPHUFHௗ$VXUYH\´Int. J. Comput. Sci. Inf. Technol., 5 (4), 5068–
5070 (2014).
24. &7HFKQRORJ\$$JDUZDODQG3+%KDWDZDO³0&RPPHUFHLQ,QGLDௗ3URPLVHDQG3UREOHPV´Int. J. Res.
Comput. Commun. Technol., 4 (4), 2013–2016 (2015).
25. R. Yazdanifard and M. Elkhabir, “Mobile Commerce and Related Mobile Security Issues,” Proc. Int. Conf, 9,
198–201 (2011).
26. M. M. Rahman and T. Sloan, “Opportunities and challenges of M-commerce adoption in Bangladesh: An
empirical study,” J. Internet Bank. Commer., 20 (3), (2015).

020058-7