USABILITY EXPERIENCE ON TOURISM
WEBSITE USING THE USE QUESTIONNAIRE
APPROACH
Tirza Madah Pratidina
Magister Informatika
Universitas Atma Jaya Yogyakarta
Yogyakarta, Indonesia 55281
tirzapratidina@gmail.com
Abstract—Tourism is an important component of social and
economic activities in many countries as it creates employment
and business opportunities. Tourism has been revolutionized
with many innovation. Technological innovation in the tourism
2021 International Conference on Electrical Engineering and Informatics (ICEEI) | 978-1-6654-0555-3/21/$31.00 ©2021 IEEE | DOI: 10.1109/ICEEI52609.2021.9611159
industry serves as a means to meet visitor satisfaction in
carrying out tourism activities. One form of the application of
technology in tourism is a travel website. In order to invite many
people to use it, tourism websites need to pay attention to the
usability aspect. It is also important to understand how users
perceive or react to websites. In this study, the author applies
the USE Questionnaire approach, where USE stands for
Usefulness, Satisfaction, and Ease of Use (Ease of Learning and
Ease of Use) to evaluate Indonesian tourism website.
Respondents will be asked to run a scenario that has been
designed by the author and then respondents will be asked to fill
out a questionnaire. The results of this study can be considered
for developing a travel website in terms of usability.
Keywords—Usability, Tourism, USE Questionnaire
I. INTRODUCTION
The tourism sector is one of the largest sectors in the
world. Tourism is an important component of social and
economic activities in many countries as it creates
employment and business opportunities. The tourism industry
continues to grow every year, with tourist arrivals reaching
1.19 billion in 2015, from 528 million in 2005, while it is
expected to grow to over 1.8 billion by 2030 worldwide [1].
Tourism has been revolutionized with tremendous innovation.
Information technology and the development of the internet
have changed the tourism industry and how users interact with
tourism organizations [2].
Technological innovation in the tourism industry serves as
a means to meet visitor satisfaction in carrying out tourism
activities. The application of technology in the tourism sector
can help develop and provide a satisfying tourist experience
for visitors [3]. In the use of tourism websites, users expect
informative, interactive and interesting features [4]. Other
aspects identified in the use of the website are the design,
security, functionality and ease of use of the website [5].
Effective tourism sites can enable users to obtain travel
information [6]. Through such websites, tourism
organizations can offer interactive experiences to their users
[7], while increasing online sales [8]. Usability can affects the
success of a tourism site especially the level of usability and
acceptance and increases user interaction with the tourism
website [9]. Even so, the current condition of the tourism web
still faces problems in the usability aspect and can negatively
affect its use. We also need to understand how users perceive
or react to websites, not only because tourism is an
experience-based product, but also because the Internet is a
XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE
Authorized licensed use limited to: Zhejiang University. Downloaded on May 16,2024 at 02:58:35 UTC from IEEE Xplore. Restrictions apply.
Djoko Budiyanto Setyohadi
Magister Informatika
Universitas Atma Jaya Yogyakarta
Yogyakarta, Indonesia 55281
djoko.budiyanto@uajy.ac.id
means and communication that strongly influences consumer
buying behavior.[10].
This research uses an Indonesian tourism website, namely
goindonesiatour.id [11]. This website offers tourist attractions
in Indonesia, equipped with details of the events that will be
carried out if you choose a tourism place. The method for
usability testing on the website uses the USE (Usefulness,
Satisfaction, and Ease of Use) Questionnaire [12]. We used
this method because while some tools exist to assess software
usability they don't focus on assessing usability across
domains. The USE Questionnaire is a short questionnaire that
can be used to measure the dimensions of usability that are
most important to users, and to measure those dimensions
across domains. The age of the selected respondents was in
the range of 17 to 50 years. The selection of these respondents
was based on statistical data from domestic tourists in 2019
[13] which shows that this age group do travel a lot.
Researchers collected data on respondents' perceptions by
filling out web-based questionnaires after running the
scenarios carried out on the website. This study focuses on the
application of the USE Questionnaire to the analysis of the
usability of tourism websites, especially in Indonesia, while
other studies are still using the ancient usability approach and
have not focused on Indonesian tourism. The purpose of this
research is to help develop travel website in terms of usability.
This research can also obtain direct feedback from consumers
regarding the level of satisfaction of their use of the tourism
website.
II. RELATED WORK
This kind of research has been conducted several times
before. Chiao in 2018 [3] conducting usability research on a
virtual tour guide application. This research applies the
Cultural Tourism Digital Guiding Platform (CTDGP) to
create 3D virtual designs and create games based on places
that are often tourist attractions. Usability analysis in this
paper uses the Unified Theory of Acceptance and Use of
Technology (UTAUT). Researchers collected data on
respondents' perceptions and actual implementation of
CTDGP by filling out web-based questionnaires before and
after running the game. The respondents' data analysis was
carried out by applying correlation analysis, ANOVA
(analysis of variance), multiple regression, structural
equation modeling and path analysis. The result of the test is
that the interaction interface and the student's own experience
of the new technology have a significant impact on students'
willingness to learn, hands-on practice, and the
implementation that students make on the game or platform.
Hendra, et al [14] in 2019 calculate the level of usability
on a website-based student value information system. This
study applies a USE questionnaire in the process of obtaining
user satisfaction data. The usability measurement in this
study uses three independent parameters, namely usefulness,
ease to use, and ease to learn and the dependent variable,
namely Satisfaction. The respondents involved were 25
teachers at the learning institution. The results of the analysis
carried out show that the measurement results in a feasibility
percentage of 75.23% which indicates that the results of the
measurement of the usefulness of the web-based student
value processing information system have a "Eligible" value.
It can also be concluded that partially the usefulness variable
and the ease of use variable have a significant effect on the
satisfaction variable. While the ease of learning variable has
no significant effect on the satisfaction variable. With the
same method, Purwinarko, et al [15] also measures usability
in the university's student assessment system. The usability
measurement have a feasibility percentage value of 85.2%
which indicates that this system is very feasible to use. Then,
there is a significant effect between the independent variables
(usefulness, ease of use, and ease of learning) on the
dependent variable (satisfaction).
Alcantara, et al in 2017 [10] conducted a study on whether
the level of consumer satisfaction when conducting online
searches, consumers' online perceptions of risk, and
perceptions of the usefulness of travel websites can predict
future consumer attitudes towards the destination of a trip to
be taken. In this study, a fictitious website was used
specifically as a research medium. A total of 228 respondents
who took part in this experiment were directly selected by the
researcher who would have the task of completing the task
scenario and filling out a questionnaire about the respondent's
experience when doing the task. To analyze the accuracy of
the predictions produced, the researcher used a regression
model. The results of this study are the higher the level of
consumer satisfaction in searching for tourist attractions
online, the more positive it will be on the choice of tourist
attractions to become consumer tourist destinations.
Lee in 2018 [16] do usability research on an online travel
agency. The main purpose of this study is to determine the
difference between user expectations and the usability value
possessed by an online travel agent and to analyze the reasons
and goals of consumers when visiting an online travel agent.
The dimension analyzed in this experiment is the calculation
of the user's expected level of performance. This experiment
is divided into three parts. First, a survey was conducted to
respondents about the history of purchasing products related
to travel or travel. Then secondly, respondents will be asked
about their reasons and goals for visiting the hotel booking
section related to online travel agents. Finally, respondents
focused more on answering surveys regarding the level of
usability expectations and usability values of the agent's page.
The results of the analysis carried out descriptively, show that
the usability value of the online travel agent page exceeds the
expectations of the respondents.
Chung-sing, et al in 2020 [17] examines the effectiveness
of scenario games in distributing education about ecotourism.
The parameters analyzed in this usability experiment are the
knowledge, attitudes of the respondents, and the level of
usability of the system being analyzed. Researchers evaluate
the scenario game about ecotourism from student respondents
who use the system totaling 35 people with discussion group
respondents as many as 31 people. The results of the
Authorized licensed use limited to: Zhejiang University. Downloaded on May 16,2024 at 02:58:35 UTC from IEEE Xplore. Restrictions apply.
questionnaire were analyzed using a paired-sample t-test. The
level of satisfaction with the game and the discussion
experience is positive, while the level of possibility of
participants to recommend this game to others is low. The
results of the analysis show that the dimensions of knowledge
and usability of the system are positive, but the level of
attitude change in respondents who run the game does not
have a significant difference compared to those who conduct
group discussions.
Lund in 2001 [12] conducted a USE Questionnaire that
have a concept that Ease of Use and Usefulness influence each
other, so an increase in Ease of Use increases the Usefulness
rating and vice versa. And both of them encourage
Satisfaction. Ease of Use is less important when the system is
an internal system that must be used by the user. Users varied
more in their Usefulness ratings when they had only limited
exposure to a product. Satisfaction is closely related to actual
or predicted usage. For internal systems, the factors that
contribute to Ease of Use can be separated into two factors,
there are Ease of Learning and Ease of Use.
III. METHODOLOGY
This research divided into several step that we can see in
Fig. 1.
Fig. 1. Methodology
A. Literature Study
At this stage, the author conducts a study on the topic to
be studied. Literature studies are carried out in accredited
journals.
B. Data Collection
The age of the selected respondents was in the range of 17
to 50 years. The selection of these respondents was based on
statistical data from domestic tourists in 2019 [13] which
shows that this age group do travel a lot. The number of
respondents who participated in this study was calculated
using the factor analysis, the size of the sample depends on
the number of indicators that are minimized multiplied by 5
times [18]. In this case, the minimum number of sample is 80
because there are 16 indicators that will be analyzed.
There are several scenarios that respondents must do
before they start to fill out the questionnaire
• See the list of tour packages offered
o Go to website goindonesiatour.id
o Go to menu where we go
o See each tour package option
• Make a tour booking
o Go to website goindonesiatour.id
o Go to menu contact us
After all of the scenarios are done by the respondents, then
the next step is to fill out the USE Questionnaire [12] in Table
1.
C. Data Analysis
At this stage, data processing will be carried out from the
results of filling out questionnaires by respondents. In this
study, we only used 16 of the 30 items contained in Lund's
original paper. The reason for the difference in these items is
that in the paper itself, Lund said that there were several items
that only have a relatively less strongly effect on the variable
in questionnaire [12]. Based on the statement, we only use
items that have a strong influence on each variable.
Researcher will use SmartPLS (Partial Least Square) as an
analysis tool
TABLE I. USE QUESTIONNAIRE
E. Conclusion
At this stage, an analysis of the results of data analysis is
carried out and then conclusions are drawn
IV. RESULT AND DISCUSSION
The number of respondents who participated in this study
was calculated using the factor analysis [18]. There were a
total of 80 participants who took part in this study, but only
56 questionnaire were valid. All of the respondents are an
undergraduate student between the age of 17 to 50 years old.
The demographic data of my respondent can be seen in Table
2.

Aspect USE TABLE II. RESPONDENT’S DEMOGRAPHIC

Percentage of Respondent (%)
Aspect Usefulness
Male 41.3
1 It helps me be more effective Gender
2 It helps me be more productive Female 58.7
3 It is useful
4 It gives me more control over the activities in my life. Total 100
Aspect Ease of Use
5 It is easy to use
6 It is simple to use The model analysis of the questionnaires can be seen in
7 It is user friendly Figure 3. The next step is to analyze the outer model. From
8 It requires the fewest steps possible to accomplish what I the results obtained that can be seen in Table 3, the resulting
want to do with it.
Aspect Ease of Learning outer loading value is > 0.5 which is in accordance with the
9 I learned to use it quickly validity limit. Next is the Consistency Reliability in the Table
10 I easily remember how to use it. 4 which measures the level of the indicator's ability to
11 It is easy to learn to use it. measure its latent construct [19].
12 I quickly became skillful with it.
Aspect Satisfaction
13 I am satisfied with it.
14 I would recommend it to a friend.
15 It is fun to use.
16 It works the way I want it to work.
D. Hypothesis Test

Fig. 3. Model Analysis

Fig. 2. Model Hypothesis in This Research Using USE
The Hypothesis that conducted in this research in Figure
2. as follows:
H1: Variable Ease of Use (EU) positively influences on
variable Satisfaction (S) on the use of tourism website.
The Composite Reliability value must be above of 0.7
[20]. This can be seen in the Table 4 that all of the Composite
Reliability has a value > 0.7. The Cronbach's Alpha value is
must be above 0.6 and is even expected to be more than 0.7
[21].

H2: Variable Usefulness (U) positively influences on TABLE III. OUTER LOADINGS VALUE
variable Satisfaction (S) on the use of tourism website.
Ease of Use Satisfaction Usefulness
a1 0.901
a2 0.899
a3 0.866
a4 0.791

Authorized licensed use limited to: Zhejiang University. Downloaded on May 16,2024 at 02:58:35 UTC from IEEE Xplore. Restrictions apply.
 b1 0.758 d4 0.525 0.886 0.686
b2 0.849
b3 0.862
b4 0.818 The cross loading value of each construct was evaluated
c1 0.783 to ensure that the correlation of the construct with the
c2 0.820 measurement item was greater than that of the other
c3 0.810
c4 0.810
constructs. The expected cross loading value is greater than
d1 0.853 0.7. In this study, all construct correlations with measurement
d2 0.897 items were greater than other constructs and each cross
d3 0.901 loading value was more than 0.7 as we can see in Table 5.
d4 0.886
Based on the results obtained in Table 4, it can be seen
that the Cronbach's Alpha criteria have been met, so that all TABLE VI. F-SQUARE
the constructs are reliable. Next is to observe the Convergent Satisfaction
Validity section which is evaluated by Average Variance
Extracted (AVE). The AVE value should be 0.5 or more. An Ease of Use 0.053
AVE value of 0.5 or more means that the construct can Usefulness 1.134
explain 50% or more of the item variance [22]. Based on
these conditions, all existing constructs have reached the
convergent validity requirements because the AVE values are Furthermore, the calculation of the Inner Model is carried
all greater than 0.5. out. In the inner model analysis, we can see the relationship
between variables and also we can calculate the R-square
TABLE IV. MEASUREMENT MODEL model. The relationship between variables can be seen in Fig.
Cronbach's rho_A Composite Average 3. Where the R-square value for the Satisfaction variable is
Alpha Reliability Variance 0.693. This shows that the Ease of Use variables and
Extracted Usefulness variables affect the Satisfaction variable by
(AVE)
Ease of Use 0.928 0.937 0.940 0.663 69.3%. Then we do the calculations with the F-square model.
The results of this calculation that can be seen in Table 6
Satisfaction 0.907 0.909 0.935 0.782
show how much influence one variable has on other
Usefulness 0.888 0.900 0.922 0.749 variables. The Usefulness variable has a large influence on
the Satisfaction variable because it has a value of 1.134
(greater than 0.35) and the Ease of Use variable has a small
The next step is to measure discriminant validity.
effect on the Satisfaction variable with a value of 0.053
Discriminant validity determine whether a reflective
(greater than 0.02 and less than 0.15).
indicator is really a good measure of its construct based on
The next step is to test the hypothesis. A hypothesis will
the principle that each indicator must be highly correlated
be accepted if it has a T-statistic value greater than 1.96 [22].
with its construct only. Cross-loading is another method to
Based on Table 7, H1 is unaccepted because it has a T-
determine discriminant validity, namely by looking at the
statistic value of less than 1.96, so the Ease of Use variable
value of cross loading [22].
does not positively influence Satisfaction on tourism
TABLE V. CROSS-LOADING websites. This shows that the level of ease in using the
tourism website has no effect on user satisfaction in using the
Ease of Use (b, c) Satisfaction (d) Usefulness (a)
tourism website. While H2 is accepted because it has a T-
a1 0.552 0.723 0.901 statistic value of more than 1.96, so the Usefulness variable
a2 0.374 0.775 0.899 positively influences the Satisfaction on tourism website
variable. This shows that the level of usefulness in the use of
a3 0.637 0.758 0.866
tourism websites has a large effect on user satisfaction in
a4 0.486 0.565 0.791 using tourism websites.
b1 0.758 0.407 0.489
TABLE VII. T-STATISTIC
b2 0.849 0.463 0.478
No Hypothesis T Sample Result
b3 0.862 0.555 0.616 Statistics Mean
(M)
b4 0.818 0.595 0.578
1 EU -> S (H1) 1.468 0.169 Unaccepted
c1 0.783 0.434 0.437 2 U -> S (H2) 8.377 0.723 Accepted
c2 0.820 0.335 0.363
c3 0.810 0.417 0.336 Judging from the results obtained, Usefulness plays a
major role in the level of user satisfaction. This shows that the
c4 0.810 0.521 0.452
level of usability of a tourism website is something that must
d1 0.622 0.853 0.734 be considered more in developing a tourism website.
d2 0.521 0.897 0.788
Developers should be able to pay more attention to the
usability aspect of the website. While ease of use only has a
d3 0.398 0.901 0.691 very small level of influence on the level of user satisfaction
with a website. User satisfaction does not pay too much

Authorized licensed use limited to: Zhejiang University. Downloaded on May 16,2024 at 02:58:35 UTC from IEEE Xplore. Restrictions apply.
attention to the level of ease of using the website and the level
of ease of learning the website. This gives instructions to
developers so that in making a tourism website they must
focus more on the level of usability of the website compared
to the level of ease of use of the website.
V. CONCLUSION
The main purpose of this research is to help develop travel
websites in terms of usability. By looking at the results of this
study, it was found that the ease of use of a tourism website
has no effect or can be considered to have a very small effect
on the level of satisfaction of tourism website users, because
it has a T-statistic value of less than 1.96. Users are more
satisfied with a tourism website if the website has good
usability and is in accordance with what users expect. This can
be taken into consideration for tourism website developers to
pay more attention to the usefulness aspect when developing
a tourism website. This usability research can be applied not
only to tourism websites, but can also be applied to websites
that manage the needs of other fields. The limitations of this
study can be seen in the reduction in the amount of data that
can be processed, due to the invalidity of the contents of the
questionnaire belonging to several respondents. To minimize
the reduction in this number, several ways are needed to be
able to maintain the number of respondents recorded.
ACKNOWLEDGMENT
This research is supported by Magister Informatika,
Universitas Atma Jaya Yogyakarta.
REFERENCES
[1] A. Kontogianni and E. Alepis, “Smart tourism: State of the art and
literature review for the last six years,” Array, vol. 6, no. February,
p. 100020, 2020, doi: 10.1016/j.array.2020.100020.
[2] Z. Huang, “Usability of tourism websites: a case study of heuristic
evaluation,” New Rev. Hypermedia Multimed., vol. 26, no. 1–2, pp.
55–91, 2020, doi: 10.1080/13614568.2020.1771436.
[3] H. M. Chiao, Y. L. Chen, and W. H. Huang, “Examining the
usability of an online virtual tour-guiding platform for cultural
tourism education,” J. Hosp. Leis. Sport Tour. Educ., vol. 23, no.
August 2017, pp. 29–38, 2018, doi: 10.1016/j.jhlste.2018.05.002.
[4] T. Albayrak, S. Karasakal, Ö. Kocabulut, and A. Dursun,
“Customer Loyalty Towards Travel Agency Websites: The Role
of Trust and Hedonic Value,” J. Qual. Assur. Hosp. Tour., vol. 21,
no. 1, pp. 50–77, 2020, doi: 10.1080/1528008X.2019.1619497.
[5] M. M. Jeon and M. Jeong, “Influence of Website Quality on
Customer Perceived Service Quality of a Lodging Website,” J.
Qual. Assur. Hosp. Tour., vol. 17, no. 4, pp. 453–470, 2016, doi:
10.1080/1528008X.2015.1127193.
[6] R. Tavakoli and S. N. R. Wijesinghe, “The evolution of the web
and netnography in tourism: A systematic review,” Tour. Manag.
Perspect., vol. 29, no. August 2018, pp. 48–55, 2019, doi:
10.1016/j.tmp.2018.10.008.
Authorized licensed use limited to: Zhejiang University. Downloaded on May 16,2024 at 02:58:35 UTC from IEEE Xplore. Restrictions apply.
[7] G. Wu, “Official websites as a tourism marketing medium: A
contrastive analysis from the perspective of appraisal theory,” J.
Destin. Mark. Manag., vol. 10, no. September, pp. 164–171, 2018,
doi: 10.1016/j.jdmm.2018.09.004.
[8] S. Liasidou, “Representation of cultural tourism on the Web:
critical discourse analysis of tourism websites,” Int. J. Cult. Tour.
Hosp. Res., vol. 12, no. 3, pp. 327–347, 2018, doi:
10.1108/IJCTHR-07-2017-0078.
[9] A. M. Martínez-Sala, J. Monserrat-Gauchi, and D. Alemany-
Martínez, “User Usable Experience: A three-dimensional
approach on usability in tourism websites and a model for its
evaluation,” Tour. Manag. Perspect., vol. 33, no. August 2019, p.
100579, 2020, doi: 10.1016/j.tmp.2019.100579.
[10] J. M. Alcántara-Pilar, F. J. Blanco-Encomienda, T. Armenski, and
S. Del Barrio-García, “The antecedent role of online satisfaction,
perceived risk online, and perceived website usability on the affect
towards travel destinations,” J. Destin. Mark. Manag., vol. 9, no.
September, pp. 20–35, 2018, doi: 10.1016/j.jdmm.2017.09.005.
[11] Politarius, “Go Indonesia Tour.” goindonesiatour.id.
[12] A. M. Lund, “Measuring usability with the USE questionnaire,”
Usability interface, vol. 8, no. 2, pp. 3–6, 2001.
[13] B. P. Statistik, Statistik Wisatawan Nusantara 2019. Jakarta: BPS-
Statistics Indonesia, 2020.
[14] S. K. Hendra and S. K. Yulyani Arifin, “Web-based Usability
Measurement for Student Grading Information System,” Procedia
Comput. Sci., vol. 135, pp. 238–247, 2018, doi:
10.1016/j.procs.2018.08.171.
[15] A. Purwinarko, M. Subagja, and A. Yanuarto, “The Evaluation of
Final Assignment System Using the USE Questionnaire
Approach,” vol. 7, no. 2, pp. 3–10, 2020.
[16] K. W. Lee, “An Investigation of The Online Travel Agencies’
Usability,” Culin. Sci. Hosp. Res., vol. 24, no. 2, pp. 44–50, 2018,
doi: 10.20878/cshr.2018.24.2.006006006.
[17] C. Chung-Shing, C. Yat-hang, and F. Tsz Heung Agnes, “The
effectiveness of online scenario game for ecotourism education
from knowledge-attitude-usability dimensions,” J. Hosp. Leis.
Sport Tour. Educ., vol. 27, no. August, p. 100264, 2020, doi:
10.1016/j.jhlste.2020.100264.
[18] R. U. I Gusti Bagus, Teknik Sampling dan Penentuan Jumlah
sampel. 2016.
[19] M. A. Memon, R. Sallaeh, M. N. R. Baharom, S. Md Nordin, and
H. Ting, “The relationship between training satisfaction,
organisational citizenship behaviour, and turnover intention: A
PLS-SEM approach,” J. Organ. Eff., vol. 4, no. 3, pp. 267–290,
2017, doi: 10.1108/JOEPP-03-2017-0025.
[20] J. Hair, C. L. Hollingsworth, A. B. Randolph, and A. Y. L. Chong,
“An updated and expanded assessment of PLS-SEM in
information systems research,” Ind. Manag. Data Syst., vol. 117,
no. 3, pp. 442–458, 2017, doi: 10.1108/IMDS-04-2016-0130.
[21] J. F. Hair, M. C. Howard, and C. Nitzl, “Assessing measurement
model quality in PLS-SEM using confirmatory composite
analysis,” J. Bus. Res., vol. 109, no. August 2019, pp. 101–110,
2020, doi: 10.1016/j.jbusres.2019.11.069.
[22] M. Sarstedt, C. M. Ringle, and J. F. Hair, Handbook of Market
Research, no. September. 2020.