Validity and reliability of end-user computing satisfaction in Indonesian language to

measure digital nursing assessment tool

Intansari Nurjannah, and Sri Warsini

Citation: AIP Conference Proceedings 1755, 070001 (2016); doi: 10.1063/1.4958496

View online: https://doi.org/10.1063/1.4958496
View Table of Contents: http://aip.scitation.org/toc/apc/1755/1
Published by the American Institute of Physics

Articles you may be interested in

Modified relevance frequency feature weighting for email spam classification
AIP Conference Proceedings 1755, 070002 (2016); 10.1063/1.4958497

Effect of modification ZSM-5 catalyst in upgrading quality of organic liquid product derived from catalytic cracking
of Indonesian nyamplung oil (Calophyllum inophyllum)
AIP Conference Proceedings 1755, 050002 (2016); 10.1063/1.4958485

Effect of dengue virus infection on the permeability of vero cells line

AIP Conference Proceedings 1755, 030003 (2016); 10.1063/1.4958474

Phantom image dimension analysis on computed tomography image

AIP Conference Proceedings 1755, 020004 (2016); 10.1063/1.4958469

Tunneling 6LoWPAN protocol stack in IPv6 network

AIP Conference Proceedings 1755, 070006 (2016); 10.1063/1.4958501

The complexity calculation for group decision making using TOPSIS algorithm
AIP Conference Proceedings 1755, 070007 (2016); 10.1063/1.4958502
Validity and Reliability of End-User Computing Satisfaction
in Indonesian Language to Measure Digital Nursing
Assessment Tool
Intansari Nurjannah1, a, Sri Warsini1,b
1
School of Nursing Faculty of Medicine Gadjah Mada University, Indonesia

a)
Corresponding author:intansarin@ugm.ac.id
b)
sri.warsini@ugm.ac.id

Abstract. Background. Rapid technological advances have affected health sector, one of which is the presence of digitizing
system in terms of health care documentation, which is known as the Electronic Health Record (EHR). In addition, there
are various software or applications which are developed to facilitate users to perform their duties in the health sector. One
of the applications of digital assessment system which has been developed is the application Intan’s Screening Diagnoses
Assessment (ISDA) mobile version that can be used by nurses to enforce nursing diagnosis and collaborative diagnosis.
User satisfaction with an application or software can be assessed by using the End-User Computing Satisfaction. This
instrument is designed not only for users who directly interact with the computer, but can also be used on a computer
replacing gadget such as a mobile phone. However, this instrument is still in English, and no one has measured validity
and reliability in the Indonesian language. Aims. This study is at identifying the instrument’s validity and reliability of
End-User Computing Satisfaction which is used to measure user satisfaction with the ISDA mobile version prototype 1, in
relation to content, accuracy, easy to use and timeliness of the application of ISDA mobile version prototype 1. Methods.
This study is a psychometric study which is used to measure the validity and reliability of the instrument of End-user
Computing Satisfaction that is modified to measure the application of ISDA mobile version prototype I. The instrument of
the End-User Computing Satisfaction was translated and modified in Indonesian language by the researchers according to
the needs related to the application of ISDA mobile version and also consulted with nurse lecturers and Information and
Technology (IT) experts. Respondents of this study were nurse practitioners who received nursing diagnosis training in
June 2015 numbering 58 people. The validity of this instrument was analyzed using the Pearson product-moment
correlation. The reliability in this study was assessed the instrument’s internal consistency using Cronbach’s Alpha. Results.
The results indicated that Indonesian version of End-User Computing Satisfaction was valid and reliable to measure user
satisfaction of ISDA mobile in term of content, accuracy, easy to use, and timeliness. The result revealed that the
Cronbach’s alpha of 0.957 for the total score. Conclusion. The End-User Computing Satisfaction can be considered a
reliable and valid tool for assessing the ISDA mobile application’ user satisfaction in Indonesia.

INTRODUCTION
Technological advances can be utilized to improve the quality of health services. One of the uses of such
technology is various applications of information systems that will help healthcare workers to improve their quality
of service. This information system will assist in the data collection that will eventually establish the Evidence-Based
Nursing (EBN). EBN is one of the factors that affect clinical decision making by nurses [1].
The improvement of EBN is necessary for healthcare personnel. The establishment of EBN is greatly influenced
by the documentation system performed by nurses who are part of the health information system. Until the present
time, the documentation systems in hospitals are generally still manual. As a consequence, it is difficult to analyze
documents in order to establish an EBN. Therefore, a documentation digitizing process is needed, especially to
determine nursing diagnosis. The literature generally mentions the digitization of health information system as the
Electronic Health Record (EHR) [2].

Advances of Science and Technology for Society

AIP Conf. Proc. 1755, 070001-1–070001-5; doi: 10.1063/1.4958496
Published by AIP Publishing. 978-0-7354-1413-6/$30.00

070001-1
 The digitization of nursing documentation should refer to international standards. One of oldest standardizations
in the nursing diagnoses labels is the North American Nursing Diagnoses Association-International (NANDA-I)
which has been used as a Nursing Standard Language (SNL) in the majority of countries throughout the world
[3].Currently, NANDA has been translated into Indonesian (by EGC publisher) and has even been used by some
academics and clinicians but its use is still manual. Therefore, digitization program is deemed important and should
be done immediately so that the nurses would be able to access more easily and use or apply the process of NANDA-
based diagnosis enforcement more easily.
The enforcement of nursing diagnosis is not an easy process, especially there are a lot of symptoms and signs that
need to be considered by nurses to enforce nursing diagnosis and collaborative diagnosis. One of the processes in
diagnosis begins with nursing assessment. The nursing assessment is the process of obtaining data as a consideration
to enforce nursing diagnosis. There are various frameworks for organizing the assessment process. To date, there have
been various digital assessment methods, such as Nursescope and Intan's Diagnoses Screening Assessment (ISDA).
ISDA [4] has been made in the form of a mobile version. ISDA can be accessed through the app store and installed on
a mobile phone. Since its introduction in June 2015 until September 2015 it has been downloaded over 1000 times.
In response to the tremendous attention to this software, it is necessary to evaluate the satisfaction of users of this
software. However, currently there is no instrument in Indonesian whose validity and reliability have been measured
in order to measure the satisfaction of the use of ISDA mobile version prototype I. One of the instruments to measure
user satisfaction related to specific applications is the End-User Computing Satisfaction [5]. This instrument is in
English and it has been tested for its validity and reliability however still in English [5]. Validity and reliability are
the key in measurement the quality of instrument [6]. There are several methods to determine the reliability of
assessment. They are test-retest, Equivalent-Forms Method, Split-Half Method, coefficient alpha r, Inter-rater
Consistency/inter-rater reliability [7]. This study was conducted to identify the validity and reliability of the instrument
of the End-User Computing Satisfaction which has been adapted to the ISDA mobile version prototype I in regard to
the content, accuracy, easy to use and timeliness.

MATERIAL AND METHODOLOGY

This study is a psychometric research aimed at identifying the validity and reliability of the End-User Computing
Satisfaction translated into Indonesian and used in the application of ISDA mobile version prototype I. The content
analysis was conducted on the results of the translation by means of translation by consulting this instrument to one
colleague who had understood the process of diagnosis in nursing and an expert in Information and Technology (IT).
Respondents of this study were 58 nurses who participated in the two-day training regarding how to diagnose.
Content validity has been applied to the End-User Computing Satisfaction in Indonesian by asking lecturer in
nursing school who has the ability to diagnoses. The items of the End-User Computing Satisfaction were modified to
this instrument before the reliability was measured. Data were collected on the second day after ISDA mobile version
prototype I was introduced to participants. After the respondent had tried this application, then they were given a
questionnaire of End-User Computing Satisfaction in Indonesian.
The instrument of this study was a questionnaire of End-User Computing Satisfaction which has translated into
Indonesian. The instrument of End-User Computing Satisfaction consists of 18 questions. The number of items is
similar to the original instrument, i.e. 18 items. These items measure several things including content, accuracy,
format, ease of use, time of use of the application5.
The translation process of this instrument was not through the process of translation and back-translation, and this
is a weakness of this study. Although back-translation process was not performed, the whole item of this instrument
is valid and reliable. This may be because the items in this instrument can be considered simple so that it does not
require a lot of modifications in its translation into Indonesian. It should be kept in mind that this instrument is just
used to measure the ISDA mobile version which is the prototype of I. Currently, the prototype II has been completed
and in the near future instruments are ready to evaluate the application of ISDA mobile version prototype II.
The data were processed by using SPSS version 20. Descriptive statistics was conducted to identify the
demographic characteristics of the respondents. The validity was analyzed using the product-moment correlation. The
reliability in this study was assessed using by the internal consistency, namely one-time instrument testing. This study
employed method of analysis using Cronbach’s Alpha.

070001-2
 RESULTS AND DISCUSSION

Demographic characteristics of the respondents

The majority of participants were women (95%) aged 25-57 years with a standard deviation of 7.18. All of the
respondents were is nurse practitioners. The range of participants works experience were from one year to 32 years.
The majority of participant’s work experience are in medical surgical area (72% participants). The rest are working
on gerontology, radiology, pediatric, maternity and psychiatric area.

Results of measurement
Internal consistency and Reliability

The Cronbach’s alpha coefficient for the Indonesian End-User Computing Satisfaction total score was 0.957
suggesting very good internal consistency. The Cronbach alpha for The Indonesian End-User Computing Satisfaction
dimensions ranged from 0.813 to 0.885. The results of Pearson’s correlation coefficients between the dimensions were
all significant at 0.01 level (Table 1)
TABLE 1.Pearson’s correlation coefficients between the Indonesian version of End-User Computing Satisfaction dimensions
based on the survey results of 58 nurses in Indonesia, June 2015
Dimension of End User Computing Cronbach’s
Content Accuracy Format Easy to use Timeliness Alpha
Content 0.860
Accuracy 0.819** 0.885
Format 0.768** 0.760** 0.855
Easy to use 0.724** 0.711** 0.751** 0.871
Timeliness 0.690** 0.651** 0.794** 0.759** 0.813
** Significant

Validity

The results of the inter-item correlation analysis indicated that all questions had a significant correlation with r
values ranged from 0.593 to 0.832. Therefore all items in the instrument can be considered valid (r> 0.254). From the
above results, it can be seen that the entire items were valid and had a high-reliability value. These results also indicated
that the items generally show that this measurement can be used for all types of applications like previously been
studied by Doll and Torkzadeh [5].

Discussion
Electronic health record basically is documentation for clinical purposes. Documentation can be in the manual of
digital form. In order to measure whether this documentation is considered good or poor, there are several methods.
In all method, the instrument is needed for measurement. The advance of technology in health system also forces
health professional to use electronic documentation which usually also related to the software application.
The instrument is needed to measure whether the software application is applicable to be used by health
professional. The instrument can be divided a general or specific instrument. Specific instrument mostly related to
how to measure in specific field, for example in nursing documentation. Even though this is the case, there is also a
general instrument which can be used to measure the more general application of software or documentation in the
health system. This research focused on psychometric testing on general instrument but is used to measure specific
software in the nursing field.
The result shows that this general instrument is valid and reliable to measure software in the nursing field. High
reliability of this instrument may be because of several factors with are: respondents, rater, the atmosphere in data
collection process, the instrument and how to analyze data [8]. In the process of data collection, respondents were
learning about a new technology and had a good spirit to find out something new. The instrument also has been
consulted to an expert of IT person in order to find out ‘fit’ term that generally used by IT user. This term needs to be

070001-3
something that is familiarized by IT user. Although the result of this study shows promising result, there was also
possibility of respondents did not pay attention to each question of this questionnaire and they only wanted to fill the
form as soon as possible [9].
Generally, the result of this study is consistent with other study related to developing an instrument to measure its
validity and reliability [9-11]. The different of this study with another study was related to setting, the type of
instrument and the type of respondents. One of a study conducted by Paulina [10] was developing an instrument to
measure user satisfaction toward electronic health record. This study, however, has been conducted in a longer time
than this study. The different setting also has been used to collect data, as well as different type of respondents. Health
professionals and also graduate students were subject to this study. Health professional were required to fill the survey
about electronic health record that has been applied about three years in their institution.
Another study was conducted by Ning [12] who is developing an instrument to measure a specific field that is in
nursing documentation. The instrument of this study was called QANDAC. Ning [12] claimed that this instrument
could be applied both in manual or electronic documentation to find out the system capability of the documentation.
This instrument however, locally developed based on the need in aged care homes. Even though the result of this study
showed that QANDAC is valid and reliable instrument, but this instrument may only fit to be applied in specific
setting.
Nurses’ satisfaction related to electronic medical records in hospitals is also the topic of research in this field.
Mehmet [11] collected data from 487 nurses with response rate as much as 74.92% to develop instrument related to
this topic. The result showed that this instrument has good validity and reliability to evaluate effectiveness and
satisfaction in using electronic medical system [11]. However, this instrument did not measure specific item for
example content, accuracy, format, easy to use and timeliness.
One research on a big scale has been conducted by Ochieng [13] which involved 1,666 nurses from 42 hospitals
in Japan in 2006. The result of this research shows that ‘use’ and ‘quality’ construct correlated with ‘user satisfaction’.
Ochieng [13] claimed that instrument could be used in a measure the quality of the electronic medical record.
The limitation of this study is that there is still subjectivity the item in the instrument. Meanwhile respondents also
only have about 2 hours opportunity to familiar with the software which may influence how they fill the questionnaire.
Sample size also very small in this study. However, this instrument has additional value as it is not only focused on
specific setting and it can be used in more general software not just in the nursing field.

CONCLUSION
The End-User Computing Satisfaction can be considered a reliable and valid tool for assessing the ISDA mobile
application’ user satisfaction in Indonesia. This tool will contribute to measuring nursing application in the future,
regarding content, accuracy, format, easy to use and timeliness of this tools.This instrument has a simple and general
item in the content and it can be used not just for nursing application software but it may also be used for general
software in the different field.

REFERENCES
1. N. Cullum, D.Ciliska, R. B. Hayne, and S. Marks, Evidence-based Nursing An introduction
(Blackwell Publishing, UK, USA, Australia, 2008).
2. K. Hayrinen, K. Santo, and P.Nykanen, International Journal of Medical Informatics 77, 291-304 (2008).
3. T.H. Herdmanand, and E. Kamitsuru, NANDA International Nursing Diagnoses: Definition & Classification,
2015-2017 (Wiley Blackwell, Oxford, UK, 2014).
4. I. Nurjannah, S. Warsini and J. Mills, Journal of Nursing and Health Care 1, 89-96 (2013).
5. W. J. Dolland, and G. Torkzadeh, MIS Quarterly 12, 259-274 (1988).
6. V.A. Scholtes, C.B Terwee, and R.W Poolman, Int J. Care Injured 42, 236-240 (2011)
7. M. Miller. R. Linn. N. Gronlud. Measurement and Assessment in Teaching (Pearson, United States of America,
2008).
8. J. Weiner, Measurement: Reliability and Validity Measures (Johns Hopkins University. United States, 2007).
9. J. Everson, S-YD. Lee, and CP.J Friedman, Am Med Inform Assoc 21, 257-263 (2014).
10. S. S. Paulina, P.W. Jonathan, B. Kathryn, and P.L Harold, Comput Inform Nurs 29, 574-585 (2011).
11. T. Mehmet, Y. Ali, K. Erdem, G. O. Ochieng, S. Melahat, B. Sevgi, and T. Sumbul, J Med Syst 39, 67 (2015).

070001-4
12. W. Ning, B. Catrin, H. David, and Y. Ping, Australasian Journal of Ageing 33, 18-24 (2014).
13. O.G. Ochieng, T. Hinako, A. Motohiro, K.P Masako, and N. Keiko, Journal of Advanced Nursing 60, 209-219
(2007).

070001-5