See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/298807422

Assessment of the hospital information system in compliance with Certification

Commission for Healthcare Information Technology standard at Isfahan
University of Medical Sciences'...

Article  in  International Journal of Educational and Psychological Researches · January 2016

DOI: 10.4103/2395-2296.178866

CITATION READS

1 1,311

3 authors, including:

Maryam Jahanbakhsh Asghar Ehteshami

Isfahan University of Medical Sciences Isfahan University of Medical Sciences
40 PUBLICATIONS   243 CITATIONS    34 PUBLICATIONS   181 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Article Developing the Medication Reminder Mobile Application Seeb View project

health information technology View project

All content following this page was uploaded by Asghar Ehteshami on 25 March 2017.

The user has requested enhancement of the downloaded file.

[Downloaded free from http://www.ijeprjournal.org on Saturday, March 11, 2017, IP: 176.102.234.146]

Original Article

Assessment of the hospital information system

in compliance with Certification Commission for
Healthcare Information Technology standard at
Isfahan University of Medical Sciences’ academic
hospitals
Maryam Jahanbakhsh, Ashgar Ehteshami1, Abbas Shafiei2
Health Management and Economic Research Center, 1Health Information Technology Research Center, 2Department of Health Information
Technology and Management, School of Health Management and Information Sciences, Isfahan University of Medical Sciences, Isfahan, Iran

Address for Correspondence:  and technical management areas in compliance with CCHIT standard were 66.1, 63.5, and 64.6 percent,
Dr. Asghar Ehteshami,
Assistant professor,
Health Information Technology
Research Center, Isfahan
University of Medical Sciences,
Isfahan, Iran.
E-mail: ehteshami@mng.mui.
ac.ir
ABSTRACT
Introduction: Hospital information system (HIS) plays an important role in improving the quality
of treatment in hospitals. These systems are considered as an information system that can have a
great influence on the efficiency and effectiveness of hospital management. Assessment system as
one of the most important debates in the hospital management system has received great emphasis.
The present study was aimed to assess HISs of Isfahan academic hospitals and was performed using
Certification Commission for Health Information Technology (CCHIT) standard. Methods: This study
is an applied descriptive cross‑sectional study, in which 11 academic HISs of Isfahan University of
Medical Sciences were surveyed by using a CCHIT standard checklist with three areas (information
management, patient management, and technical management). Obtained data were analyzed with  SPSS
software V. 18 (IBM), and descriptive statistics was used to examine measures of central tendencies.
Results: Data analysis revealed that the mean score of information management, patient management,
respectively. Conclusions: According to the results of this study, it can be expressed that the criteria
of HIS has been implemented in a rather favorable way. Hence, to achieve the desired mode, planning
must be done on designing systems to increase the efficiency of information management, patient
management, and software and technical infrastructures.
Key words: Academic hospital, assessment, Certification Commission for Health Information
Technology, hospital information system

Introduction information, which healthcare personnel require them for

Considering the mass volume of producing information
and its diversity, health care structure, especially health
centers is certainly needed information, communication,
and science technologies and the product of this knowledge
and technology is the creation of hospital information
system (HIS).[1] A HIS plays an effective role in improving
quality and increase of efficiency, effectiveness, and
safety of services.[2] Its purpose is the management of
Access this article online
their tasks.[3] It also provides opportunities for universal
access to health care services and medical knowledge
and in a simple sense it is a help to improve quality and
give efficient health care to patients, which at first was
advertent to the patient and then better management of
This is an open access article distributed under the terms of the Creative
Commons Attribution‑NonCommercial‑ShareAlike 3.0 License, which allows
others to remix, tweak, and build upon the work non‑commercially, as long as the
author is credited and the new creations are licensed under the identical terms.

Quick Response Code:

For reprints contact: reprints@medknow.com
Website:
www.ijeprjournal.org
How to cite this article: Jahanbakhsh M, Ehteshami A, Shafiei A.
Assessment of the hospital information system in compliance with
DOI: Certification Commission for Healthcare Information Technology standard
at Isfahan University of Medical Sciences’ academic hospitals. Int J Educ
10.4103/2395-2296.178866
Psychol Res 2016;2:105-10.

© 2016 International Journal of Educational and Psychological Researches | Published by Wolters Kluwer - Medknow 105
[Downloaded free from http://www.ijeprjournal.org on Saturday, March 11, 2017, IP: 176.102.234.146]
Jahanbakhsh, et al.: Healthcare Information Technology
medical care, nursing care, and support are of this type
of health care service.[4] The majority of HISs is made in
a gradual process, commensurate to the primary needs of
the organization instead of being made in a planned and
controlled process.[5] Hence, before implementing these
systems, logical development of information systems in
accordance with the needs of health services at all levels
must be addressed.[1] In this regard, there are four main
steps for the creation and development of HISs, including
the definition of objectives, design, implementation, and
evaluation.[6] Therefore, evaluation is an ongoing process
in establishing a HIS with comprehensive framework
and this system includes all various sections and groups
and meet all the problems of organization,[7] and its
purpose is to ensure that system activity is consistent
with expectations.[8] According to Marashi’s finding, the
importance of reporting criteria and required outputs
(57 percent), prevention of errors (52 percent), technical
knowledge (59 percent), technical and educational
documentation (43 percent), and interactions of doctors
with system stated that 2.28 percent of users are dissatisfied
with HISs in Iran while the satisfaction of users influences
on productivity,[9] return of capital, and acceptance of
the system.[10] In the HISs of Iran’s hospital 2.46 percent
criteria related to the suitability of software to customize it
according to the demands and expectations, have not been
met also, the suitability of the software for performing
tasks of users, software capacity to be controlled by
the user, software fallibility and self‑descriptiveness of
software 31.4, 30, 30.6, 32.2 percent, respectively, does
not comply the standard.[7] Furthermore, in the evaluation
of implementation of HISs, human aspects, training and
informing must be considered and user’s abilities must
be used for improving quality of information, patient
treatment,[11] and setting of technical standards.[12] Jeremy
in 2003 on the evaluation of health information systems
concluded that often inefficiency and lack of flexibility of
these systems causes information to be useless, so internal
and external evaluations that will be preferably useful for
both of these cases causing changes and remove defects
in the content of information systems.[13] Hamborg et al.
in 2004, in a research entitled assessment of the HIS,
using isometric measures has referred to support of health
care employees, software compliance with the users’ need
considering performance improvement, prevention of
errors, interferences, and reduction of cost.[14] Isfahani
et al. mentioned that observing standards of electronic
registration of narcotics, drug interactions, and allergies
were far acceptable results and ignoring system in support
of drug management and drug interactions have been
observed.[15] Raeisi et al. stated that observing criteria in
the regional information system in comparison with the
criteria of WHO is not desirable which adversely has
106 International Journal of Educational and Psychological Researches / Vol 2 / Issue 2 / April-June 2016
affected the management of information system and health
care.[16] Ahmadi and Shahmoradi in a research entitled
six‑step process evaluation of health information system
concluded that evaluators should be completely aware of
all steps of evaluation.[17] Saeedbakhsh concluded that HIS
developers must pay attention to factors reducing user’s
learning time, providing better and more appropriate
descriptions, providing more appropriate training,
designing software in a way that user does not need much
details to use it, easy ways to use the software command
in the evolution of software.[6] Ahmadi et  al. study,
29.5 percent of users, mentioned that lack of automated
monitoring of data by software is the cause of poor
quality of information system and its non‑coordination
with the needs of users of HIS. [18] Furthermore, in
the evaluation of implementing HIS human aspects,
education, and informing must be considered and users’
abilities in implementing HISs must be used in future
to improve information quality care and treatment
of patients.[11] Certification Commission for Health
Information Technology (CCHIT) is an independent,
nonprofit licensing organization for electronic medical
records (EMR) of patients and includes a set of standard
criteria for health information systems regarding systems’
interaction, safety, security, and performance of HIS
that most doctors by the existence of standard criteria
feels more confident about improvement of health
information systems in the future. CCHIT has provided
more confidence in the markets of electronic health
records for patients that formerly there was no such
confidence.[19]
With regard to the development of information and
communications technologies in health systems, Changes
in the health system softwares is inevitable, therefore to
avoid redoing and to help improve the quality of care
and decrease cost, ongoing evaluation of HISs must be
made[20] to determine whether we have been able to achieve
predetermined results or not? And whether the HIS project
has been completed or not? And in fact, we can recognize
past mistakes and avoid them by using systems evaluation[1]
because in the health care system trial and error method is
wrong and irrecoverable along with imposing additional cost
on organization and before the occurrence of any mistake,
deficiencies and problems must be identified and necessary
activities must be taken to resolve them. Therefore, by
continuous evaluation of the system and using its feedback
to the organization, irreversible errors can be prevented and
by identifying the strengths and weaknesses of the system and
removing its deficiencies system quality can be improved.
The aim of this study was to assess the information
systems of academic hospitals in Isfahan based on
[Downloaded free from http://www.ijeprjournal.org on Saturday, March 11, 2017, IP: 176.102.234.146]
Jahanbakhsh, et al.: Healthcare Information Technology
standard of Licensing Commission of Health Information
Technology that this assessment was done based on three
areas information management, patient management,
and technical management to determine the rate of
conformity of studied systems with three specified
standard areas.
Methods
This research is an applied descriptive cross‑sectional
study, in which HIS in 11 academic hospitals of Isfahan
University of Medical Sciences (including Shahid Beheshti,
Shahid Chamran, Noor, Ali‑Asghar, Imam Mousa‑Kazem,
Al‑Zahra, Kashani, Feyz, Sayedoshohada, Farabi, and
Amin) were surveyed by using a CCHIT standard checklist
which contains 41 criteria that include 301 elements.
Structural and content validity of the checklist was
confirmed by academic members of the Isfahan University
of Medical Science. Data were collected by returning to
academic hospitals of the Isfahan University of Medical
Science, observation of HISs, interview and completing
the checklist. In this study, sampling is not done, and all
of the 11 HISs implemented in the academic hospitals
were assessed. Obtained data were analyzed with  SPSS
software V.18 (IBM) and descriptive statistics was used to
examine measures of central tendencies.
Results
From the total of 11 examined hospitals, four different
systems of Pooyasamane Diva, Sayan Rayan, Kowsar, and
Rayavarane‑Toos were studied. Generally, the results of the
study have been presented in Tables 1 and 2. The mean rate
of compliance of HIS of the Isfahan University of Medical
Science based on the standard of Licensing Commission
for Health Information Technology was 64.6 percent.
The highest rate of compliance with these criteria is
related to information management (66.1 percent), and
the lowest rate of compliance with these criteria is related
to patient management (63.5 percent). Furthermore,
the rate of compliance with the criteria at technical
management was calculated as 64.6 percent, and the
total mean compliance of assessment criteria in HIS was
calculated as 64.7 percent [Table 1].
Based on the findings outlined in Table 2, among
criteria of information management area identification
and maintenance of the patient’s profile with mean
87.7 percent, criteria of recording incoming documents
with mean 84.8 percent, data and outputs reporting
criteria with mean 80 percent compared with other criteria
had the highest mean values. Among these, the lowest
rate of compliance of information management criteria
International Journal of Educational and Psychological Researches / Vol 2 / Issue 2 / April-June 2016
Table 1: Rate of compliance of hospital information system
in academic hospitals of Isfahan University of Medical
Sciences based on the standard licensing commission of
health information technology
Evaluation criteria Rate of compliance (%)
Information management 66.1
Patient management 63.5
Technical management 64.6
Total mean compliance of criteria 64.7
is related to development instruction criteria with mean
18.1 percent and data assessment with mean 44.1 percent.
The highest rate of compliance with criteria of the patient
management area is related to the electronic prescribing
of drugs, medical referrals, and measurement of patients’
clinical data with mean 100 percent and the lowest
rate relates to patient‑family‑nurse training with mean
9 percent and the patient’s preferences, language, religion,
and culture were calculated with mean 18.1 percent.
Furthermore, the highest rate of compliance of technical
management criteria relates to information backup, restore
and maintenance, access, privacy, and destruction with
mean 100 percent and the lowest rate relates to warnings,
notifications, and criteria range with mean 0 percent and
technical service criteria with 26.1 percent.
Discussion
According to the findings, the rate of compliance of
information management criteria in HIS of selected
academic hospitals was relatively desirable (66.1 percent)
which had a distant from its desirable mode (100 percent).
The results of the present study are in line with Kimiyafar’s
findings that the calculated information quality of HISs
was about 45.6 percent.[18] The study is also in line with
Habibi’s results that reported information management
was 60.8 percent,[1] Wright et al. expressed that most
information system support of incoming data to the
system is desirable[21] and Oroviogoicoechea and Watson
in his study stated that the interaction of Information
Technology system in patient’s data and information
analysis from the nurses’ view equals to 92.5 percent,[22]
results of the present study are in line with results of these
two mentioned studies so it can be said that evaluation of
HIS based on information management area was relatively
desirable but hospital software designers and experts must
pay more attention to the following criteria to improve
information quality in HIS: Support of clinical document,
record management, complete documentation of clinical
terms and codes by hospital software, and evaluation of
clinical data recorded by the user.
107
[Downloaded free from http://www.ijeprjournal.org on Saturday, March 11, 2017, IP: 176.102.234.146]

Jahanbakhsh, et al.: Healthcare Information Technology

Table 2: Rate of compliance of hospital information system criteria of academic hospitals of Isfahan University of Medical
Science based on the standard of health information licensing commission separated based on the evaluation area
Area Studied criteria Rate of compliance (%)
Information management Identify and maintain a patient record 87.8
Capture external documents of medical records 84.8
Reporting/data exporting 80
Information management 77.7
Clinical documentation of patients 72.7
Manage clinical documents and notes of patient medical records 65.7
Manage terminology and codes 63.6
Assessment of patient’s clinical data 44.1
Registration of advance directives of disease 18.1
Patient management Medications/electronic prescribing 100
Registration of the patient’s health referrals to another provider 100
Patient clinical data measurements 100
General ordering requirement 95.4
Manage the medication list 94.6
Report generation 81.8
Management medication 80
Manage practitioner/patient relationships 75
Patient’s treatment program documentation 74
Patient’s history documentation 65.6
Order medication registration 6.3
Provider management 60
Manage allergy and the adverse reaction list 54.5
Problems list 52.2
Patient’s complication management 45.4
Support drug interactions 41.3
Registration of patient’s legal information 38.6
Patient appointments schedule 22.7
Patient’s preferences, language, religion, culture 18.1
Patient, family, and caregiver education 9
Technical management Backup/recovery of record data 100
Data retention, availability, and destruction 100
Patient clinical audit 93.3
Security and access 85.1
Case loading medical profile 81.8
User authentication 80
Access control 68.8
Confidentiality 100
Clinical decision support 40.9
Technical services 26.1
Communication throughout the system domain 0
Alerts and notifications 0

The rate of compliance of patient management in the
studied HIS has been relatively desirable (63.5 percent).
Regarding patient’s medical decisions, Kimiyafar et al.
stated that the rate of compliance of existing criteria by
hospitals’ software equals to 48.9 percent.[18] Isfahani
et al. also reported that patient’s drug information
record in academic hospitals equals to 44.7 percent,[15]
and Azizi et al. stated that the patient’s drug information
record in clinical wards, pharmacy, and medical
documents equals to 34.5 percent, 36.8 percent,
and 29.7 percent, respectively. These results are not
compatible with the results of the present study.[23]
Traverse pointed to increase of employees’ productivity,
interoperability between systems, health care quality
through improved patient safety, more adherences to
clinical practices, reduction of medical errors, and
drug prescription.[24] According to the studies and the
present study, it can be concluded that the rate of
compliance with patient management criteria in studied
HISs is desirable and somehow are considered in HISs.
However, the following criteria need Information
Technology professionals’ special attention: Presenting
a list of the patient’s allergies and adverse reactions,
list of patients’ problems in the system, management
of patients’ complications, support drugs interactions
and patient’s legal information, support patients
schedule with medical staff, support patient training,
and support clinical decisions and patient’s preference.
The rate of compliance of technical management in the
studied HISs was (64.6 percent) that was far from its
desirable mode. The results of this study are in line with

108 International Journal of Educational and Psychological Researches / Vol 2 / Issue 2 / April-June 2016
[Downloaded free from http://www.ijeprjournal.org on Saturday, March 11, 2017, IP: 176.102.234.146]
Jahanbakhsh, et al.: Healthcare Information Technology
results of Kimiyafar’s study who reported that compliance
of technical and hardware cases with data concentration
was equal to 69.2 percent.[18] Saeedbakhsh et al. reported
the rate of software’s suitability for performing tasks was
about 68.6 percent[7] and Sharifian et al. reported this rate
was equal to 42.8 percent.[25] Littlejones mentioned some
reasons for project’s failure of using HISs in South Africa
were lack of appropriate technical infrastructure, users’,
designers’, and project managers’ different expectation,
lack of attention to the cultural characteristics of the
patient care, after qualitative and quantitative examining
they stated that there was no significant difference
between project failure in this country in comparison
to other countries. [26] Young stated that technical
infrastructures are important factors in creating EMR,[27]
this result is in line with the result of the present study
and it can be concluded that compliance of technical
management criteria of the studied software is relatively
desirable and somehow are considered in software
development; However, to increase the capabilities of
the studied software, promotion of the following criteria
seems necessary: Clinical decision support by hospital
software, support promotion of technical services and
ranges in software, and software support for the warnings
in hospital systems.
Conclusions
Assessment of HISs based on standard and known models
to have qualified information system seems necessary.
Altogether, the results of this study showed that compliance
with CCHIT criteria in selected HISs of Isfahan University
of Medical Sciences has been desirable. Meanwhile, experts
and programmers paid more attention to information and
patient management areas, but the technical management
area was ignored somewhat. Considering that a successful
HIS should be at the high and acceptable levels of the
studied criteria, and must be designed for information
systems, taking steps for development and promotion of
HIS is necessary. Therefore, compliance with the criteria of
the CCHIT standard in the development of information
system is effective. To improve the situation, we suggest
the followings:
• Before developing a system, need and feasibility
assessments must be done by the relative organization
• Operation of national and international standards for
the exchange of patient data between HISs
• Further evaluation of HISs using different criteria
• Using the same meanings and definitions and units
throughout the HISs
• Idolizing international standards and establish a
national standard for the design and development of
HISs.
International Journal of Educational and Psychological Researches / Vol 2 / Issue 2 / April-June 2016
Financial support and sponsorship
This Article was part of a master thesis supported by Isfahan
University of Medical Sciences (Grant No: 393330). We
extend our sincere thanks to the information technology
employees of Isfahan academic hospitals.
Conflicts of interest
There are no conflicts of interest.
References
1. Saghaeiannejad S, Saeedbakhsh S, Jahanbakhsh M, Habibi M.
Assessment and comparison of hospital information systems in
Isfahan Hospitals Based on Adjusted Delone and Mclean model.
Health Inf Manage 2011;8:620‑9.
2. Yucel G, Cebi S, Hoege B, Ozok AF. A fuzzy risk assessment
model for hospital information system implementation. Expert
Syst Appl 2012;39:1211‑8.
3. Kaipio J. Usability in Healthcare: Overcoming the Mismatch
between Information Systems and Clinical Work [Doctoral
Thesis]. Helsinki: School of Computer Science and Engineering,
Aalto University; 2011.
4. Haux R. Health information systems – Past, present, future. Int J
Med Inform 2006;75:268‑81.
5. Dargahi H, Ghazisaeeidi M, Safdari R, Hamedan M. Evaluation
of clinical information systems public hospital Tehran University
of Medical Sciences. Health Payaward 2010;4:31‑43.
6. Saeedbakhsh S, Sadoughi F, Ehteshami A, Kasaei M. Assessment
of ability of user education in medical records module of selected
HIS in Isfahan University of Medical Sciences. Spec Educ Dev
2011;5:877‑85.
7. Saeedbakhsh S, Ehteshami A, Kasaei M. Evaluation the
medical records module of the selected hospital information
systems software in hospital of Isfahan University of Medical
Sciences, Iran, According to ISO 9241‑10. Health Inf Manage
2012;9:490‑501.
8. Sadoughi F, Meydani Z, Shahmoradi L. Management Information
Systems. Tehran: Jafari; 2011.
9. Marashi P, Ebadifard‑Azar F, Ansari H, Zohour A. User coments
about the computerized hospital information systems in hospital
of Tehran. Payesh J 2006;6:11‑8.
10. Sadoughi F, Khoshkam M, Farahi SR. Usability evaluation
of hospital information systems in hospitals affiliated with
Mashhad University of Medical Sciences, Iran. Health Inf Manage
2012;9:317.
11. Zare‑Fazlollahi Z, Lotfnejad‑Afshar H, Jebraeeli M, Maleki M. An
evaluation of hospital information system implementation in Imam
Hospital in Urmia. Health Inf Manage 2011;8:731‑7.
12. Al‑Shorbaji N. E‑health in the Eastern Mediterranean Region: A
decade of challenges and achievements. East Mediterr Health J
2008;14 Suppl: S157‑73.
13. Wyatt JC, Wyatt SM. When and how to evaluate health information
systems? Int J Med Inform 2003;69:251‑9.
14. Hamborg KC, Vehse B, Bludau HB. Questionnaire based usability
evaluation of hospital information systems. Electron J Inf Syst
Eval 2004;7:21‑30.
15. Isfahani SS, Raeisi AR, Ehteshami A, Janesari H, Feizi A,
Mirzaeian R. The role of evaluation pharmacy information system
in management of medication related complications. Acta Inform
Med 2013;21:26‑9.
16. Raeisi AR, Saghaeiannejad S, Karimi S, Ehteshami A, Kasaei M.
District health information system assessment: A case study in
Iran. Acta Inform Med 2013;21:30‑5.
109
[Downloaded free from http://www.ijeprjournal.org on Saturday, March 11, 2017, IP: 176.102.234.146]

Jahanbakhsh, et al.: Healthcare Information Technology

17. Ahmadi M, Shahmoradi L. Evaluation of Health Information Systems:
Six Basic Stages. Hospital. Journal of Hospital 2007;6:16‑23.
18. Kimiyafar K, Moradi G, Sadoughi F, Sarbaz M. Views of users
towards the quality of hospital information system in training
hospitals affiliated to Mashhad University of Medical Sciences.
Health Inf Manage 2007;4:43-50.
19. Gold JA, Wang J. Implementing electronic health records in
physician offices: Lessons learned. WMJ 2007;106:163‑4.
20. Mehraeen E, Shajarat M, Khoshkam M. Evaluation of hospital
information systems in selected hospital of Tehran. Health
Payaward 2013;6:458‑66.
21. Wright A, Sittig DF, Ash JS, Sharma S, Pang JE, Middleton B. Clinical
decision support capabilities of commercially‑available clinical
information systems. J Am Med Inform Assoc 2009;16:637‑44.
22. Oroviogoicoechea C, Watson R. A quantitative analysis of the
impact of a computerised information system on nurses’ clinical
practice using a realistic evaluation framework. Int J Med Inform
2009;78:839‑49.
23. Azizi A, Hajavi A, Haghani H, Baghini MS. Respect rate of
hospital information system criteri a of American College of
Physicians in Educational Hospitals of Iran, Tehran, and Shahid
Beheshti Medical Science Universities. Health Inf Manage
2010;7:323‑9.
24. Travers D. Selecting an electronic medical record system for small
physician practices. N C Med J 2009;70:399.
25. Sharifian R, Nematollahi M, Monem H, Ebrahimi F. Evaluation the
security safeguards in hospital information system according to
the health insurance portability and accountability act of university
hospital in Shiraz University of Medical Sciences, Iran. Health Inf
Manage 2013;10:36‑46.
26. Littlejohns P, Wyatt JC, Garvican L. Evaluating computerised
health information systems: Hard lessons still to be learnt. BMJ
2003;326:860‑3.
27. Park YT, Lee J. Factors affecting electronic medical record
system adoption in small Korean hospitals. Healthc Inform Res
2014;20:183‑90.

110 International Journal of Educational and Psychological Researches / Vol 2 / Issue 2 / April-June 2016

View publication stats