Download as pdf or txt
Download as pdf or txt
You are on page 1of 7

2019 2nd International Conference on Artificial Intelligence and Big Data

Evaluate Accessibility and Usability Issues of Particular China and Pakistan


Government Websites

Muhammad Bilal, Zhi Yu, Shuyi Song, and Can Wang


Alibaba-Zhejiang University Joint Institute of Frontier Technologies
Zhejiang Provincial Key Laboratory of Service Robot
College of Computer Science, Zhejiang University, Hangzhou 310027, China
e-mail: {mbilal, yuzhirenzhe, brendasoung, wcan@zju.edu.cn}

Abstract—Today the Web has become an essential part of The Web accessibility framework comprises of the user
people’s life. Everyone has the right to access websites, interface, navigation, contents, HTML standards and
including seniors and disabled people. Government websites scripting for improving Web design issues of people with
contain public information for the whole society, but affect disabilities [12]. There are three necessary guidelines of
mostly disabled people. In this paper, we have analyzed the WAI (called a three-pillar model for Web accessibility):
accessibility and usability issues of selected top 60 government Authoring Tools Accessibility Guidelines (ATAG) for
websites of Pakistan and China. We analyzed all websites’ authoring tools, User Agent Accessibility Guidelines
loading time, page size, mobile friendliness, broken links, (UAAG) for user agents, and Web Content Accessibility
HTML5 and ARIA, Web Content Accessibility Guidelines Guidelines (WCAG) for Web contents receptively [13, 14].
(WCAG) 2.0 (Level A, Level AA, Level AAA) conformance, The foundation of WCAG is based on four essential
and contrast errors through online accessible testing tools and principles: perceivable, operable, understandable, robust
human inspections. We designed an interactive evaluation
and three conformance levels: A, AA and AAA [2, 11, 15,
framework for Web accessibility and usability, which is used
16]. The WCAG 2.1 standard was published in June 2018.
to evaluate both countries’ government websites. According to
The ATAG 2.0 was finalized in September 2015 and
the evaluation 22 out of 30 Chinese government websites have
UAAG was finalized in December 2015 [11, 13, 17-19].
less than about 50% of the accessibility issues, while Pakistan
Web accessibility components are more clearly identified in
in contrast only 12 out of 30 government websites have less
than about 50% of accessibility issues. The Chinese
the restructured guideline set of Web accessibility, in
government websites are much better than Pakistani
accordance with the new technology that supports
government websites with respect to accessibility and usability implementation in a better way according to user needs [13].
features. The all-inclusive results indicate that Pakistan Accessibility field is also considered to be an Artificial
government websites have poor conformance levels to meet Intelligent (AI). Due to the development of several
WCAG standards. We gave some valuable recommendations, technologies especially designed for people with disabilities
which will be helpful for web designers, developers and also in this present year [20].
web users particularly disabled people performance in the According to the learning aspects, the main goals of Web
newest Artificial Intelligence (AI) area in a decent way. accessibility are: learning standards (Web accessible well-
known standards), building different abilities (multimedia
Keywords-evaluation; web accessibility; usability; disabled contents utilized for disabled people) and building different
people; artificial intelligence (AI) devices (Web accessibility’s latest technology should be for
disabled persons, such as screen reader device) [21].
I. INTRODUCTION
Accessibility is considered an essential part of usability [22].
The Internet has played a vital role in every aspect of The main purpose of this study is to evaluate Web
people’s lives. Everyone, including seniors and disabled accessibility issues of China and Pakistan government
people has the right to access the Internet for using a variety websites for providing public services to disabled people of
of different websites [1-3]. both countries in the same way as to the abled citizens.
The Web is an essential part of people’s lives [4, 5]. It is China and Pakistan’s government websites accessibility and
an important part of our education and daily life routine usability aspects are examined due to a rapid increase in the
work. We are faced with the urgent needs to provide access ratio of disability in both countries. In both countries, most
services on the Web. To ensure each and every member of of the disabled individuals as intellectual as abled citizens.
our society receives equal rights [5-8]. The Web has Are both countries government’s websites equally
increasingly become full of multimedia contents [9]. accessible by everyone including disabled persons? In this
Information and Communication Technology (ICT) plays a evaluation framework, famous government websites from
vital role in delivering public service information to the both countries have been compared. How can better Web
society [8, 10]. accessibility and usability of both countries’ government
Web accessibility means that people with disabilities can websites be evaluated?
easily use the Web [4, 11]. Working under World Wide The rest of this paper is organized in the following way:
Web Consortium (W3C), the Web Accessibility Initiative Section 2 describes the related work of accessibility and
(WAI) aims to make the Web accessible for disabled people.

978-1-7281-0831-5/19/$31.00 ©2019 IEEE 316


usability evaluation of other countries contribution. Section WAVE tool presents the website pages with the help of
3 signifies the methodology of both countries’ government embedded icons and indicators (Contrast errors, HTML5
websites accessibility and usability evaluation relevant and ARIA issues present) [24].
issues. Section 4 deliberated the results and discussion The aim of a good website is how efficient, effective,
about both countries performance. Section 5 describes the error-free, memorable and satisfactory according to the
limitations and recommendations for China and Pakistan needs of the user is. Usability of e-government websites is
government websites. Section 6 concludes the main theme to learn easily, efficiently and the level of satisfaction is
of the paper and guidelines provided to designers and attained effectively. The aim of a website to be usable is to
developers of Pakistan government websites to perform be accessible. The main factors that affect the website
better in the future. usability point of view are loading speed, page size, broken
links and accessibility errors [10].
Sri Lanka’s government websites investigate usability
II. RELATED WORK and accessibility guidelines through online evaluation tools
China is one of the largest countries within the world for consisting of WAVE, google page speed insight, google
people with disability issues. According to the 2nd National Mobile-Friendly test, Pingdom and PowerMapper. The
China Survey on disability, 85 millions disabled people live websites’ loading speed was evaluated, adaptivity to mobile
in China’s cities. Due to the rapid growth of disabled devices and basic accessibility standards as well [22].
persons, China has made considerable efforts starting in Mobile-Friendly is an online testing tool used for checking
2013 on its specific Chinese government websites for the website’s pages of accessibility on mobile devices [25].
evaluating Web accessibility standard requirements The accessibility and quality of government ministry
(YD/T1761-2012) [4]. websites were comparatively analyzed in four countries:
The data has increasingly grown from 2012 to 2015 in Turkey, Kyrgyzstan, Azerbaijan, and Kazakhstan. In this
the form of board data, data science, and big data. analysis, few ministry websites achieved AA conformance
Underlying Web is defined with the help of this data and level of accessibility and most of the websites failed in the
applications, which supports and makes sense for everyone conformance level of A and AA. Turkish websites were
requiring it and leads to accessibility research [5]. better than the other three countries’ ministry websites. In
The World Health Organization (WHO) produced a this study, online available testing tools were used for
world report on disability with the collaboration of the evaluating websites’ performance (broken links), loading
World Bank based on scientific evidence. This world report time and page size [26]. GTmetrix is used for optimizing
produced with the support of policy makers, development website with clear and easy with Google PageSpeed and
agencies, researchers, practitioners, and academians. Yahoo rulesets analysis. In terms, GTmetrix is analyses
According to the WHO 2011 survey report, approximately websites’ loading time and page size [27]. Dead Link
one billion disabled persons faced difficulties to access Checker is an online testing tool used for identifying the
different versatile websites. In the 2004 study illustration; websites’ broken links. In other words, help out developers
9.6% of disabled people are live in Pakistan and 7.7% in avoid damaging the website usability and ranking [28].
China [23]. The Kyrgyz Republic 55 e-government websites were
The Web accessibility significant problems is seriously examined in terms of accessibility, usability and security
faced in China due to some laws and courses of Web aspects through online available testing tools such as
accessibility implemented. One of the researchers, Shi [1], WebsiteOptimization and EvalAccess 2.0 [7].
observed the accessibility of 324 Chinese local government Bangladesh government ministry websites accessibility
websites with Web accessibility standard WCAG 1.0 and guidelines on the basis of the W3C standard were analyzed
also evaluated Australian government websites. Both and evaluated through online testing tools AChecker, and
Chinese and Australian government websites were EvalAccess [6]. AChecker is an online testing tool that
compared with each other. The results showed that Chinese checks only HTML pages for conformance with respect to
e-government websites are less accessible as compared to accessibility standards to ensure the content is accessible for
Australian websites. The researchers analyzed 38 popular everyone including disabled people [29].
Chinese government websites in 2009 in terms of the Turkey e-government websites’ accessibility issues
improvement of the Web accessibility issue. In 2013, were evaluated on the basis of WCAG 1.0 and WCAG 2.0
researchers examined 50 websites that are more accessible guidelines. In this evaluation 25, e-government websites of
as compared to the 2009 analysis [1]. Turkey were chosen for the disabled people. Evaluation
In Tanzania, e-government websites’ accessibility, results showed that the accessibility priority-1 barrier was
usability, and security aspects were evaluated through present and the most important text alternative for the non-
automatic diagnostic tools such as WAVE, google speed text element was absent. These websites failed to update
insight, and w3c checker [10]. WAVE is an online web static contents which need to be changed a dynamic one [8].
accessibility evaluation-based tool which helps the web
developers make websites contents accessible. Simply, the

317
III. METHODOLOGY the top 30 China government websites that previously
The empirical research methodology is used in our evaluated through Web automatic system by LI, Liang-
research. The main objective of this article is evaluating cheng in 2016 were selected [4].
accessibility and usability issues of 60 useful e-government The online evaluation method was conducted between
websites from China and Pakistan and providing some November 14, 2017, and until February 20, 2018. The main
recommendations to designers and developers of these issues of Web accessibility and usability are loading time,
websites. page size, mobile friendliness, broken link, HTML5 and
In this evaluation study 30, government websites were ARIA, contrast errors, and WCAG 2.0 performance
selected from Pakistan government official gateway address evaluated through relevant online testing tools and disabled
(http://www.pakistan.gov.pk/). These websites are mostly human inspections. The online websites’ evaluation through
used by regular and disabled people in Pakistan. Similarly, online testing tools and disabled people’s opinion
framework for both countries are shown in Figure 1.

Figure 1. Subjective web accessibility and usability evaluation framework.

According to design Web accessibility and usability GTmetrix. We analyzed the loading time in Hong Kong,
evaluation framework for China and Pakistan government’s China test server region for way better results utilized. The
websites issues evaluated through online testing tools and loading time is rated High, Average and Low. The
human inspections are shown in Figure 1. In this Web evaluated results of both countries’ websites are shown in
accessibility and usability evaluation framework, we used Table 1.
online tools for evaluation of both countries’ websites to
TABLE 1. HOME PAGE LOADING TIME COMPARISON
reduce the cost of larger websites. First, we selected 60
Websites Loading China Pakistan
government websites (30 from Pakistan and 30 from China) Time Rated Government Government
and sample pages for evaluation. The page execution from (Seconds =S) Websites Websites
the HTML file is necessary before evaluating the pages High (<=5 S) 21 14
through online tools and human-based inspection used for Average (<=10 S) 04 10
Low (>10 S) 02 06
checking the level of Web accessibility and usability. In this
framework, we analyzed inclusive loading time, page size,
mobile-friendly, broken links, HTML5 & ARIA, contrast The loading time of Chinese government websites is
errors and accessibility performance issues identified relatively better than from Pakistani government websites.
through accessible testing tools. We performed systematic The loading time of Pakistani government websites can be
analyses and compared the evaluated results of both improved if the image size is optimized and lightweight
countries’ websites framework. We evaluated completely framework of CSS3 and HTML5 is used.
all raw results in an intelligent way and provided some Next, the page size is also evaluated through the online
suggestions to designers and developers of both countries’ tool GTmetrix. The recommended page sizes for the
websites to minimize accessibility costs of websites and website are considered: high (<=12 Kb), average (<=2 MB)
also make the evaluation possible for larger websites. and low (more than 2 MB). According to both countries’
evaluated results, no government’s website page size is
IV. RESULTS AND DISCUSSIONS high. 19 websites’ (63%) page size is average and 11
First, we analyzed the loading time of Pakistani and websites (37%) page size is low in Pakistani government
Chinese government websites through online testing tool websites. Compared to China; 8 websites’ (30%) page size

318
is average and 19 websites’ (70%) page size is low. TABLE 3. COMMON SUCCESS CRITERIA FOR LEVEL (A, AA AND AAA)
MANUAL EVALUATION (PAKISTAN WEBSITES FAILED)
Pakistani government’s websites page size is revealed to be
Checkpoints Criteria
Chinese government websites.
1.1.1 Non-text Content (Level A)
Whether both countries’ government websites are
mobile friendliness or not can be evaluated through the 1.3.1 Info. & Relationships (Level A)
online Google Mobile-friendly tool. The evaluated results 1.4.1 Use of Color (Level A)
of both countries’ government websites mobile-friendliness 1.4.3 Contrast (Minimum) (Level AA)
are shown in Table 2. 1.4.4 Resize Text (Level AA)
2.1.1 Keyboard (Level A)
TABLE 2. MOBILE-FRIENDLY TEST
Mobile China Pakistan 2.2.2 Pause, Stop, Hide (Level A)
Friendly Government Government 2.4.4 Link Purpose (Level A)
Test Websites Website 2.4.6 Headings and Labels (Level AA)
Pass 09 09
Fail 14 21 3.1.1 The language of Page (Level A)
3.3.2 Labels or Instructions (Level A)
Only 30% of both countries’ government websites are 4.1.1 Parsing (Level A)
mobile friendly. In this evaluation, 6 Chinese government’s
website pages cannot be evaluated at all. The overall failure There is not even one selected Pakistani government
rate is higher in Pakistani government websites as compared website to fulfill the common success criteria of Web
to Chinese government websites. accessibility performance Level AAA. The most common
In this evaluation, the most common issues are revealed accessibility performance success criteria for Level A and
to be the viewport is not set, the text is too small to read, AA failed Pakistan government websites are explained.
clickable elements are very close together and the content is Text alternative describes the non-text content in an
wider than the screen. Pakistani government websites would equivalent way with a brief description of data. In non-text
be much more mobile-friendly if the viewport area is based content, the state faced CAPTCHA, decoration, and
on dynamic design and there is some space between the formatting form issues. Information and relationship are not
clickable elements and other the rest of the text. presented in an easy and informative method. The
Broken links of both countries’ government websites are foreground and background colors are not used easy form.
evaluated through the online testing tool Deadlink Checker. Large-scale text and images’ contrast ratio are not set
According to evaluation results, there occur more than 10 properly. The text is not sized. All keyboard functionality is
broken link errors occur on 77% of the Pakistani not available in an easy way to operate. Automatic scrolling,
government websites and 87 % of the Chinese government moving, stopping, hiding functionality are missing. Some of
websites. The most common issues in Pakistani government the links are not ambiguous. Heading and labels of some
websites are 403 Forbidden, 404 not found, -1 Invalid URL, topics are not described properly. The default language of
- 1 Timeout, 500 internal server error and 401 unauthorized. each page is not set properly. Some labels and instructions
The HTML5 and ARIA issues were analyzed through are not provided when content requires user inputs. Some
the WAVE online testing tool. According to the analyzed content start and end tag are missing in parsing? Chinese
results, 19 (63%) Pakistani government websites have no government websites both Levels (A and AA) are much
issues with HTML5 and ARIA structure, but 11 (37%) better than Pakistani government websites according to the
websites still have experience problems. According to the AChecker testing tool.
analyzed results, 24 (89%) Chinese government websites Pakistani and Chinese government’s websites are ranked
have no issues with HTML5 and ARIA structure, but 3 according to a small number of contrast errors in both
(11%) websites still have issues. These common issues are a countries’ websites shown in Table 4.
header, footer, navigation, content, ARIA tabindex, and
TABLE 4. PAKISTAN AND CHINA GOVERNMENT WEBSITES RANKING
ARIA label. The inclusive HTML5 & ARIA particular both Ranking Pakistan Government China Government
countries’ websites structure is better as compared to other Websites Websites
websites usability and accessibility issues. 1 Tourism of Pakistan Guizhou
The accessibility performance was evaluated for both 2 NDMA Hebei
3 Supreme Court of Shaanxi
countries through the online tool AChecker. According to Pakistan
evaluated results, there are more than 50 known 4 Sindh Province Shanghai
accessibility errors, which violated conformance level A, 5 Senate Liaoning
AA, and AAA checkpoints on 18 Pakistani government 6 Pakistan Railway Zhejiang
websites and 6 Chinese government websites. The most 7 NEPRA Heilongjiang
common mistakes that occur in all selected Pakistani 8 Punjab Province Henan
9 Ministry of Commerce Beijing
websites in WCAG 2.0 are shown in Table 3.
10 Nation Database Yunnan
Information System

319
11 Ministry of Religion Xinjiang 23 Ministry of Finance Jiangxi
Affairs 24 Ministry of Interior Shandong
12 Ministry of Defence Ningxia
25 National Assembly Gansu
13 IPC Hubei
14 FPSC Qinghai 26 Ministry of Foreign Sichuan
15 Government of Jiangsu Affairs
Pakistan 27 Lahore High Court Hainan
16 Ministry of Law & Tianjing 28 PTA Anhui
Justice 29 DGIP (Interior Inner Mongolia
17 Ministry of Planning & Shanxi Ministry)
Development 30 Kyber Pakhtunkhwa Jilin
18 Cabinet Guangxi
19 Federal Tibet We categorized both countries websites by ranking them
OMBUDSMAN of to represent the contrast in errors easily represnt. Both
Pakistan
20 NTC Guangdong countries’ website performance is improves if the contrast
21 Punjab Emergency Fujian errors are less. The contrast errors are shown in Figure 2.
Service
22 Ministry of Chongqing
Energy/Power

200
180
160
140
120
100
80
60
40
20
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Contrast Error (China Government Websites)


Contrast Error (Pakistan Government Websites)

Figure 2. contrast errors of both countries websites (comparison)

Both countries’ websites are nearly balanced in contrast • Most of the Pakistani government’s websites have a
errors occurring according to a comparison in Figure 2 with colorful background and text size that is not readable
the exception of some Chinese websites. The contrast errors for the user as compared to Chinese government’s
can be improved in Pakistani government’s websites to websites.
diminish accessibility issues if the developers choose the • Most of Pakistan government’s websites are static
right colors combinations in websites in consideration of pages (not dynamic pages) that are not suitable for
disabled people. Pakistani government’s websites have disabled users to easily access these government’s
more colorful background images, text writing, forms, websites as compared to Chinese government’s
buttons, links, and navigations point of view. A more websites.
suitable way for disabled users would be a medium level • The most common violation accessibility
text, separation between buttons and normal text and a performance criteria of WCAG 2.0 guidelines cannot
white background. successfully meet in both countries’ government
websites are non-text content (Level A) and Contrast
V. LIMITATIONS AND RECOMMENDATIONS
(Level AA) issues.
The main limitations in the field are: We recommend some guidelines on the basis of the
• The lack of awareness found in the design and accessibility performance issues analyzed by testing tools.
development stage in Pakistani government’s According to the AChecker tools, results for common issues
websites on the basis of WCAG 2.0 guidelines as in Web accessibility principle are highlighted in Table 3. If
compared to the Chinese government’s websites. designers and developers address these issues then disabled

320
people can easily access these websites. We would suggest DEStech Transactions on Social Science, Education and Human
Science, no. icssd, 2016.
to Pakistani government’s websites, developers, to
[5] H. Takagi and C. Asakawa, "New challenges in web accessibility," ed:
incorporate more dynamic web pages as compared to static Springer, 2017.
pages because web users use different devices to access [6] M. K. Baowaly and M. Bhuiyan, "Accessibility analysis and
these websites. We recommend that the homepage of the evaluation of Bangladesh government websites," in Informatics,
Pakistani government’s websites be simple and attractive to Electronics & Vision (ICIEV), 2012 International Conference on,
all users. Web accessibility performance of Pakistani 2012, pp. 46-51: IEEE.
government’s websites can be improved if the developers [7] R. Ismailova, "Web site accessibility, usability and security: a survey
of government web sites in Kyrgyz Republic," Universal Access in
follow the four main principles of WCAG 2.0 in an accurate the Information Society, vol. 16, no. 1, pp. 257-264, 2017.
way. [8] Y. AKGÜL and K. Vatansever, "Web accessibility evaluation of
government websites for people with disabilities in Turkey," Journal
VI. CONCLUSION AND FUTURE WORK of Advanced Management Science Vol, vol. 4, no. 3, 2016.
This empirical research work shows that the Pakistani [9] L. Moreno, M. Gonzalez, P. Martí nez, and A. Iglesias, "A study of
accessibility requirements for media players on the Web,"
government’s websites need to be improved in terms of International Conference on Universal Access in Human-Computer
accessibility and usability for all members in the society. In Interaction, pp. 249-257, 2011.
China 22 out of 30 government’s websites have less than 50% [10] E. Noe, "Usability, Accessibility and Web Security Assessment of E-
issues as compared to 12 out of 30 Pakistani government’s government Websites in Tanzania," International Journal of
websites which have less than 50% issues. Chinese Computer Applications, vol. 164, no. 5, 2017.
government’s websites are prove to be better than Pakistani [11] W. W. W. Consortium, "Web content accessibility guidelines
(WCAG) 2.0," 2008.
government’s websites. Furthermore, Chinese government’s
[12] M. Bakhsh and A. Mehmood, "Web accessibility for disabled: a case
websites’ developer properly follows WCAG 2.0 guidelines study of government websites in Pakistan," Frontiers of Information
and basic principles as compared to Pakistani government’s Technology (FIT), 2012 10th International Conference on, pp. 342-
websites. 347, 2012.
In the future, we could suggest evaluating Web [13] M. Cooper, "Web accessibility guidelines for the 2020s,"
Proceedings of the 13th Web for All Conference, p. 5, 2016.
accessibility and usability issues will be improve by
[14] R. Rutter et al., Web accessibility: Web standards and regulatory
modifying predefined accessibility and usability compliance. Apress, 2007.
questionnaires of W3C to both countries disabled users [15] B. Caldwell, M. Cooper, L. G. Reid, and G. Vanderheiden, "Web
inquire [30]. Our design evaluation framework Web Content Accessibility Guidelines (WCAG) 2.0. W3C
accessibility and usability particular Pakistan government Recommendation 11 December 2008," URL: www. w3.
websites will be helpful for designers, developers and also org/TR/WCAG20, pp. 1-41, 2008.
disabled people in the future. In the future, we will plan to [16] H. S. Al-Khalifa, I. Baazeem, and R. Alamer, "Revisiting the
accessibility of Saudi Arabia government websites," Universal
develop Web accessibility evaluation-based system for Access in the Information Society, vol. 16, no. 4, pp. 1027-1039, 2017.
Pakistani government’s Websites analyze on the basis of [17] M. A. Aziz, W. A. R. W. M. Isa, and N. Nordin, "Assessing the
Web accessibility guidelines and standards. accessibility and usability of Malaysia Higher Education Website,"
User Science and Engineering (i-USEr), 2010 International
ACKNOWLEDGMENT Conference on, pp. 203-208, 2010.
[18] S. Kurt, "Accessibility of Turkish university Web sites," Universal
Special acknowledgment is to all Zhejiang University Access in the Information Society, vol. 16, no. 2, pp. 505-515, 2017.
teachers for their support in this paper. This work is [19] (Sep. 12, 2017). Web Content Accessibility Guidelines (WCAG) 2.1
supported by Alibaba-Zhejiang University Joint Institute of [Online]. Available: https://www.w3.org/TR/2018/REC-WCAG21-
Frontier Technologies, Zhejiang Provincial Natural Science 20180605/
Foundation of China (Grant No: LZ13F020001), National [20] S. Abou-Zahra, Judy Brewer, and Michael Cooper, "Artificial
Intelligence (AI) for Web Accessibility: Is Conformance Evaluation a
Science Foundation of China (Grant No. Way Forward?," in Web4All 2018, Lyon, France, 2018: ACM.
61173185,61173186) and National Key Technology R&D [21] C. Hitchcock, "Balanced instructional support and challenge in
Program (Grant No. 2012BAI34B01). universally designed learning environments," Journal of Special
Education Technology, vol. 16, no. 4, pp. 23-30, 2001.
REFERENCES [22] S. Gopinath, V. Senthooran, N. Lojenaa, and T. Kartheeswaran,
"Usability and accessibility analysis of selected government websites
[1] P.-L. P. Rau, L. Zhou, N. Sun, and R. Zhong, "Evaluation of web in Sri Lanka," Region 10 Symposium (TENSYMP), 2016 IEEE, pp.
accessibility in China: changes from 2009 to 2013," Universal Access 394-398, 2016.
in the Information Society, vol. 15, no. 2, pp. 297-303, 2016.
[23] W. H. Organization, "The World Bank. World report on disability.
[2] S. Sánchez Gordón and S. Luján Mora, "Web accessibility 2011," WHO Library Cataloguing-in-Publication Data, 2011.
requirements for massive open online courses," 2014.
[24] (Jan. 12, 2018). WAVE Help [Online]. Available:
[3] S. Luján-Mora, R. Navarrete, and M. Peñafiel, "Egovernment and https://wave.webaim.org/help
web accessibility in South America," in eDemocracy & eGovernment
[25] (Nov. 22, 2018). Mobile-Friendly Test [Online]. Available:
(ICEDEG), 2014 First International Conference on, 2014, pp. 77-82:
https://search.google.com/test/mobile-friendly
IEEE.
[26] R. Ismailova and Y. Inal, "Web site accessibility and quality in use: a
[4] L.-c. LI, J.-j. BU, Y. Zhi, W. Wei, and W. Can, "An Overview of
comparative study of government Web sites in Kyrgyzstan,
Web Accessibility Evaluation of Government Websites in China,"

321
Azerbaijan, Kazakhstan and Turkey," Universal Access in the [29] (Feb. 04, 2018). We Accessibility Checker [Online]. Available:
Information Society, vol. 16, no. 4, pp. 987-996, 2017. https://achecker.ca/checker/index.php
[27] (Nov. 14, 2017). GTmetrix Features [Online]. Available: [30] shawn. (Aug. 12, 2018). WAI Site Usability Testing Questions
https://gtmetrix.com/features.html [Online]. Available:
[28] [28] (Dec. 27 , 2017). Dead Link Checker [Online]. Available: https://www.w3.org/WAI/EO/Drafts/UCD/questions.html#tasks
https://www.deadlinkchecker.com/

322

You might also like