Lifelong independent learners

Title of proposal – Systematic Literature Review on Intelligence Tutoring System in

Student Language Learning Outcomes: Is ITS Useless for Language Learners?
Short Description - (75 words).
This presentation reports the findings from a systematic review of 52 peer-reviewed journals
published between 2009-2021. It is focusing on the student language learning outcomes
through Intelligent Tutoring System implementation. This study will be beneficial in
providing a better understanding for language educators, researchers, developers,
programmers, and non-programmers about the effects and challenges in ITS implementation
for language learners. Thus, they can use ITS as an effective means to improve student
language learning outcomes.
Keywords: Learning Outcomes, Technology Integration
Session type: Concurrent Session
Equipment needs: LCD Projectors

Abstract - (750 -1000 words).

Intelligent Tutoring System (ITS) is a computer program broadly used to provide

guidance and feedback in a customized environment for students as they complete tasks and
solve problems instantaneously (Dermeval et al., 2018: Seo et al., 2021). As depicted in
several systematic review research, such as the use of ITS for learners can sufficiently
enhance the outcome from hard skill training in various domains ranging from biology to
engineering subject (Laine et al., 2022). It can be achieved since learners customized their
own questions that are answered correctly by the agents. Not only the agents’ features, but
Dermeval et al. (2018) also analysed that artificial intelligence tools used in ITS can tailor
personalized learning environments through a systematic review of ITS’s peer-reviewed
articles from 2009 until 2016. Aligned with Mousavinasab et al. (2021) systematic review
findings from 2007 to 2017, the implementation of ITS with various artificial intelligent
characteristics has helped learners improve their reasoning in the learning process suited to
their needs. However, in the same article, it is shown that the implementation of ITS in
language learning is only 7.54% among other subjects. This low number of ITS
implementation encourages the researcher of this study to conduct a systematic review to
identify the pattern of how ITS is developed to improve students’ language learning
outcomes that will be beneficial to improve the implementation of ITS for language learners.
Thus, through the integration of ITS into the language learning process, it can personalized
language learning activities that meet individual students’ needs and enhance student
learning outcomes (Klašnja-Milićević et al., 2018; Wang et al., 2021).

My research proposal was conducted in Systematic Literature Review (SLR) as a

method to identify and interpret the recent research outcomes that address the research
questions and topic interest in the context of ITS research (Dermeval et al., 2018). Those
research questions are 1) How has ITS for the language learning process been presented in
the published research?; 2) How does an intelligent tutoring system affect students’ language
learning outcomes?; 3) What are the challenges with integrating intelligent tutoring systems
in teaching languages? To collect the evidence before the conclusion of this study, the SLR
process applies the PRISMA principles to select the articles (Liberati et al., 2009).
To begin with, this systematic review is guided by several systematic reviews related to
ITS in other fields. First, as the foundation of my current review, Dermeval et al. (2018) have
set their systematic review on the ITS authoring tools fields from 2009-2016 literature to
develop the ITS authoring tools for non-programmers. Moreover, having the same starting
year to search the literature, Alabdulhadi & Faisal (2021) examine the effectiveness of ITS in the
STEM fields. Thus, to provide general insights into language fields, a research paper database
from Academic Search Premier in this systematic review were searched from 2009-2021.
The search string used in this study was adopted from Mousavinasab et al. (2021), who
applied the population, intervention, comparison, and outcome (PICO) criteria to define the
search string in their systematic review for evaluating the ITS characteristics, applications,
and evaluation. Thus, in this study, the population was ITS. Interventions included students,
and the outcomes were language learning, while comparison was excluded. In conclusion, the
search string was: (ITS* OR Intelligence Tutoring System*) AND (student* OR learner*)
AND (language*). The use of “*” is applied to expand the data resources.
The first search result yielded 3941 articles. These articles were downloaded and
scanned with the aid of the Covidence application. Afterward, I scanned similar articles
which excludes 364 duplicated papers. Also, the filtering of journals has been done by
identifying based on educational top journal publishers and excluding the dissertation and
conference proceedings. The filtering process of reputable journals was conducted based on
Google Scholar and excluded 462 articles (Zawacki-Richter et al., 2019). Moreover, since
this study was focused on students’ competencies and the implementation of ITS in the
language learning fields, the author excludes 2985 articles. In the final screening, the 130
 articles were reviewed more thoroughly in the full text to ensure that the articles met the
following criteria: ITSs with evaluation, not used for the commercial, and focused on the ITS
language learning outcomes. This selection process resulted in 41 articles for the systematic
reviews.
In my presentation, I will share the findings and summarize them here. Underlying the
language learning domain, I categorized the articles into four modalities: speaking, listening,
reading, and writing (Schoonen, 2019). Among those modalities, reading is the most
dominant language skill addressed by ITS, representing over 60% of all the literature in this
domain and followed by writing, speaking, and listening skills sequentially. This number is
aligned with the effectiveness of ITS in improving ten academic reading skills. They
committed fewer errors and improved their reading comprehension significantly more than
the instructions without the aid of ITS due to the advice generator that can respond to the
learner’s errors and guide learners toward improving their performance over time. However,
there are challenges in using ITS in teaching writing and speaking skills due to the existence
of many different types of genres and culture-specific nuanced words that do not clearly
express the same meaning in different contexts of writing and speaking. This diversity poses
a challenge for an ITS in terms of providing tailored and meaningful responses to learners’
problems.
Moreover, the distribution of language taught by ITS is diverse: while most teach
English as a second language (ESL), others focus on Japanese and Chinese. Especially in the
Japanese and Chinese domains, ITS has successfully been deployed into classrooms and has
been used by teachers at different levels of education to teach thousands of Japanese and
Chinese characters. Furthermore, I also found that in 2016, there were higher publications
compared to the previous years, with at least nine studies evaluating the effectiveness of ITS
for language learning, and they increased gradually in the following year. This aligns with the
massive research and development focus on the feedback component in Intelligent computer-
assisted language learning (ICALL) tutoring systems as the most used ITS approach in this
systematic review databases.

References

Alabdulhadi, A., & Faisal, M. (2021). Systematic literature review of STEM self-study related ITSs.
Education and Information Technologies, 26(2), 1549–1588. https://doi.org/10.1007/S10639-020-
10315-Z/TABLES/14
Dermeval, D., Paiva, R., Bittencourt, I. I., Vassileva, J., & Borges, D. (2018). Authoring Tools for
Designing Intelligent Tutoring Systems: a Systematic Review of the Literature. International
Journal of Artificial Intelligence in Education, 28(3), 336–384. https://doi.org/10.1007/S40593-
017-0157-9/TABLES/13
Laine, J., Lindqvist, T., Korhonen, T., & Hakkarainen, K. (2022). Systematic Review of Intelligent
Tutoring Systems for Hard Skills Training in Virtual Reality Environments. International Journal
of Technology in Education and Science, 6(2), 178–203. https://doi.org/10.46328/ijtes.348
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M.,
Devereaux, P. J., Kleijnen, J., & Moher, D. (2009). The PRISMA statement for reporting
systematic reviews and meta-analyses of studies that evaluate healthcare interventions:
explanation and elaboration. BMJ, 339. https://doi.org/10.1136/BMJ.B2700
Mousavinasab, E., Zarifsanaiey, N., R. Niakan Kalhori, S., Rakhshan, M., Keikha, L., & Ghazi Saeedi,
M. (2021). Intelligent tutoring systems: a systematic review of characteristics, applications, and
evaluation methods. Interactive Learning Environments, 29(1), 142–163.
https://doi.org/10.1080/10494820.2018.1558257
Schoonen, R. (2019). Are reading and writing building on the same skills? The relationship between
reading and writing in L1 and EFL. Reading and Writing, 32(3), 511–535.
https://doi.org/10.1007/S11145-018-9874-1/TABLES/4
Wang, X., Wang, Y., Yang, Y., & Wang, L. (2021). Investigating Chinese University Students’
Enjoyment in a Web-Based Language Learning Environment: Validation of the Online Foreign
Language Enjoyment Scale. Https://Doi.Org/10.1177/00315125211041714, 128(6), 2820–2848.
https://doi.org/10.1177/00315125211041714
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research
on artificial intelligence applications in higher education – where are the educators? International
Journal of Educational Technology in Higher Education 2019 16:1, 16(1), 1–27.
https://doi.org/10.1186/S41239-019-0171-0