Intelligent tutoring systems: a systematic review of characteristics, applications, and evaluation

methods

Method:

- The systematic review followed the PRISMA guidelines for data collection and
analysis.
- The search strategy included databases such as PubMed, ProQuest, Scopus, Google
Scholar, Embase, Cochrane, and Web of Science.
- The search covered the period from 2007 to 2017 (up to 10th October).
- The PICO criteria (Population, Intervention, Comparison, Outcome) were used to
define the search string.
- The population of interest included Intelligent Tutoring Systems (ITSs), intelligent or
intelligent adaptive educational systems, applications, or software.
- The intervention focused on students' or staff education or training in any field.
- No specific comparison was included in the search.
- The outcomes of interest were the characteristics of developed ITSs and their
methods of evaluation.
- Inclusion criteria involved English-language original papers that evaluated intelligent
tutoring systems as training, education, or educational assistance tools for learners in
any field.
- Excluded from the review were game-based education systems and non-intelligent
adaptive education systems.
- Certain educational systems labeled as ITS were also excluded if their structure did
not align with the classical architecture of ITS or if they used commercial adaptive
platforms for development.
- Two reviewers independently screened papers based on titles and abstracts,
categorizing them into groups: 1) meeting inclusion criteria, 2) uncertainty about
inclusion criteria, and 0) not meeting inclusion criteria.
- Papers in Group 0 were excluded, while those in Groups 1 and 2 proceeded to the
next phase.
- Full-text of included papers was obtained for further assessment by two reviewers.
- Data extraction involved extracting eight variables: system name, study population, AI
techniques and purposes, learner's characteristics, types of evaluation, system
performance criteria, and field of education.

Key takeaways
1. Domains and Audience:
 - Computer science education, particularly computer programming, was the most
frequent field, accounting for 55% of the studied ITSs. The majority of the audience in
this field were university students (75%).
- Health/medical and mathematics fields were in the second rank.
- ITSs designed for school student education in mathematics were the third most
frequent systems.

2. AI Techniques:
- Fuzzy-based techniques (20%) and condition-action rule-based reasoning (20%)
were the most frequent AI techniques used for the education of computer programming.
- Bayesian-based techniques were used in 50% of the ITSs in health/medical fields.
- Condition-action rule-based reasoning was primarily used (83.77%) in ITSs designed
for school student education in mathematics.
- Other AI techniques included case-based reasoning (CBR), intelligent multi-agent
approaches, and data mining methods.

3. Student Modeling:
- Student modeling played a crucial role in categorizing ITS types.
- Learner's performance (ability or procedural knowledge) and learner's behaviors in
the learning process were the most frequent characteristics used in developing the
student model.
- Combining multiple characteristics of learners was commonly applied, while in some
cases, a single characteristic was the determinant factor for the student model.

4. Customization and Personalization:

- The use of learner information, such as performance and behaviors, in the student
model aimed to enable customization and personalized learning experiences.
- Incorporating more information about the learner's current status was seen as an
approach to improve customized learning.

5. Potential Applications:
- Condition-action rule-based reasoning was the most frequent technique used,
particularly in computer programming and mathematics education.
- Case-based reasoning (CBR) showed potential for medical case reasoning tasks.
- Further exploration and application of AI techniques in different educational domains
could lead to improved ITS design and effectiveness.
Conclusion:
- This review examined factors of Intelligent Tutoring Systems (ITSs) such as
educational fields, AI techniques, learners' characteristics, evaluation methods, and
user interface.
- ITSs are increasingly used as adaptive learning tools in various educational fields.
- Adaptive learning in ITSs primarily focuses on learners' knowledge and performance,
with limited application in problem-solving and decision-making courses.
- Action-condition rule-based reasoning, Bayesian networks, and data mining are the
most frequently used AI techniques in ITSs.
- Evaluation methods for ITSs mainly involve learner-based approaches and system
performance evaluation.
- The review recommends the development of mobile-based ITSs and their evaluation,
considering the growing use of mobile devices and the potential for personalized
learning.