A Learning Analytics Tool for Monitoring and
Improving Students' Learning Process
Yusep Rosmansyah1, Nunung Kartikasari2, Aciek Ida Wuryandari3
School of Electrical Engineering and Informatics
Bandung Institute of Technology
Bandung, Indonesia
1
yusep@stei.itb.ac.id, 2nu2ng.kartikasari@gmail.com, 3aciek@lskk.ee.itb.ac.id
Abstract— A learning analytics tool that facilitates teachers in
monitoring students’ learning process and enables them to early
recognize the signs of failure was developed. The tool was added
to an e-learning system based on Moodle operated by a primary
school. Two parallel classes (i.e., an experimental group and a
control group) of students were involved in the research for
duration of one semester. The statistical test results showed that
better learning improvement occurred to the experimental group
with an increase in the average marks by 5.3 (out of 100 points).
The use of learning analytics has helped improve the effectiveness
of the students' learning process.
Keywords—Learning analytics, e-learning
I. INTRODUCTION
E-learning (electronic learning) is a type of learning that
allows the delivery of teaching materials to students using the
computer, multimedia, and Internet technologies. E-learning
offers various features that can improve availability, flexibility,
effectiveness, and efficiency of a learning process. In
particular, it can have a special feature to monitor the learning
activities of students, known as learning analytics. In the
business world, learning analytics is similar to customer
analytics that becomes an important part of a company's
marketing strategy. Similarly, learning analytics becomes a
potential tool to promote effective learning, since it provides
tools, technologies, and platforms to empower teachers to
involve, inspire, and prepare students to achieve success in the
learning process [1].
II. LITERATURE STUDY
A. Learning Analytics
Learning analytics can be defined as the process of
measuring and collecting data on students and learning with the
aim of improving teaching and learning processes through the
analysis of data [2]. Learning analytics is at the level of
understanding of the overall learning system (holistic) to
support the decision-making process. Another definition was
proposed by Arnold [3] which is data interpretation activity
related to students that is acquired during the academic activity
process to predict the performance and detect potential
problems that may arise in the future. There is also a wide
range of data explicitly drawn from diverse sources such as:
test results, discussion records,
978-1-5386-0475-5/17/$31.00 ©2017 IEEE
Authorized licensed use limited to: b-on: Instituto Politecnico de Viana do Castelo. Downloaded on May 10,2020 at 11:41:41 UTC from IEEE Xplore. Restrictions apply.
paperID_99_iceei2017.pdf
an d interaction with teachers or
fellow students, extracurricular activities, and so forth.
Learning analytics simply focuses on improving the
performance of students so that the data collected might be
used by teachers or instructors and heads of educational units
in fixing or revising ecosystems or components of learning
environment in the surrounding areas, such as curriculum,
educational programs, instructional design, and so on [4].
B. Domains of Competencies of the 21st Century Learning
According to Pellegrino et al. [5], there are three domains
in developing competencies required by the current students
living in the 21st century, namely:
1. The cognitive domain that includes expertise in critical
thinking, information literacy, logical thinking,
argumentation, and innovation.
2. The intrapersonal domain that covers self-management
which includes competencies such as flexibility,
initiative, appreciation of diversity, and metacognition
(the ability to reflect on one’s own learning and make
adjustments accordingly).
3. The interpersonal domain that includes competencies
such as communication, cooperation, collaboration,
responsibility and conflict resolution.
Referring to [5], the study of learning analytics reported
in this paper was limited to the cognitive and intrapersonal
domains.
C. Related Work
Huang et al. [7] developed a real-time learning analytics
based on a database log. This represents a sophisticated
learning analytic system that was too complex to be operated
by teachers of primary schools in Bandung areas. A similar
system was developed by Dunbar at al. [8], where data log was
collected continuously so that the analysis can be applied as
further as for revising the curriculum. In this research, a
simple-to-use learning analytics tool by ordinary Indonesia
teacher was set as the main objective.
 III. DESIGN AND DEVELOPMENT
Overall research project followed the method described in
[9]. Starting with the need analysis, the following are
functional purposes of the system, which can be:
1. Saving time and reducing operational complexity for
teachers.
students learned while its teacher used the proposed learning
analytics tool.
The statistical testing proceeded as follow: first, each group
was given the same questions (pre-test) to find out the average
value between groups. Pre-test results for each group can be
seen in Fig. 6. All scores range from 0 to 100.

2. Helping teachers to monitor the progress of the

learning process of students.
3. Providing additional assistance to students who need
more attention and learning support.
4. Accelerating the students' learning process.

Based on the conducted analysis, a system architecture was

needed, covering aspects of business processes and system of
interface design. In short, adopting the method described in [6],
in this paper only the main use case diagram of the developed
learning analytics is shown, such as in Fig. 2.

Inspect Individual Student

Progress Fig. 3 Managing group of students.

Manage Group of Students

based on Progress

Treat Group of Students

Accordingly

Manage Schedule of
Learning Ac vi es

Fig. 2 Use case diagram

IV. IMPLEMENTATION AND EVALUATION

Implementation of the learning analytics was performed by
modifying the Moodle web application. Fig. 3-5 show Fig. 4 Individual student progress.
screenshots of main menus available to a teacher. When all
stages of development and testing had been done and the
learning analytics tool was ready to use, it was time to apply
the tool to students. Two classes of 5th grade students in UPI
Primary School, Bandung, Indonesia, were chosen as
respondents. The cognitive domain of [5] was elaborated to
Science subject. The proposed tool was used for a period of
one semester. The respondents were from middle class of
economy.
The main objective of this paper is the students'
improvement in learning outcome, which will be evaluated by
means of statistical analysis. Meanwhile, other objectives such
as time saving, complexity reduction for teacher in monitoring
the students' progress, and pinpointing certain students who
need more attention, were evaluated by way of interview, in
which both guardian teachers of the two classes confirmed
positively. Fig.5 Managing treatments for students in different groups.
Statistical testing was conducted using a sample of 53
students coming from 2 parallel clases: control group (27
students of Class 5A) and experimental group (26 students of
Class 5B). The control group was the students attending the
learning process normally. Meanwhile, the experimental group

Authorized licensed use limited to: b-on: Instituto Politecnico de Viana do Castelo. Downloaded on May 10,2020 at 11:41:41 UTC from IEEE Xplore. Restrictions apply.

Fig.6 Pre-test results of control and experimental groups.
Average scores of 74.2 and 75.3, respectively for the
control and experimental group, it can be seen that both groups
have very similar competence at the beginning of semester.
After both groups have gone through the learning process, a
post-test was given to determine the effectivity of the students'
learning progress, with and without using the proposed tool.
The results of the average value of the post-test of the two
groups can were 77.6 and 83.8, respectively, as depicted in Fig
7.
Fig.7 Post-test results of control and experimental groups.
From the average scores of pre-test and post-test, it can be
seen that both groups improved in terms of scores. The average
scores of the control group was up (77.6 - 74.2) = 3.4 points,
and it was up (83.8 - 75.3) = 8.6 for the experimental group.
There was a difference of 5.1 points between the two groups. It
can be concluded that there was an increased competence of
the students in the experimental group, which used the
proposed application.
A. Hypothesis Testing
Following [10], hypothesis testing was conducted using
unpaired t-test (independent t-test). T-test can be conducted if
Authorized licensed use limited to: b-on: Instituto Politecnico de Viana do Castelo. Downloaded on May 10,2020 at 11:41:41 UTC from IEEE Xplore. Restrictions apply.
groups of data to be tested are homogenous (have the same
variances) and have normal distribution. To determine the
variance similarity (homogeneity) in an unpaired groups of
data, the Levene test will be performed. A criterium for groups
of data to be classified as having same variance is if sig. > 0.05,
and otherwise. If the variance is the same, then the t-test can be
performed (by using the Equal Variance Assumed).
To test the normality, Shapiro-Wilk test will be conducted,
since the number of respondents from each group is less than
30.
When the homogeneity and normality tests have been
determined, decision-making criteria for the t-test results are as
follows:
Ho: There is no significant improvement difference of learning
progress between the control group and the experimental
group.
Ha: There is a significant improvement difference of learning
progress between control group and the experimental
group.
The deciding factor is the result of the t-test, as follow:
If t-test's sig. (2-tailed) > 0.05
then Ho is accepted and Ha rejected
else Ho is rejected and Ha accepted.
To make this hypothesis testing simpler, a pseudocode can be
presented as follow:
If (t-test's sig. (2-tailed) > 0.05)
then (there is NO significant improvement)
else (there is SIGNIFICANT improvement).
B. Results and Analysis
During the pre-test, the result of Shapiro-Wilk tests showed
values of sig. of 0.616 for control group and 0.100 for
experimental group. The corresponding values for the post-test
were 0.228 and 0.061. Since all values are > 0.05, it was
concluded that all data were considered to be normally
distributed. The remaining tests are homogeneity and t-tests.
Referring to Table I, it is seen that the results of Levene test
in the pre-test is sig. = 0.826, so it can be concluded that
variances of the control and experimental groups are
considered the same. Based on the decision-making criterium
and looking at the results of t-test calculation in Table III, it
was shown that the sig.(2-tailed) = 0.763 (which is > 0.05). It
means that Ho is accepted (and Ha is rejected), which means
there is no significant difference of scores between control
group and the experimental group during the pre-test.
Looking at the post-test results in Table II, it can be seen
that the Levene test to determine the variance of the data group
is sig. = 0.181 (which is > 0.05) so that it can be concluded that
data group variance is categorized the same. Based on the
decision-making criterium and the results of t-test calculation
in Table IV, it shows that the value of t-test is sig.(2-tailed) =
0.440 (which is < 0.05), so it can be concluded that Ho is
rejected and Ha is accepted. This means that there are
significant differences of results between control group and
experimental group. In other word, statistical test
 TABLE I
LEVENE TEST RESULTS OF PRE-TEST

Levene's Test for

Equality of
Variances t-test for Equality of Means
95% Confidence
Sig. Mean Interval of the
(2- Differe Std. Error Difference
F Sig. t Df tailed) nce Difference Lower Upper
Equal
variances .049 0.826 .304 51 .763 .936 3.081 -5.250 7.122
assumed
Equal
variances .303 50.148 .763 .936 3.087 -5.263 7.135
not assumed

TABLE II
LEVENE TEST RESULTS OF POST-TEST

Levene's Test for

Equality of Variances t-test for Equality of Means
Sig. 95% Confidence Interval
(2- Mean of the Difference
taile Differe Std. Error
F Sig. T df d) nce Difference Lower Upper
Nilai Equal variances
assumed
1.836 0.181 2.069 51 .044 6.365 3.076 .189 12.540

Equal variances not

2.059 47.254 .045 6.365 3.091 .148 12.581
assumed
____________________________________________________________________

2. From these tests, it indicated that the group using

TABLE III
the application has higher learning improvement
INDEPENDENT T-TEST RESULTS OF PRE-TEST than that who did not.
Equal 3. From the post-test results, it can be concluded that
Equal Variances
Variances
Assumed
Not Assumed there is a significant difference of learning progress
F .049 between the experimental group and the control
Sig. .826 group.
t .304 .303
df 51 50.148
Sig.(2-tailed) 0.763 .763 TABLE IV
Mean Difference .936 .936 INDEPENDENT T-TEST RESULTS OF POST-TEST
Std. Error Difference 3.081 3.087
95% interval of the confidence Equal
Lower -5.250 -5.263 Equal Variances
Variances
Upper 7.122 7.135 Not Assumed
Assumed
F 1.836
Sig. .181
t 2.069 2.059
Based on the obtained resuls, some findings can be df 51 47.254
discussed as follows: Sig.(2-tailed) 0.440 .45
1. In this research, the application of the proposed Mean Difference 6.365 6.365
Std. Error Difference 3.076 3.091
learning analytics has helped improve the learning 95% interval of the confidence
of cognitive and intrapersonal domains of the 21st Lower .189 .148
century skills. Upper 12.540 12.581

Authorized licensed use limited to: b-on: Instituto Politecnico de Viana do Castelo. Downloaded on May 10,2020 at 11:41:41 UTC from IEEE Xplore. Restrictions apply.

V. CONCLUSION
Based on the research design and implementation in the
field, some conclusions and future work can be expressed as
follows:
1. A simple-to-use learning analytics tool has been planned,
designed and implemented for a semester to 2 classes of
the 5th grade students. Tests and evaluation of the
impacts have indicated that there is a significant
improvement in terms of scores (5.3 points). This
indicated that cognitive and intrapersonal aspects of
learning have been improved.
2. The proposed learning analytic tool has helped teachers
to treat the "slowing-down" students to catch up with the
rest. It saved teachers' time and reduce complexity
compared to traditional method in monitoring the
students' learning progress.
3. In the subsequent research, better improvement of
learning effectivity is expected. This may require the
development of learning analytics using predictive and
prescriptive analysis. Both of these analyses may assist
teachers in predicting things that will happen in the
future based on the patterns available at present and help
in the decision making process to determine the best
treatment for the students.
Authorized licensed use limited to: b-on: Instituto Politecnico de Viana do Castelo. Downloaded on May 10,2020 at 11:41:41 UTC from IEEE Xplore. Restrictions apply.
REFERENCES
[1] Elias, Tanya. Learning analytics: Definitions, processes and
potential. 2011. (Accessed from
http://learninganalytics.net/DefinitionsProcessesPotential.pdf,
December 2016.)
[2] Long, P. & Siemens, G. Penetrating the Fog: Analytics in Learning
and Education. Educause Review, 46(5), 31-40, 2011.
[3] Arnold, K. “Signals: Applying Academic Analytics". 2010. Accessed
from: http://er.educause.edu/articles/2010/3/signals-applying-
academic-analytics)
[4] Macfayden, L. P., and S. Dawson. “Mining LMS Data to Develop an
‘Early Warning’ System for Educators: A Proof of Concept.”
Computers & Education 54 (2): 588–599. 2010.
[5] Pellegrino, J.W. & Hilton, M.L.Education for Life and work:
Developing Transferable Knowledge and Skills in the 21st
Century.Washington, DC: National Academies Press., Eds.2013.
[6] A. Dennis, B. H. Wixom, and D. Tegarden, Systems analysis and
design: An object-oriented approach with UML. John Wiley & Sons,
2015.
[7] L. H. Huang, C. R. Dow, Y. H. Li, P. Hsuan and W. L. Koo, "A
Prompt Response Monitoring System for the Guiding Support of
Learning Activities," Information Technology: New Generations
(ITNG), 2010 Seventh International Conference on, Las Vegas, NV,
2010, pp. 1080-1085.doi: 10.1109/ITNG.2010.74
[8] Dunbar, Robert L.; Dinger, Molly J.; PRAT-RESINA, Xavier.
"Connecting Analytics and Curriculum Design: Process and
Outcomes of Building a Tool to Browse Data Relevant to Course
Designers." Journal of Learning Analytics, [S.l.], v. 1, n. 3, p. 223-
243, Aug. 2014. ISSN 1929-7750
[9] L. T. Blessing and A. Chakrabarti, DRM: A Design Reseach
Methodology. Springer, 2009
[10] Fraenkel, Jack R, and Norman E. Wallen. How to Design and
Evaluate Research in Education. 7th ed. Boston: McGraw-Hill, 2009.