IDT 6800 Learning Analytics/Data Driven Decision Making

Description/Overview
In order to garner stakeholder “buy in” for projects and/or training
initiatives it is important to provide clear communication of Learning
Analytics data that will benefit stakeholders and/or an organization as a
whole. Learning Analytics is a multistep process that involves the
gathering of data, processing of said data, and the delivery/presentation of data in a digestible
way to inform stakeholders of the importance of the project initiative and drive decision-making.
For my professional project I used Learning Analytics data from the learning management
system to help inform stakeholders of the quality of course development/design and to
demonstrate to stakeholders the importance of Learning Analytics to drive the design initiative to
create more robust and quality courses for the Cyber Security program.
Skills Gained
I gained the ability to analyze learning analytics and synthesize the data to demonstrate learning
achievement occurred for students in an online learning environment. I also gained useful
experience working with raw data from a learning management system to illustrate what each
component means and relate the information to stakeholders in an understandable manner.
Lessons Learned
It’s incredibly important to incorporate collaborative practice (Standard 4) into any Learning
Analytics analysis since data alone cannot inform the “how” and “why” students have not met
and achieved the learning outcomes of a course/program. I also learned that although Learning
Analytics alone can show proof of usage and raw statistical data, you must be able to interpret
the data and relay the results in a clear efficient manner to stakeholders.
Impact/Importance
Learning Analytics provides incredibly useful statistical data to inform stakeholders decisions,
however, numbers alone cannot solve design issues of making a robust and quality course. It’s
important to incorporate other forms of data, such as focus groups, interviews, and observations
to address design questions about why students are not reaching learning outcomes and/or to
drive increased quality course material.
Final Paper
1

Definition

Learning analytics is a multistep process in which data is gathered, processed, and then shared to

stakeholders to better inform decisions that affect student learning outcomes and ultimately drive change

for student success (Elias, T., p.2, 2011). Although learning analytics and educational data mining are

used interchangeably, it's important to note that educational data mining emphasizes system generated and

automated responses to students (U.S. Department of Education, 2012, p. 13), while learning analytics

enables human tailoring of responses, adaptation of instructional content, risk intervention, and feedback.

Setting

The Learning and Design department at the University seeks to improve course quality and

decrease student attrition for the online Cyber Security program by capturing historic and concurrent

student performance learning analytics data. Learning analytics data will be collected from two live

courses while concurrently capturing historical degree program data to inform and improve curriculum

content redesign, which will begin Spring 2021 and end by Summer 2022. Concurrent and historic

student performance learning analytics will help inform data-driven decisions that impact overall course

content difficulty, program and course consistency, attrition, and course academic impact on student GPA.

This project will begin Spring 2021 with the first set (2) of courses in the degree program, and proceed

until all courses have been redesigned by Summer 2022.

Analytics and Decision-Making

In order to effectively inform decisions to drive change and improve the course quality and

redesign of the Cyber Security program, instructional designers and project participants will gather and

analyze student GPA over the duration of the program to inform course difficulty, collect and analyze

student academic assessment performance per course, and collect student learning management data (user

clicks & age), and attrition rates. Gathering data on student GPA and grades per course (Figure 1) will

help inform designers about the trends of student academic performance over the duration of the program,
Final Paper
2
which will provide in depth analysis of specific courses that pose difficult to students, thus informing

intervention strategies as well as course redesign. Real-time and historical student academic assessment

performance data will be gathered and analyzed for every course in order to monitor and illustrate

granular assessment results and concepts that proved difficult for students to learn. Student academic

assessment analytics would help inform decisions of tailoring content and/or addressing specific concepts

in more depth to students. Student learning management system data is pertinent to the study as it

illustrates the content students clicked and engaged most in and to determine the effectiveness of the

initial course design, impacting the redesign of content and curriculum in future versions of the course.

Figure 1

Learner Grades

Collecting student demographic data on age and professional experience will inform stakeholders

of the target audience of the Cyber Security program and will help inform course redesign. Analyzing age

and professional demographics allows designers another way to differentiate learners that may have

struggled or succeeded in the program, and based on the data inform course redesign elements that can be
Final Paper
3
updated with supplemental documentation to bridge a learning gap and/or inform the need for student

support services (Dawson et al., 2010, p.121). Lastly, the analysis of student attrition rates illustrated over

the duration of an entire program (2 years) will inform researchers the various points in time students

dropped from the program. Student attrition learning analytics illustrated over a 2-year period via

PivotChart, specifically a line graph chart to inform stakeholders of the trends of dropout rates in relation

to course enrollment at that time.

Analytics Project Goals

One of the strategic goals of the Learning and Design department at the University is to improve

the course quality of the online degree programs currently available. Therefore, conducting a learning

analytics initiative of the Cyber Security program will inform pertinent course redesign elements specific

to each course as well as the program as a whole, but most importantly to provide ongoing data to help

inform designers of what to improve on or what supports to provide students in real time. According to

Elias (2010) learning analytics seeks to combine historical and current user data to predict what services

specific users may find useful now (p. 4). Therefore, the collection of real-time and historical learning

analytics data will satisfy this requirement.

Learning analytics offers granular data related to each individual student as well as providing

insight into specific course impact on students’ academic performance throughout the Cyber Security

program. Implementing this learning analytics initiative would better inform designers to make

data-driven decisions and provide students with useful information that is more personalised, relevant and

timely experience and therefore, provide the company with a better bottom line” (Dawson el al., 2010).

Although end of course evaluations provide insight into the student experience, this data may be skewed

depending on the time and effort students spend on their evaluation. Learning analytics offers an

abundance of data, however, this initiative will focus on aggregate student GPA and attrition rates over

the duration of the program, collection and analysis of trends in student academic performance data per

course, and lastly, LMS (usage clicks & age) data to inform course redesign and content tailoring.
Final Paper
4
Business intelligence and web analytics have proven the importance of website visitor usage data

collection and how data informs strategic thinking and decision making capabilities (Baker, 2007, p.2).

All data collected in this initiative will be used strategically to help inform possible barriers for students,

whether academic of social in nature, and used determine the best course of action for program redesign

elements.

Data & Schema

Learner records on student historical GPA data will be collected from those enrolled in the Cyber

Security program, which consist of 10 courses. Attrition rates will also be collected from the enrollment

services department, and will continue to be collected throughout the duration of this initiative. Human

resources demographic records will be captured for student age and professional experience for those

enrolled in the program. Learning management system analytics will be collected from each course within

the Cyber Security program and will focus on capturing student performance grades and trends of

activities and assignments within the course, as well as high usage areas of the system. High usage data

from the learning management system will be collected and focus on area clicks, date/time clicks, and

user clicks within each course.

Final Paper
5
Bibliography

Baker, B/ (2007). A conceptual framework for making knowledge actionable through capital

formation. D.Mgt. dissertation, University of Maryland University College, United States -

Maryland. Retrieved October 19, 2010, from ABI/INFORM Global. (Publication No. AAT

3254328).

Dawson, S. (2010).’Seeing” the learning community: An exploration of the development of a

resource for monitoring online student networking British Journal of Educational Technology,

41(5), 736-752. doi:10.1111/j.1467-8535.2009.00970.x.

Elias, T., (2011, January). Learning analytics: Definitions, processes, and potential. Retrieved

from http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf on April 12,

2015.

U.S. Department of Education Office of Educational Technology. (2012, October). Enhancing

Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue

Brief.

Wikipedia. (n.d.). Learning analytics. en.wikipedia.org. Retrieved October 20, 2020, from

https://en.wikipedia.org/wiki/Learning_analytics