Rabeeya Arif

01-235191-026
Erum Raheem
01-235191-102

Student behavior analysis during E-Learning Environment

Supervisor: Ma’am Mehwish Parvaiz

Co-supervisor:

Department of Computer Science

Bahria University, Islamabad
 CERTIFICATE

We accept the work contained in the report titled “Student behavior analysis
during E-Learning Environment ”, written by Mr.’s Rabeeya Arif and Ms.’s
Erum Raheem as a confirmation to the required standard for the partial fulfillment
of the degree of Bachelor of Science in Computer Science.

Approved by …..

Supervisor: Name of the Supervisor (Title)

Internal Examiner: Name of the Internal Examiner (Title)

External Examiner: Name of the External Examiner (Title)

Project Coordinator: Name of the Project Coordinator (Title)

Head of the Department: Name of the HOD (Title)

Chapter 1

Introduction
Learning has traditionally been believed to occur in a classroom or face-to-face when the
instructor and students are physically present. But today the era in which we are living right now
and with the on-going pandemic e-learning is increasingly being adopted. E-learning also known
as web based communication has become more easy for students and reliable as they can access
the lectures or course content at any time and as per student’s convenience. But real time
analyzing behavior of students is a crucial part, classroom monitoring has become a necessity.
Though there are several works on student’s and teacher’s behavior but there is no such work
which extracts and predicts the behavioral pattern of students in real time or during online
classes such as actions like fighting, falling down, or any other absurd actions which are
prohibited or allowed during the e-learning environment. From the existing systems we could
observe that the surveillance cameras in classrooms obtained data of any unobtrusive activities
which were performed seldom.

Taoufik et al. [1] proposed a method for classifying student behavior in classroom based on
facial expressions. They aimed is to identify whether the students are understanding or not
understanding the lesson. Initially, they pre-trained the model tuned (Model α) on the facial
expression from video sequences of the large dataset 1. Then, they transferred and fine-tuned the
model to classify students’ behavior in classroom (Model β) using video sequences of the small
dataset 2. It consisted of three main steps: (i) face detection and tracking, (ii) facial expression
recognition, and (iii) student behavior classification. Identifying the good or bad behaviors of a
large number of students was considered a big challenge. To address this challenge, he proposed
a new students’ behavior approach based on deep transfer learning.
Banerjee et al. [2] deployed a model using image frames as the input. They created and
annotated the dataset with augmented image frames according to the everyday activities found
inside the classrooms to analyze the students’ and teachers’ behaviors. But using deep learning
techniques, they trained his model and thus detected various class labels, which are generally
observed in computer-enabled teaching laboratories. The input image is obtained from the
cameras (video surveillance cameras or CCTV cameras). Then they detected the multimodal
data through deep learning model using object detection and localization. They proposed a
method with better accuracy and also developed a faster detection model that can detect the
students’ and teachers’ behavioral patterns. They created a dataset and tested the results with the
proposed method and obtained an accuracy of 76.5%.

Zhou et al. [3] developed a system which consisted of about 100 surveillance videos captured
by cameras installed in a corridor without overlapping. These cameras started recording student
activities during a pre-set period of time, about ten minutes after class. Then they sampled these
videos every 0.5 seconds and produced latent keyframes of danger behavior dataset. After
removing irrelevant monitoring areas, we cropped all the frames to a specific size i.e
(2400×1600). They used LabelImg, a popular open-source annotation tool in object detection, to
finish the annotation work. Specifically, they selected three representative dangerous behavior
categories to label which were fight, tumble, and squat. Through this model they evaluated the
dangerous activities which were performed by the students. They presented a novel method
DangerDet which integrates both object detection and pose estimation for detecting three
dangerous behaviors of students on campus. By designing efficient feature aggregation methods
and adjusting the loss function, the training robustness and final accuracy of DangerDet was
improved. But in the above mentioned systems there were some limitations like occlusion, gaze
direction and adverse lighting conditions. Detecting various behavioral patterns were hard to
detect in the learning environment. A specific threshold was not set for detection. Or in action
fight got less accuracy in detecting dangerous activities performed.

Hence from the above proposed systems we can conclude that no such application has been
made that will evaluate student behavior in real time during online classes so we proposed a
method to evaluate the actions and activities performed by the students in an online learning
scenario. We used our own data sets for our application which helped in monitoring the
behaving patterns of students and alert the teacher or instructor about the prohibited actions.

2.Objective:
"To design a real-time application to detect student behavior patterns during e-learning
environment".

3. Problem Description
The reason why we developed this project is because as there is a pandemic going on the classes
are sometimes online or physical. During physical system it is easier for the instructor or mentor
to monitor the students and keep a check of the students who are attentive and who are not. But
during online classes the students do not pay much attention to the lecture, and it becomes
difficult for the teacher to see which student is attentive and who is not. So this application
became helpful for the teacher as it alerts them whenever any prohibited action is done.

4. Methodology
We created and used different data sets for training our model architecture. Datasets provided the
information of the actions of the students which helped in detecting and alerting the teacher. The datasets
were given as input in the form of images. Firstly, our application removed the background or noise in the
data set. After this phase it verified the student if it really is a student, a computer or a machine. After the
verification process the extra features were determined of student like the actions which they are
performing. In the end the classifiers were given and the output was generated. Following is a flowchart
which represents the methodology
5. Project Scope:
The main target of our project is the classroom environment. The purpose of our project is to
understand student behavior about online classes. Acknowledging the importance of
investigating faculty attributes about online education, our goal is to move the spotlight onto
students as well, second, our focus is on attitudes of students. The project defines a way to
improve interaction between teacher and a student during online classes. For it made it easier for
the instructor to teach and deliver the lecture.

But all the systems are prone to errors. The risks which are involved in our project are:
Internet connectivity issue. Camera not being in the proper position where it cannot detect the
image or data set properly. A fixed threshold of images be defined to detect datasets and verify
them easily

6. Feasibility Study
We tried our best to meet the schedule of our project as we worked through out our semesters
and used the best technology for our project as we wanted to use this project for future use as
well. We used simple languages and logic so it became easy for other users to understand and
make it more feasible.

i. Risks Involved:
 Internet connectivity issue
 Wrong camera position
ii. Resource Requirement:
 Camera
 A proper system for the application to run

7. Solution Application

The main areas of domain which we have targeted is educational domain as it is focused on e-
learning environment. Which determined the performance of student if they are attentive or
performing any unethical actions or not. We used this application in classroom environment

8. Tools/Technology
The tools which we used are Colab, Jupyter Notebook and GPU Processor and the technologies
used are Python, Deep learning and Artificial Intelligence using libraries like Scikit, terson flow,
Pycharm, Pandas, Keras etc
Chapter 2

Literature Review:
Literature review is a systematic method of identifying, evaluating and interpreting the work
(similar to yours) produced by others. This chapter should set the project into context and give
the proposed layout for achieving the project goals. It is an important chapter especially if the
project involves significant amount of groundwork. Review prior work critically, identify gaps in
knowledge/areas of application and build an argument for your own work. When referring to
other pieces of work, cite the sources where they are referred to or used, rather than just listing
them at the end.

The aim of this study is to identify and analyze student behavior on a specific platform. Reason
behind this research is that the past studies have not presented sufficient literature to the body of
knowledge.

As stated by Cucchiara et al. [1] in their study, postures matter more for the students as their
attention and focus can be interpreted and the feedback obtained from them through this can be
more accurate. They aimed to monitor the behavior of people in the home (especially for elders
with limited autonomy) and define potential alarm situations. They defined the event “fallen
person” that is recognized whenever an object classified as person changes its posture to “laying”
and remains in that posture for a given period.

Similarly, Jalal et al. [2] in their study used hybrid Spatio-Temporal algorithm to key body points
and to measure UT-Interaction in martial arts training in e-learning environment. They used UT-
Interaction dataset which contains 20 video sequences of six different behavior patterns between
two individuals. They used methods like Silhoutte Detection and Silhoutte Segmentation as an
early pre-processing stage to detect sequence based on the student’s heights and width pd
spatially connected components. They used different features for detection of behavior. But their
study does not have enough knowledge on the behavioral patterns of students which could alert
the trainer if the student is performing the prohibited actions or not.

Bo sun et al. [3] stated that research like pedagogy and educational psychology requires a
technology which could automatically recognize, detect and caption students’ behaviors. They
used datasets like UCF101, HMDB and MSVD. They collected videos of 128 classes in different
disciplines and in 11 classrooms. For detection purposes they used temporal detection data
module with a total number of 4542 samples, for the detection of actions they used 3343
samples, 4276 for recognition part and 4296 for captioning part. But as students may
simultaneously perform different actions which may seem similar hence to detect these actions
became a challenge for the researchers. Also the classroom environment could contribute to a
low video quality which could result in occlusions, the presence of a number of subjects or a
huge difference in subject size. These were some challenges faced by the researchers and hence
it could not reach a high level of accuracy.

Morris et al. [4] proposed a system to examine the student engagement in online classes or
courses to determine how many students completed the course or how many will withdraw the
course through an analysis of student behavior in online classes. Three hundred and fifty-four
students were used in the data analysis for training phase. They interpreted the participation of
students by accessing computer. They used multiple regression analysis to evaluate the
performance of students and approximately 31% of variability was achieved. The researchers
used t-tests for determining whether the completers and withdrawers differed significantly from
each other in frequency and duration of participation. For this purpose they used Multiple
Regression which raised a question of how the system will accurately predict the linear
combination of student participation measures. But the system could not fully explain the
concept of withdrawal behavior also it consists of a gap in determining the lack of attention to
the quality of the online discussion. This could not help in recognizing the level of students focus
and their participation in class.
Maria et al. [5] stated in their study that social interactions are building blocks of a certain
society. The subjects in these interactions can be in different ways for instance human to human
or human to mobile etc. They used the HAR. BIT-Interaction datasets which demonstrated eight
common occurring interactions in outdoor or indoor settings. The procedure for recognition was
carried out in two modules where each module has four phases which were detection,
segmentation, feature extraction and classification. The dataset used contains eight different
human activities or interactions commonly performed for example bow, handshake, high five,
hug, kick, push, pat, boxing. They separated the data into 60% for training of their model and
40% for testing of the proposed methodology. Their methodology targets the domain of human
computer interactions, virtual reality, surveillance cameras and e-learning. However, in their
video sequence background objects are also involved which could result in occlusion. Hence
their system does not have much information on behavior of humans.

For better exchange and cooperation with other nations around the world, English is very
necessary, in this modern era with the diversity of English learning methods ,its’s essential that
data mining be applied to online English learning Education .Due to the steady application of
data mining techniques and the improvement of the online learning system, its application in
education is also more extensive. In the large amount of learning data and student behavior data,
the traditional methods have the problems of low clarifying efficiency, memory requirements,
and large prediction error. Therefore, this paper proposes a method of student behavior analysis
in online English education based on data mining. The student behavior data is collected, and
online English education learning behavior model is fixed. The data mining model is built to the
obtained behavior data through data preparation, data statistics, and analysis. the apriorism
algorithm is used to mine association rules and calculate the similarity of data followed by the
application of a fuzzy neural network to mine the behavior data of English online education
students. The experiments show that this method has high data processing efficiency, takes less
space, and produces a low prediction error.

Students have fabricate a number of learning records and a number of educational data through
online learning. In order to collect useful information from these huge information, teaching
activities, improve teaching quality, provide learners with personalized learning support, and
teach students according to their bent , educational big data analysis has Charm  substantial
observation of educators. Under the collision of informatization, education and information
technology are greatly integrated, and network education is in the ascendant. At theother hand ,
the theories and methods of artificial intelligence are gently applied on the educational field.
Learning analysis, data mining, and other analysis methods provide effectual analysis methods
for online learning process. Therefore, this paper proposes a design of online learning early
warning model based on artificial intelligence. When system is implemented, based on existing
evaluation index system, the online learning early warning model of analytic hierarchy process is
introduced that gives different evaluation indexes and unequal weights to confirm the fairness
of evaluation. On the basis of understanding the artificial intelligence algorithm, the
corresponding detection indicators also be selected on the basis of objectives of the online
learning early warning model and the need of actual work of network operation and
maintenance. Then, follow the corresponding data flow of artificial intelligence technology and
analyze data preprocessing, feature selection, and model training.

This research is based on examining attitudes high school students’ towards e-learning. Also,
the current study look into the effects of online learning on high school students’ general
English. To achieve these goal , the Oxford Quick Placement Test (OQPT) was direct to 73
students, and 50 of them who are in the intermediate level are selected as the study candidates.
Then, the selected candidate were randomly divided into two groups: one experimental group
(EG) () and the other is control group (CG) (). Then, a general English pretest was allot to both
groups, and then, the candidates of the EG obtain the treatment through online instruction
(WhatsApp). Three lessons of their course (Vision book 3) were taught to the candidates
through the WhatsApp application. At the same time , the candidates of the CG were destitute
of online instruction. They were instruct traditionally, through counducting a face-to-face class.
After teaching all the 3 lessons, both groups grab the posttest of general English. In addition, an
attitude questionnaire was administered to the candidates of the EG to examine their attitude on
e-learning, and then ten students were interviewed. The study results shows that the EG
performed better then the CG. The results of one-sample-test showed that EFL students held
positive attitude on e-learning in teaching English. The interview results shows that digital
literacy, incapacity to focus on screen for a long time, and lack of availability to high-speed
Internet were the problems of e-learning.

The paper shows the moderating role of teachers’ academic support between students’
satisfaction with online learning and academic motivation during the COVID-19 in Pakistan. It
was belief that teachers’ academic support is likely to temperate the relationship between
students’ satisfaction with online learning and the academic motivation of undergraduate
students. A correlational research design was used and a sample of 406 students within the age
18–22 years (M = 21.09, SD = 1.41 (male); M = 20.18, SD = 0.71 (female)) were included. The
sample was select through the judgmental strategy from different universities Pakistan.
Students’ Satisfaction with Online Learning Application , Teacher’s Academic Support Scale,
and Academic Motivation Scale were used.As the results of moderation analysis through process
macro 3.5 disclose that teachers’ academic support plays a moderating role in students’
satisfaction with online learning and the academic motivation of undergraduate students.
Conclusions will provide support to educational administrators, policymakers, course designers,
and curriculum developers for planning the curriculum and Express a system to identify that
students need different support perfectly in digital learning environment.

With the pandemic’s general effect, the “New Normal” in instruction in Philippine education
embraces online learning. This study considered the effect of online learning on students’
motivation, self-efficacy, and anxiety in mathematics. The study used quasi-experimental
research, specifically a one-group pre-post-test research design. Two adapted research
questionnaires on mathematics motivation, self-efficacy, and anxiety were employed. Results
show that students’ mathematics motivation and self-efficacy have considerably decreased over
the 6-week pilot test of concurrent online learning. Students’ anxiety remained “High” before
and after the implementation, signifying fear and uncertainty of the new normal in instruction.
The paper shows that slow and unstable Internet connection, less motivation to do self-study,
plenty of activities at home, and chores were some of the main reasons students got difficulties in
learning the subject matter and adjusting to the “New Normal.” Results indicate the betterment
of online learning processes, emphasize government projects for faster Internet connectivity.
Also, focus on engaging classroom activities, specifically in mathematics, must be established,
enhancing learners’ motivation and self-efficacy and decreasing anxiety.
Chapter 3

Requirement Specifications
Existing System
Prior systems related to our project had somewhat similar features and working but they are
different is such a way that some focused on the postures or some focused on daily routines of
students in physical classes like a fight or pranking or bullying.

Those systems were not able to overcome the problem of analyzing students’ behavior during
online classes. From recent studies some drawbacks or gaps were concluded like the video
quality could become a challenge. Hence, we developed a system which will overcome all the
challenges like these.

Proposed System

Our system is an application which will detect student’s behavior based on their movements and
actions. This approach is towards E-learning environment, in which the system will determine
the student’s behavior and alert the teacher about their participation during the class which is to
enhance the classroom environment.

Requirement specifications:

 Functional requirements:

1. Input:

Datasets will be given in the form of images as input.

These data sets will detect the frames of specific threshold.
Size of the images will be fixed as it will help in detecting the image clearly.

2. Pre-processing

After the frames are detected the images will be processed.

In this process the we will import the videos, read the content and extract frames
from the videos and save them as temporary memory.
 3. Detection of objects:

After processing the frames, the objects in the dataset like pictures on walls or
home furniture will be first detected and then removed.

4. Verifying humans:

The system will then verify the students.

This will be done in following ways:
 Template matching
 Movement based
 Kalman filter

5. Extracting features:

Extra features of the dataset will be extracted.

Which includes:
 Orientation
 Velocity
 Movement.

6. Adding classifiers:

Two types of classifiers (SVM) will be added i.e Allowed and Prohibited.
If the movements are detected the application will generate an alert.

7. Alerting the user:

The system will generate an alert whenever any unprohibited action is detected.

 Non-functional requirements:

• Security
• Flexibility
• Integrity
• Efficient
Use case diagram:
Table definition:

Use case Object detection

Pre-condition The picture should be big and clear enough to
detect the objects.
Post-condition The objects are detected, and the noise and
background objects are removed
Basic path The user will provide a picture as input. Then
the system will then process it and if the
picture is clearly detected the system will
remove the objects and display it.
Alternative path If the image is not pre-processed in case the
image is big than the given threshold, then it
is detected again
Exceptional path If the system crashes or system failure occurs.

Use case Human verification

Pre-condition The objects are detected successfully.
Post-condition The system verified correctly that it is a
human not a dummy based on the
movements.
Basic path The system detects human and verifies it by
determining the movements of human.
It will verify by monitoring the movements of
human within a specific period of time.
Alternative path If the system does not detect any movements
within 10-15 seconds, then the verification
process will start over.
Exceptional path In case the human is wearing a face mask a
hat which can make it difficult to verify.

Use case Feature extraction

Pre-condition The verification process is completed
successfully, and orientation, velocity and
movements will be detected.
Post-condition The features are extracted successfully
Basic path The system will check the orientation of
frames in the dataset, and it will check if the
features are correct and clear. After this the
 features will be extracted
Alternative path Extracting
Exceptional path In case the pixels are not clear the features
can’t be extracted.

Use case Alert user

Pre-condition
Post-condition
Basic path
Alternative path
Exceptional path

SEQUENCE DIAGRAM:
Chapter 4

Design
This chapter discusses the different aspects related to the design of the system.

System Architecture
Our project has no hardware requirements as its working is purely based on predefined datasets
(prerecorded videos) . We used libraries scikit, numpy or pandas.

Design constraint
Design Methodology
The aim of this system is to analyze videos and perform the task of detection and recognition of actions
from a set of predefined undesired situations. The development of this system is based on the following
activities:
• Select the video from the respective dataset (sleep, jump, stand, laying etc).
 Perform frame extraction from the selected video.
 Extract the features to characterize various actions.
 Recognition outcomes shown to the users.

In The first step frame extraction from the previously selected video is performed using frame extraction
method.
In second step extracting the features to characterize various actions
Then lastly the result or an alert will be displayed to the user.

High Level Design

This section will further describe in detail elements of the project architecture.

Low level design

Database design
GUI Design
External interfaces:
References:

 Cucchiara, R., Grana, C., Prati, A., & Vezzani, R. (2004). Probabilistic posture
classification for human-behavior analysis. IEEE Transactions on systems, man, and
cybernetics-Part A: Systems and Humans, 35(1), 42-54.

 Jalal, A., & Mahmood, M. (2019). Students’ behavior mining in e-learning environment
using cognitive processes with information technologies. Education and Information
Technologies, 24(5), 2797-2821.

 Sun, B., Wu, Y., Zhao, K., He, J., Yu, L., Yan, H., & Luo, A. (2021). Student Class
Behavior Dataset: A video dataset for recognizing, detecting, and captioning students’
behaviors in classroom scenes. Neural Computing and Applications, 33(14), 8335-8354.

 Morris, L. V., Finnegan, C., & Wu, S. S. (2005). Tracking student behavior, persistence,
and achievement in online courses. The Internet and Higher Education, 8(3), 221-231.
 Jalal, A., Mahmood, M., & Hasan, A. S. (2019, January). Multi-features descriptors for
human activity tracking and recognition in Indoor-outdoor environments. In 2019 16th
International Bhurban Conference on Applied Sciences and Technology (IBCAST) (pp.
371-376). IEEE.