Evaluating Mental Workload Among 3rd Year

BSIE Evening Students in Cebu Technological University Main Campus

During Their 1st Semester S.Y. 2021-2022

In Partial Fulfillment

of the Requirements in

IE-PC 3212 Ergonomics 2

BSIE 3-5

Amistad, Joseph I. Jr

Armenton, Exchel

Dibdib, Christiam Rey C.

Tindugan, Juleven A. Jr.

June 17, 2022

 INTRODUCTION

Covid 19 has caused havoc in people's lives all around the world and it changed the way
people live. One of the changes globally is the shift of colleges and universities to an online mode
of education. This transition has advantages like no transportation costs and students can learn in
the comfort of their homes. However, it has also disadvantages like long exposure to radiation to
gadgets and mental health problems. Even if students save time especially since they will not
require to go to school, there is also an increasing demand for their mental activity since education
needs to sustain and improves its quality. One of the important things to look at is the amount of
cognitive capacity required for students to perform a given task and that is the mental workload.
According to (Gopal, Singh, & Aggarwal, 2021), the factors affecting students’ satisfaction during
online classes are the instructor’s quality, students’ expectations, feedback, and the design of the
course content. Mental workload (MWL) can affect human performance and is considered critical
in the design and evaluation of complex human-machine systems (Tao, et al., 2019). Furthermore,
researchers suggest that MWL is a multidimensional concept in nature and it is different from the
physical workload and task load. For example, MWL differs from the physical workload in that
MWL emphasizes stress caused by task demands, while physical workload focuses more on strain
imposed on the human body. MWL is also distinguished from task load in that MWL reflects
individuals’ subjective experience in performing particular tasks under certain environments and
time constraints, while task load refers to external duties or the amount of work that individuals
have to perform.

In the study of (Tao, et al., 2019) using systematic reviews and meta-analyses (PRISMA)
guidelines, the findings identified 78 physiological measures from 91 original studies, which were
distributed in cardiovascular, eye movement, EEG, respiration, EMG, skin, and neuroendocrine
categories. Furthermore, Cardiovascular, eye movement, and EEG measures were the most widely
used across varied research domains, with 76%, 67%, and 73% of times reported significant
associations with MWL, respectively. In addition, according to (Rubio-Valdehita, López-Higes,
& Díaz-Ramiro, 2014), the workload (WL) perceived by college students represents the main
factor in generating academic stress. The consequences of such stress on the psychological well-
being and performance of students make it necessary to assess and monitor the presence and
intensity of stress factors in the education field. Furthermore, the researchers use NASA TLX in
evaluating Mental Workload among psychology students and the results indicate that the activities
with the greatest WL were, in this order, personal study and work, lecture attendance, group work
outside the classroom, and practical attendance. On the other hand, the study of (Atalay, Can,
Erdem, & Müderrisoglu, 2016) entitled “Assessment of mental workload and academic motivation
in medical students” using NASA TLX to measure MLW shows that MWL has a significant
impact on the dimensions of Academic Motivation (AM) for the individual. To make sure if NASA
TLX is a reliable method in measuring MWL, the study of (Longo & Orrú, 2020) of investigating
the reliability, validity, and sensitivity of three existing self-reporting mental workload measures
when applied to long learning sessions, namely, the NASA Task Load index, the Workload Profile,
and the Rating Scale Mental Effort, in a typical university classroom shows that these selected
measures are reliable and their moderate validity is in line with results obtained within
Ergonomics. Another study that was conducted by (Prastawa, Susanto, & Nofri, 2018) entitled
“Mental workload of undergraduate student (a case study in Industrial Engineering Department of
Diponegoro University” revealed that the level of mental workload that occurred among students
of the Industrial Engineering Department of Diponegoro University semesters 3 and 5 is 80.04
(high level) and the dominant scale is the Temporal Demand scale which is related to working
time of a task or job as well as the strict deadline of the report.

This study aims to evaluate the mental workload among 3rd year BSIE evening students in
Cebu Technological University-Main Campus during 1st-semester S.Y. 2021-2022. Furthermore,
the NASA-TLX method is used to complete the study aims.
 ANALYSIS

To learn more about how mental workload affects and influences people, we, as the
researcher will conduct a NASA TLX rating scale among the 3rd Year BSIE Evening students.
Further, the primary goal of this case study is to analyze and evaluate mental workloads among
3rd Year BSIE Evening students throughout their 1st Semester S.Y. 2021- 2022. All of the
members of the group are Cebu Technological University-Main Campus Industrial Engineering
students. They'll use the NASA-TLX rating scale and questionnaire to assess their various mental
workloads. Below is the NASA-TLX rating scale and questionnaire.

Figure 1: NASA-TLX Rating Scale & Questionnaire

 The NASA-TLX was created in response to the advent of subjective workload measuring
demands that are easier but more sensitive (Hancock, 1988) . The following are the steps in
measuring mental workload using the NASA-TLX method;

1. Explanation of mental load indicator to be measured

2. Weighting
3. Rating calculation
4. Product calculation: Product = rating x Factor weight
5. Calculation of Weighted Workload (WWL): ∑ WWL = Product
∑( 𝑤𝑒𝑖𝑔ℎ𝑡 𝑥 𝑟𝑎𝑡𝑖𝑛𝑔 )
6. Calculation of mental workload; score = 15

7. Interpretation of score

This tool asks student to make rating on six subscales which are the following:

1. Mental Demand (MD) measures how much thinking, deciding, or calculating was
required to perform the task
2. Physical Demand (PD) measures the amount and intensity of physical activity
required to complete the task.
3. Temporal Demand (TD) the amount of time pressure involved in completing the
task.
4. Performance (PR) measures the level of success in completing the task.
5. Effort (EF) how hard does the participant have to work to maintain their level of
performance
6. Frustration level (FR) measures how insecure, discouraged, or secure or content
the participant felt during the task

The result in conducting NASA-TLX is based on numerical value in which the mental
workload of a specific task can be determined. The interpretation of the value is shown below:

Scores Category

0 – 20 Very Low
 21 – 40 Low

41 – 60 Medium

61 – 80 High

81 – 100 Very High

Table 1: Interpretation Score of NASA-TLX

In conducting the NASA-TLX in the assessment of the mental workload of the student, a
pairwise comparison factor is used each dimension is paired with another, giving in a total of 15
combinations, and the students, who are the respondents, have to choose the dimension that
provided the most important source of workload for each combination. The result in evaluating
mental workloads among 3rd Year BSIE Evening students throughout their 1st Semester S.Y.
2021- 2022 are shown below:

Frustration
Mental scores Physical Temporal Performance Effort Level
1 90 90 98 75 89 90
2 70 65 50 65 80 80
3 50 50 50 80 80 60
4 70 75 70 70 80 70
5 80 80 80 60 70 80
6 80 80 100 80 60 80
7 85 90 81 30 90 80
8 95 61 98 40 81 81
9 70 70 75 78 78 75
10 85 65 70 60 80 95
11 80 70 70 90 75 80
12 80 80 100 40 80 100
13 90 90 100 80 80 100
 14 70 70 70 70 70 70
15 85 87 90 85 85 90
16 100 65 100 40 70 100
17 80 80 85 80 83 79
18 70 70 80 50 80 80
19 75 80 80 90 95 80
20 90 60 78 95 100 80
21 60 60 60 60 60 60
22 80 60 80 60 60 100
23 75 80 90 60 90 90
24 90 85 88 60 81 94
25 95 85 95 90 95 80
26 95 90 90 100 100 90
27 95 60 96 98 100 98
28 65 65 70 80 75 65
29 95 50 80 80 85 55
30 99 75 80 70 79 100
31 75 75 75 75 75 75
32 85 60 81 80 100 95
33 80 90 80 80 100 90
Total 2684 2413 2690 2351 2706 2742
Average 81.33333333 73.12121 81.51515 71.24242424 82 83.09

Table 2: Rating of 33 Respondents for Each of the NASA TLX Dimensions

The table above shows the rating of 33 respondents for each of the 6 dimensions/ subscales.
 Graph 1: Mean Rating for the 33 Respondents

The graph shows the mean rating of the 33 respondents for the 6 NASA TLX
Subscales/Dimensions. It is evident that the frustration has the highest mean rating while the
performance has the lowest mean rating. Based only the rating, we can say that the students of
Bachelor of Science in Industrial Engineering Evening Program have lowest mean rating but still
high performance and experience very high frustration.

Table 3: Frequency of the Respondents for 15 Pairs of Pairwise Comparison

The table above shows the frequency of the selection of the respondents to a dimension
over the other dimension. The highlighted cells are the higher frequency of selection versus its pair
dimension.

Table 4: Pairwise Comparison for the 15 Pairs of Subscales/Dimensions

The selected dimensions were indicated with 1 and must be in the rows while the unselected
dimensions were in the column. The summation of each dimension in the rows are the weight.

Table 5: Unweighted TLX Score and Weighted TLX Score

After getting the mean rating and the weight of each dimension, the two values will then
be multiplied to get the weighted rating for each dimension. Hence, Mental Demand becomes the
rank 1 which has the highest weighted mean of 406.67, while the physical demand becomes the
rank 6 which has the lowest weighted mean rating of 0.
 Graph 2: Weighted Mean Rating for the 33 Respondents

The graph shows the Weighted mean rating of the 33 respondents for the 6 NASA TLX
Subscales/Dimensions. Obviously, the mental demand has the highest weighted mean rating while
the physical demand has the lowest mean rating. Based on the rating, we can say that the students
of Bachelor of Science in Industrial Engineering Evening Program are experiencing high mental
demand and lowest weighted mean rating in physical demand which means that performing their
task/activities in the first semester required high mental demand.

The unweighted and weighted mean formula are as follow:

• Unweighted TLX Score
∑𝑅
6
• Weighted TLX Score
∑𝑹 𝒙 𝑾
𝟏𝟓

Where:
R = Rating for each dimension
W = weights or the number of times the associated dimension was
chosen in the pairwise comparison
 Unweighted TLX Score 78.72
Weighted TLX Score 79.71

Table 6: Unweighted and Weighted TLX Score

Finally, Weighted TLX or the NASA TLX for the mental workload of the students of
Bachelor of Science in Industrial Engineering Evening Program is 79.71. While, the unweighted
TLX is 78.72.

Reliability

In this section, a reliability test was performed to determine the consistency or reliability
of a set of tests or scales. A measurement's reliability refers to how consistent it is in measuring a
concept, and Cronbach's alpha is one way of determining how strong that consistency is.
Cronbach's alpha is a metric for internal consistency, or how closely a group of items is related. It
is regarded as a scale reliability indicator. The presence of a "high" alpha value does not show that
the average value is unidimensional. Below is the Alpha Cronbach Value and the interpretation of
each value.

The Alpha Cronbach Value (Konting et al.,2009)

Alpha Cronbach Interpretation

Value
0.91-1.00 Excellent
0.81-0.90 Good
0.71-80 Good and
Acceptable
0.61-0.70 Acceptable
0.01-0.60 Non acceptable
Table 7: The Alpha Cronbach Value

The formula for calculating Cronbach Alpha is:

𝑘 ∑ 𝑠2 𝑦
∝= (1 − 2 )
𝑘−1 𝑠 𝑥
 Where: 𝑘= number of test item

∑ 𝑠 2 𝑦= the sum of the item variance

𝑠 2 𝑥 = is the variance of total score

Given the formula of Cronbach alpha, we can now test if the NASA TLX Subscales
results from 3rd year BSIE Evening Students is acceptable or not. The table below is the result of
Reliability test using the Cronbach Alpha.

Alpha Level of Internal Consistency

NASA TLX Subscales 0.66 Acceptable
Table 8: Level of Internal Consistency
Table shows that the reliability level of the instrument represented by Alpha Cronbach
(Creswell, 2010), states Alpha Cronbach's value above 0.6 is considered high reliability and
acceptable index (Nunnally and Bernstein, 1994). In connection, the alpha coefficient for the
NASA TLX Subscales is 0.66, suggesting that the subscales are ACCEPTABLE.
 RECOMMENDATION

In this section, the group discuss the possible reason on resulting of high mental workload
of the student in performing their task. The recommendation are also present in order to cope up
with the high mental workload in performing the task or job especially for the College students:

Figure 2: Fishbone Diagram

In this figure, it shows the possible reason resulting of the high mental workload of the
students. Moreover, high mental workload can lead to mental exhaustion, which can lower a
student's performance in school and, in turn, have long-term consequences. Therefore, it is
important to cope up with high mental workload in performing the task in the workplace in order
enhance the mental health of the students and become more productive which result in better
performance.

The following are the recommendation in coping with high mental workload:

Self-Discipline

Having a self-discipline allow you to stay focus on your studies and achieving your goals
in life. It enables you to stay in control of yourself and the reaction of every situation you
encounter. This is the way in overcoming your negative habits and it gives you the ability to not
give up after failure and setbacks as a college student.

Motivation & Minimizing Distraction

This one is the hardest, but also the most rewarding. Everyone has days where they struggle
staying especially motivated. However, successful teachers and students know that staying
motivated throughout their education program ensures that they get the most out of their learning.
The researcher recommends to stay motivated and avoid distraction through this, it will increase
your productivity in doing your activities everyday and achieve your personal goals.

Time Management

Managing your time correctly can offer you the discipline you need to manage both your
academic and personal to-do list while learning online, from getting up early and prioritizing tasks
to avoiding distractions and setting goals. It's a time-consuming stress-reduction technique that
will benefit many elements of your life for many years to come.

Get enough sleep

Getting enough sleep is the first step in teaching yourself how to manage stress. A good
night's sleep is essential not only for online learning performance, but also for managing stress.
This allows your body and mind to recharge, leaving you refreshed and alert when you wake up.

Take time to relax or Unwind

Taking a break and take time to unwind can help you manage stress while you pursue your
online education. It's not beneficial for your mind or body to sit in one place for hours on end when
learning online. Stretch and stand up. Take a walk. Make a coffee date with a friend. Taking a
break does not have to be a difficult task. It's a simple, efficient method of stress management that,
like exercise, will improve your mood.

Another recommendation for teachers and instructors:

According to the study of (Gopal, Singh, & Aggarwal, 2021), the instructor’s quality is the
most prominent factor that affects the student’s satisfaction during online classes. Therefore,
instructors need to understand students’ psychology to deliver the course content prominently. If
the teacher can deliver the course content properly, it affects the student’s satisfaction and
performance. The teachers’ perspective is critical because their enthusiasm leads to a better online
learning process quality. Is also important for instructors to not just focus on the results, they must
also show care and understand the human well-being of the students in order to make students feel
safe and motivated even if there is a higher demand for mental activity in performing their tasks.
Through this, students will more likely to thrive hard and do their best because they feel the
leadership and support of their instructors through not only focusing on results but also looking
into the well-being of the students and showing empathy.
 CONCLUSION

The conclusion that can be taken from this study is the level of mental workload among

3RD year BSIE evening students in Cebu Technological University-Main Campus during 1st-

semester S.Y. 2021-2022 using NASA-TLX method and its six dimensions is high which has an

unweighted and weighted TLX Score of 78.72 and 79.71 respectively. The dominant scale is the

Mental Demand which means the task to be done during 1st Semester of BSIE evening students

requires higher demand of thinking, deciding, or calculating. The weighted rating of Mental

Demand is 406.67.
REFERENCE

Atalay, K. D., Can, G. F., Erdem, S. R., & Müderrisoglu, I. H. (2016). Assessment of mental
workload and academic motivation in medical students. 574-578.

Gopal, R., Singh, V., & Aggarwal, A. (2021). Impact of online classes on the satisfaction and
performance of students during the pandemic period of COVID 19. Education and
Information Technologies.

Longo, L., & Orrú, G. (2020). Evaluating instructional designs with mental workload assessments
in university classrooms. Behaviour & Information Technology.

Prastawa, H., Susanto, N., & Nofri, T. (2018). Mental workload of undergraduate student (a case
study in Industrial Engineering Department of Diponegoro University).

Rubio-Valdehita, S., López-Higes, R., & Díaz-Ramiro, E. (2014). Academic Context and
Perceived Mental Workload of Psychology Students. Spanish Journal of Psychology.

Tao, D., Tan, H., Wang, H., Zhang, X., Qu, X., & Zhang, T. (2019). A Systematic Review of
Physiological Measures of Mental Workload. International Journal of Environmental
Research and Public Health.