Sultan Qaboos University

Collage of Education
Department of Instructional and Learning Technologies

Report on

Evaluation of Online
Learning

Submitted for: Dr.Alaa Sadik

Submitted by: Athra AlYazeedi (61542)

Fatma AlNaabi (66754)
Introduction:

Most educational institutions shifted their trends toward the student is the center of
educational process. Therefore, Students' performance has been considered as a major
concern of today's education. According to this shift tools that used to evaluate
students' performance has been expanded from just focusing on quizzes and tests tools
to more authentic and innovative ways like projects, discussion, assignments and ..etc.
This expansion is appeared due to the importance of assessing students' performance
that perceived by educational institutions which provide their education either in
traditional or online learning environments. As online learning environments have
been emerged beside traditional classrooms, more traditional strategies have been
transferred to be online. Online Interaction is one of those important strategies in
which students' performance can be emerged. Many different tools either synchronous
and a synchronous can reflects the interaction of students during online learning
environments and discussion forum is one of those. Discussion forum is a
synchronous tool that can represent three types of interactions: 1)student-content in
which student can summarize, critique and paraphrasing the information, student-
2)teacher in which student can teacher questions and negotiate meaning with him and
3)student-student interactions in which student can reviews other students' works,
negotiate meaning with them and collaborate in projects with them. Due to the
importance of discussion forum, it has been used by many of educational institutions
that offers online or blended courses and SQU in the Sultanate of Oman is one of
those. Most of the courses in SQU that used discussion forum used a moodel learning
management system as a platform for providing this interactive tool. Although many
SQU subjects have been supported by this tool, the usage of this tool has been differ
from one subject to another and that is why this report was conducted.

Purpose:
The report aimed to investigate the effectiveness of the discussion forum in
representing students' performance through teachers' perception.

Research questions:

1. To what extent does the Degree of participation in discussion forum represent

students' performance?

2. To what extent does the Interaction in discussion forum represent students'

performance?

3. To what extent do the content related posts in discussion forum represent

students' performance?

Research methodology:

Sample:
The sample of this report is consisted of 8 faculty members at Sultan Qaboos
University. Six of them are from Instructional and learning technology department,
one from educational psychology department and the other one is from foundation of
education department. All of those Faculty members use discussion in their courses.
Measure:
To answer the research questions, survey was build and distributed to the research
population. This questioner focused on three areas to examine effectiveness of
discussion forum in representing students' performance. These areas are: Degree of
participation, Interaction and content related posts. Degree of participation
concentrated on the number and length of students posts. Interaction focused on the
students' participation in discussion forum like: Students’ statements that comment in
another message and Responding to the tutor's views or advises. Content related posts
discuss the following items: The ideas that students posted, students' clarification, The
relevance of students posting and Drawing conclusion.
See a copy of the survey on appendix A

Survey procedures:

1. A significant amount of time is devoted to construct an effective

questionnaire as the main instrument of the study.
2. The surveys administered to participants electronically via Survey console
website.
3. Statistical analysis of the participants’ responses conducted by Survey
console website was then obtained.
4. Results gained from the analysis is then discussed and interrupted.

Findings and Discussion

Total of 8 surveys were completed and returned; a completion rate of 100%.

The survey statistical analysis was then obtained from Survey Console website (see
appendix B ) and interpreted as following:

1. Degree of Participation
The results showed that 80% of faculty members disagreed with all the statements on
this part where 57% of the members disagree with the statement “The number of posts
by a particular student reflects its high achievement within the course” and the other
43% agreed with. In the other hand 40% of faculty members disagreed and other 40%
agreed with the statements “The more messages posted in the discussion forum the
more likely the students understand the topics discussed.” And “Number of statements
directly related to learning per student.”
These results indicate that the degree of participation in the discussion forum is
perceived by the faculty members as secondary aspect that indicates the students’
performance truthfully. This may due to the awareness of faculty members in that the
students shouldn’t be evaluated by just counting the number and the length of their
posts on the forum. The evaluation process should goes away beyond that to measure
the effect of other factors on student performance.
2. Interaction
In controversy with the abovementioned point (Degree of Participation) almost all the
participants agreed on the statements under this section. 80% of them viewed Students
interaction in the discussion forum encouraging for peer collaboration, reducing
students’ tension and anxiety, reflecting students’ proficiency level and indicate
students possession of social skills and represent student’s ability to share and
exchange knowledge.

These responses should indicate that interaction in the discussion forum provides
reliable source of input regarding students’ performance because if the students
interaction (as perceived by the faculty members) encourage peer collaboration and
reduce students tension and anxiety, indicate students possession of social skills and
represent student’s ability to share and exchange knowledge it will assist for sure their
achievement since learning is a social process and without interacting with its
members students learning can be affected. Moreover, being comfortable and not
worried in the learning environment will empower the students to perform their bests
as they motivated to.
And thus interaction should be considered as a key element on evaluating students’
performance.

3. Content related posts

In this section also almost all the participants agreed on its statements. 85% of them
see that content related posts indicate students understanding for the content, represent
their critical thinking ability and their achievement of the course objectives.

So as a result, if the interaction reflecting students’ proficiency level, indicates

students understanding for the content, represent their critical thinking ability and
their achievement of the course objectives then directly it reflected the students’
performance.
And thus for hence, Content related posts is reflecting students’ performance and
should be used to evaluate it in an online environment.
Appendix A: Faculty members Survey

This survey designed to evaluate the effectiveness of the discussion forum in

representing students' performance through teachers' perception. Please take time to
answer this questioner candidly. Your responses will be confidential and used for
research purposes. Thank you for completing this evaluation forum. Its usefulness
depends solely in your cooperation.
 Agree Disagr Neutr
ee al

Degree of participation

1 The number of posts by a

particular student reflects its high
achievement within the course.
2 Length of student posts represent
his mastery of the topic discussed
3 The more messages posted in the
discussion forum the more likely
the students understand the
topics discussed.
4 Number of statements directly
related to learning per student.
Interaction
5 Students’ participation in the
discussion forum provides reliable
source of input regarding
students’ performance.
6 Students interaction in the
discussion forum encouraging peer
collaboration
7 Students interaction in the
discussion forum reducing students’
tension and anxiety
8 Students’ participation in the
discussion forum accurately reflecting
students’ proficiency level
9 Students’ statements that
comment in another message
indicate students possession of
social skills
10 Responding to the tutor's views or
advises represent student’s ability
to share and exchange
knowledge.
Content related posts
The ideas that students posted in dissection
form really represent their thinking ability.

The clarification to the points posted

reflecting student understanding of the
content.

The relevance of students posting to the

discussion topic indicates their
understanding for the content.
The questions asked by students represent
their critical thinking ability.
Asking for more clarification represent that
students perceived content value.

Repeating of the course content through

posting in the discussion forum represent
that students' achieved course objectives.
Drawing conclusion in discussion forum
represent that students' achieved course
objectives.
Appendix B: Survey Statistics
Survey Statistics

Viewed
21

Started
8

Completed
8

Completion Rate
100%

Drop Outs (After Starting)

0

• Average time taken to complete survey : 4 minute(s)

Degree of Participation
Overall Matrix Scorecard

Question
Count
Score
Disagree
Agree
Neutral

1.
The number of posts by a particular student reflects its high achievement within the
course.
7
1.571

2.
Length of student posts represent his mastery of the topic discussed
7
1.286

3.
The more messages posted in the discussion forum the more likely the students
understand the topics discussed.
7
1.714
The number of posts by a particular student reflects its high
achievement within the course.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
4
57.14%

2.
Agree
2
28.57%

3.
Neutral
1
14.29%

Total
7
100%

Key Analytics
Length of student posts represent his mastery of the topic
discussed
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
5
71.43%

2.
Agree
2
28.57%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
The more messages posted in the discussion forum the more
likely the students understand the topics discussed.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
3
42.86%

2.
Agree
3
42.86%

3.
Neutral
1
14.29%

Total
7
100%

Key Analytics
Number of statements directly related to learning per student.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
3
42.86%

2.
Agree
3
42.86%

3.
Neutral
1
14.29%

Total
7
100%

Key Analytics
Interaction
Overall Matrix Scorecard

Question
Count
Score
Disagree
Agree
Neutral

1.
Students’ participation in the discussion forum provides reliable source of input
regarding students’ performance.
7
1.857

2.
Students interaction in the discussion forum encouraging peer collaboration.
7
2.000

3.
Students interaction in the discussion forum reducing students’ tension and anxiety
7
2.143

4.
Students’ participation in the discussion forum provides reliable
source of input regarding students’ performance.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
1
14.29%

2.
Agree
6
85.71%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
Students interaction in the discussion forum encouraging peer
collaboration.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
0
0.00%

2.
Agree
7
100.00%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
Students interaction in the discussion forum reducing students’
tension and anxiety
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
0
0.00%

2.
Agree
6
85.71%

3.
Neutral
1
14.29%

Total
7
100%

Key Analytics
Students’ participation in the discussion forum accurately
reflecting students’ proficiency level
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
2
28.57%

2.
Agree
4
57.14%

3.
Neutral
1
14.29%

Total
7
100%

Key Analytics
The relevance of students posting to the discussion topic
indicates their understanding for the content.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
1
14.29%

2.
Agree
6
85.71%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
The questions asked by students represent their critical thinking
ability.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
1
14.29%

2.
Agree
6
85.71%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
Asking for more clarification represent that students perceived
content value.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
1
14.29%

2.
Agree
6
85.71%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
Repeating of the course content through posting in the
discussion forum represent that students' achieved course
objectives.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
2
28.57%

2.
Agree
3
42.86%

3.
Neutral
2
28.57%

Total
7
100%

Key Analytics
Drawing conclusion in discussion forum represent that students'
achieved course objectives.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
0
0.00%

2.
Agree
7
100.00%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
Content related posts
Overall Matrix Scorecard

Question
Count
Score
Disagree
Agree
Neutral

1.
The ideas that students posted in dissection form really represent their thinking
ability.
7
2.000

2.
The clarification to the points posted reflecting student understanding of the content.
7
1.857

3.
The relevance of students posting to the discussion topic indicates their
understanding for the content.
7
1.857
The ideas that students posted in dissection form really
represent their thinking ability.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
0
0.00%

2.
Agree
7
100.00%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
The clarification to the points posted reflecting student
understanding of the content.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
1
14.29%

2.
Agree
6
85.71%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
The relevance of students posting to the discussion topic
indicates their understanding for the content.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
1
14.29%

2.
Agree
6
85.71%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
The questions asked by students represent their critical thinking
ability.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
1
14.29%

2.
Agree
6
85.71%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
Asking for more clarification represent that students perceived
content value.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
1
14.29%

2.
Agree
6
85.71%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
Repeating of the course content through posting in the
discussion forum represent that students' achieved course
objectives.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
1
14.29%

2.
Agree
4
57.14%

3.
Neutral
2
28.57%

Total
7
100%

Key Analytics
Drawing conclusion in discussion forum represent that students'
achieved course objectives.
Frequency Analysis

Answer
Count
Percent
20%
40%
60%
80%
100%

1.
Disagree
0
0.00%

2.
Agree
7
100.00%

3.
Neutral
0
0.00%

Total
7
100%

Key Analytics
Analytics Powered by SurveyConsole