Professional Documents
Culture Documents
Cohens Kappa
Cohens Kappa
ON
RESEARCH METHODOLOGY FOR
MANAGEMENT
QUALITATIVE STUDY
SECTION B
Submitted By – GROUP - 9
Our aim is to understand how students learn is crucial for effective course design. This study
investigates student views on incorporating practical data analysis tools into coursework,
compared to a purely theory-based approach.
We employed a qualitative approach, interviewing two students and conducting thematic
analysis on the transcribed data using NVivo software.
INDEX
S.NO PARTICULARS PAGE.NO
1. Problem Statement 4-5
2. Objective 6
3. Methodology 6
4. Description of Statistics 7-8
5. Data Analysis 9-11
6. Organizational Impact 12-13
7. Limitations of the study 14
8. Conclusion 14
Many instructional tools have been made available by the digital age, such as NVivo for
qualitative data analysis and SPSS Statistics for quantitative data analysis. With the use of
these technologies, students can: Interact directly with data, testing and visualizing
theoretical ideas to gain a deeper comprehension. By using these resources, students can
develop transferable analytical skills that are highly valued across a wide range of
professions.Compared to strictly theoretical lectures, interactive tools have the ability to
increase student engagement and stimulation during the learning process.
Teachers can gain important insights from this research by knowing how students see
educational materials. Teachers are able to create courses that successfully combine theory
and tool-based learning, promoting the development of more profound comprehension and
useful skills.Teachers can design more dynamic and engaging learning experiences by
having a better understanding of the learning styles and preferences of their students.Making
the most of technology's advantages: Best practices for integrating instructional technologies
into the curriculum can be found in the research, enabling teachers to use technology to get
the best possible learning results.
This study looks into how students feel about particular technologies as well as how
they affect learning as a whole. We can work to make the learning environment for
students more productive and interesting by addressing the harmony between theory
and real-world application.
The interview questions are strategically designed to cover several key areas. Firstly, our aim
to stimulate discussions around the perceived effectiveness of using tools versus traditional
theory-based teaching approaches. By prompting students to reflect on the advantages and
drawbacks of each method, researchers can gain valuable insights into how students assess
the impact of these differing instructional techniques on their learning experiences.
Furthermore, the interview questions are designed to elicit preferences and recommendations
from students regarding the integration of tools into coursework. By exploring students'
preferences for more hands-on tool-based learning or traditional teaching methods,
researchers can discern patterns in students' learning preferences and potential areas for
improvement in curriculum design.
Moreover, the questions delve into the perceived impact of using tools on students' learning
outcomes and academic performance. By probing into whether students believe that the
incorporation of tools enhances their understanding of course material and contributes to
their academic achievements, researchers can gauge the effectiveness of these tools in
facilitating student learning and success.
Additionally, the interview questions aim to identify barriers and challenges that students
may encounter when using these tools in their coursework. By understanding the obstacles
that hinder students' full utilization of these tools, educators and curriculum developers can
devise strategies to address these challenges and optimize students' learning experiences.
Lastly, the questions explore students' perspectives on the future trajectory of tool integration
in coursework, aiming to uncover their expectations and predictions regarding the evolving
role of tools in education.
METHODOLOGY
2. OBJECTIVE OF THE STUDY
Understanding student views on the use of different tools (like, SPSS, Statistics,
NVivo, etc.) in courses over theory- based teaching in the course.
2.1 We have done qualitative research by taking interviews of the two members (member
1, member 2) of the group. Then we transcribed the video and then two other
members (member 3, member 4) of the group did thematic analysis using the tool
NVivo.
To measure the inter rater reliatibility of the two members, we applied the Cohen’s
Kappa statistic and then analyzed the findings.
The validity and the reliability of the instruments were ensured while conducting the
interviews. Further, ethical considerations were promised to the interviewees.
.
2.2 Questions asked during the interview:
According to you, how should the course be taught to improve the learning and
performance of students?
3. DESCRIPTIONS OF THE STATISTICS USED IN THE STUDY
Percentage Agreement:
DEFINITION FORMULA
Percentage agreement or The percentage agreement is calculated using the
Pr(a) is a measure that following formula:
indicates the proportion of
agreement between two
Pr(a)=
∑ of the total number of agreement *10
raters or methods. It is Total no . of observation
calculated by dividing the 0
number of agreements by the
total number of observations,
then multiplying by 100 to
express it as a percentage.
Expected Agreement:
DEFINITION FORMULA
Expected agreement or Pr(e) represents The Expected agreement is calculated using
the agreement that would be expected by the following formula:
chance alone. It is calculated based on cm 1∗rm1 cm 2∗rm2
+
the marginal frequencies of the observed Pr(e)= n n `
categories. n
Where ,
cm1 represents column 1 marginal
cm2 represents column 2 marginal
rm1represents row 1 marginal
rm2 represents row 2 marginal
n represents the number of
observations (not the number of
raters).
Cohen’s Kappa
DEFINITION FORMULA
Cohen's Kappa (𝜿) is a statistical The Cohen’s Kappa is calculated using the
measure of inter-rater agreement for following formula:
categorical items. It adjusts for the
agreement occurring by chance. It ranges Pr ( a )−Pr ( e )
from -1 to 1, where 1 indicates perfect 𝜿= ¿ ¿
1−Pr ( e )
agreement, 0 indicates agreement
equivalent to chance, and -1 indicates
perfect disagreement.
The standard significance of Cohen's Kappa is used to determine whether the observed
agreement (Kappa value) is significantly different from what would be expected by chance
alone. It is often used to assess the statistical significance of the Kappa value.
SE𝜿 =
√ Pr (a)( 1−Pr (a))
n∗(1−Pr ( e ) )
Question 1 Demand extensive mental efforts Expressing ideas becomes subjective since it relies
and impede efficiency solely on writing.
Question 2 Analysis and Visualization Streamlines the process, saving time and effort.
Interviewee 2
Question 2 Combination of theories and real Enables individuals to articulate their perspectives
time application effectively
Question 3 Practice is vital for proficiency A combination of theory, practical experience, and
ongoing practice is vital for proficiency and
successful application
5. Cohen's Kappa Calculation -
Calculation of Percentage
Agreement
Person Person
1 2
Interviewee 1
(A) 1 0 1
(B) 1 1 0
(C) 0 0 0
Interviewee 2
(A) 0 0 0
(B) 1 1 0
(C) 1 1 0
Number Of Zeroes 5
Total no. of Questions 6
6. Confusion Matrix
Confusion Matrix
Person 2
Positiv Negativ Row
e e Marginals
Person Positive 3 1 4
1 Negative 0 2 2
Column
Marginals 3 3 6
7. Findings of the study: The study's results indicate a moderate level of agreement,
with a 66.67% consensus observed between the two raters. The Kappa value falls
within the range of 0.16 to 1.16 and is statistically significant at a confidence level of
90%. These findings suggest that practical knowledge plays a vital role in ensuring the
tool's efficient and effective utilization.
Recommended solutions completely and accurately consider effects and implications for
other similar scenarios.
In considering the effects and implications for other similar scenarios, it's essential to
recognize that the questions provided are tailored to gather insights specifically on theory-
based versus tool-based courses within the context of the study objectives. However, the
underlying principles can be applied to other scenarios involving comparisons between
different instructional approaches or methodologies. Here's how the recommended solutions
can be adapted to similar situations:
The Cohen's Kappa value of 66% indicates a substantial level of agreement between theory
and practical learning. However, it's important to note that this doesn't inherently i mply that
practical-based learning is definitively better than theory-based learning. Instead, it suggests
that there is a significant alignment between the two approaches.
That said, here are some potential business implications:
Continuous Improvement: Businesses can use this insight to continually refine their
training and educational programs. By regularly assessing the effectiveness of
theoretical versus practical learning methods, organizations can adapt their approaches
to maximize learning outcomes and stay competitive in their respective industries.
The survey only focuses on students' views regarding theory-based teaching versus
tool-based teaching, neglecting other potential factors that could influence learning
and performance, such as teaching methods, course materials, instructor expertise,
student engagement, etc.
The survey does not define what is meant by "performance" of students, leaving room
for interpretation. Performance could encompass various aspects such as grades,
understanding of concepts, application of knowledge, etc. Without clear definitions,
responses may vary widely and be difficult to compare.
\The survey questions provide only two options for respondents to choose from (easy
to understand or difficult to understand), which may oversimplify the complexity of
students' experiences and perspectives. It does not allow for nuanced responses that
could capture the diversity of student experiences.
The main limitation of this research is the limited number of sample sizes. The
reliability of the result could have increased if the number of sample sizes or the
participant giving the interview were more in numbers.
11. CONCLUSION
The study investigates student perspectives on incorporating practical data analysis tools into
coursework compared to a purely theory-based approach. It utilizes qualitative research
methods, including interviews and thematic analysis, to explore student views. The analysis
reveals a moderate level of agreement (66.67%) between raters regarding the alignment
between theory and practical learning.
Key findings suggest that while practical knowledge is crucial for efficient tool utilization, it
doesn't imply superiority over theory-based learning. Instead, there's a need to integrate both
approaches to create a balanced curriculum that caters to diverse learning styles. Potential
business implications include the importance of balanced curriculum design, skill
development programs, informed hiring decisions, continuous training improvement, and
effective marketing strategies for educational products.
However, the study has limitations, including its focus solely on student perspectives,
oversimplified survey questions, and a small sample size, limiting the generalizability of
findings. Recommendations for improvement include increasing sample size, defining
performance metrics, using a mixed-methods approach, including diverse perspectives, and
refining survey questions for richer data collection. Addressing these improvements can lead
to a deeper understanding of effective teaching and learning strategies in educational
settings.