Item Analysis Using CITAS

Classical Item and Test Analysis Spreadsheet

March 17, 2021 ȁ 1: 00 PM − 2: 00 PM

Lecturer: Mr. Fritz M. Ferran

To Cite: Ferran, F. M. (2021, March 17). Item Analysis Using CITAS. [Virtual In-Service Training]. Bayuin National High School, Mindoro, Philippines
 Mr. Fritz M. Ferran
Statistical Software
▪ Adept with statistical tools and can perform data analysis using different statistical
treatments with the aid of SPSS, JAPS, WarpPLS, MINISTEPS, jMetrik, XCalibre,
CITAS, MaxQDA, and MS Excel.

Specific Skills
Can perform the following data analysis:
▪ Exploratory Data Analysis and Basic parametric and nonparametric tests
▪ Nonparametric tests (Kruskal-Wallis, Mann-Whitney U, Wilcoxon, Kendall’s W,
Friedman)
▪ ANOVA, ANCOVA, MANOVA, MANCOVA, Repeated Measures of ANOVA
▪ Simple and Multiple Linear Regression
▪ Path Analysis and Structural Equation Modeling (SEM)
▪ Confirmatory Factor Analysis, Exploratory Factor Analysis
▪ Mediation Analysis and Multilevel Analysis
▪ Convergent, Discriminant, and Reliability Tests
▪ Statistical Power Analysis and Sample Size Determination
▪ Item Analysis using CTT, IRT, and Rasch Analysis

BSMT MSM WORK

 Agenda

Lesson 1: Lesson 2: Lesson 3:

Understanding Fundamentals of Item Analysis
Classical Test Theory Item Analysis Using CITAS

Adopted from the lecture of Professor Johnny T. Amora, the President of the Philippine Association of Researchers and Statistical Software Users
Lesson 1:
Understanding Classical Test Theory
 Lesson 1: Understanding Classical Test Theory
1. Approaches to Measurement

Adopted from the lecture of Doucette, A. (2015). Examining How Rasch/IRT Can Increase Measurement Precision.
 Lesson 1: Understanding Classical Test Theory
2. What is Classical Test Theory?

1. Traditional method of item analysis.

2. CTT assumes that each person has a TRUE score which can be obtained if there
are no errors in measurement.
3. A person’s TRUE SCORE is defined as the expected number-correct score over an
infinite number of independent administrations of the test.
4. However, we can only observe the person’s score known as OBSERVED SCORE.
5. Thus, errors will score in estimating the person’s TRUE SCORE using his
OBSERVED SCORE.
6. These errors can be due to test-takers, environment, questionnaire, etc.

Adopted from the lecture of Professor Johnny T. Amora, the President of the Philippine Association of Researchers and Statistical Software Users
Lesson 2:
Fundamentals of
Item Analysis
Lesson 2: Fundamentals of Item Analysis

ITEM DIFFICULTY
Which among the items are easy, difficult, and very difficulty?

ITEM DISCRIMINATION
Do the items discriminate between students who really knew the material and those
who did not?

VALIDITY
Does the test measure what it is supposed to measure?

REALIBILITY
Is the test reliable? That is, can the test produce reliable scores?

Adopted from the lecture of Professor Johnny T. Amora, the President of the Philippine Association of Researchers and Statistical Software Users
Lesson 2: Fundamentals of Item Analysis
ITEM DIFFICULTY
1. It is the proportion of test-takers who correctly answered an
item.
2. Item difficulty index = c/n
a. c = number of test-takers with correct answer
b. n = total number of test-takers

EXAMPLE
1. If 12 out of 48 test-takers correctly answered a test
item, then the Item Difficulty Index is 12/48 = 0.25.
2. If 40 out of 50 test-takers correctly answered a test
item, then the Item Difficulty Index is 40/50 = 0.80.

Adopted from the lecture of Professor Johnny T. Amora, the President of the Philippine Association of Researchers and Statistical Software Users
Lesson 2: Fundamentals of Item Analysis
ITEM DIFFICULTY
1. It is the proportion of test-takers who correctly answered an
item.
2. Item difficulty index = c/n
a. c = number of test-takers with correct answer
b. n = total number of test-takers

RULE OF THUMB

Adopted from the lecture of Professor Johnny T. Amora, the President of the Philippine Association of Researchers and Statistical Software Users
Lesson 2: Fundamentals of Item Analysis
ITEM DISCRIMINATION
1. Get the correlation between the item score (0/1) and the total scores of
the test-takers. The correlation coefficient is called POINT BISERIAL
CORRELATION.
2. The Point Biserial Correlation serves as the Item Discrimination Index.

Adopted from the lecture of Professor Johnny T. Amora, the President of the Philippine Association of Researchers and Statistical Software Users
Lesson 2: Fundamentals of Item Analysis - ITEM DISCRIMINATION
RULE OF THUMB

Adopted from the lecture of Professor Johnny T. Amora, the President of the Philippine Association of Researchers and Statistical Software Users
Lesson 2: Fundamentals of Item Analysis
VALIDITY AND RELIABILITY
Does the test measure what it is supposed to measure? Is the test reliable? That is, can the test produce reliable scores?

Adopted from the lecture of Professor Johnny T. Amora, the President of the Philippine Association of Researchers and Statistical Software Users
Lesson 2: Fundamentals of Item Analysis
CRONBACH’S ALPHA
1. It is a measure of internal consistency (reliability).
2. It is commonly used when you have multiple Likert questions in a survey/questionnaire that form a scale and you
wish to determine if the scale is reliable.

Its Cronbach’s alpha is 0.980.

RULE OF THUMB
The general rule of thumb is that a Cronbach's alpha of . 70 and above is good, . 80
and above is better, and . 90 and above is best

Adopted from the lecture of Professor Johnny T. Amora, the President of the Philippine Association of Researchers and Statistical Software Users
Lesson 2: Fundamentals of Item Analysis
KUDER-RICHARDSON 20 (KR-20)
1. KR-20 is a special case of Cronbach’s Alpha in which items are binary variables (usually scored as 0 or 1).

Its Cronbach’s alpha is 0.655.

The KR-20 is also 0.655.

RULE OF THUMB
The general rule of thumb is that a Cronbach's alpha of . 70 and above is good, . 80
and above is better, and . 90 and above is best

Adopted from the lecture of Professor Johnny T. Amora, the President of the Philippine Association of Researchers and Statistical Software Users
Lesson 3:
Item Analysis
using CITAS

Classical Item and Test Analysis Spreadsheet

Demonstration
Lesson 3: Item Analysis Using CITAS
STEP 1: Open the CITAS and the Dataset

This is the only space

that is editable.
Lesson 3: Item Analysis Using CITAS
STEP 2: Transfer the Keys (in orange row) and the dataset (in blue region) to CITAS.
Lesson 3: Item Analysis Using CITAS
STEP 3: Interpret the Results provided by CITAS.
Lesson 3: Item Analysis Using CITAS
STEP 3: Interpret the Results provided by CITAS.
Lesson 3: Item Analysis Using CITAS
STEP 3: Interpret the Results provided by CITAS.

Option N refers to the number of students who

answered the particular answer such as ABCD.
Lesson 3: Item Analysis Using CITAS
STEP 3: Interpret the Results provided by CITAS.

Option P refers to the proportion of students who

answered the particular answer such as ABCD.
Lesson 3: Item Analysis Using CITAS
STEP 3: Interpret the Results provided by CITAS.

Option Rpbis refers to the point biserial correlation.

1. For item 1, majority of the high performing students chose A or C (not B and D). This is called
DISTRACTOR analysis.
2. For item 1, the key is C. Note that if few students got the correct answer, it does not mean the item is a
bad item. To analyze this, check the point biserial correlation. If the key has a positive coefficient, then the
performing students got it right and the item is still a good item.
Lesson 3: Item Analysis Using CITAS

Option Rpbis refers to the point biserial correlation.

1. For item 21, the correct answer is A. However, there were high performing students who chose C and D
due to the positive value of Rpbis. Thus, this item or the options of this item require revision
Lesson 3: Item Analysis Using CITAS
Group Activity:
Using the CITAS software, analyze the 15 items in
the Exercise1 with Keys.xlsx.
1. Access the BNHS Google Shared Drive Folder: bit.ly/BNHS_StatAnalysis_Inset202.
2. Download and open in MS Excel the Item Analysis using CITAS-Demonstration (empty).
3. Download and use the excel file “Exercise1 with Keys” to perform the item analysis.
4. Use the sample item analysis report (PDF) to report your item analysis.
Mr. Fritz M. Ferran
“I do not see my classroom as a class of 30 students but a class
of 30 innovative minds. And connecting them to information
through technological platforms, their innovativeness becomes a
service to their community.
Thus, even as early as a student, they can be depended on.”
-Ferran, Fritz M.
fritz.ferran@dlszobel.edu.ph

/fritz.ferran bit.ly/FATsEducSite bit.ly/FATsYTChannel