[EDP 380D-4] FA19 Kang

EDP 380D-4: Psychometric Theory and Methods

Fall 2019
Time: TTH: 12:30-2:00 PM Location: SZB 286
Instructor: Hyeon-Ah Kang Office: 538G
Email: hkang@austin.utexas.edu Office hours: TH: 2:30–4 PM

Course Description
This course introduces fundamental theories and methodologies used to evaluate the quality of
psychological measurements in research and applied settings. It covers three broad topics in
psychometrics: (i) classical test theory, (ii) item response theory, and (iii) factor analysis.
Throughout the semester, you will learn the basic concepts of these psychometric theories and
practice the methods using statistical analysis software.

Learning Goals
After completing this course, you will be able to:
× Discuss and apply the theoretical fundamentals of classical test theory;
× Discuss the foundations of testing (reliability, validity, fairness) and utilize psychometric
modeling approaches to evaluate those foundations for specific tests;
× Describe item response theory models and their applications in measurement;
× Understand the similarities, differences, and links between classical test theory and modern test
theory; and
× Understand and apply the basics of factor analysis and principal components analysis for
purposes of test construction and validation.

Pre-requisites
× The pre-requisite for this course is successful completion of Fundamental Statistics (EDP380C-
2). Basic statistical knowledge is essential for understanding the concepts and conducting the
analysis in psychometrics.
× Programming knowledge in R and writing skills in APA style are highly recommended.
§ For elementary programming in R, see here 1 and here 2.
§ For APA style writing, see here 3, here 4, and here 5.

Texts
× (main) Furr, R. M. (2018). Psychometrics: An Introduction (3rd Ed). SAGE Publications, Inc.
× Kline, T. (2005). Psychological Testing: A Practical Approach to Design and Evaluation. Sage
Publications.
× Allen, M. J. & Yen, W. M. (1979). Introduction to Measurement Theory. Waveland Press. Inc.

1
[EDP 380D-4] FA19 Kang

Other Materials
× Lecture notes and handouts: I will post all class materials on Canvas, including lecture notes,
supplemental resources, announcements, and assignments. Lecture notes are designed to help
your learning and require guided note-taking. This means that you will need to fill in blanks
with key definitions and write out your own answers to questions and exercise problems. Note
that students are responsible for all materials presented in lectures. If you miss a class, you are
responsible for obtaining both the class material and any class announcements from fellow
classmates.
× Calculator: You will need to use a calculator both during lectures and for in-class exams. The
calculator must be able to perform key mathematical functions (e.g., taking the natural log, the
exponent, square root, etc.).
× Statistical software: We will use R as a primary programming language. While the R software
itself is sufficient for use, I recommend using R-Studio as it provides a convenient tool for
editing. Please refer to the R installation guide I posted on canvas. Note that R-Studio does not
run by itself; you need to install R before using R-Studio.

Tentative Course Schedule

× See page 5.

Assessment
× Homework (60%): There will be a total of five or six homework assignments (depending on
your performance on quiz), which are designed to give you an opportunity to apply and
practice the concepts and techniques learned in class.
- If you pass the quiz, you will receive a full credit for homework 2 (i.e., no need to submit
homework 2). If you don’t, you will need to submit homework 2 and receive the credits
based on your answers.
- For each assignment you will need to submit your answers to canvas showing all relevant
process of solving the problems (presenting only the final answers will lead to partial
credits). The relevant process includes equations, interim values, R code, etc.
- All numeric final answers should be rounded off to three decimal places. Do NOT round off
when you are deriving the final answers; doing so will propagate the rounding error and will
result in incorrect answers.
- You may discuss the homework assignments with classmates, but the work you turn in must
be your own. No late homework will be accepted without prior approval of the instructor.
- Each assignment will carry varying weight toward the final grade due to the different
coverage and total credits awarded. The weight for each assignment is calculated by dividing
the full score of the corresponding assignment by the total score across all assignments.
e.g., Suppose you have two homework assignments.
HW1 HW2 Total/Average
Full score 15 20 35
Your score 12 (80%) 18 (90%) 30 (85%)
Weight 15/35 20/35 1
Your weighted score (12/15)×(15/35) (18/20)×(20/35) 85.714%

2
[EDP 380D-4] FA19 Kang

× Exams (40%): There are two take-home exams (20% each), each consisting of conceptual,
computational, and application problems. For the in-class exam, you are allowed to bring your
own cheat-sheet and calculator. For the take-home exams, you are allowed to use books,
relevant materials, and statistical software to complete the exams, but you must work on your
own and must not collaborate with other classmates.

Grading Policy
× Note that the grades in canvas do not take the attendance scores and different weights for
homework assignments into account. Consult with the instructor if you want to predict a letter
grade.
B+ [87, 90) C+ [77, 80)
A [93, 100] B [84, 87) C [74, 77) D [65, 70)
A- [90, 92) B- [80, 84) C- [70, 74) F <65

Class Expectations
× As a student, you are responsible for being present in class and for all material covered during
class meetings. If you miss a class, it is your responsibility to obtain and review both notes and
announcements from your fellow classmates without regard to the reason for being missed.
Frequent or unexcused absences and tardiness will adversely affect your grade.
× The major determinants of success in this course are i) whether you understand basic concepts
of psychometric theory, ii) apply adequate statistical analysis techniques to a given context,
and iii) interpret the outcomes of the analysis appropriately. While I will keep the scope of the
course within the graduate introductory level, I also encourage you to develop deeper
understanding of the materials by studying advanced topics, which are stamped with Ⓐ in the
lecture notes.
× The classes in this course are moderately collaborative, and a large amount of learning occurs
during the class meetings. Come ready to participate and work. Be prepared to be active in
class.
× Respect your instructors and your classmates. Respect includes creating an environment
conducive to learning, which means being on time, turning off cell phones, avoiding online
activities (e.g., Facebook, YouTube, email), listening, and contributing.

Academic Integrity
Students are expected to abide by the University's honor code and maintain absolute integrity in
scholastic work. All assignments must be completed with the utmost honesty, which includes
acknowledging the contributions of other sources to your scholastic efforts, avoiding plagiarism,
and completing assignments independently unless expressly authorized otherwise. Assignments
containing any plagiarized material will not be accepted. Any evidence of cheating and sharing
answers will result in a zero point for all students involved.

3
[EDP 380D-4] FA19 Kang

Student Accommodation
The university is committed to creating an accessible and inclusive learning environment
consistent with university policy and federal and state law. Please let me know if you experience
any barriers to learning so I can work with you to ensure you have equal opportunity to
participate fully in this course. If you are a student with a disability, or think you may have a
disability, and need accommodations please contact Services for Students with Disabilities
(SSD). Please refer to SSD’s website for contact and more information:
http://diversity.utexas.edu/disability/. If you are already registered with SSD, please deliver your
Accommodation Letter to me as early as possible in the semester so we can discuss your
approved accommodations and needs in this course.

Religious Holidays
You must notify the instructor of a pending absence due to religious observance at least fourteen
days in advance. If you must miss a class, a work assignment, or a presentation, you will be
given an opportunity to complete the missed work within a reasonable time after the absence,
provided that you have properly notified the instructor ahead.

Use of Class Materials

The materials used in this class, including, but not limited to, exams and homework assignments
are copyright protected works. Any unauthorized copying of the class materials is a violation of
federal law and may result in disciplinary actions being taken against the student. Additionally,
the sharing of class materials without the specific, express approval of the instructor may be a
violation of the University's Student Honor Code and an act of academic dishonesty, which could
result in further disciplinary action. This includes, among other things, uploading class materials
to websites for the purpose of sharing those materials with other current or future students and/or
for making profits.

Continued enrollment in this course assumes that you have read and understand the syllabus and
abide by the rules outlined in the syllabus.

4
[EDP 380D-4] FA19 Kang

Tentative Course Schedule

Date Lesson Subtopics Reading
Aug L1. Intro to ⋅ Syllabus ⋅ Furr: Ch1, 2
29, Psychometrics ⋅ Terms ⋅ Kline: Ch.1 (pp.1-4, 24-
Sep 3 ⋅ Phases in test development 27), Ch.2 (pp.29-35),
⋅ Designing tests Ch.3
⋅ Analysis of binary response scores
Sep 5, L2. Review of ⋅ Terms ⋅ Furr: Ch3
10, 12 Basic Statistics ⋅ Variability and distributions of ⋅ Kline: Ch1 (p.4-24)
scores ⋅ A&Y: Ch2
⋅ Association between distributions
⋅ Interpreting item and test scores
⋅ Lab
Sep 19 Quiz (In Class)
Sep 17 L3. Classical Test ⋅ Test theory ⋅ Furr: Ch5. (p,111-119;
Theory ⋅ Assumptions of CTT p.129-140)
⋅ Parallel tests ⋅ Kline: Ch5 (p.91-94)
⋅ Tau-equivalent tests ⋅ A&Y: Ch3
Sep L4. Item Analysis ⋅ Descriptive statistics ⋅ Kline: Ch5 (p.95-105)
24, 26 in CTT ⋅ Item difficulty ⋅ A&Y: Ch6
⋅ Item discrimination
⋅ Distractor analysis
⋅ Lab
Oct 1, L5. Reliability ⋅ Definition of reliability ⋅ Furr: Ch5. p.119-129;
3, 8, ⋅ Reliability coefficients Ch6
10 ⋅ Internal consistency reliability ⋅ Kline: Ch7
⋅ Factors affecting reliability ⋅ A&Y: Ch4
coefficient
⋅ Standard errors of measurement
and confidence intervals for true
score
⋅ Lab
Oct L6. Validity ⋅ Definition of validity ⋅ Furr: Ch8, 9
15, 17, ⋅ Pre-1999 definition (content, ⋅ Kline: Ch9 (p.201-224);
22 criterion, construct) Ch10 (p.286-287)
⋅ Post-1999 definition (content, ⋅ A&Y: Ch5
predictive/concurrent,
convergent/discriminant)
⋅ Lab
Oct 24 Review
Oct 31 Exam 1 (Due on Nov 6)

5
[EDP 380D-4] FA19 Kang

Oct L7. Item ⋅ Basic concepts of IRT ⋅ Furr: Ch14

29, Response Theory ⋅ IRT models ⋅ Kline: Ch6 (p.107-128;
Nov 5, ⋅ Item and test information 162-165)
7 ⋅ IRT applications ⋅ A&Y: Ch11 (p.239-
⋅ Connection to CTT 242; 253-269)
⋅ Lab
Nov L8. Factor ⋅ Test dimensionality ⋅ Furr: Ch4, 12
12, 14, Analysis ⋅ Exploratory factor analysis ⋅ Kline: Ch10 (p.241-
19 ⋅ Confirmatory factor analysis 269)
⋅ Lab
Nov L9. Test Bias ⋅ Construct bias ⋅ Furr: Ch10, 11
21, 26 ⋅ Predictive bias ⋅ Kline: Ch9 (p.224-233)
⋅ Response bias ⋅ A&Y: Ch9 (p.193-200)
⋅ Lab
Dec 3 Review
Dec 5 Exam 2 (Due on Dec 11)

This syllabus is subject to minor change at the discretion of the instructor and/or college. If so,
changes will be communicated via class and email announcements.