Name: Edson Louis A.

Para
Program/Course: MAED - Language Test
Professor: Dr. Jonas V. Palada

Topic: Evaluation of Tests

Learning Outcomes:
At the end of the lesson, students should be able to:
1. Define and differentiate between validity and reliability in testing, demonstrating their
understanding through examples and explanations.
2. Analyze different aspects of validity (test purpose, suitability of format and features,
and test difficulty) and different types of reliability tests (test-retest, Alternate Form,
and Internal Consistency) and apply these concepts to evaluate the quality of
assessments.
3. Interpret test scores effectively, taking into account factors such as norm-referenced
vs. criterion-referenced scoring, as well as raw, standard, and percentile scores.

The Purpose of The Test: Test Validity

What is test validity?

Conversely, test validity refers to what characteristic the test measures and how well the test
measures that characteristic. Formed by Truman Lee Kelley, Ph.D. in 1927, the concept of
test validity centers on the concept that a test is valid if it measures what it claims to
measure. For example, a test of physical strength should measure strength and not measure
something else (like intelligence or memory). Likewise, a test that measures a medical
student’s technical proficiency may not be valid for predicting their bedside manner.

Test validity: How do you ensure test validity?

The validity of an assessment refers to how accurately or effectively it measures what it was
designed to measure, notes the University of Northern Iowa Office of Academic Assessment.
If test designers or instructors don’t consider all aspects of assessment creation — beyond
the content — the validity of their exams may be compromised.

Establish the test purpose.

“Taking time at the beginning to establish a clear purpose, helps to ensure that goals and
priorities are more effectively met” (Gyll & Ragland, 2018). When building an exam, it is
important to consider the intended use for the assessment scores. Is the exam supposed to
measure content mastery or predict success?

THE CHARACTERISTICS OF THE EXAMINEES: TEST DIFFICULTY

Analyze student performance on the test to see if the results align with expectations based
on their learning progress and objectives. Understanding the characteristics of the
examinees helps ensure that the assessment accurately measures what it intends to assess.
Tailoring the test to match the knowledge, skills, and abilities of the students enhances the
validity of the assessment.

DECISION ACCURACY

Suitability of Format and Features

Testing the format and features of the assessment ensures that it is suitable for the intended
purpose and target audience. The format should align with the content being assessed and
the learning objectives, ensuring that the assessment accurately measures the desired skills
or knowledge.

Test Reliability and Replicability

Here are three types of reliability, according to Fiona Middleton’s, that can help determine if
the results of an assessment are valid:

 Test-Retest Reliability measures “the replicability of results.”

 Alternate Form Reliability measures “how test scores compare across two similar
assessments given in a short time frame.”
 Internal Consistency Reliability measures “how the actual content of an assessment
works together to evaluate understanding of a concept”

Using these three types of reliability measures can help teachers and administrators
ensure that their assessments are as consistent and accurate as possible.

Scoring and Reporting: Test Interpretability

Scoring, reporting, and test interpretability in evaluating an assessment are crucial

components that ensure the validity, reliability, and effectiveness of the assessment process.
Here is a detailed breakdown of each aspect:

Test Interpretability:

1. Norm-Referenced Tests: Scores are compared to those of a norm group to assess

performance relative to others. It ensures that the norm group aligns with the target group to
ensure accurate comparisons.

2. Criterion-Referenced Tests: Scores indicate the level of skill or knowledge in a specific

area, focusing on the test-taker's competence rather than comparison to others.

Interpreting Test Results:

1. Raw Scores: Initial unadjusted scores, which are converted into standard scores or
percentiles for comparison.
2. Standard Scores: Converted raw scores that show how an individual's score compares to
a reference group.
3. Percentile Scores: Indicate the percentage of people in the reference group who scored
below the test-taker.

Cost of Procurement: Test Economy and Scoring

Test Economy Procurement involves the efficient acquisition of assessment tools or tests
while considering cost-effectiveness. It focuses on obtaining assessments that provide value
for money, align with the assessment needs, and fit within budget constraints.

Importance of Considering the Cost of Test Procurement in Evaluating the Effectiveness of a

Classroom Assessment:

1. Resource Optimization: Evaluating the cost of test procurement ensures that resources
are optimized effectively. By considering costs, schools can select assessments that offer
value for money and meet educational goals without overspending.
2. Budget Management: Understanding the cost of test procurement is crucial for effective
budget management. Schools need to assess the financial implications of assessments to
allocate funds wisely and sustainably.
3. Impact on Teaching: Cost-effective test procurement can positively impact teaching
quality. By investing in assessments that are reasonably priced, educators can access
valuable tools to enhance their teaching methods and student learning outcomes.
4. Assessment Quality: Considering the cost of test procurement helps in balancing
affordability with assessment quality. Schools can select assessments that provide accurate
and meaningful data without compromising on the assessment's effectiveness due to budget
constraints.
5. Evaluation Frequency: Cost evaluation in test procurement influences the frequency of
assessments. Schools can determine how often assessments can be conducted based on
their affordability, ensuring regular assessment practices within financial limits.
6. Equity in Education: Cost-effective test procurement promotes equity in education. By
choosing assessments that are affordable and within budget, schools can ensure that all
students have equal access to quality assessment tools and opportunities for academic
success.
7. Long-Term Sustainability: Assessing the cost of test procurement contributes to the long-
term sustainability of assessment practices in schools. It enables educators to plan
strategically for future assessments while considering financial implications and
sustainability.
8. Alignment with Goals: Considering costs in test procurement ensures that assessments
align with educational goals. Schools can select assessments that provide the necessary
data to evaluate student progress and instructional effectiveness while staying within
budgetary constraints.

In conclusion, evaluating the cost of test procurement is essential for optimizing resources,
managing budgets effectively, enhancing teaching quality, maintaining assessment quality,
determining assessment frequency, promoting equity, ensuring sustainability, and aligning
assessments with educational goals in the classroom.

Test Administration and Scoring

Test administration and scoring are crucial processes in the field of assessment and
evaluation. Here is a detailed breakdown based on the search results:

Test Administration:

1. Preparation:

- Preliminary Administration: Before administering the test, it should be reviewed by experts

for feedback and modifications. An experimental tryout, involving a sample size of 100, helps
identify weaknesses, item difficulty, and appropriate length of the test.
- Proper Tryout: Involves delivering the test to a sample of 400 to conduct item analysis,
focusing on item difficulty, discriminatory power, and effectiveness of distractors.
- Final Tryout: The test is administered to at least 100 participants in its final form to identify
any minor defects that were not detected in the previous stages.

2. Test Administration Procedures:

- Ensure a congenial testing environment.

- Provide written instructions.
- Avoid unnecessary announcements and interruptions during the test.
- Keep time accurately and record any significant events.
- Motivate students to do their best and ensure a fair testing environment.
3. Avoiding Disruptions:

- Refrain from talking unnecessarily.

- Minimize interruptions during the test.
- Avoid giving hints to students.
- Discourage cheating practices.

4. Environmental Considerations:

- Control the physical environment by managing light levels, temperature, noise, ventilation,
and distractions.
- Ensure all aspects are suitable for examination to maintain consistency.
- Administer the test at the same time and location for all participants to ensure fairness.

Test Scoring:

1. Scoring Process:

- Standardized Scoring: Involves norm-referenced or criterion-referenced score

interpretations.
- Human vs. Computer Scoring: While human scoring can be variable, computer scoring is
preferred for consistency.
- Descriptive Statistics: Summarize and describe a large group of data for analysis.
- Central Tendency: Represents values around which data cluster.
- Variability: Indicates the dispersion of scores within a group.

By following standardized procedures and ensuring a fair and conducive testing

environment, accurate test administration and scoring can be achieved to yield reliable and
valid results.

Sources: The Intact One, Scribd - Administration, Reporting and Scoring, Scribd - Test
Administration and Scoring, Quizlet - Chapter 13: Administration, Scoring, and Interpretation
of Selected Tests, ETS, Schreyer Institute for Teaching Excellence