COMPETENCY-BASED ASSESSMENT AND EVALUATION

DACUM SERIES 3 ROBERTO N. PADUA

LECTURE 1: TYPES OF ASSESSMENT

A norm-referenced test (NRT) is a type of test, assessment, or evaluation which yields an estimate of the position of the tested individual in a predefined population, with respect to the trait being measured. This estimate is derived from the analysis of test scores and possibly other relevant data from a sample drawn from the population. That is, this type of test identifies whether the test taker performed better or worse than other test takers, but not whether the test taker knows either more or less material than is necessary for a given purpose.

 Norm-referenced assessment can be contrasted with criterion-referenced assessment and ipsative assessment. In a criterion-referenced assessment, the score shows whether or not the test takers performed well or poorly on a given task, but not how that compares to other test takers; in an ipsative system, the test taker is compared to his previous performance.

Robert Glaser originally coined the terms normreferenced test and criterion-referenced test. Standards-based education reform is based on the belief that public education should establish what every student should know and be able to do. Students should be tested against a fixed yardstick, rather than against each other or sorted into a mathematical bell curve. By assessing that every student must pass these new, higher standards, education officials believe that all students will achieve a diploma that prepares them for success in the 21st century.

USES OF CRITERION-REFERENCED TESTS Most state achievement tests are criterion referenced. In other words, a predetermined level of acceptable performance is developed and students pass or fail in achieving or not achieving this level. Tests that set goals for students based on the average student's performance are norm-referenced tests. Tests that set goals for students based on a set standard (e.g., 80 words spelled correctly) are criterion-referenced tests.

CRITERIONREFERENCED=COMPETENCY-BASED
Competency-Based Assessment = CriterionReferenced Assessment We DO NOT use the Normal Curve here. University will base the criterion on CHEDs policies and standards for specific disciplines.

 Test-takers cannot "fail" a norm-referenced test, as each test-taker receives a score that compares the individual to others that have taken the test, usually given by a percentile. This is useful when there is a wide range of acceptable scores that is different for each college. .

IN THE U.S. OF A

By contrast, nearly two-thirds of US high school students will be required to pass a criterionreferenced high school graduation examination. One high fixed score is set at a level adequate for university admission whether the high school graduate is college bound or not. Each state gives its own test and sets its own passing level, with states like Massachusetts showing very high pass rates, while in Washington State, even average students are failing, as well as 80 percent of some minority groups. This practice is opposed by many in the education community such as Alfie Kohn as unfair to groups and individuals who don't score as high as others

FUNDAMENTAL ASSESSMENT BELIEF IN COMPETENCY-BASED EDUCATION

A rank-based system only produces data which tell which average students perform at an average level, which students do better, and which students do worse. This contradicts the fundamental beliefs, whether optimistic or simply unfounded, that all will perform at one uniformly high level in a standards based system if enough incentives and punishments are put into place.

 This difference in beliefs underlies the most significant differences between a traditional and a standards based education system.

 Examples IQ tests are norm-referenced tests, because their goal is to see which test taker is more intelligent than the other test takers.
Theater auditions and job interviews are normreferenced tests, because their goal is to identify the best candidate compared to the other candidates, not to determine how many of the candidates meet a fixed list of standards. TOEFL, GRE, APTITUDE TESTS and other such tests are criterion-referenced tests. There are standards that should be met in these tests.

WORKSHOP 1 (2 hours)
A. Answer the following: 1. Evaluate your current system of assessment in the University. Is the system criterion-referenced or norm-referenced? 2. In your daily quizzes, do you adjust the cut-off for passing? How do you adjust this cut-off score? Why do you adjust the cut-off scores? Is your attention called if many students fail your subject? 3. Cite the advantages and disadvantages of a criterion-referenced assessment system.

 B. The following are average test scores of students over 10 quizzes in CALCULUS. There are 100 students. The first column are the test scores while the second column are the sorted values (from smallest to highest). 1. Give six letter grades to the students (A (excellent) to F(fail)) using the mean as basis. 2. Let the competency criterion be: students fail (F) if their scores fall below 50. Give the same six letter grades. 3. Compare the grade distribution in the two cases.(i.e. How many got As , Bs, Cs, Ds, Es and Fs?) What is the implication of the shift from norm-referenced to criterion referenced grading? NOTE: Mean = 26.89, sd = 26.86

te st score 11.4513 0.0055 0.823 70.2593 72.4788 44.6912 51.3586 3.2331 33.837 0.7574 11.2074 2.9996 4.0455 53.1852 0.4855 39.1936 37.0756 53.974 23.2271 0.0938 13.6035 73.7325 26.1218 50.7973 70.7066 85.6004 9.7895 11.4519 18.7118

te st1 0.0006 0.0015 0.0055 0.023 0.0597 0.0938 0.1436 0.1581 0.1828 0.4385 0.4855 0.4984 0.618 0.6513 0.6518 0.7574 0.823 1.4855 1.584 1.6943 1.695 2.2001 2.2327 2.5885 2.6377 2.9996 3.15 3.2331 3.7615

8.7776 11.8127 0.4385 1.6943 12.9613 2.2327 9.7632 16.5586 67.7283 2.2001 0.1828 13.178 0.0597 4.9041 64.5033 0.1436 55.1 97.2394 24.523 32.3522 4.5439 59.3966 21.296 69.0119 41.9706

4.0455 4.5439 4.9041 5.9875 6.1109 8.7776 8.9835 9.7632 9.7895 11.2074 11.4513 11.4519 11.8127 12.5232 12.9613 13.178 13.6035 16.0854 16.5586 17.3451 17.7711 18.4555 18.7118 19.6792 20.8089

57.6791 0.4984 3.7615 91.0837 80.7673 16.0854 8.9835 47.3524 17.3451 0.0006 44.8074 26.8091 47.9398 25.9045 0.0015 2.5885 1.4855 79.2659 20.9764

20.9764 21.296 22.0152 23.2271 24.0156 24.523 25.9045 26.1218 26.8091 28.674 30.4836 32.3522 33.837 36.3833 37.0756 37.7408 39.1936 41.9706 44.6912

28.674 5.9875 19.6792 0.618 12.5232 30.4836 66.7573 0.023 18.4555 69.2863 17.7711 1.695 22.0152 64.8766 1.584 2.6377 0.6518 61.1118 0.1581 20.8089 6.1109 64.5372 36.3833 0.6513 24.0156 3.15

47.3524 47.9398 50.7973 51.3586 53.1852 53.974 55.1 57.6791 59.3966 61.1118 64.5033 64.5372 64.8766 66.7573 67.7283 69.0119 69.2863 70.2593 70.7066 72.4788 73.7325 79.2659 80.7673 85.6004 91.0837 97.2394

LECTURE 2: ASSESSING A COMPETENCY

Given a competency, break it down into its smaller tasks. Example: Proficiency in the Use of a Gavel Competency:
Task 1: Number of gavel bangs for Call to order Task 2: Number of gavel bangs for approval of resolution Task 3: Number of gavel bangs for restoring order. Task 4: Number of gavel bangs for stand or sit Task 5: Number of gavel bangs for adjourn Task 6: Match number of gavel bangs with specific purposes

 For each task, either a student possesses the skill or not Set the standard for possessing the competency. Example: A student is proficient in the use of a gavel if he is able to perform 5 out of 6 skills. The students competency score is expressed as a percentage of the tasks successfully performed:
Example: if student is able to do 4 out of 6 skills, his 4 competency score is or 67%. If able to do 5 out of 6 skills, 6 5 his competency score is or 83%.
6

 the smaller the number of skills that comprise a competency, the more stringent becomes the competency score. We agree: (Ad Hoc Rule)
A competency should comprise no less than 10 skills, but no greater than 20 skills.

If a competency has more than 20 skills, this means that you are measuring an ABILITY. Break it down into two or more sets.

 The standard for each competency must be agreed by all faculty numbers teaching the subject. This standard will define the competitiveness of the graduates of the University.

 Example:

Proficiency in factoring the difference of two squares

Task 1: Identify a variable raised to the second power Task 2: Identify a binomial. Task 3: Multiply two Binomials. Task 4: Find the product of ( + ) and ( ) Task 5: Obtain the product: 2 2 =(x + Y)(x - y). Task 6: Replace x and y by other letters (a, b, e etc.) Task 7: Perform Tasks 4 and 5 using different letters Task 8: Represent 2 as the area of square with side x. Task 9: Express the difference 2 2 as the difference of the areas of two squares. Task 10: Replace x by and y by to express as the product: + =

Assessing A Task
Provide at least three (3) ways to assess a task before declaring a task as successfully done or not done. Example: Task: Identify a variable/number raised to the second power.
Assessment 1: Which of these expression is a square?
1 2 1 2 4 2 3 2

(a) (b) (c) (d) Assessment 2: How will you express 16 as a power of 4?
(a) (b) 4 4 (c) 3 (d) 2 Assessment 3: What is the area of a square whose side is 2 3? (a)6 (b) 12 (c) 18(d) 24 Note: If student gets any of A1 to A3 correctly, then the task is successfully performed.
1 2

2 2

 We can make use of various task assessment combinations Task: Realize that 2 2 is the difference of the areas of Example: two Squares. Assessment 1: Cut out two squares the difference of whose areas is 12. What are the sides of the two squares? (Note: there are many possibilities here). Assessment 2: Shade the areas represented by 2 2 below: y

Workshop 2 (3 hours)
Go back to your syllabus, break down each competency found in your syllabus into tasks Please be reminded of our ad hoc rule.

Lecture 3: THE ASSESSOR

The Assessor must not be the Teacher of the subject. This is a basic principle in Competency Based Assessment. It is preferable if the Assessor is external to the system. If this is not feasible, the University can establish a pool of Certified Assessors.

 The Vice President for Academic Affairs instructs the Deans to establish a pool of Certified Assessors per subject. The Deans begin the process of Certification and submits the list to the VP - AA

 The Certification process is as follows:

Step 1: For a given subject, the Dean chooses a random number of competencies (at 50% of total number of competencies for the subject). Step 2: The Dean requests a group of 10 individuals to participate as students to be assessed. Step 3: The participants are instructed to pretend they do not possess a random number of skills. Step 4: the faculty applicant/assessor assesses the 10 participants. Step 5: The Faculty is certified if he is able to correctly identify the skills not possessed by the 10 participants.

Workshop 3
Choose a subject. (You can make use of chapter 1 for this exercise.) The Dean organizes the participants to the Certification process including the candidate assessor. For this purpose only 5 participants + 1 faculty candidate will be needed. Perform the Certification process

LECTURE 4: PREPARING THE ASSESSMENT INSTRUMENTS

It is possible to prepare the Assessment Instruments ahead of time The Assessment Instrument of Tasks per competency.

Sample: Competency: Proficiency in factoring the difference of two squares.

Task Present Absent Method

STANDARD ASSESSMENT FORM

We agree to adopt the STUDER form. The students are provided the opportunity to rate themselves through demonstrable performance or evidence. Students must show EVIDENCE of their proficiency in a given competency. Teacher must specify ACCEPTABLE EVIDENCES. The evidences are gathered in a PORTFOLIO.

ON ASSESSMENT KEY
We modify the STUDER form by changing the assessment key: 1 = needs improvement 2 = proficient in 90% of the tasks 3 = proficient in 100% of the tasks

WEIGHTS ON ASSESSORS RATING AND STUDENTS PORTFOLIO

We agree to put 60% weight on assessors rating and 40% weight on students portfolio: Competency rating = 60% assessment
+

40%students portfolio

GRADING SYSTEM
The students are declared either PROFICIENT or NOT PROFICIENT in a given competency. Each subject has a set of ABILITIES to be developed. Each ABILITY has several COMPETENCY components. The students grade in a subject is therefore equal to the proportion of COMPETENCIES where he is PROFICIENT to the TOTAL NUMBER OF COMPETENCIES.

 The teacher REPORTS not only the GRADE based on successfully performed tasks, but also the ABILITIES DEVELOPED by the student. Example: Juan got a grade of 85% in College Algebra. The abilities developed by Juan are: 1. Ability to do algebraic operations on variables. 2. Ability to translate real-world problems in algebraic terms and solve the problems. Juan is unable to prove abstract THEOREMS.

NAME: JUAN DELA CRUZ SUBJECT: COLLEGE ALGEBRA GRADE: 85% ABILITIES DEVELOPED ABILITIES NOT DEVELOPED REMARKS 1. ABILITY TO DO 1. ABILITY TO PROVE Juan needs coaching in Logic and ALGEBRAIC OPERATIONS ABSTRACT THEOREMS IN Inference. ON VARIABLES. ALGEBRA. 2. ABILITY TO TRANSLATE 2.ABILITY TO ANALYZE Juan needs to concretize the REAL WORLD PROBLEMS PROBABILITY PROBLEMS concept of randomness as IN ALGEBRAIC TERMS opposed to determinism. AND SOLVE THEM.

WORKSHOP 4 (3 hours)
Using Chapter 1 as basis for your discussions, prepare a STUDER chart for assessing competencies. For each TASK identify ACCEPTABLE EVIDENCES to be presented by the student.

For each TASK provide at least three(3) assessment methods.