Professional Documents
Culture Documents
Exam Scanning Interpretation
Exam Scanning Interpretation
October 2008
CUM. FREQ Cumulative Frequency. This number is a running sub-total of the FREQ COLUMN. It is useful for determining how many students fell below a certain RAW SCORE of percentile rank, PCT. RANK. PERC. RANK The Percentile rank for any candidate with score A is: Perc Rank = sum * 100 / Number of candidates taking the exam Where sum = the number of candidates with raw scores lower than A plus one-half the number of candidates with the same score, A PERC RIGHT Item repeated for convenience MEAN SCORE Arithmetic Average of all raw scores in the column SCORE STANDARD DEVIATION A commonly used indicator of the degree of dispersion and a source of estimation of the variability in the total population from which the sample came. A statistical tool that enters into many statistical computations. If all the candidates have the same raw score, the standard deviation will be zero. If this happens, the standard deviation is changed to one and the following message is printed: ****WARNINGSTANDARD DEVIATION IS CHANGED TO 1.0--**** If the standard deviation is changed to one, the Kuder Richardson Formula #20 and the Standard Error of Measurement are not calculated. DISCR Discrimination Index. A measure of how well the questions differentiates the highscoring student from the low-scoring student. A maximum value of 1.00 indicates that students who scored high on the test also answered this question correctly while students who scored low on the test tended to miss the question. A .40 is an indication that the question discriminated wee while a .20 or less means that the question tended to be a poor discriminator. items DISC = I = 1 ISPRD (ISUM * RMS) / SAS * ITVAR
Where: ISPRD = Sum of raw scores times the number of correct answers for a particular question. ISUM = Sum of correct answers for all questions. RMS = Mean of scores. SAS = Standard deviation of scores
October 2008
ITVAR = (number of candidates * ISUM) ISUM2 Items = Number of questions. RBIS A correlation index dichotomizing the candidates into a top 27% and measuring students who scored high and were in the top 27% as opposed to those who scored low and were in the lower 27%. For Example, if: 1. 12 candidates answered the question correctly who are in the top 27%. 2. 12 candidates were in the top 27%. 3. 0 candidates answered the question correctly who are in the bottom 27%. 4. 12 candidates were in the bottom 27%. 12/12 0/12 = 1.0 items RBIS = i=1 IHIGH / ITOT2 items i-1 ILOW / ITOT1
Where: IHIGH = number of people who answered question correctly that are in the upper 27%. ILOW = number of people who answered question correctly that are in the lower 27%. ITOT1 = the number of people in the lower 27%. ITOT2 = the number of people in the lower 27%. Items = number of questions. DIFF Difficulty Index. This number represents the difficulty of the item based on the number of students who answered the question correctly or incorrectly. This index is the percentage of students answering the correct alternative right. KEY The correct response alternate identified on the key sheet. ALT Alternative (1-5). These alternatives correspond to the response positions on the key sheet. The numbers in the ALT columns represent the proportion of students who answered the question by choosing a given alternative. EXAMPLES: A 1.00 (100%) indicates that all students chose this response. A 0.0 indicates that no student chose this response. RESPONSE NUMBER This is a table showing the number of persons choosing one of the responses (misleads) numbered 1-5 for a given item number. NO RESPONSE shows the total for the number of persons leaving an item blank.
October 2008
KUDER RICHARDSON FORMULA #20 This is a reliability coefficient based upon interitem consistency. It is of greatest value when all of the items of a test are equal or nearly equal in difficulty. In such a case it gives an indication of how consistently the students answered the questions. A lack of consistency is an indication of error in measurement. This coefficient varies from 1.00 to 1.00. Ideally it should be above .90. However, a value or .70 or above is acceptable for teacher made tests. If the coefficient is too low it is an indication that either the test is too short or that the items have very little in common. The Kuder Richardson formula is not calculated if the standard deviation has been changed to 1.0 (see STANDARD DEVIATION(S)). STANDARD ERROR OF MEASUREMENT The KR20 formula above gives an indication of the degree of accuracy. The standard error of measurement gives an indication of the amount of error. Each students score is assumed to be an estimate of his true score since our measurements are never exact. By adding and subtracting one standard error of measurement from a students score the user can get an indication of the range within which the true score probably lies. For example, if a students score on a 70-item test was 50 and the standard error of measurement was 4 it is probably that his true score is somewhere between 46 and 54. This measure of error is calculated by the following formula: S.E. = S.D 1 r tt Where SD is the standard deviation of the test scores and RTT is the reliability coefficient (kr). The amount of error as indicated by the S.E of measurement is inversely related to the reliability. When reliability is small the amount of error is large.
October 2008