GUIDELINES ON TEST BLUEPRINT, TEST CONSTRUCTION PRINCIPLES, ITEM EDITING, SETTING THE

PASSING MARK

GUIDELINES ON THE TABLE OF SPECIFICATIONS (TEST BLUEPRINT), TEST CONSTRUCTION,

DETERMINING THE PASSING SCORE, ITEM EDITING, AND ITEM ANALYSIS

I. COMPETENCY-BASED TABLE OF SPECIFICATIONS (Test Blueprint)

A. Rationale

a. Examinations must have content validity, emphasizing on the “must know”

topics.
b. Examinations must measure what must be learned.
c. The skills and abilities to be tested must be identified.
d. Proper distribution of items must be ascertained.
e. Trivial items are avoided

B. Process

a. Use the behavioral objectives as main guide in determining the ability /

competency to be tested.
b. Use the content or subject matter as basis for identifying specific topics for
testing.
c. Assign the distribution of items assuring proper distribution of knowledge,
application and problem-solving questions.
d. Make sure that the test blueprint is before you. Periodically refer to it as you
formulate the examination questions.
e. Prepare more examination questions than what are actually necessary. Make
allowances for items discarded or rejected during the review.
f. Allow adequate time for formulating questions. The quality of the items suffers
when they are hurriedly written down. There must be enough time for review
and editing by the examiner and his/her peers

C. Components

1ST column: Competency

2nd column: content or subject matter (main topics and subtopics)

3rd column: weight in percentage

4th column: number of items

Last 3 columns: distribution of items per learning dimension/Bloom’s taxonomy

and level of difficulty

D. GENERAL SCHEME (SAMPLE) FOR ITEM DISTRIBUTION (%)

a. Knowledge: 15%
i. Recall…………………5% (Easy)
ii. Comprehension………..10% (Moderate)
b. Application: 25%

i. Moderate………………25%

c. Problem-solving: 60%
i. Easy……………………10%
ii. Moderate………………30%

1
 iii. Difficult............……….20%

E. SAMPLE TOS (100 questions in Surgery)

COMPETENCY CONTENT WT NUMBER KNOWLEDGE APPLICATION PROBLEM-

OF RECALL / COMPRE SOLVING
ITEMS (Easy) (Moderate) (Difficult)
Given clinical conditions: 1.Head and 9% 9 1 3 5
1. Identify neck surgery
incidences (age,
sex,etc) 2. Thoracic 7% 7 1 2 4
2. identify and
presenting signs cardiovascular
and symptoms, surgery
physical exam
findings 3. Abdominal 8% 8 1 2 5
3. identify etiology, surgery I
causative and
contributory 4. Abdominal 8% 8 1 2 5
factors surgery II
4. Explain
mechanisms of 5. Abdominal 8% 8 1 2 5
disease surgery III
5. Select
appropriate 6. Endocrine 5% 5 1 1 3
diagnostic work- surgery
up
6. Interpret results 7. Urology 5% 5 1 1 3
of diagnostic
procedures and 8.
laboratory Neurosurgery 3% 3 1 1 1
exams
7. Formulate 9. Pediatric 5% 5 1 1 3
diagnosis surgery
8. Determine
extent and stage 10. Plastic 3% 3 1 1 1
of the disease surgery
9. Decide on most
appropriate 11.
management Orthopedics 5% 5 1 1 3
(medical,
surgical, RT, 12.Gynecology 3% 3 1 1 1
chemoTx, etc.)
10. Predict course 13.Skin, soft
and behavior of tissue, breast, 8% 8 1 2 5
disease hernia surgery
11. Identify
complications 14.Anesthesia 3% 3 1 1 1
from the disease
and 15.Surgical
complications basic 20% 20 1 3 16
from treatment principles
and procedures
12. Decide when to
refer to
specialist

2
 TOTAL 100 100 5 10 25 60
%

II. TEST CONSTRUCTION (MCQ FORMAT)

A. COMPONENTS

1. The ITEM

-the complete test question

-the question and all possible answers

-contains the idea to be tested

2. The ITEM IDEA

-the fact or concept to be tested by the item

-it must be a single coherent thought

-it must relate directly to the objectives of the program

-it must be worth testing, avoiding trivial items

3. The STEM

-sets the problem, question, stimulus

-contains the clear task that the examinee is expected to perform

-should be concise

-preferably worded positively

4. The ANSWER SET

-all the answers from which a selection is made

-include only one best or correct answer

-option should be grammatically consistent with the stem

-option should belong to the same class

-place options in random order unless they possess logical or numerical order

-avoid “none or all of the above”

5. KEYED ANSWER

-the correct or best answer

-first to be determined

-basis for selecting distracters

3
 -there should be consensus that the keyed answer is the best or correct answer

-its length, qualification, position does not cue the correct answer

-contains no cues (always, never) or use of same word in stem

-avoid use of stereotyped phrases

6. FOILS or DISTRACTORS

-the set of wrong answers

-the closer they are to the keyed answer, the more difficult is the item

-must be plausible and logical options, otherwise, they do not function as

distractors; will not be selected by examinees

B. PRINCIPLES

Principle 1: Avoid multiple true-false items

In spontaneous pneumothorax:

A. Majority occurs in the elderly

B. Located mostly in the right side

C. Preceded by vigorous exertion

D. Occurs only in pathologic lungs

Poor question because the stem does not contain what the examinee is supposed to do (task)

The options deal with different aspects of the condition, not with a single concept or idea.

Principle 2a: Avoid negatively-worded stems

Which is NOT a common finding in SLE?

A. purpura

B. splenomegaly

C. alopecia

D. joint swelling

Poor question because it emphasizes what is NOT common. It will be better to deal with
what is common.

Principle 2b: Avoid double negatives

The following are uncommon manifestations of acute rheumatic fever,

EXCEPT:

A. skin rashes

4
 B. sore throat

C. joint swelling

D. body malaise

Having UNCOMMON and EXCEPT in the stem confuses the examinees. And because they
are uncommon, they may be dealing with TRIVIAL, rather than important aspects of the disease.

Principle 3: All options should belong to the same class

A 12-year-old girl presents with acute, severe attack of difficulty of breathing

characterized by wheezing. The most appropriate management:

A. steroids

B. nebulization

C. epinephrine

D. Bronchodilators

Make the examinees choose which option belonging to the same class is the best answer. If
you want to test if they know what drug to use in the clinical situation, then all options should
be drugs.

Principle 4:Avoid the options “none of the above” and “all of the above”

Genetic predisposition to breast cancer is carried by chromosome number:

A. 5

B. 7

C. 17

D. 21

E. none of the above

Even if the examinees can choose letter E as the correct answer, it does not mean that
they know which chromosome number carries the genetic predisposition. So, better include the
correct answer in the options.

Hemoptysis is most likely to occur in which of the following conditions?

A. mitral stenosis

B. tuberculosis

C. bronchiectasis

D. lung abscess

E. all of the above

5
 In this question, if the examinee is able to determine that hemoptysis is present in just TWO
of the conditions (example: bronchiectasis and tuberculosis), even if they do not know anything
about mitral stenosis and lung abscess, they will already answer ALL of the ABOVE.

Principle 5: All alternatives should be grammatically consistent with the stem. Reading
thru the option should complete the sentence

A patient develops oliguria after nephrectomy. The most appropriate management is:

A. Administering diuretics

B. Refer to nephrologist

C. IV fluid increase

D. Check patency of foley catheter

Pay attention to the grammatical construction of your question. If the stem is an

incomplete statement, reading through the options should complete the sentence.

Principle 6: Words that repeatedly appear in the options should be transferred to the
stem

Response to anti-thyroid treatment is most frequently monitored through:

A. decrease in exophthalmos

B. decrease in heart rate

C. decrease in tremors

D. decrease in appetite

Transfer the word DECREASE to the stem. The question will now be revised to:

Response to anti-thyroid treatment is most frequently monitored through decrease in:

A. exophthalmos

B. heart rate

C. tremors

D. appetite

Principle 7: Place options in random order, varying the position of the correct answer.
When the situation calls for it, they should be placed in a logical or numerical order

The urine output of a 60 kg. man should be maintained at what minimum amount
(cc/hr)?

A. 15

B. 30

C. 60

D. 120

6
 Arrange the options from lowest to highest value/number.

Principle 8A: Avoid antonyms (opposites)

The most characteristic finding in typhoid fever is:

A. headache

B. bradycardia

C. hypothermia

D. tachycardia

Bradycardia and tachycardia are opposites of each other. The examinee will now know that the
answer is one of the two options, so that they will not consider the other options anymore
(headache and hypothermia) because it is not possible to have two opposite conditions to be
present at the same time.

Principle 8B. Avoid synonyms (similar terms)

A 60-year-old male, chronic smoker, presents with multiple, matted cervical nodes of 6
months’ duration. Most probable diagnosis:

A. TB

B. Lymphoma

C. Koch’s

D. Carcinoma

TB and Koch’s are the same. The examinee will be confused because there two correct answers
in this question. So, think of another clinical condition as distractor.

Principle 9: Use acceptable standards to avoid confusion

Vehicular accidents most frequently result in injuries to which of the following?

A. head and neck

B. thorax

C. abdomen

D. extremities

Revise: according to the WHO statistics on trauma, the most common………

Principle 10: There must be at least three plausible, logical options

Newborns in respiratory distress should always be intubated and connected to a

respirator.

A. True

B. False

Convert to TRUE / FALSE format.

7
 Principle 11: Avoid the cascade effect when asking a series of questions

A 30 year-old male presents with a two-week history of cough, yellowish sputum, fever,
chills and chest pain. The most probable diagnosis is:

A. bacterial pneumonia

B. tuberculosis

C. viral influenza

D. bronchial asthma

The most appropriate management is:

A. antibiotics

B. quadruple anti-TB regimen

C. anti-viral agents

D. steroid inhalation

Cascade effect means that when the examinee commits an error in question number 1,
he will also commit a mistake in question number 2, because they are connected or inter-
related.

Principle 12: When using a case, the question should be answered based on the case.

A 70-year-old male presents with a 1 week history of abdominal distention, obstipation

and recurrent abdominal pain.

The most common cause of intestinal obstruction is:

A. post-operative adhesion

B. hernia

C. carcinoma

D. volvulus

You can remove the first part of the stem (clinical data about the 70-year-old patient)
and you will still be able to answer the question.

III. DETERMINING THE PASSING SCORE

A. Criterion Reference

 Use of a predetermined cut-off standard

8
  Based on the principle that when professional competence is to be tested,
there should be a minimum standard imposed, below which examinees should
not be allowed to be admitted to the profession

 Main advantage: less chance of passing an incompetent examinee, with its

dire consequences on future clients/patients and on society

 Not affected by performance of examinees based on the profile of the batch

taking the exam (this is found in norm- referenced standards where the
passing could be significantly pulled down when the batch is weak)

 Main disadvantage: possible high failure rates due to high expectations of

those setting the standards

B. Nedelsky Method

 Computation of Minimum Pass Level (MPL) or Acceptability index (Ai)

 Process: place yourself in the position of a borderline examinee. For each item,
determine which option/s a borderline examinee can reject outright as wrong
option/s

 Compute for the Ai for the item

 Ai = 1 ÷ no. of options not rejected

 Ex. The most probable diagnosis is:

A……

B……

C……

D……

 Borderline examinee rejects D outright.

 4 options minus 1 = 3

 Ai = 1 divided by 3

 Ai = 0.3

 If borderline examinee rejects D and B outright

 4 options minus 2 = 2

 Ai = 1 divided by 2

 Ai = 0.5

9
  If borderline examinee rejects D, B and A outright

 4 options minus 3 = 1

 Ai = 1 divided by 1

 Ai = 1.0

 Determine the MPL of the exam where:

% MPL = Ai for all items x 100 / Total no. of questions

Make summary table of your formulated questions (sample):

MPL / Ai Value Number of Items Level of difficulty

1.0 20 Easy
0.5 60 Moderate
0.3 20 Difficult
Computation 1.0 x 20 = 20
for 0.5 x 60 = 30
0.3 x 20 = 6
MPL for the Exam 56

IV. EDITING MCQ QUESTIONS

The ITEM IDEA

1. Can it be identified? What is the idea, fact, concept tested?

2. Is it important? Must know?

3. Is it a single coherent concept/thought?

4. Is it included in the content to be tested?

5. Did it follow the test blueprint?

The ITEM STEM

1. Is the idea clearly and concisely stated? Is the stem too long?

2. Did it pose a specific question/problem?

3. Is it properly constructed?

4. What ability / competency is it testing?

The ANSWER SET

1. Is one best or correct answer included?

2. Is there consensus regarding the keyed answer?

10
 3. Are all distractors logical and plausible?

4. Do all options belong to the same class?

5. Were the options “all and none of the above” avoided?

6. Are all options grammatically consistent with the stem?

7. Are options placed in random order, or if there is a need, in logical/numerical order?

8. Are antonyms/synonyms avoided?

9. Are acceptable standards used?

10. Is the cascade effect avoided?

11. Are the answers based on the cases given?

NOTE: Based on the answers to these questions, revision and improvement of the items are effected
either through revision of the stem or the options, or the keyed answer.

Lecturer/Facilitator: Jose Y. Cueto Jr., MD, MHPEd

Dean, MMSU College of Medicine
Former Faculty Member, UP National Teacher Training Center for the Health
Professions

11