The Pediatric OSCE Collaboration of Canada

(POCC)
Authors: Moyez Ladhani (McMaster University, Pediatrics,Hamilton, Canada);
A Atkinson (University of Toronto, Pediatrics, Toronto, Canada);
H Writer (University of Ottawa, Pediatrics, Ottawa, Canada);
S Lawrence (University of Ottawa, Pediatrics, Ottawa, Canada);
A Jeffries (University of Toronto, Pediatrics, Toronto, Canada);
CPPD Canadian Pediatric Program Directors of Canada; (CPPD)

BACKGROUND SUMMARY OF WORK

The Royal College of Physician and Surgeons of Canada requires
residents to pass a comprehensive examination at the end of their training,
which includes a ten station OSCE. All seventeen pediatric training
programs in Canada have implemented some form of practice OSCE,
usually occurring biannually, which provides the learner with valuable
formative feedback.
Successful implementation is, however, resource intensive - requiring
question bank development and maintenance, extensive human
resources, faculty development and physical resources. Many of our
programs lack the infrastructure and manpower to run successfully a
comprehensive and meaningful OSCE for their learners.
With the objectives of distributing resources and improving standardization, in 2009 three programs in
Ontario began collaborating on OSCE administration. Station development responsibilities were spread
across programs, reducing individual centre resource strain, and allowing for the administration of a
more standardized exam at each centre. A national OSCE was the logical expansion, with the creation
in 2012 of the Pediatric OSCE Collaboration of Canada (POCC).
All 17 residency programs now participate in this nationwide standardized process. Responsibilities for
OSCE blueprint development, station development and review; language translation and data collection
and distribution are spread nationwide. Collaboration is key to the success. Data for individual stations
and resident peer group performance is distributed nationally, allowing each program to benchmark
resident performance against a national cohort, and to identify strengths and weaknesses with regard to
specific content.

Step 1: Blueprint development – 10 stations each for Fall and Spring OSCE
Step 2: Programs assigned station(s)
Step 3: Stations reviewed by vetting committee
Step 4: Stations translated English/French
Step 5: Programs run OSCE and submit average scores for each station and Post Graduate year
Step 6: National results distributed to each program

OSCE BLUEPRINT EXAMPLE OF RESULTS SUMMARY

STATION 1 STATION 2 …….STATION 10*

Content/Topic EKG teaching Critical Appraisal RESP PROGRAM X STATION 1 STATION 2 ……STATION 10*
(AVERAGE
Teach EKGs to Written – Critical Asthma History &
Brief Description
medical student Appraisal Smoking Education SCORE)

Type: Structured Oral, **SP/SHP or Written SHP Written Structured Oral PGY 1 6.8 6.8 5.6
PGY 2 7.1 7.0 5.8
History Taking X
PGY 3 7.7 7.9 7.5
Physical Exam
PGY 4 7.9 8.8 7.4
Tests and Procedures
Data Interpretation X X
PROGRAM STATION 1 STATION 2 ……STATION 10
Management X Y (AVERAGE
SCORE)
Communication and Patient Education X X
PGY 1 6.1 5.9 7.0
CanMEDS Medical Expert X X
PGY 2 7.1 6.8 7.8
CanMEDS: Communicator X X
PGY 3 7.3 8.1 8.3
CanMEDS: Collaborator
PGY 4 7.2 8.7 7.9
CanMEDS: Health Advocate X
CanMEDS: Professional
NATIONAL STATION 1 STATION 2 ……STATION 10
CanMEDS: Scholar X X
AVERAGE
CanMEDS: Manager X
PGY 1 6.9 6.1 7.2
Program Responsible
PGY 2 7.5 6.9 7.5
Weighing of Station Check List: Global Rating 30:70 100:0 50:50
PGY 3 7.7 7.2 8.0
*In this example only 3 stations are shown, there are actually 10 stations
PGY 4 7.9 8.3 8.2
**SP = Standardized Patient
SHP = Standardized Health Professional *In this example only 3 stations are shown, there are actually 10 stations

SUMMARY OF RESULTS

POCC has successfully implemented a national standardized formative OSCE and highlights an effective collaboration of the Canadian Pediatric Program Directors.
Residents can now receive feedback compared with their peer group nationwide. Task distribution across programs has decreased individual program resource
challenges, and organization and standardization have improved.