Creating a Competency-Based OSATS

Conceptual Framework:
• Identify PAEA competency (or competencies) to assess
• Identify the technical skill(s) to assess the competency

Practical Considerations
• Space, cost, time, supplies, etc.
• Standardized patient (SP) availability

Guide to Developing an OSATS

• Develop set of objectives
o What do you want students to achieve by completing this OSATS?
 Demonstration of problem-solving skills?

ty
 Reflect on progress towards non-cognitive attributes?
• List competencies to assess

si
o What non-cognitive attributes will you be assessing to evaluate that competency?
• Determine specific technical skills associated
o Selection of materials
o Demonstration of proper technique
v er
o Patient & provider safety
ni
• Create a patient scenario
o Are you going to challenge students with a procedural complication during the OSATS?
rU

o Will this be a routine procedure?

• Develop special instructions for standardized patients
o Be clear and concise with expectations for SP
tle

• List materials and plan for set-up

o What supplies are needed for students to complete the task presented?
o How do you want to set up each student station?
Bu

• Rubric for student evaluation of technical skills

o Start with creating a step-by-step list of the procedure(s) students are tasked with
 Should each step have the same point value, or will they have different weights?
 Is the point criteria “yes” or “no,” or will you have a range of point values?
 What does “adequate” vs. “mastery” look like?
• Determine evaluation method for non-cognitive competencies
o Student reflection
 Survey?
 Written/open-ended response?
 Large group debrief?
o SP reflection survey
 From the “patient” perspective, did the student display the non-cognitive attributes being
evaluated?
• Provide students with feedback
o Large group debrief, including discussion of non-cognitive attributes demonstrated in OSATS
o Written personalized feedback on rubric, when possible, focusing feedback on non-cognitive
attributes