A CONCEPTUAL FRAMEWORK OF THE CLINICAL LEARNING ENVIRONMENT IN

MEDICAL EDUCATION

By

Jamie Sue Padmore

Dissertation submitted to the Faculty of the Graduate School of the

University of Maryland University College, in partial fulfillment
of the requirements for the degree of
Doctor of Management
2015

Advisory Committee:
Dr. Thomas J. Mierzwa
Dr. Denise A. Breckon
 ProQuest Number: 10041765

All rights reserved

INFORMATION TO ALL USERS

The quality of this reproduction is dependent upon the quality of the copy submitted.

In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.

ProQuest 10041765

Published by ProQuest LLC (2016). Copyright of the Dissertation is held by the Author.

All rights reserved.

This work is protected against unauthorized copying under Title 17, United States Code
Microform Edition © ProQuest LLC.

ProQuest LLC.
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106 - 1346
 © Copyright by
Jamie Sue Padmore
2015
 ABSTRACT

Title of Dissertation: A CONCEPTUAL FRAMEWORK OF THE

CLINICAL LEARNING ENVIRONMENT
IN MEDICAL EDUCATION

Jamie S. Padmore, Doctor of Management,

2015

Dissertation Directed By: Dr. Thomas J. Mierzwa and Dr. Denise A.

Breckon, Doctor of Management

The hospital setting provides an environment for patients to receive medical care, for medical
professionals to provide treatment, and for medical students and residents to learn the practice of
medicine through supervised patient encounters. Education provided at the point of care allows
students and residents to apply knowledge and develop clinical skills needed for medical
practice. The hospital environment is also a confluence of learning and work, where applied
learning takes place in an integrated and simultaneous manner with work duties. This setting,
referred to as the clinical learning environment (CLE), is a focus for educators, scholars,
administrators, regulators and accrediting agencies to understand, measure and improve it. While
several instruments have been developed to measure the CLE, they suffer from great variation in
subscales and content. The purpose of this study is to deconstruct the CLE, apply theories from
related fields, and frame those theories in the context of the hospital setting to develop a
conceptual framework for the CLE. A systematic review of the literature and thematic synthesis
of existing research about the CLE provided evidence to inform and test a learning environment
framework in the clinical setting. Data from qualitative CLE assessments, the ACGME Clinical
Learning Environment Review (CLER) Pathways to Excellence, and existing CLE measurement
instruments informed these results. Findings showed that a CLE framework consists of three
mediating factors: learning, people, and change. As the clinical setting is a unique environment
for learning, the people dimension (as a community of practice) was found to be the most
influential on learning outcomes for students. The dimension of change was found to be most
influential from the perspective of improving organizational or work outcomes, including patient
care, clinical quality and patient safety. Findings from this study provide researchers and
scholars with a framework to for developing measures of clinical learning environment
effectiveness, and informing practitioners of CLE components and relationships that impact both
learning and organizational outcomes.

Keywords: Clinical learning environment, community of practice, learning outcomes, graduate

medical education, medical students, organizational outcomes, residency, teaching hospital.

ii
 Dedication

To my seventh grade teacher Judith Kern Reed, who instilled in me a lifelong passion for
learning. Thanks, Mom!

iv
 Acknowledgements

The doctoral journey is one that is unpredictable, all-consuming, fulfilling, and

exhausting. It literally takes a village to support someone completing a dissertation process.

Before I started this endeavor I asked my family if they would support me in this process, and in

hindsight, it was one of the most important discussions we had. Thus, I must begin this section

by thanking the two people who were most supportive and encouraging – my husband Eric and

my son Robert. Without your support every step of the way, this dissertation would have never

been finished. Your constant love, encouragement, and willingness to accept my distractions and

time away from our family allowed me to succeed in this journey. Eric, you succeeded in getting

me over the finish line! I literally could not have done it without you. And Robert, your

dissertation ―hugs‖ were the best to inspire me to keep working one more hour…

The other person that literally made this possible is my boss, mentor, and role model, Dr.

Stephen R. T. Evans. Steve, you set high expectations every day, lead by example, and never

expect anything less than spectacular. You could not have been more generous with allowing me

time away, supporting my scholarly activities, and being a constant voice of reason. For all of

these reasons, I am indebted and forever grateful.

To my staff at MedStar and Georgetown – you are the absolute best. Each and every one

of you have stepped up, filled in when needed, and never let one single thing fall through the

cracks. It is truly a gift to work with a team like you. And I would be remiss if I didn‘t call out

my assistant individually, Ms. Jennifer Pritchett. Jenn, you literally put up with me every day. I

couldn‘t get through the day without you.

To the MedStar Health Teaching Scholars program, my professional love. Neil, Peg,

Tamika, Sarah, Amy and Allison – you are incredible, smart, thoughtful, and generous with your

v
feedback and encouragement. To the actual teaching scholars and scholar alumni, thank you for

putting up with me, indulging my ―theorist of the month‖ discussions, and provocative discourse

on conceptual frameworks. I can only have those conversations with you.

To Michele Malloy, previously the Research Services Librarian at Georgetown‘s

Dahlgren Memorial Library – thank you. You are a gifted and talented librarian who shares a

love of systematic review methodology with me. Your support with search strategies, data and

organization of articles was critical to the success of this research.

Throughout the doctoral journey, many things happen in life. Along the way we lost two

very important people, Ms. Thelma Padmore, my mother-in-law, and Dr. David Reed, my step-

father. I wish you were both here to see this process come to a conclusion, as I know how both of

you were proud and looking forward to my completion of this dissertation and of the doctoral

program.

And finally, to my classmates and UMUC cohort… I‘m going to miss seeing you all

every semester! I couldn‘t have asked for a better cohort, and I thank you all for your support.

Likewise, our faculty advisors Dr. Tom Mierza and Dr. Denise Breckon – thank you for keeping

all of us on track, challenging us to do better, and helping to get us through.

Finally, I want to acknowledge my dog Ruffis. He is so tired of sitting by my feet in my

office while I write. Now, Ruffis, we get to go for some walks. And walk we will.

vi
 Table of Contents
Dedication .......................................................................................................................... iii
Acknowledgements ............................................................................................................ iv
Table of Contents ............................................................................................................... vi
List of Tables ................................................................................................................... viii
List of Figures .................................................................................................................... ix
List of Appendices ...............................................................................................................x
List of Abbreviations ......................................................................................................... xi
Chapter 1: Introduction and Research Problem ...................................................................1
Introduction ..................................................................................................................1
Background ...........................................................................................................2
The Clinical Learning Environment .............................................................................4
ACGME and the Next Accreditation System (NAS) ...........................................7
CLE Measurement Instruments ..........................................................................10
Problem Statement .....................................................................................................12
Research Question ......................................................................................................12
Purpose of the Study...................................................................................................13
Significance of this Study to Management.................................................................13
Scope and Limitations ................................................................................................15
Organization of Dissertation Chapters .......................................................................15
Chapter 2: Literature Review and Interpretive Model .......................................................17
Critical Analysis of Literature Themes ......................................................................19
Influence of Learning Environments ...............................................................19
The Physical Environment in Learning ...........................................................22
The Social Environment in Learning ...............................................................25
Participation in Learning Environments ..........................................................33
Application of Clinical Learning Environment Studies...................................37
Interpretive Model ......................................................................................................39
Thesis Statement.........................................................................................................44
Conclusion ..................................................................................................................45
Chapter 3: Methodology ..................................................................................................47
Systematic Review Process ........................................................................................48
Search Strategy ................................................................................................51
Pilot Search ..................................................................................................52
Inclusion and Exclusion Criteria ......................................................................53
Identification of Additional Articles ................................................................54
Literature Search Results .................................................................................55
Quality Appraisal .............................................................................................56
Considerations from Expert Review Panel ................................................................58
Summary of Results ..................................................................................................59

vii
 Data Coding ..................................................................................................61
Data Triangulation ...........................................................................................62
Data Aggregation .............................................................................................63
Synthesis of Results ..................................................................................................69
Chapter 4: Analysis of Findings ........................................................................................71
Findings of Evidence ..................................................................................................71
The Learning Dimension .................................................................................71
The People Dimension .....................................................................................76
The Change Dimension....................................................................................80
Relationships Between Dimensions .................................................................83
Reinterpreted Model ..................................................................................................87
Summary of Findings .................................................................................................91
Limitations and Interpretations of Findings ...............................................................91
Chapter 5: Conclusions and Implications for Management Practice.................................93
Study Conclusions ..................................................................................................93
Implications for Management Practice.......................................................................97
Implications for Educators .........................................................................................99
Implications for Further Research ............................................................................100
Summary Observations ............................................................................................102
References ........................................................................................................................103
Appendices .......................................................................................................................120
Curriculum vitae ..............................................................................................................144

viii
 List of Tables

Table 1 Moos‘ Framework for Characteristics of Learning Environments ..............................28

Table 2 Scales Developed by Rentoul and Frasier and Classification
According to Moos‘ Domains of Human Environments .............................................31
Table 3 Summary of Propositions Derived from the Literature ...............................................40
Table 4 Databases Utilized for Searching Existing Literature ..................................................53
Table 5 Inclusion and Exclusion Criteria..................................................................................54
Table 6 Learning environment framework descriptions utilized for initial coding and
searching ......................................................................................................................62
Table 7 Frequency Counts by Domain and Data Set ................................................................65
Table 8 Sample Group Data Extractions Coded by Domain ....................................................66
Table 9 CLE Instrument Subgroup Classification Extractions by Domain ..............................68
Table 10 ACGME CLER Pathways to Excellence Group Data Extractions
Coded by Domain ........................................................................................................69
Table 11 Categories, Subcategories, and Frequency Counts of the Learning Dimension ..........73
Table 12 Categories, Subcategories, and Frequency Counts of the People Dimension .............77
Table 13 Categories, Subcategories, and Frequency Counts of the Change Dimension ............81
Table 14 Codes Identified with the Physical Environment.........................................................87

ix
 List of Figures

Figure 1 Components of Classroom Environment Assessments................................................36

Figure 2 Interpretive Model .......................................................................................................41
Figure 3 PRISMA Diagram ........................................................................................................56
Figure 4 The three dimensions of the social context of the clinical learning environment .......84
Figure 5 Conceptual Framework of the Clinical Learning Environment ...................................88

x
 List of Appendices

Appendix A Inventory of Clinical Learning Environment Instruments, Scales

and Subscales ....................................................................................................121
Appendix B Literature Search Keywords and Boolean Operators........................................123
Appendix C MeSH Subject Headings Utilized in PubMed Medline search .........................124
Appendix D Search String Utilized in the Systematic Review .............................................125
Appendix E MERSQI Appraisal Instrument.........................................................................126
Appendix F Subject Matter Expert Feedback Summary ......................................................127
Appendix G Sample Set of Studies Meeting Inclusion and Exclusion Criteria ....................128
Appendix H Clinical Learning Environment Measurement Instruments and Subscales Coded
to Social Dimensions ........................................................................................133
Appendix I ACGME CLER Pathways to Excellence (2014) Coded to Social
Dimensions .......................................................................................................136

xi
 List of Abbreviations
AAMC – Association of American Medical Colleges
ACGME – Accreditation Council for Graduate Medical Education
CLE – Clinical Learning Environment
CLER – Clinical Learning Environment Review
CMS – Centers for Medicare and Medicaid
DIO – Designated Institutional Official
LCME – Liaison Committee for Medical Education
MedPAC – Medicare Payment Advisory Commission
NAS – Next Accreditation System

xii
 Chapter 1: Introduction and Problem Statement

Introduction

The American health care system, and hospitals in particular, are undergoing

unprecedented change due to the rapid innovation in technology and the science of medicine. At

the same time, public scrutiny of health care delivery is at an all-time high. In 1999, the Institute

of Medicine published the landmark report, To Err is Human, exposing and drawing attention to

the projected 98,000 unnecessary deaths occurring each year as a result of errors occurring in the

delivery of care and calling for a national effort to make health care safer (Kohn, Corrigan &

Donaldson, 1999). The report raised public awareness and focused attention on issues such as

patient safety, medical errors and effective care delivery. Subsequently, hospitals have been

pressured to implement publicly reported patient outcome and safety measures. In response to

calls for greater transparency, attention is now focused on high reliability, communication, and

the impact of an inter-professional team of providers in the provision of care.

There are many secondary effects and consequences resulting from this increased public

attention and concomitant accountability of health professionals. One example of secondary

effects is the increased rigor imposed by regulatory groups and accreditation organizations to

hold hospitals accountable for implementing change and demonstrating improved patient

outcomes. The role of medical residents as student physicians, or trainees, learning the practice

of the art and science of medicine is highlighted as an area that raises vulnerability and risk of

error, in the delivery of care, potentially impacting patient safety and clinical outcomes (Rice,

2010; Sifferlin, 2013). To that end, the clinical learning environment (CLE), or the environment

in which medical residents engage in applied learning activities and hone the skills of clinical

medicine, is an area of focus critical to the delivery of safe and effective patient care. Medical

1
school and residency training are separate but related aspects of education, both of which are

fundamental learning experiences in the continuum of medical education. The nuances between

medical students and residents are important to fully understand as their roles within the clinical

learning environment are separate but integrated.

Background

Medical school and residency. During medical school, students learn requisite

knowledge required by the Liaison Committee for Medical Education (LCME) to earn the degree

Doctor of Medicine. The first two years of medical school are comprised of basic science

education, while the final two years are clinically oriented and are comprised of a series of

significant patient encounters under the supervision of medical residents and clinical faculty.

These clinical encounters take place primarily within teaching hospitals where actual patients are

the core teaching material. There are 145 accredited medical schools in the United (LCME,

2015). At the conclusion of medical school, after the degree of medical doctor is conferred,

continued training in the form of residency education is required for eligibility for licensure and

board certification. Residency training, also known as graduate medical education (GME), is

specialty specific and ranges from three to seven years depending on the respective specialty; for

example, internal medicine is three years in duration and neurosurgery is seven years in duration.

The ACGME website contains requirements for each medical specialty, including the required

duration of training (www.acgme.org). GME programs, also referred to as residency or

fellowship programs, are formal clinical education and training programs that follow medical

school and lead to one‘s ability to develop skills in a specific domain or specialty of medicine,

leading to board certification. GME training involves the application of clinical knowledge

through the applied practice of medicine in specific specialties. Residency and fellowship

2
education takes place in various clinical environments, generally including a mix of teaching

hospitals, ambulatory outpatient settings, and private doctors‘ offices. Sponsors of GME

programs, typically hospitals, are accountable for the oversight, outcomes, and administration of

the programs.

Upon successful completion of an accredited residency program, the resident earns the

privilege to practice medicine independently and test for board certification in the respective

clinical specialty. Following residency, some physicians enter practice, while others continue

with further subspecialty training known as fellowship.

In 2014, there were 120,108 medical residents enrolled in 9,527 Accreditation Council

for Graduate Medical Education (ACGME) accredited training programs in the U.S. (ACGME,

2014). This training occurs in 4,687 hospitals. Over 37,000 medical residents and fellows

complete their training programs each year and enter independent medical practice (ACGME,

2014). Medical residents and fellows, by nature of their training programs, are the front-line

providers of patient care reporting to a supervising physician or faculty member. In teaching

hospitals, medical residents and fellows have primary direct patient care responsibility for the

vast majority of patients. Given that more than 120,000 residents and fellows are currently

enrolled in hospital GME programs across the country, and each resident typically carries a

patient care load of eight to ten patients, their impact on the actual delivery of safe and effective

medical care rendered to patients is on the forefront of regulatory and public accountability

group agendas (Eden, Berwick, & Wilensky, 2014; Kohn, Corrigan & Donaldson, 2000;

Rieselbach, Sundwall, Shine, Epperly & Crouse, 2015).

Accreditation. The Liaison Committee for Medical Education is responsible for the

oversight and accreditation of medical schools. LCME‘s scope is limited to medical school

3
education, or the curriculum leading up to the degree of doctor of medicine. The ACGME is the

regulatory organization responsible for the oversight and accreditation of GME programs.

The ACGME is responsible for setting the standards and requirements for accredited

training in all residency and fellowship programs, and also serves as the agency that reviews and

accredits compliance with such standards. In 1998, in an effort to heighten the visibility of GME

programs in hospitals, the ACGME required all sponsors of graduate medical education

programs to appoint a senior-level individual as the designated institutional official (DIO) to

have authority and responsibility for oversight of the graduate medical education programs

(Bellini, Hartman, & Opas, 2010; Riesenberg, Rosenbaum, & Stick, 2006). This requirement

was initially created to increase institutional accountability for medical education, and for the

ACGME to have a primary individual identified for administrative purposes, but this position has

since evolved into a primary institutional leadership role for GME programs and hospitals

(Riesenberg et al., 2006).

The Clinical Learning Environment

The clinical learning environment is a term commonly used by medical educators when

referencing the location or setting in which medical students and medical residents engage in

clinical education and patient care, and is a term widely used in the medical education

scholarship and peer-reviewed literature. The term is also referred to by the ACGME in

published accreditation standards and is now the actual term used in the new assessment process

of the Next Accreditation System (NAS) – the Clinical Learning Environment Review (CLER).

However, in none of these circumstances does the term clinical learning environment have a

universal definition or shared understanding of formal components or constructs.

4
 While the CLE has been more rigorously studied in related fields such as nursing, the

term has been used indiscriminately in the profession of medicine to refer to aspects of the

culture, environment, or the learning context (Malling, Mortenson, Scherpbier, & Ringsted,

2010). Shochet, Colbert-Getz, and Wright (2015) define the CLE as ―the physical, social, and

psychological contexts in which medical students learn and grow professionally, and it

influences how students develop behaviors and form identities as future physicians‖ (p. 810).

During residency and fellowship training, the CLE is attributed to the environment which clinical

knowledge learned during medical school is subsequently applied to practice by delivering

medical care to actual patients in a supervised teaching environment. Generally used as a term to

encompass the space in which clinical learning takes place, typically a hospital or clinic setting,

CLE is used liberally by medical educators and policy makers in the context of referencing an

important aspect of medical education, training, and the safe delivery of direct patient care,

which is largely provided by residents.

The term learning environment has many other connotations ranging from physical space

to personal feelings, but the all-embracing nature of the term provides ambiguity and leads to

disparate perspectives and a lack of ability to measure or weigh the importance of various

aspects (Isba & Boor, 2011, p. 101). This inability to describe and quantify learning

environments in general is noted as far back as Bloom (1964) as one of the significant failures of

educational research (in Traub, Weiss, Fisher & Musella, 1972, p. 69), and also by Schulman

(1970) noting, ―social scientists are dramatically impotent in their ability to characterize

environments‖ (p. 374). Learning environments have been noted as early as primary school as

influencing learning (Isba & Boor, 2011, p. 100). The environment of learning is an important

factor throughout adult development, particularly in new settings outside of classrooms in which

5
adult learners transition from focusing on knowledge acquisition to knowledge application, and

where learning is applied in practical settings, such as a hospital.

This applied learning in the context of patient care in a hospital setting comes at a

substantial cost. It is estimated that teaching hospitals in the United States spend over $30 billion

annually on mission-based costs related to the training and education of medical residents and

fellows (Dunn & Becker, 2013). Considered a societal good to invest in a robust healthcare

system, Medicare reimburses teaching hospitals for a portion of these costs each year, accounting

for about $9.8 billion of the overall Medicare budget in 2014 (Association of American Medical

Colleges [AAMC], 2015). Given the private and public investment in the training and education

of our future physicians, the critical study of the CLE is paramount in understanding how

learning and patient care can be optimized.

Many formal efforts and initiatives are underway to enhance the clinical learning

environment, although little research and few publications have explored the actual components

and none have approached the understanding of the CLE with a basis or rationale rooted in

theory (Isba & Boor, 2011, p. 101; Schonrock-Adema, Bouwkamp-Timmer, van Hell, & Cohen-

Schotanus, 2012). For example, the AAMC launched a formal and multi-faceted initiative on the

learning environment in 2014, noting its importance in ―developing and sustaining a

professional, capable, compassionate and diverse healthcare workforce that will meet the needs

of all‖ (AAMC, 2015). Despite this major initiative and subsequent impact on the organization‘s

resources, no acknowledgement of defining the CLE or creating working groups to study and

research the CLE exist. The LCME requires schools to regularly evaluate the learning

environment, but without a framework or valid assessment tool, many schools find this

compliance requirement difficult (LCME, 2015). Leaders of the ACGME have published studies

6
in the peer-review literature regarding the learning environment without providing definition or

guidance regarding the components or aspects of the CLE (Philibert, 2012; Holt, Miller,

Philibert, Heard & Nasca, 2010).

Asch, Nicholson, Srinivas, Herrin, and Epstein (2009) provide one of the few studies

linking GME training to longitudinal patient outcomes. This retrospective study of 4,124

obstetrician/gynecologists who trained in 107 different residency programs, and now practice

medicine in two states (Florida and New York) analyzed the hospital clinical outcomes and

residency programs from which these physicians trained. Even after adjusting for differences in

individual licensing exam scores, the results provided empirical evidence that the hospital and

training program of the physician has a direct correlation on that physician‘s future practice as

measured by patient care outcomes over time. Organizations such as the AAMC and ACGME

are placing a focused emphasis on the learning environment in an attempt to improve learning

experiences by measuring and improving certain components, presuming an enhanced or

improved learning environment will lead to enhanced or improved learning outcomes, and

subsequently improved patient care (Nasca, Philibert, Brigham, & Flynn, 2012; Weiss, Bajan, &

Nasca, 2013). If the factors of the CLE are defined, measured, and better understood, then

hospitals can make improvements that will ultimately enhance patient care.

ACGME and the Next Accreditation System (NAS). As the organization responsible

for the approval and accreditation of GME programs, the ACGME accredits both individual

medical specialty residency and fellowship programs (e.g., internal medicine, general surgery,

obstetrics, etc…) and also accredits the actual institution or hospital as an approved sponsor of

GME programs. Historically, accreditation of both the individual programs and the sponsoring

7
institutions, predominantly teaching hospitals, has been very process-oriented (Nasca, Philibert,

Brigham & Flynn, 2012).

Recent heightened public attention to clinical quality, patient safety and the perceived

risk to patients associated with medical residency education has raised increasing attention on the

GME community and prompted ACGME officials to approach accreditation standards and

review processes more rigorously. As a result of the changing environment and increased

pressures, the ACGME responded with an overhaul of its approach to accreditation and

introduced a more rigorous accreditation process, referred to as the Next Accreditation System

(NAS). There are three aims of NAS: (a) to enhance the peer-review system of physician

assessment and preparedness for the future practice of medicine, (b) to accelerate outcomes-

based accreditation, and (c) to reduce the administrative burden currently associated with the

process-based system (Nasca et al., 2012).

One component of this new approach is an assessment of sponsoring institutions referred

to as the Clinical Learning Environment Review, or CLER. CLER visits were initiated and beta-

tested in 2012, and are now in the second phase of reviews. CLER visits are conducted at each

sponsoring institution approximately every 18 months. The purpose of the CLER visit is to

assess the hospital‘s compliance in six core focus areas: Patient safety, health care quality

(including reduction of health disparities), transitions of care, supervision, duty hours and fatigue

mitigation, and professionalism (Weiss, Baigan & Nasca, 2013; www.acgme.org). CLER visits

are focused on both observational assessments and data collected from a hospital‘s many wards,

settings and specialties. Visits are conducted by trained site visitors who spend time in these

various clinical settings interacting with medical residents, faculty, nurses, and other care

providers.

8
 Some of the most important objectives of graduate medical education programs are to

foster improvements in patient safety, clinical quality of care, and reduction of unnecessary harm

to patients (Jenson, et al., 2009). It is speculated that the CLE has a direct impact on safe and

reliable patient care in teaching hospitals, and despite overwhelming evidence, studies like Asch

et al. (2009) provide rigorous support of speculation. In 2009, the Medicare Payment Advisory

Commission (MedPAC) issued a report to Congress suggesting that graduate medical education

reform is a key component in the transformation of the U.S. health care delivery system to one

that is patient-centered, conserves resources, and demonstrates clinical quality and patient safety

(MedPAC, June 2009). A recent survey of hospital leaders highlighted the critical need to

educate U.S. physicians and medical residents in quality improvement, specifically citing their

inability to communicate effectively and work in teams, leading to adverse patient outcomes

(Combes & Arespacochaga, 2012). Thus, the ACGME developed the CLER process to align

with factors related to quality and safety in medical education. Despite all of the heightened

attention to the CLE, investment in more robust accreditation requirements and inspections, and

the subsequent resources required by hospitals and GME programs to meet and measure these

requirements, a framework for the CLE continues to be absent.

Two years into the CLER program, ACGME reports the initial data from site visits

demonstrate residency training in teaching hospitals is fragmented, specialty-specific, and limits

exposure of the learner to safe and effective care delivery due to separation between individual

programs and the leadership of the hospital needed for integrative care and enhanced learning

environments (Nasca, Weiss, & Bagian, 2014). While these conclusions were based on

thousands of observed encounters, a measurement instrument or objective methodology remains

elusive and limits the ability of ACGME or researchers to fully understand where deficits are

9
most critical, and how to effectively compare clinical learning environments between hospital

settings.

Fraser (2001) estimates students spend up to 20,000 hours in classroom environments by

the time they graduate from college (Fraser, 2012, p. 1191). This estimate does not include

graduate school or medical school coursework, or the education and training time following

when physicians engage in residency training. General internal medicine training lasts three

years, with the hospital unit or ward being the classroom environment for physicians engaging in

programs that demand 80+ hours a week of training time. Thus, for a three-year residency

education program alone, the hospital clinical learning environment is the classroom for the

resident physician for 12,480 hours during this time. Thus, the study of the environment

affecting medical residents‘ applied learning experiences and reactions are significant.

CLE measurement instruments. European medical educators have developed

instruments to assess the CLE including the well-known Postgraduate Hospital Educational

Environment Measure (PHEEM), the Dutch Residency Educational Climate Test (D-RECT), the

Medical School Learning Environment Survey (MSLES), and the Dundee Ready Education

Environment Measure (DREEM) (Boor, van der Vleuten, Teunissen, Scherpbier & Scheele,

2011; Roff, McAleer & Skinner, 2005; Marshall, 1978; Roff et al., 1997). Despite the creation of

these instruments to assess and measure aspects of the clinical learning environments, each

instrument approaches the domains and constructs of the environment from different

perspectives and there is no uniformity or continuity of the components or the elements being

assessed. Furthermore, the instrument structures and scales are also varied, underscoring the lack

of consensus regarding what to measure and how to measure it (Colbert-Getz, Kim, Goode,

10
Shochet, & Wright, 2014; Schonrock-Adema, Boukamp-Timmer, van Hell & Cohen-Schotanus,

2012).

The nursing literature is more robust and provides empirical research informing

constructs and definition to the elusive CLE in the context of nursing education. Saarikoski,

Isoaho, Leino-Kilpi and Warne (2005) published a validated scale for measuring the clinical

learning environment which includes five domains: patient relationships, staff-student

relationships, leadership style, hierarchy and ritual, and supervisory relationships. It is important

to establish that there are key distinctions between nursing education and physician education,

including differing roles, educational background, supervisory requirements, and expectations

for applied learning; these distinctions must be considered and analyzed before applying the

nursing constructs and scales in the context of physician education.

The nursing literature provides important constructs in understanding and defining the

components of the CLE. However, even with this knowledge and growing body of literature,

physicians and medical educators have failed to adopt known information to apply constructs to

the CLE in medical education, or to determine how organizational factors may affect the CLE.

And while nurses serve in a critical role in the care of hospitalized patients, it is the physician

who is the ultimate decision-maker and individual responsibility for the diagnosis, treatment

plan, and management of the patient. This lack of consensus regarding the domains and

constructs of the CLE in medicine has been examined in one study (Schonrock-Adema et al.,

2012) which determined the absence of theoretical frameworks in the development of these

instruments may be the cause of the ambiguity.

11
 Problem Statement

Given the many formal efforts and initiatives that are underway to enhance the learning

environment, little research and few publications have explored identifying the actual

components and none have approached the understanding of the learning environment with a

basis or rationale rooted in theory (Isba & Boor, 2011, p. 101; Schonrock-Adema et al., 2012).

Hospital leaders are being increasingly pressured to demonstrate improved patient care outcomes

through decreased errors and improved quality of care, and the emphasis on the integration

between medical education programs and patient care outcomes is the subject of many studies. A

conceptual framework based on theory and developed through empirical evidence could inform

both researchers and practitioners. Researchers could utilize a conceptual framework to develop

measurement tools and instruments that would more accurately describe, measure, and weight

CLE elements. Educators could utilize a conceptual framework to enhance the approach to

learning and assessment in clinical settings. Finally, administrators could utilize a CLE

framework and subsequent measurement tools to better understand the hospital teaching

environment, and perhaps improve those facts linking GME and patient care outcomes.

Research Question

One of the most critical components to a research study is the development of a research

question which is clear and applicable (Denyer, Tranfield & van Aken, 2008). While the clinical

learning environment seems at face value like a term that is easily understandable, commonly

referenced, and frequently used in both conversation and scholarly literature, a clear

understanding of CLE components remain evasive. This study seeks to inform the development

of a conceptual framework of the CLE, which includes the identification of defined components,

and an analysis to determine which components may be most important to clinical learning.

12
Therefore, the research question for this study is: How does the clinical learning environment

influence clinical learning?

Purpose of the Study

The purpose of this study is to utilize existing theory and research to deconstruct the CLE

and create a conceptual framework of the clinical learning environment in medical education to

be used for medical education delivery, practice, and research. First, in Chapter 2, existing theory

and theoretical frameworks utilized in other settings will be identified and examined, such as

open classroom environments in higher education. These frameworks will be used to develop an

interpretive model that can be tested for applicability in a new setting--the teaching hospital.

Second, in Chapter 3, existing primary research in medical education, in addition to other

scholarly sources, will be identified and synthesized to extract components of the CLE and test

the interpretive model. Finally, Chapter 4 will present an analysis of existing research, which

will be utilized and analyzed to determine which components of the CLE may have more or less

importance in the clinical learning process. This study will be completed by a systematic review

of the literature and narrative/thematic synthesis.

Significance of the Study for Management

Hospitals face increasing financial pressures from reduced payments from insurers, pay

for performance initiatives, increased regulatory requirements, and requisite risk associated with

poor patient outcomes. Additionally, the public attention that has been focused on patient safety

and quality provisions of care are pressuring hospitals to be transparent about clinical outcome

measures and report safety issues to regulators, accrediting agencies, and the public. These

challenges require hospitals to improve efficiency, patient care outcomes, and quality

improvement efforts while facing decreased funding and payment for care services. These

13
financial pressures are significant to the average teaching hospital that has an overall profit

margin in the 1-4% range (Truven Health Analytics, 2015), and negative Medicare margins

(MedPAC, 2015). Hospital leaders are questioning whether or not the expense of GME programs

truly add value to the provision of clinical care, or instead just add risk from trainee trial-and-

error method of learning application on patients. This study will be important to managers and

leaders of teaching hospitals who are responsible for the delicate balance of patient care, reduced

errors, improved quality, and financial stability in an environment where clinical learning is also

a priority.

Accrediting agencies such as the ACGME are responding to these pressures in an attempt

to assure the public that the matriculating physician workforce is prepared to care for future

populations, and to integrate medical education programs and medical residents more into the

front-end clinical operations and strategic development of hospitals (Weiss, Bagian & Nasca,

2013). A theory-based and evidence-driven conceptual framework of the CLE will allow

researchers to create measurement instruments that are more valid and reliable. The

understanding of the elements of the CLE will inform policy makers, accreditation agencies such

as the ACGME, and advocacy organizations such as the AAMC, as they embark on their journey

to assess and measure this environment in clinical settings and the relationship to clinical quality

and patient safety efforts. Practitioners of medical education and hospital leaders will be able to

approach measurement and understanding of institutional effectiveness in education, pedagogical

approaches, curricular delivery methods. Likewise, consensus around the dimensions and

constructs of the CLE will provide a framework for medical education researchers to have a

common language for continued scholarship and investigation.

14
 Scope and Limitations

The scope of this study is limited to the clinical learning environment in teaching

hospitals. Other clinical settings such as clinics, physicians‘ offices, ambulatory settings, or

outpatient centers are not included. The unit of analysis is perceptions by medical students (third

and fourth year students in clinical rotations), residents, and fellows. Perspectives of first and

second year medical students, faculty, and other members of the care team are not included. This

is a secondary research study, and data will be collected utilizing existing published research of

subjects‘ impressions and characterizations of clinical learning environments in teaching

hospitals.

The conclusions are limited to existing data sources and published evidence. The

conclusions may not generalize to other clinical learning environments such as outpatient

settings or community hospitals. Likewise, the conclusions may not apply to other individuals in

the CLE such as the clinical teaching faculty.

Organization of Dissertation Chapters

This study examines and identifies the constructs and components of the clinical learning

environment in the hospital setting. In Chapter 2, the researcher will review the relevant

literature and identify foundational theories of learning environments, open classroom settings,

social exchange theories, and current research on the CLE. The literature review will yield a set

of propositions. The functional relationships will be described in an interpretive model. The

interpretive model drives the research questions and frames the methodology for the study.

The research methodology is presented in Chapter 3. The methodology review allows for

the research to be reproducible and demonstrates consistency with the scientific method. Results

of the data sets and narrative/thematic synthesis are presented at the end of this chapter. In

15
Chapter 4, the study findings and evidence are interpreted and presented in the form of a

conceptual framework. Finally, Chapter 5 provides a discussion the study findings in the context

of management implications for practice, educational implications for pedagogy, and the

implications for further research and scholarship.

In the next chapter, existing theory and literature is examined in the context of learning,

organizational, and learning environments. This theory and literature will inform the

development of an interpretive model of the learning environment, which will serve as a basis for

the research methodology.

16
 Chapter 2: Literature Review & Interpretive Framework

Introduction

“There is nothing as practical as a good theory”. – Kurt Lewin

Chapter 1 provided an overview of the topic and highlights the importance studying the

clinical learning environment to inform scholars, educators, and practitioners on a topic that is of

critical importance in today‘s medical field. Now, in Chapter 2, an extensive review of the

literature informs the reader of supporting theory, research resulting from that theory, drivers of

the learning environment, social-cultural aspects of learning in situ, and an explanation of the

phenomenon being researched. The literature presented in Chapter 2 provides the reader with the

basis for which existing knowledge is integrated, and identifies gaps in the literature that will be

addressed by this study. The literature review and knowledge integration provides the basis for

the development and presentation of an interpretive model explaining how components and

drivers of general learning environment work together, forming the basis for the research

methodology presented in Chapter 3.

The purpose of this study is to develop a conceptual framework of the clinical learning

environment in medical education. In this chapter, literature from the fields of secondary and

higher education, organizational management, and medical education provide a basis for

research propositions informing this study. First, literature on social environments and

organizational climates are presented in a broad context, then narrowed to focus on the context of

higher education, with a particular focus on the role of the environment in learning. Second, the

literature on the social context of learning including the impact of the environment on behavior,

communities of practice, and social exchange aspects are presented. Finally, an exploration of

clinical learning environment literature from related fields such as nursing is presented, along

17
with current clinical learning environment (CLE) literature in medical education including

published measurement instruments. Collectively, this literature review provides a context for

the learning environment in the clinical (hospital) setting and leads to the development of a

interpretive framework that guides the research methodology in Chapter 3.

In the context of medical education, the current body of literature is fragmented and

provides little insight into the clinical learning environment. Traditionally, medical education

research is remiss of supportive literature from the social sciences, which leaves a gap in the

understanding of human learning mechanisms. Sociology literature provides context of how an

organizational environment affects culture and climate, as well as underlying theories in which

research on organizational environments is based. Social ecology literature integrates the

organizational environment and human behavior. The education literature contributes robust

theory and subsequent research of learning environments in settings such as primary and higher

education, and provides a foundation for the context of studies in the clinical setting. Educational

literature also addresses the role of environments in traditional classroom settings, but falls short

of addressing settings such as a teaching hospital, where traditional classrooms and fixed cohorts

of students are replaced by medical units and transient learning cohorts of people in varied roles

and of differing levels of competence. In the medical field, research of the CLE focuses

primarily on the development of instruments to measure the CLE, with particular emphasis in the

field of nursing. Medical education literature is the least developed, and also focuses on the

development and validation of CLE measurement, but fails to integrate educational theories or

the context of the organizational environment. Qualitative studies of stakeholder perceptions of

the clinical learning environment in medicine also inform the field, but are limited in scope, and

the results have not been integrated with existing knowledge or with each other.

18
 Gaps in the literature include: (a) the absence of social science knowledge integrated

with medical education and medical pedagogy; (b) the absence of theory-based literature

informing the development of CLE components, constructs, and measurement instruments in

medical education; (c) the absence of literature synthesizing existing qualitative studies of the

CLE in medical education, and; (d) a lack of integration of existing qualitative studies of the

CLE into the development of conceptual frameworks. This study addresses these gaps.

Critical Analysis of Literature Themes

Influence of Learning Environments

The environment in which one participates in learning has been identified as an

influential factor of the learning process as early as primary school (Isba & Boor, 2011, p. 100).

The conceptualization and assessment of human environments and the subsequent effect on

human behavior, referred to as social ecology, is rooted in social theories of Lewin (1936) and

Murray (1938), who emphasize the importance of environmental and social determinants of

behavior (Fraser, 2012, p 1192; Rentoul & Fraser, 1979, p. 233-234). Lewin‘s early work on

field theory (1936) laid the groundwork for subsequent theory positing human behavior as a

product of the physical environment (or space in which people exist), together with the personal

characteristics of the individual; thus, human behavior is a product of both the person and their

environment. Based on Lewin‘s work (1936) and the subsequent development of the Lewinian

(heuristic) formula, research strategies and studies addressed behavior as an outcome of both the

person and the environment, leading to the current body of knowledge of learning environments

(Fraser, 2012, p. 1192-1193). In the hospital or clinical setting, the role of the person in the

context of an environment which combines both work and learning, is important to

19
understanding learning and behavior. Lewin‘s theories provide a relevant context and basis to the

development of learning environment studies in a variety of settings.

Rudolph Moos, a psychiatrist at Stanford, initiated the study of human environments in a

variety of settings. Moos (1973) distinguishes the previous work of personality theorists such as

Lewin and Murray, noting their emphasis on the person variables in behavior, more so than the

environmental variables. This lack of emphasis on the environmental variables affecting

behavior, as well as the lack of approaches in determining the environmental variables relating to

human behavior, prompted Moos to extend and develop theories and descriptors of the learning

environment. Moos and colleagues studied at least nine different types of social environments

including psychiatric wards (Moos, 1973), psychiatric treatment programs in the community

(Moos, 1972), correctional institutions (Moos, 1968), military training companies (Gerst &

Moos, 1972), university residences (Gerst & Moos, 1972), junior and high school classrooms

(Trickett & Moos, 1973), social and therapeutic groups (Moos & Humphrey, 1973), work

environments (Insel & Moos, 1972), and family environments (Moos, 1973). The application of

learning in the context of the variety of environments ranging from psychiatric wards to

classrooms provide a strong basis for extending this research to a learning setting such as a

hospital. In particular, settings that are both work-based (correctional institutions, military

training) and educational (classrooms, university residences) provide a strong rationale for

examination of the hospital setting, which combines work and education in a single environment.

Anderson and Walberg (1974), Walberg (1978), and Rentoul and Fraser (1979) expanded

on Moos‘ scholarship and extended the theories and frameworks presented on social

environments to the learning setting. Measurement tools such as the Classroom Environment

Scale (CES) (Moos & Tricket, 1974) and the Learning Environment Inventory (LEI) (Anderson

20
& Walberg, 1974) were soon developed in an attempt to identify, define and measure a learning

environment and the subsequent impact on student behavior and outcomes. Marshall (1978) was

the first to translate learning environment scales to the medical school setting, creating the

Medical School Learning Environment Survey (MSLES). It was not until the late 1990‘s that

scales were developed to measure the learning environment in a clinical context, such as a

hospital. Despite the development of environmental measurements in the clinical setting, the

theoretical basis provided by Moos (1968, 1973, 1974), Rentoul and Fraser (1974), and

Anderson and Walberg (1974) were not used to inform the creation of these instruments, such as

the MSLES, or any of the subsequent instruments (Schonrock-Adema et al., 2012). Instead,

measurement instrument such as the MSLES were developed based on perceptions, observations

and assumptions of learning, but not on theoretical underpinnings. Bloom (1964) made a strong

case for educational research of the learning environment, arguing it critical for effective

teaching and learning approaches and modifications, as well as learning outcomes.

Two types of learning environments exist within medical education – the traditional

classroom and the hospital or clinical setting which constitutes the clinical learning environment.

The traditional classroom is the primary setting during the first two years of medical school,

where the focus is on basic science and the aim is learning or knowledge acquisition. Beginning

in the third year of medical school, the primary learning setting shifts from the classroom to the

hospital or clinical facility, where the focus is shifted to application of knowledge and the aim is

learning to work as a practitioner (Isba & Boor, 2011, p. 101). A final shift occurs during

residency training, where the hospital or clinical setting is almost the exclusive location for

learning, and the physician serves in a dual role of student and employee/practitioner, thus the

21
physical and social aspects of the environment are key factors in learning and in developing

identity.

The Physical Environment in Learning

The physical environment or material aspects of the learning setting are most commonly

presumed when one speaks of a learning environment. Material aspects such as bricks and

mortar, technology, seating arrangements, and even availability of natural light are all

components of the physical setting that are important to learning (Fraser, 2012). Investment in

learning environments are often initially focused on physical space and material objects as these

are easy to identify and fix, but do not always lead to an improved learning outcomes (Isba &

Boor, 2011, p. 103). In the clinical setting, the physical environment serves as both space to

deliver patient care and for learning to take place. Efforts to make space more appealing to

patients and more functional for care delivery are often prioritized over learning requirements. In

fact, many times learning aspects are not even considered when hospital leaders design, renovate,

or update physical space.

The physical environment and behavior. Understanding and defining the context and

components of learning environments emerged through research in the social sciences in the late

1960s. Moos‘ initial research (1968) focused on the social climates and atmospheres of

psychiatric wards and correctional institutions (Moos & Houts, 1968). Building on prior research

by Endler, Hunt and Rosenstein (1962), who concluded both individuals and settings contribute

to behavioral variance in anxiety, Moos furthered this work by developing research on the role of

the environment in various other settings. Moos and Houts (1968) posited that social

atmospheres can be separately identified and measured as a contributor of behavioral effects of

individuals, and those effects are differential. Determining that the environment was a significant

22
contributor to the behavior of psychiatric patients, Moos and Houts (1968) developed, tested, and

validated the Ward Activities Scale (WAS), an instrument to differentiate between social

atmospheres and the physical environment in psychiatric wards. In the context of education, the

physical space of the classroom is associated with the satisfaction and outcomes of the learning

process (Anderson & Walberg, 1974; Wang, Haertel, & Walberg, 1990). Fraser, Treagust &

Dennis (1986) validated quantitative research methods were effective in measuring the physical

environment of a classroom separately from the social dimensions. This is an important

differentiation in the clinical setting because the ability to measure the dimensions separately can

be critical in understanding the separate but related impacts of physical space and social contexts

on in the clinical learning setting.

Physical and material aspects of the clinical environment are often overlooked by

researchers as being less important to study, assuming identification of need and resources or

cash investments to address improvements such as space renovations or new

equipment/technology can be done (Isba & Boor, 2012). Residency training, however provides a

unique context to learning, as a resident physician is both a student and an employee of the

hospital. In higher education, students are clearly students, demonstrated by paying tuition and

submitting to academic requirements leading to a degree. However, in medicine, residents are

both seeking an academic credential and submitting to employment by the hospital in exchange

for the clinical services they provide during the course of applied learning. Despite this dual

identity, the resident‘s primary role is that of learner, and employment functions are secondary to

the goal of knowledge acquisition, knowledge application, and the achievement of clinical

competence (Padmore, Karpovich, Shaver & Richard, 2010; Padmore et al., 2009). However, it

is difficult, if not impossible, to separate learning from work in contexts such as medicine, where

23
the provision of service (patient care) is the basis of applied learning and demonstration of

clinical competence (Padmore et al., 2009).

Herzberg‘s (1976) motivation-hygiene theory addresses job factors influential to the

motivation of employees. Herzberg concluded job satisfaction is the consequence of motivation

factors, and dissatisfaction is the consequence of hygiene factors. Hygiene factors are principally

those inherent to the physical environment and material resources of the work environment, such

as compensation, benefits, and working conditions. Herzberg‘s theory approaches satisfaction

and dissatisfaction as two separate functions that are not on a continuum. For example, hygiene

factors do not necessarily improve job satisfaction, but they prevent dissatisfaction. Dissatisfiers

are associated with one‘s relationship to the working environment, and relate to the work

situation or work role.

Herzberg‘s theory has been applied in the medical education setting in the context of the

CLE. Syptak, Marsland and Ulmer (1999) and Seabrook (2003) found hygiene factors such as

physical space within the hospital for teaching, educational resources and technology needed to

support learning, staffing, and compensation were identified areas of dissatisfaction in a hospital

setting, and overshadow social structures, investment and support for learning. Philibert (2012)

conducted a retrospective review of data acquired from the ACGME‘s annual resident survey to

determine if the survey itself captures aspects identified by residents as being important to their

learning environments. This study concluded certain aspects of the physical environment receive

consistently negative comments, such as parking, food, and call rooms; however, despite

dissatisfaction of these factors, residents placed significantly more weight on social determinants

such as interactions with faculty and availability of learning opportunities. Likewise,

Yudkowsky, Elliott and Schwartz (2002) concluded the interpersonal culture of a residency

24
program may be the most important component for learning satisfaction. Benbassat (2013)

conducted a narrative review of clinical learning environments to identify undesirable features,

but limited the parameters of the study to social factors, and did not address hygiene factors

associated with dissatisfaction. Despite the limitations of the study design, Benbassat (2013)

concluded that social factors such as egalitarian attitudes, trust, and social support are factors

important to a positively perceived learning environment in medicine. While the physical

environment can be isolated and measured separately from the social context, both factors are

interrelated and together comprise the learning environment. The physical space, resources and

related elements impact the learner‘s perceptions of the overall environment, especially in

settings that combine work and learning.

Proposition 1: Positive physical settings in a learning environment

are associated with positive learning experiences.
Proposition 2: The physical context and the social context are
separate but integrated components of a learning
environment, the influence of each affecting the
other.
Proposition 3: Perceptions of the physical environment impact
perceptions of the social environment in learning.
The Social Environment in Learning

Moos (1968) further posited that in addition to physical settings, individuals, and their

social interactions with others, also contributed to behaviors. The psychosocial environment in

learning includes psychological and social factors including one‘s satisfaction, relationships

between students and faculty, and the ability to perform to one‘s optimal ability (Moos, 1973,

1979). Moos (1968) hypothesized people involved in events may interpret those events

differently from others who are observing from an external environment, referred to as detached

25
observers. Moos developed and tested his Social Climate Scale (SCS) to measure twelve

dimensions of social climates and demonstrated that perceptions of individuals participating in

those environments were in fact different from those of the detached observer. Moos‘

conclusions provided a basis for research of learning environment as assessed from the vantage

point of the students. Thus, engagement of learners to elicit feedback, perceptions, and attitudes

of the learning environment are important to informing researchers regarding learning factors. In

clinical education programs, engaging medical students and residents regarding the learning

environment is an important aspect to understanding perceptions of this setting.

The organizational climate in the hospital setting is the basis for this study of the clinical

learning environment. Katz and Kahn (1966) defined five characteristics of social organizations:

(a) maintenance, production, and production-supportive structures; (b) elaborate formal role

patterns; (c) authority structure; (d) regulatory mechanisms and adaptive structures which

include feedback to the institution concerning its own operation and the changing character of its

environment; and (e) explicit formulation of an ideology that provides system norms and

supports the authority structure. These characteristics are utilized as the basis of environmental

scales created by Moos to characterize and discriminate among the nine environments he studied

(Moos, 1973). The hospital setting meets these five characteristics, underscoring the validity of

this setting as a social organization.

Learning environment frameworks. Moos (1973, 1974) provides a theoretical

framework for social environments, describing dimensions of the human environment in three

domains: Personal development or goal orientation, relationships, and system maintenance and

system change. Moos isolated these dimensions based on comparative studies of nine different

environmental contexts. Subsequent research by Insel and Moos (1974) validated that these three

26
descriptive dimensions can be used as a framework for characterizing a broad range of

environments, as noted by the applicability of the contexts in which they have been applied

(Rentoul & Fraser, 1979, p. 238). Although specific research has not been conducted in the

context of the hospital setting, research has been done in other clinical contexts and in learning

contexts such as classrooms. Therefore, application in the clinical learning environment is a

logical progression. The setting is important to consider with regard to nuance in components

and outcomes. While the dimensions are similar across disparate environments, different settings

impose unique variations within each category (Insel & Moos, 1974).

Education and work environments are two such categories where variations occur and

certain elements may be more or less dominant than in other environments. Since residents are

both students and employees, the dual contexts of work and education environments are

important to distinguish from variations in other settings. Insel and Moos (1974) compared

organizational climate scales with Moos‘ dimensions to examine patterns within differing

environments, and to determine which factors are most related to contexts such as education and

work (Table 1).

27
Table 1.

Moos‟ Framework for Characterizing Organizational Environments and Characteristics in the

Settings of Education and the Workplace

Domain Description Characteristics (*Education, +Work)

Personal development or Variables relating to specific functions Autonomy (self sufficient and independence)*+
Goal orientation dimensions of environments and to the potential Practical orientation (working toward concrete goals)
for personal growth and development Personal problem orientation (one seeks to understand
feelings and problems)
Academic achievement*
Intellectuality*
Task orientation (ability to prepare for next job or
role)+
Concern for improvement, learning or advancing
knowledge*
Scholarship, awareness
Relationship Dimensions Nature and intensity of personal Involvement-affiliation*+
relationships and the extent to which Support*+
individuals are involved and support Expressiveness
each other Peer cohesion (social and interpersonal growth and
relationships)*+
Group cohesion
Community
System maintenance and Extent of order and control in the Order and organization*+
change dimension environment and its responsiveness to Clarity and control*+
change Innovation*+
Work pressure+
Leadership and role delineation+
Hierarchy*+
Note: Adapted from ―Conceptualization of enquiry-based or open classroom learning environments‖ by A. J.
Rentoul and B. J. Fraser, 1979, Journal of Curriculum Studies, 11(3), p. 240. Copyright 1979 by Taylor & Francis.
Also adapted from ―Conceptualization of human environments‖ by R. H. Moos, 1973, American Psychologist,
28(8), pp. 657-658. Copyright 1973 by the American Psychological Association. Also adapted from ―Psychological
environments: Expanding the scope of human ecology‖ by P. M. Insel and R. F. Moos, 1974, American
Psychologist, 29(3), p. 184. Copyright 1974 by the American Psychological Association.

Personal development and goal direction. Moos‘ first domain, personal development or

goal direction, relates to the personal growth of the individual, and the related aims, or goals and

objectives, of the environment (1973, 1974). Characteristics of this dimension include autonomy,

achievement and personal orientation. This domain considers potential or opportunity for

personal growth and development of the individual, including confidence (Insel & Moos, 1974,

p. 181). Unlike the other two domains, personal development and goal direction have dimensions

28
that vary the most among different environment contexts, and are associated with the goals of

each environment. Outcomes associated with personal development and goal direction are

typically aligned with individual learning outcomes, such as achievement and aptitude (Walberg

& Anderson, 1968).

Studies assessing learning environments, such as the Learning Environment Inventory

(Walberg, 1969) and the College and University Environment Scale (Pace, 1969) associated the

factors of concern for learning and advancing knowledge, intellectual aesthetic, intellectual

climate and achievement standards to be more strongly aligned with this dimension. Likewise,

work environment studies including the Work Environment Inventory, associated task

orientation and autonomy to be more aligned in the workplace environment. The personal

development and goal direction domain focuses on independence in decision-making, and how

well the environment supports individual preparation for one‘s next role (Insel & Moos, 1974).

Relationships. Moos‘ next domain, relationships, defines the nature and intensity of

personal relationships within an environment (Moos, 1973, 1974; Insel & Moos, 1974, p. 181).

This dimension is defined by the involvement of individuals and the extent to which the various

stakeholders of the environment support and help each other. The variability and context of the

environment does not provide significant variation in the descriptors of this dimension, including

involvement, peer and group cohesion, staff support, and community (Insel & Moos, 1974).

System maintenance and system change. Moos‘ third dimension, system maintenance

and system change, is described as the clarity of the environment‘s expectations, control, and

response to change (Moos, 1973 1974; Insel & Moos, 1974, p. 181). Components of this

dimension include order, organization, clarity, control and innovation. Clarity accounts for the

extent to which the stakeholders in the environment know what to expect, what the rules are, and

29
how policies are communicated (p. 183). An organization‘s ability to respond to change based on

input from stakeholders is the other major piece of this dimension. In studies of the educational

and work environments, innovation and influence were uniquely emphasized in these contexts

(Insel & Moos, 1974).

While Moos‘ categories are descriptive dimensions of learning environments, this work

was later expanded by Rentoul and Fraser (1979) in the context of educational environments to

include a normative framework, or actual defined characteristics. Rentoul and Fraser (1979)

created this work in the context of open classrooms in the K-12 setting, however, its applicability

in the medical education setting has been examined as a useful basis for the clinical learning

environment (Schonrock-Adema et al., 2012). The normative framework and scale developed by

Rentoul and Fraser (1979) consists of five dimensions: Personalization, participation,

investigation, differentiation, and independence. Rentoul and Fraser (1979) mapped each of their

five dimensions to Moos‘ three domains of personal development or goal direction, relationships,

and system maintenance or system change (Table 2).

30
Table 2.

Scales Developed by Rentoul & Fraser and Classification According to Moos‟ Domains of
Human Environments

Scale Description Moos’ Domain Examples

Independence Students are allowed to make decisions Personal development or Academic achievement
about and have control over their own Goal orientation Practical orientation
learning and behavior Competition
Task orientation
Investigation Emphasis on skills and processes of Personal development or Problem solving
inquiry and their use of problem Goal orientation Inquiry
solving and investigation Feedback and evaluation
Learning objectives

Personalization Opportunities provided for individual Relationships Involvement

students to interact with the teacher; Affiliation
concern for personal welfare and social Staff Support
growth of the individual

Participation Students are encouraged to actively Relationships Peer cohesion

participate rather than be passive
learners

Differentiation Emphasis on the selective treatment of System maintenance and system Order and organization
students based on ability, learning change Clarity of expectations
style, interest and rate of learning Control
Innovation
Individualization
Control over one‘s learning
Responsiveness to change
Note: Adapted from ―Conceptualization of enquiry-based or open classroom learning environments‖ by A. J.
Rentoul and B. J. Fraser, 1979, Journal of Curriculum Studies, 11(3), p. 240. Copyright 1979 by Taylor & Francis.

Investigation. Rentoul and Fraser (1979) describe investigation as the ―skills and

processes of inquiry related to problem-solving and independent investigation‖ (p. 240). This

domain focuses on one‘s intake and application of knowledge, ability to draw conclusions from

information, and assessment of learning. Investigation is mapped to Moos‘ dimension of

personal development and goal orientation.

Independence. Independence describes the ability of students to make decisions about

and have control over their learning, based on their individual achievement, needs and learning

31
styles. Also grouped to Moos‘ dimension of personal development and goal orientation,

normative examples include measured achievement, practical orientation,

Personalization. The scale measures include opportunities for the students to interact

with the teacher individually (personalization) and the extent to which students are encouraged to

be active participants in learning instead of passive listeners (participation). This domain is

focused on the people engaged in the learning process, teacher and student, and the interactions

and relationships that form around the process of learning.

Participation. Participation addresses the extent to which one actively engages in

learning, as opposed to being a passive participant in an existing learning process. Important

aspects of this domain includes peer and group cohesion.

Differentiation. System maintenance and system change. Measures include the emphasis

of students‘ learning ability and learning style as the basis of selective treatment (differentiation)

and students‘ ability to make decisions and have control over their own learning and subsequent

behavior (independence). This domain is focused on the ability of the environment to change

through influence and response of the individuals involved in the learning.

Together, Moos‘ three domains and Rentoul and Frasier‘s five scales create a descriptive

and normative construction of the social context of a learning environment. The three domains

can be simply characterized as learning (personal development and goal direction), people

(relationships) and change (system maintenance and system change). Together, these three

domains moderate the social context of the learning environment. Each of the three domains are

separate but also interconnected, and align or influence either learning or organizational

outcomes.

Proposition 4: Moos‟ three descriptive domains of relationships,

personal development/goal direction, and

32
 change/adaptation are moderating factors of the
social context of the learning environment.
Proposition 5: Learning aspects (personal development, goal
direction) and people (relationships) most influence
learning outcomes.
Proposition 6: Change aspects (system maintenance, system
change) most influence work or organizational
outcomes.
Participation in Learning Environments

Socio-cultural learning theories offer important perspectives in medical education

research, as they approach learning in the context of the social setting, occurring through

participation and active learning methods. While social cognitive theory provides

complementary perspectives of how learning occurs within individuals, theories arising from

situated learning and communities of practice inform research on learning environments in the

social context (Contu & Willmott, 2003; Hodges & Kuper, 2012; Mann, 2011).

Communities of practice. Lave and Wenger (1991) developed the concept of

communities of practice, describing learning as ―distributed among co-participants, not a one-

person act‖ (Lave & Wenger, quoted in Isba & Boor, 2012, p. 34). Co-participation as a concept

of learning has particular applicability in medicine, where a team (doctors, nurses, pharmacist,

and other care providers) work together to care for, diagnose and heal the patient. A community

of practice is defined as ―groups of people who share a concern, a set of problems, or a passion

about a topic, and who deepened their knowledge and expertise in this area by interacting on an

ongoing basis‖ (Wenger, McDermott & Snyder, 2001 as quoted in Li et al., 2009, Background,

p. 4). Li et al. (2009) conducted a systematic review of communities of practice to compare

effectiveness and best practices in business and in health care. This synthesis concluded that

33
social interaction, knowledge sharing, and knowledge-creation are the common characteristics of

communities of practice in the health care setting, which is consistent with the environment for

learning in medicine.

The resident, as a physician member of the care team, must approach learning from two

aspects. Individual learning, continuous knowledge acquisition, and self study together form one

aspect. Learning from other members of the team, or from peers and others within the academic

program, constitute another aspect, referred to as the community of practice. The social process

of learning in medicine integrates participation between students or residents (novices), faculty

(experts), nurses and other ancillary staff, and even cultural artifacts and culture (Lave &

Wegner, 1991). Lave and Wenger (1991) coin the term legitimate peripheral participation to

refer to the ―relations occurring between newcomers and old-timers, and about activities,

identifies, artifacts, and communities of knowledge and practice‖ (p. 29). Novices enter

communities of practice through the periphery, due to their lack of experience, but then move to

the center as full members of the community as they continue to acquire knowledge and

experience (Lave and Wegner, 1991; Mann, Dornan, & Teunissen, 2012). Through this

participation, students develop identity and competence.

Resident physicians are both students and employees in the hospital setting, with respect

to hygiene factors and the physical and material environment. This dual role of student and

employee can often create unease among faculty and practitioners who must balance learning

with service to the hospital. However, Lave and Wenger draw no distinction between work and

learning, noting, ―learning is an integral and inseparable aspect of social practice‖ (Lave &

Wegner, quoted in Mann, Dornan & Teunissen, 2012, p. 35).

34
 Situated learning theory. Lave and Wenger‘s research form the basis of situated

learning theory, which emphasizes the social-cultural dynamic of learning, or learning in situ.

Situated learning theory is one of the core theories applied to medical education research

(Hodges & Kuper, 2012). The workplace as a context of learning through participation and

communities of practice is the basis for situated learning theory, which provides belonging and a

sense of commitment. A different perspective from conventional approaches of individual

learning, this theory posits learning is socially-constructed, directly linked to the organizational

context, and the social relationships and practices that occur within that context (Handley,

Sturdy, Fincham & Clark, 2006; Mann, 2012). Social relationships within an organization give

rise to power, affecting learning practice and one‘s ability to apply knowledge within the setting

or context (Contu & Willmott, 2003). In medical education, the culture socializes and transforms

medical students into physicians throughout the continuum of learning.

Communities of practice and situated learning are particularly relevant in medical

education, as physicians engage in a learning curriculum, or opportunities for development

where the learning occurs in many ways, and the community in itself becomes an important

learning resource (Mann, 2011). Teunissen et al. (2007) concluded participation is a critical

aspect of learning during residency training, thus proposing a model of learning by doing.

Workplace learning is described by Billet (2001) as inseparable with participation, as individuals

interact with others to increase one‘s capacity to learn and perform. Likewise, Eraut (2007)

describes learning at work as occurring through experience and interactions with colleagues,

through the process of socialization.

Proposition 7: Communities of practice positively influence

workplace learning and organizational outcomes.

35
 The student as learner. Student perceptions of classroom learning are reliable and valid

predictors of learning (Fraser, 2012; Walberg, 1969). Walberg (1968) and Walberg & Anderson

(1968) identified three components of classroom environment assessments (Figure 1): inputs

(pre-tests, baseline personality and cognitive measures), throughputs (classroom climate and

group perceptions of the class as a whole), and outputs (learning, achievement and attitude). The

throughput, the learning environment, serves as a mediating factor between the inputs and the

outputs. Mediating factors explain the relationship or processes that intervene between the input

and the output in an organization. Anderson & Walberg (1974) posited that the students in the

classroom were best suited to perceiving, judging and rating the socio-emotional climate of the

classroom and the subsequent affect on their learning. Walberg (1969) further concluded the

individual students collectively form a social system that contributes to learning outcomes and

the perceived climate of the learning environment.

Subsequent studies provide empirical research showing students‘ perceptions of the

quality of the environment influences student involvement, satisfaction and success (De Young,

1977; Muller & Louw, 2004). The perception by students of a positive environment is a required

36
element to motivate student learning (Fraser & Fisher, 1983; Schonrock-Adema, Bouwkamp-

Timmer, van Hell & Cohen-Schotanus, 2012; Muller & Louw, 2004). Fraser & Fisher (1983)

concluded that achievement of the students in the class can be enhanced by changing the

classroom environment so it is more in alignment with preferences identified by the students.

The teacher has also been identified as a major contributor to the social context of the

environment. Fraser (1994) concluded that students‘ perceptions of exemplary teachers directly

correlates with the teacher‘s ability to maintain a favorable classroom learning environment.

Proposition 8: Students‟ positive perceptions of the learning

environment are predictors of higher levels of
motivation, engagement, satisfaction, and learning
outcomes.
Proposition 9: The learning environment is the „throughput‟, or the
mediating factor explaining the relationship
between learning inputs and learning outputs.

Application of Clinical Learning Environment Studies

The clinical learning environment is a focus of study within clinical disciplines such as

nursing, dentistry and allied health education. However, the clinical (medical) education

literature is devoid of studies based on theoretical underpinnings. In nursing, Chan (2002) and

Saarikoski, Isoaho, Leino-Kilpi and Warne (2005) utilize aspects of organizational and

environmental theory in the development of instruments to measure the CLE in nursing. Chan

(2002) provides one of the only studies utilizing psychosocial educational underpinnings

including reference to Moos, Fraser, and Walberg. Saarikoski et al. (2005) published their

validation of the Clinical Learning Environment and Supervision Scale (CLES), but noted ―no

coherent theory in nursing literature that underpins research into the congruence between the

clinical learning environment and the supervisory relationship‖ (Paragraph 2). Despite these

37
close linkages between nursing and medicine, the use of theory as a basis for research and

development of CLE constructs has not been translated.

In medical education, scholars reference the clinical learning environment from a

theoretical standpoint, but these published works tend to be editorials or invited commentaries in

journals. In 2001, a theme-based issue of the Medical Teacher focused on the topic of the clinical

learning environment. The focal point of the journal includes a two-part guide on the curriculum,

environment, and climate in medical education authored by Genn (2001a, 2001b), but only

briefly mentions the association between medical education curriculum and the physical, social,

and pedagogical aspects of the learning environment; however, learning theories such as

andragogy and pedagogy are included. Harden (2001) prepared an editorial on the learning

environment and the curriculum, in conjunction with Genn‘s guides (2001a, 2001b), but focuses

his opinions and expertise solely on the curriculum. Roff and McAleer‘s (2001) editorial in the

same issue focuses on the educational climate, but only references instruments used to measure

the environment. Despite the recognition of the importance of the CLE in the context of clinical

learning, lessons from sociology and sociological aspects of human interactions are often not

integrated into medical education research or practice, as evidenced by this thematic issue in a

prominent medical education journal on the topic of the CLE.

Measurement instruments. Several instruments have been developed to measure the

quality of the clinical learning environment. Examination of CLE instruments reveal significant

variation in content, scales, subscales, measures, structure, and terminology (Colbert-Getz, Kim,

Goode, Shochet, & Wright, 2014; Hooven, 2014; Schonrock-Adema et al., 2012; Soemantri,

Herrera, & Riquelme, 2010). Schonrock-Ademea et al. (2012) conclude this variation and lack of

consensus regarding concepts to measure results from the absence of a theoretical framework.

38
Colbert-Getz et al. (2014) compared four types of validity (content, response process, internal

structure, and relationship to other variables) to calculate a total validity score. Not one CLE

measurement instrument contained all four types of validity evidence, and only 61% had some

level of evidence related to content validity. This study concluded significant variation in the

validity of these widely-used CLE scales, noting the lack of a ―gold standard‖ for assessing the

learning environment is in part due to the lack of consensus on ―content, response process,

internal structure, and relationship to other variables‖ (Paragraph 4). Appendix A provides a

summary of eleven instruments, the respective number of scales and subscales for each, and an

inventory of the subscale topics utilized for the measurement.

Many of the subscale measures can be attributed to one of the three dimensions identified

by Moos (1968), or closely aligned with the normative framework provided by Rentoul and

Fraser (1979). Likewise, several of the subscale measures such as teamwork and collaboration

are critical and foundational aspects of a community of practice. However, none of these

instruments were developed based on these theories, nor have they been analyzed and mapped to

determine if the content is inclusive of the totality of the learning environment. The instruments

identified in Appendix A represent the most commonly utilized CLE measurement instruments

in the literature, as noted in review articles (Colbert-Getz, Kim, Goode, Shochet, & Wright,

2014; Hooven, 2014; Isba & Boor, 2011; Schonrock-Adema et al., 2012; Soemantri, Herrera, &

Riquelme, 2010) and by consensus of subject matter experts. This inventory can be used to

inform this research by assessing how the subscales align with Moos‘ domains.

Interpretive Model

The literature described in the first part of this chapter provides the basis for developing

an interpretive model describing the components and relationships of a learning environment.

39
Throughout this chapter, propositions formed from the literature establish how components of

the learning environment relate to each other. In some aspects of the learning environment, the

propositions explain how components of learning environments relate to each other. In other

aspects of the learning environment, the propositions explain how factors influence each other.

The interpretive model is developed based on the integration of theories, knowledge, and the

resulting propositions derived from the literature (Table 3).

Table 3

Summary of Propositions Derived from the Literature

Proposition 1 Positive physical settings in a learning environment are associated with

positive learning experiences.
Proposition 2 The physical context and the social context are separate but integrated
components of a learning environment, the influence of each affecting the
other.
Proposition 3 Perceptions of the physical environment impact perceptions of the social
environment in learning.
Proposition 4 Moos‘ three descriptive domains of relationships, personal development/goal
direction, and change/adaptation are moderating factors of the social context of
the learning environment.
Proposition 5 Learning aspects (personal development, goal direction) and people
(relationships) most influence learning outcomes.
Proposition 6 Change aspects (system maintenance, system change) most influence work or
organizational outcomes.
Proposition 7 Communities of practice positively influence workplace learning and
organizational outcomes.
Proposition 8 Students‘ positive perceptions of the learning environment are predictors of
higher levels of motivation, engagement, satisfaction, and learning outcomes.
Proposition 9 The learning environment is the ‗throughput‘, or the mediating factor
explaining the relationship between learning inputs and learning outputs.

40
 The interpretive model (Figure 2) provides a visual representation of a learning

environment, and is based on the propositions presented in Table 3. A description of the

interpretive model comprises the last portion of this chapter.

A learning environment consists of two components, the physical environment and the

social environment. Walberg (1968), and Walberg and Anderson (1968), provide context for the

role of the learning environment as the throughput of the learning process, or the mediating

factor between inputs and outputs. Mediating factors are transformational processes, intervening

between the input and output and explaining relationships (Baron & Kenny, 1986). As a

mediating factor, the learning environment is the transformational component, or process,

41
affecting learning and learning outcomes, as indicated by Propositions 8 and 9 (P8, P9). Walberg

and Anderson (1968) delineate the learning environment as a throughput consisting of the

classroom climate, student relationships, student-teacher relationships, and learning resources

(Figure 1). These factors align with literature presented earlier in this chapter by Moos (1968),

establishing factors of the social context of the learning environment being inclusive of climate,

relationships, and learning resources.

The social context and the physical context together create the learning environment. As

shown by Proposition 2 (P2), each of the two contexts influence each other, and the learning

environment as a whole. The literature leading to the development of Proposition 3 (P3) supports

the physical environment‘s impact on the social context, but not the reverse. Therefore,

Proposition 3 is limited in direction from the physical context only. The physical context of the

learning environment is associated with learning satisfaction and impacts perceptions of the

learning environment with regard to the social context. Proposition 1 (P1) shows this

relationship between the physical environment and learning satisfaction outcomes. The impact of

the physical environment on satisfaction is based on Herzberg‘s (1976) motivation-hygiene

theory. As reviewed in the literature, when applied to medical education settings, evidence does

not substantiate significant findings to indicate dissatisfaction of the physical factors overshadow

the factors in the social context. Therefore, for purposes of this study, the physical context is

identified as a core component of the learning environment, but is not a primary emphasis for

study. However, it is important to consider in the context of the learning environment, the

physical context‘s relationship with and impact on the social context, and the influence on

outputs such as learning outcomes.

42
 Moderating factors can be qualitative or quantitative, and predict the conditions affecting

a certain variable (Baron & Kenny, 1986). In the learning environment framework, moderating

factors of the social context include Moos‘ three dimensions of personal development and goal

direction, relationships, and system maintenance and system change. The influence of the

moderating factors is highlighted by Proposition 4 (P4) which shows the relationship of these

three elements on the social context. These three moderating factors are separate, but related

components, influencing the social context of learning. As such, these moderating factors also

influence the outputs of individual [learning] outcomes, program outcomes, and organizational

outcomes. Proposition 5 (P5) shows the relationship between the domain of personal

development and learning outcomes. Moos‘ domain of personal development and goal direction

includes factors such as the learning curriculum, learning opportunities, feedback, and individual

development. While there can be some cross-over with the relationship domain (for example, the

ability of faculty to provide feedback), this personal development and goal direction is most

aligned with the impact on individual learning and program outcomes. Likewise, Proposition 6

(P6) shows the relationship between the domain of system maintenance/change and

organizational outcomes. However, Proposition 7 (P7) illustrates the cross-over of outcomes

from the relationship dimension to both learning outcomes and organizational outcomes.

Moos‘ domain of relationships includes factors such as supervision, communication,

faculty support and group cohesion. It is in this dimension that the community of practice is

anchored. Relationships between all stakeholders of the learning environment – students, faculty,

and other members of a learning team, comprise the community of practice. As Lave and

Wenger (1991) posit, the community of practice is defined by groups of people, with common

concerns or common sets of problems, passion of a topic, whose shared participation deepens

43
knowledge and expertise. This framework for a community of practice is consistent with the

relationship dimension and normative factors provided by Moos (1968) and Rentoul and Fraser

(1979) to include components such as faculty supervision, faculty or teacher support, social

growth, and cohesion of the group. In the context of the CLE, the role of the patient as a member

of the community of practice will be an important consideration. It is the patient that provides the

focal point for learning, working and communicating around a common goal. The cohesion of

the group around the anchor of the patient will be a unique and integral aspect to testing this

framework in the context of medical education.

Group cohesion is inclusive of peer-to-peer, faculty-to-student, and the group as a whole.

Within these relationships and in an environment where learning and working occur

simultaneously, this dimension may have the most impact on the other two dimensions and also

on the social context. The relationships domain is aligned with learning outcomes in higher

education; however, given the unique context of the CLE and the community of practice, it will

be important to consider if the relationships dimension may also affect organizational outcomes.

Moos‘ third domain of system maintenance and system change includes factors such as

order, expectations, individualization, and responsiveness to change. Based on the research

conducted by Moos in the context of the nine different environmental settings, this dimension is

shown to align the most with organizational or work outcomes. As this framework is tested in the

learning environment, it will be important to consider if the change dimension affects

organizational outcomes such as those related to patient care.

Thesis Statement

This research explores and deconstructs the components of the clinical learning

environment. Through a narrative and thematic synthesis, these elements will be tested against

44
an interpretive model of general learning environments to develop a conceptual framework of the

CLE. Moos‘ three domains of personal development and goal direction, relationships, and

system maintenance and system change, will be utilized to determine applicability in the clinical

setting. Of these three domains, relationships may be the most important to learning, as the

community of practice is the basis for applied learning in the clinical (work) setting. The

relationships dimension is also an important consideration in an applied learning setting such as a

hospital, as the outcomes measures associated with change are most aligned with the

performance of the organization or the work environment, which in this case would include

patient care. This research will validate the three dimensions of the social context of learning,

identify the categories and subcategories of each dimension, and examine which of the

components affect the outcomes of the mediating variable, the learning environment.

Conclusion

The purpose of the current study is to develop a theoretically-based conceptual

framework to identify and describe the components of the clinical learning environment. This

chapter reviewed the organizational, social, educational, and medical literature associated with

learning environments. From this literature, research propositions are presented that drive this

study and serve as a basis for an interpretive model. This review provides the basis for a

theoretically-based model for learning environments which will now be examined through a

research methodology for applicability in the clinical (hospital) setting for medical education.

Published qualitative studies of the CLE identify factors of the clinical learning

environment and serve as a basis for deconstructing the content, analyzing learning environment

components in the context of the clinical setting, and developing themes and patterns of evidence

based on narratives. In Chapter 3, the methodology for the systematic literature review and

45
synthesis will be described, along with a description of the use of secondary data to determine

empirical findings.

46
 Chapter 3: Methodology

In Chapter 2, an iterative framework was developed to guide the research needed to

create a conceptual framework of the clinical learning environment (CLE). This chapter now

describes the approach and research methodology used in this research study. An evidence-based

research synthesis is described and justified as the most appropriate approach to informing the

research question. The justification of the selected methodology, the research method, results,

and internal validity of the methodology are addressed in this chapter.

Several qualitative studies have been conducted to assess learner (medical student,

medical resident, clinical fellow) perceptions of the CLE. While some research is quantitative,

most studies are qualitative or survey research, and are very heterogeneous in terms of type of

information gathered or specific aim or purpose of review in addressing a specific component of

the CLE. The purpose of this study is to develop a conceptual framework of the clinical learning

environment which includes descriptive categories and subcategories. This research will inform

the research question, how does the clinical learning environment influence clinical learning?

This study is based on the applicability of the descriptive framework established and

tested by Moos (1968), and the normative framework developed by Rentoul and Fraser (1972)

for general learning environments. This research utilizes these theories to extend knowledge and

develop a framework the context of the inpatient clinical setting of medical education. Three data

sets will be utilized in this research analysis. First, a systematic review of the literature and

subsequent thematic synthesis provides the basis for testing the research question, and creating

themes and categories informing the conceptual framework of the CLE. A detailed description of

the systematic review methodology including the search strategy, inclusion and exclusion

criteria, quality assessment, and synthesis is provided. Second, existing CLE measurement

47
instruments will be assessed using a similar approach of thematic synthesis of the subscales, to

compare existing measures to data from qualitative literature. Finally, the Accreditation Council

for Graduate Medical Education (ACGME) Clinical Learning Environment Review (CLER)

Pathways to Excellence (2014) will be assessed utilizing a thematic synthesis to provide a third

set of data for comparison.

Systematic Review Process

In fields such as medicine, evidence-based practice is a commonly utilized approach to

decision making. Beginning in medical school, students are expected to utilize evidence from

reputable, peer-reviewed sources, which is routinely sought and utilized for both learning and

application of knowledge as a precursor to clinical decision-making. In non-medical fields, such

as management, an evidence-based approach is less utilized for decision-making because it is

common for managers to make decisions based on experience and organizational culture.

Evidence-based management (EBMgt) is the scientifically-based practice of management

achieved through integration of information (Rousseau 2012, p. xxiii). Also referred to as

evidence-informed management, the purpose of EBMgt is to address the gap between science

and practice by providing evidence from prior research to enable informed decision-making. A

systematic review of the literature is a commonly used research methodology to inform

evidence-based management.

This approach to systematic review in medicine evolved to the adoption of evidence-

based medicine approaches to the practice of medicine (Higgins and Green, 2008). Mulrow

(1994) describes systematic literature reviews as ―invaluable scientific activities‖ that integrate

an abundance – perhaps unmanageable – amounts of information into an efficient integration of

information providing data for rational and practical decision making (p. 597). Systematic

48
reviews not only summarize data, but can define the boundaries of what is known, and can

prevent knowing less than has been proven (Cook, Mulrow, & Haynes, 1997, p. 378). In the

social sciences, systematic review provides synthesis of existing evidence that is dependable and

robust, and has the potential for the transfer of knowledge across different contexts (Denyer &

Tranfield, 2009, p. 672). Researchers have a tremendous opportunity to learn from existing

research using a transparent and reproducible approach to synthesize existing data through

systematic review. The results of systematic reviews allow the research to utilize or apply

knowledge in a new way, or to identify gaps or incongruent findings that may result in the need

for more primary studies.

To best address the primary research question, defining the components of the clinical

learning environment, the researcher will perform a systematic review of the literature utilizing

thematic evidence synthesis. While there have been several studies performed on perceptions of

the CLE, and several instruments developed to measure the CLE, the elements of the CLE

remain ambiguous. To date, there has not been a systematic review of the literature performed in

order to synthesize data regarding perceptions of the components of the CLE. Data presented in

published research thus far is both qualitative and quantitative, gathered through several

modalities including interviews, focus groups, surveys, and use of existing data collection

instruments. Thematic synthesis of this existing research through a systematic review

methodology, especially for qualitative studies, allows the researcher to assimilate data from

diverse contexts, generate new conceptual models, identify gaps in existing knowledge, and

provide evidence for further research, study, and design of assessment/outcome instruments

(Tong, Flemming, McInnes, Oliver & Craig, 2012, p. 181.)

49
 Systematic reviews in health care or medicine collect all empirical evidence (referred to

as an exhaustive review) fitting pre-specified criteria in order to answer a specific research

question (Higgins & Green, 2008, p. 6). Emerging from medical science and healthcare, an

evidence-based synthesis approach systematic review methodology allows the researcher to

collect varying forms of evidence and aggregate knowledge on a specific topic (Higgins &

Green, 2008; Mulrow, 1994; Tranfield, Denyer & Smart, 2003). Systematic review has shared

processes with the traditional scientific method (Gough et al., 2012, forward). The synthesis of

all existing evidence, not just empirical studies, allows the researcher to form evidence-based

and scientifically formulated conclusions about existing problems of phenomena (Rousseau,

2008).

In medical education, a true hybrid between traditional medical research and the social

sciences, systematic review is a common research approach utilized to synthesize knowledge and

inform practice in teaching and learning in the medical sciences. Health policy makers,

regulators, and academic accrediting agencies such as the ACGME utilize systematic reviews to

inform guidelines, accreditation regulations, and review processes (Mulrow, 1994). Agencies

such as the ACGME have commissioned calls for research utilizing systematic review to provide

evidence synthesis in order to inform decision-making on new topics, such as fatigue and duty

hours of medical residents (ACGME, 2015).

The researcher will approach the research methodology by conducting a configurative

review and thematic synthesis to arrange prior findings from primary studies in order to answer

the research question (Gough et al., p. 51). Configurative approaches are utilized to address

research questions that either generate new theory, or explore the applicability of existing theory

in particular situations (p. 51). This research addresses the configuration of the learning

50
environment in the context of the clinical (hospital) setting. This research explores the salience

of existing learning environment theory rooted in higher education to a new setting, the clinical

environment or teaching hospital. A configurative process allows the researcher to arrange

findings from heterogeneous primary studies to develop a coherent whole inclusive of all

elements of the CLE (Gough et al., 2012, p. 51). Thematic synthesis is a useful approach in

identification, description and categorization of heterogeneous research, and contributes to the

development and testing of theories with contextual variables (Popay et al., 2006, p. 12).

Search strategy. In traditional research, the sample and sampling methodology

employed by the researcher may be one of the most important underlying factors in a research

study. The sample utilized in a study should represent a representative portion of the population

being studied, and informs the quality of the inferences made by the researcher based on the

study results (Onwuegbuzie & Collins, 2007, p. 281). Rigorous sampling methods are directly

related to the external validity of the results, and the ability of the researcher to generalize the

findings. In systematic review, the sampling process and subsequent data collection is conducted

through systematic database searching and identification of applicable literature; the rigor and

transparency of this process informs the quality and inferences that will be made by the

researcher.

Systematic review methodology is approached in six steps, following initiation of the

study and formation of a review team. These steps include formulation of the research question,

framework and approach; development of a search strategy and screening of articles; description

of study characteristics; quality and relevance appraisal; data extraction and synthesis; and

interpretation of findings and communication with stakeholders (Gough et al, 2012). The

51
researcher‘s strategy for data collection is described in this section and includes the search

strategy, screening, criteria, and quality and relevance appraisal.

This research utilizes a configurative review process to generate theory from assumptions

and concepts that emerge from the data (Gough et al., p 51-52). The researcher aims to identify

sufficient studies to extract and synthesize concepts for coherent configuration (Gough et al.,

2012, p. 108); therefore, the search was not exhaustive as typically performed in the traditional

medical sciences or for an aggregative approach. The researcher aimed to select studies

providing the most valuable information for the review until saturation was achieved (Gough et

al., 2102, p. 114). Saturation is established when ―studies no longer contribute new concepts to

the synthesis‖ (p. 114).

Pilot search. An initial search of articles using the key words clinical learning

environment was conducted using Google Scholar. 12 articles were identified by the researcher

and utilized to assist with the initial development of keywords, databases and an initial search

strategy. With assistance from a librarian with expertise in medicine and evidence synthesis,

Appendix B shows the list of keywords identified and the Boolean operators utilized to develop a

search string. The primary databases utilized for this study is PubMED Medline. Due to nuances

in literature database functionality, in particular with PubMED Medline, the keywords must be

paired with medical subject headings (MeSH), listed in Appendix C. The electronic databases

utilized for searching are listed in Table 4.

52
Table 4.

Databases Utilized for Searching Existing Literature

Databases Searched Using Subject Terms Databases Searched Using Keywords Only

PubMed MEDLINE Web Of Science

ERIC ProQuest Disserations Database
EMBASE Google Scholar
Academic Search Premiere

A pilot search resulted in 953 articles. The researcher and the librarian agreed to utilize

the first 200 articles in the search results to perform a robust pilot to refine the inclusion and

exclusion criteria, and to also assess the effectiveness of the search strategy. Based on the review

of the first 200 results, the search terms and the inclusion and exclusion criteria were revised and

confirmed for a final search and selection process. The final search was run on October 17, 2014

in PubMED Medline (Appendix D). A total of 1,081 results were identified. Searches in the

remaining databases did not produce articles relevant to the research. Therefore, the researcher

exclusively utilized PubMED Medline for the search.

Inclusion and exclusion criteria. Inclusion and exclusion criteria must be developed and

explicitly stated in order for the researcher to be able to indicate how decisions were made to

include relevant research for the study. For this study, inclusion and exclusion criteria were

developed iteratively. First, the researcher created an initial list of criteria utilized to focus and

narrow the search based on the research questions to produce a manageable number of results.

For example, the researcher only included studies that assessed the clinical learning environment

in teaching hospitals, and excluded other settings such as ambulatory clinics, outpatient settings,

or nursing homes. The initial inclusion and exclusion criteria were utilized on the pilot search

53
results. The researcher maintained a list of additional reasons for exclusion and then evaluated

the list at the end of the pilot process in order to create a final list of exclusion criteria. The final

inclusion and exclusion criteria are listed in Table 5. All inclusion and exclusion criteria must be

met for a study to be included in the sample.

Table 5.

Inclusion and Exclusion Criteria

Inclusion Criteria Exclusion Criteria

Assess the overall learning environment (not
specific to single aspects).
Setting: Hospital, inpatient acute care setting
Subjects: Medical students (clinical years),
residents or fellows
Multi-specialty or > 2 specialties
Language: English
Study Site: United States, Canada, Europe &
Australia
Instrument validation studies
Studies utilizing established CLE instruments
Editorials or opinion pieces
Literature reviews or systematic reviews
Studies focused on mistreatment, disruptive
behavior, disruptive events, abuse, or moral
distress
Simulation studies
Online learning, e-Learning or virtual
learning, or Technology-based (computers,
EMRs, iPADs, smartphones)

Identification of additional articles. In addition to the articles identified through the

database search, the researcher convened a focus group of experts on November 11, 2014 to

discuss the methodology and elicit input on identification of articles that should be included in

the sample, including grey literature such as whitepapers, reports, and other organizational

briefings. The experts included in the focus group were unable to identify any additional articles

from peer-review journals to be included in the sample.

54
 Additionally, the researcher completed a process referred to as hand searching, which

involves reviewing all cited references from key articles identified through the database search to

assure thoroughness of the search process. The aim of hand searching is to address any

limitations that may arise from the established search terms and a controlled vocabulary (Gough

et al., 2012, p. 125). Hand searching also allows the researcher to assure scholarly works

including book chapters, grey literature, government reports, and other applicable documents are

identified for assessment and potential inclusion. If a scholarly work was not identified through

the electronic search, but was identified through hand searching, it was collected for further

review and assessment based upon the established inclusion/exclusion criteria.

Literature search results. The search conducted in PubMED Medline produced a total

of 1,081 results (Figure 3). All titles and abstracts were reviewed to eliminate all publications

meeting obvious exclusion criteria. A total of 992 articles were excluded based on this initial

review process. The remaining 85 articles required further analysis and were reviewed based on

a reading of the full text article in order to determine criteria for inclusion. An additional 74

articles were excluded (see Figure 2 for delineation of exclusions). The 11 remaining articles

were reviewed in detail, including a hand search of the references, for consideration of inclusion.

Four new articles were identified through the hand-searching process for inclusion, for a total of

15 articles. These 15 articles are referred to as the sample set.

55
 Quality appraisal. This next step of study appraisal accomplishes three objectives. First,

the researcher assesses the internal and external validity of the studies in the sample to determine

if studies with low validity should be included in the study sample. Second, the researcher uses

the results of the quality appraisal to uncover reasons for divergent results among studies

resulting from differences other than chance. Finally, the researcher will utilize the quality

56
assessment results to inform the reader with sufficient information to independently judge the

applicability and results of the research (Meade & Richardson, 1997, p. 533-534).

All of the articles identified can be classified as medical education research – that is,

research performed by physicians or health care educators or researchers and published in

journals that are either medical or medical education-centric. Given the context of the studies,

and the audience that will be utilizing this research, the researcher utilized the Medical Education

Research Study Quality Instrument (MERSQI) to appraise the quality of each study and to

provide a score to assess and compare study quality (Reed et al., 2007). The MERSQI was

developed to address the variation in the types of studies published in medical education

research, and specifically the lack of certain study types such as randomized controlled trials,

which are common in medicine and science. The MERSQI assesses the quality of both

observational and experimental medical education studies in order to conduct comparative

analysis in the medical education field (Reed et al., 2007). The MERSQI domains and scoring

rubric are provided in Appendix E.

Possible MERSQI scores range on a scale from 5-18. Ten items are scored in six

domains. A total of three points can be provided for each of six domains including study design,

sampling, type of data, validity of evaluation instrument, data analysis, and outcomes. While the

MERSQI is a relatively new quality assessment instrument (Reed et al., 2007), it has been

validated in both the medical education literature and the nursing literature (Yucha, St. Pierre-

Schneider, Smyer, Kowalski & Stowers, 2011) as a reliable instrument appropriate for diverse

and heterogeneous health care studies. Since its development, the MERSQI is now routinely

used by medical professional societies to score articles presented in scientific presentations.

Although there are not score ranges defined to correlate with study quality, one study was done

57
to assess the mean score of articles accepted for publication in peer review journal compared to

articles that were rejected for publication in the same journal. Reed et al., 2008, applied the

MERSQI to 100 articles submitted for publication to the Journal of Internal Medicine and found

a significant difference in the mean score of published articles (mean 10.7) versus rejected

articles (mean 9.0), with a range of 5 – 15.

Attempts to eliminate bias are done by making the research methodology transparent and

reproducible, using comprehensive database searches, establishing clearly defined inclusion and

exclusion criteria, and establishing well defined quality assessment criteria.

Considerations from Expert Review Panel

This study incorporates subject matter input from nine different perspectives, through the

input of subject matter experts (SME). Five individuals are medical education researchers, all of

whom hold the degree of either Doctor of Philosophy (PhD) or Doctor of Education (EdD). Each

of the five hold faculty appointments as either professor or associate professor at medical

schools, serve in leadership (decanal) positions within those schools, and also serve as experts in

medical education research and qualitative methods to the Association of American Medical

Colleges (AAMC). Two individuals are physicians and currently hold leadership positions with

either the AAMC or the ACGME. Both of these individuals serve in leadership roles with their

organizations related to the CLE. One individual is a PhD researcher, practitioner, and expert in

human factors engineering. This scholar-practitioner holds an academic appointment at a major

university and teaches social science research methods to clinical faculty. The final SME is a

scholar-practitioner who holds the degree of Doctor of Public Health (DrPH), serves as a vice

president for a hospital system, and is responsible for strategy and operations related to medical

education and research. This individual also serves as an appointed member of the ACGME‘s

58
Institutional Review Committee (IRC), and the AAMC‘s chair of the Section on Graduate

Medical Education.

The SME feedback summary is provided in Appendix F. The SME feedback and

guidance was extremely beneficial in refining the research question, developing the research

methodology, and approaching the qualitative analysis. One of the most important contributions

of the SMEs was in the consideration of data triangulation and validity. Based on discussion and

feedback with these experts, two additional data sets were identified for comparative data

analysis. First, an inventory of existing and predominant CLE measurement instruments was

suggested. A thematic synthesis of the subscale components of these instruments will provide a

comparative analysis of factors utilized currently by medical education researchers to assess

learning environments in medicine. Follow up discussion with several of the SMEs resulted in a

consensus inventory of 11 CLE measurement instruments. Finally, given the emphasis of the

ACGME‘s CLER process and recently developed pathways, a third thematic synthesis of this

data will be conducted to compare and triangulate data. Finally, the SMEs were helpful in

discussing the implications of this research, and areas for future focus, research, and application

in medical education.

Summary of Results

Literature was categorized and extracted to include publication details (authors, year,

source, aims), context descriptions (subjects, settings, methodologies), and findings (Appendix

G). A descriptive paragraph was developed on each included study. Fifteen studies were

identified through the literature search process as meeting all inclusion and exclusion criteria

(Appendix G). Each study was evaluated using the MERSQI and scored accordingly. While the

MERSQI quality appraisal tool allows for broad variation in study type, additional points are

59
given to studies that utilize a validated tool or instrument in the study design, thus accounting for

variance in the mean score between the two groups. The sample set yielded a mean MERSQI

score of 7.3 (range 6.0 – 10.5).

The majority of the studies (n=7, 47%) were performed in Western Europe (Netherlands

n=3; United Kingdom n=2; Belgium n=1; Sweden n=1), followed by the United States (n=5),

Australia (n=1), New Zealand (n=2). Seven of the studies were performed at a single institution.

The remainder of the qualitative studies were performed at a relatively small number of

institutions, ranging from 2-9 hospitals. Two studies did not report on the number of institutions

included in their sample.

Of the 15 studies, the majority (n=8) assessed the learning environment from the vantage

point of medical students. Four studies were resident-based, one was both resident and student

combined, one was from the perspective of faculty regarding student learning, and one was a

mixed (students, residents and faculty). The ratio of subjects was consistent between the two

groups.

The research studies are heterogeneous in nature, and therefore the configurative

synthesis utilizing a thematic approach is best suited to assess the data from the 15 studies.

Thematic analysis allows the researcher to identify core concepts both inductively and

deductively through iterative exploration (Gough et al, 2012 p. 52). Through a process of

thematic synthesis, concepts from primary studies will be identified, interpreted and configured

to create higher order concepts and a conceptual framework. Following the search and

identification of relevant studies, thematic synthesis is conducted by data extraction, data coding,

theme development, and analytical or higher-order theme generation (Thomas and Harden,

2008).

60
 Thematic synthesis has four main elements: Developing a theory of how the

[intervention] works, why and for whom; developing a preliminary synthesis of findings of

included studies; exploring relationships in the data; and assessing the robustness of the synthesis

(Popay et al., 2006).

Data coding. Code development was completed in two processes. First, a deductive

process, utilizing the theoretical frameworks and domains created in higher education by Moos

(1968) and Rentoul & Fraser (1979) were utilized to establish domain-based themes. The actual

data extraction process of the complete sample set allowed for inductive approaches based on

additional concepts identified, or variations in concepts, therefore allowing for formation of

additional codes and themes. Findings were entered into Excel and coded for meaning and

content. Codes were then grouped into hierarchical structures with descriptive themes, allowing

for further synthesis and generation of analytical themes based on patterns emerging (Cruzes and

Dyba, 2011).

61
Table 6.

Learning Environment Framework Descriptions Utilized for Initial Coding and Sorting of Data

Moos Rentoul & Fraser

Domain 1 Personal development & goal direction
 Learning Objectives
 Content
 Feedback
Domain 2 Relationships
 Open communication
 Social and interpersonal support
 Group cohesion
 Teacher support
Domain 3 System Maintenance & System Change
 Orderly environment
 Clear expectations
 Responsiveness to change
 Student influence
Investigation
 Process of inquiry
 Problem solving
 Independent learning
Personalization & Participation
 Individual teacher interaction
 Active participation in
learning
 Social growth of individuals
Differentiation, Independence
 Individualization
 Student selective treatment
 Student decision-making and
control over one‘s own
learning

Data triangulation. Data triangulation is a methodological process to demonstrate rigor

in the analysis, and addresses internal validity by using more than one method of data collection

to answer a research question (Barbour, 2001). However, triangulation through data collection

methods is difficult when using qualitative or mixed methods. Since qualitative research is

typically carried out from a relativist perspective, it does not typically conform to traditional

quantitative analysis methods such as triangulation (Barbour, 2001, p. 1117). Richardson (1991)

refers to crystallization as a better alternative for qualitative studies, allowing for a research

question to be explored through various angles. Likewise, Mays and Pope (2000) explore the

issue of quality in qualitative health services research, and conclude comprehensiveness through

use of a complete sample in a full range of settings may be the most realistic goal for internal

validity (p. 52).

62
 In this study, two additional data sets are utilized. First, an inventory of commonly

utilized CLE measurement instruments was developed (Appendix A). Identification of these

CLE instruments came from the quantitative studies that were excluded from the sample set,

instruments analyzed in other sources (Colbert-Getz et al., 2014; Isba and Boor, 2011;

Schonrock-Adema et al., 2012), and recommendations of the SMEs. Second, the ACGME CLER

Pathways to Excellence (2014) document provides a listing of all factors, and their respective

definitions, utilized in the CLER process. Data from both of these sources were extracted, sorted

and coded utilizing the same process identified above. Utilizing three sources of related data

allows for crystallization of the evidence (Barbour, 2001).

Data aggregation. Qualitative data was summarized and tabulated through grouping and

weighting, and tied back to the scoring utilized through the MERSQI scale and assessment. A

frequency count on themes and subthemes within the findings was conducted. This data will then

be compared to existing measurement instruments identified (Appendix A) and the CLER

Pathways to Excellence (2014). A description of each instrument including the respective

domains and subsets, along with the results of the thematic synthesis will be compared to Moos‘

three domains. This process will allow for a comparative analysis based on themes that will

inform the development of the conceptual framework.

Data from three independent sources contribute to the evidence and findings of this study,

informing the research question, how does the clinical learning environment influence clinical

learning? First, the 15 studies identified through the systematic review and literature search

process are referred to in the results as the sample group. Each of the studies in the sample group

utilize qualitative methodologies to identify components of the clinical learning environment

from the perspectives of the learner or faculty. Data extracted from the results of each of these

63
studies are coded and sorted to develop thematic categories and subcategories. Next, the eleven

instruments developed to measure the CLE in medical education settings are analyzed. The

subscales in each of these instruments are evaluated for fit and placed into one of the dimension

categories. Finally, the ACGME CLER Pathways to Excellence (2014) document provides a

basis for the third data set. Each pathway is evaluated based on the described pathway properties

and assigned to a dimension category. While each of the 15 studies and two additional data sets

included in the analysis do not contribute to every category, each category is informed by

multiple studies and sources.

Moos‘ three domains of provide a basis for the initial sorting of extracted data into three

groups utilizing a deductive process. Each domain is characterized by properties defined by

Moos (1968) and Rentoul and Fraser (1979), as previously described. These property

characterizations provide a structure and definition by which initial sorting was completed. After

the extracted data was assigned to each of the three dimensions, the data elements within that

dimension were examined and further grouped based on patterns that formed. These patterns

provide the basis for the development of categories and subcategories within each dimension. To

the extent that patterns fit the categories established in higher education frameworks, these same

categories were utilized. Each data element extracted from the sample group accounts for a

single unit and the basis for a frequency count. Likewise, each subscale in the CLE measurement

instruments and each factor extracted from the CLER Pathways were assigned a single unit

frequency. A frequency count for each category was calculated and utilized as a basis for an

aggregate frequency count for each dimension.

Table 7 provides a summary of the synthesized data from each of the three data sets

sorted by dimension. A total of 317 units of data comprise the total extractions from all three

64
data sets. The sample group accounted for the most data extractions (n=169, 53.8%). The

relationships domain had the highest frequency of codes assigned (n=141, 44.5%), followed by

system maintenance and system change (n=94, 29.7%) and then personal development or goal

orientation (n=82, 25.9%).

Table 7.

Frequency Counts by Domain and Data Set

Domain
Personal System
Data Set Development Relationships Maintenance Total (%)
Sample 54 85 30 169 (53.3)
CLE Instrument Subscale Classifications 15 28 16 59 (18.6)
ACGME CLER Pathways to Excellence 13 28 48 89 (28.1)
Total (%) 82 (25.9) 141 (44.5) 94 (29.7) 317 (100.0)

Data extractions and narrative synthesis was first performed on the sample group. The

results from each of the 15 articles in this group were coded and placed into one of the three

domains of personal development, relationships, or system maintenance (Table 8). There were

no data extractions that did not fit in any of the three dimensions based on the descriptive

frameworks utilized. Data extractions ranged from two to 37 codes per article, with a mean of

11.3 and a median of 10.0. Ten of the articles contributed data to all three dimensions, and just

one article only contributed data to a single dimension. The relationships domain accounted for

slightly over half (n=85, 50.3%) of all data extractions from the sample set.

65
Table 8.

Sample Group Data Extractions Coded by Domain

Domain
Personal System
Article Year Development Relationships Maintenance Total
Bernabeo 2011 1 5 0 6
Boor 2009 7 4 0 11
Cross 2006 2 6 3 11
Daelmans 2004 0 2 0 2
Deketelaere 2006 3 1 3 7
Dolmans 2008 0 4 1 5
Gallagher 2012 1 7 2 10
Gordon 2000 3 3 2 8
Hagg-Martinell 2014 14 16 7 37
Henning 2011 7 2 1 10
Mathers 2006 2 10 2 14
Philibert 2012 0 7 3 10
Philibert 2010 5 6 4 15
Seabrook 2004 4 7 1 12
Thrush 2007 5 5 1 11
Total (%) 54 (32.0) 85 (50.3) 30 (17.8) 169 (100)

The sample set informs the research question by providing qualitative data on what

learners believe to be the most important elements of the CLE. Therefore, this data set was

utilized for the initial data synthesis and as a basis for the development of the categories and

subcategories emerging from patterns in the data. Data extracted from the sample group and

placed into each dimension was grouped based on similarity and patterns. The descriptions of

each dimension were used to determine identification and alignment of CLE categories within

each dimension. In many cases, the thematic synthesis produced patterns and groupings

consistent with the descriptors utilized by Moos (1968) and Rentoul and Frasier (1979) and the

same or similar nomenclature could be used. Once the thematic categories were established,

66
subcategories were further delineated based on the narrative themes. Each of the three

dimensions contains four categories.

Next, the CLE measurement instruments identified in the literature review were

examined (Table 9). The complete inventory of the CLE measurement instruments, the intended

population (residents, students or both), items and subscales, and subscale classification sorted

by dimension and category are included in Appendix H. Eleven instruments in total are included

in this data set. Instruments were introduced as early as 1978 (Marshall, 1978) and the most

recent in 2009 (Boor, 2009). Seven of the instruments were developed in the time period of

2000-2005. Each instrument has a range of items (37-57) and subscales (3-11). Subscale

descriptors provided a basis for determining coding of each subscale into one of the three

dimensions. Of the 11 instruments, only one subscale was unable to be coded to a dimension

(Boor, 2009); the subscale item of patient sign-out was determined to be a single outlier and

excluded from the data set. Finally, each subscale was assigned a category code based on the

categories established in the thematic synthesis of the sample set.

67
Table 9.

CLE Instrument Subscale Classification Data Extractions by Domain

Domain
Personal System
Author Instrument Year Development Relationships Maintenance Total
Boor D-RECT 2009 4 6 1 11
Cassar STEEM 2004 1 2 1 4
Holt and Roff ATEEM 2004 1 2 2 5
Kanashiro OREEM 2006 1 2 1 4
Keitz LPS 2003 1 1 1 3
Marshall MSLES 1978 2 3 2 7
Mulrooney PEEM 2005 1 2 1 4
Olivera Filho DREEM-R 2005 1 3 1 5
Roff PHEEM 2005 0 2 1 3
Roff DREEM 1997 1 3 1 5
Rotem SLHS 1995 2 2 4 8
Total (%) 15 (25.4) 28 (47.5) 16 (27.1) 59 (100)

Finally, the ACGME CLER Pathways to Excellence (2014) document was utilized to

develop the third data set (Table 10, Appendix I). This document was created by ACGME to

provide hospitals with guidance regarding the creation of optimal CLEs (p. 6). The document is

organized around six focus areas (quality improvement, patient safety, transitions of care,

supervision, duty hours/fatigue management and mitigation, and professionalism). Each focus

area then contains a number of pathways delineating key properties for which hospitals can self-

assess engagement of the learning environment by key stakeholders including residents, fellows

and faculty members. Each pathway contains a variable number of properties which further

define CLE assessment. A total of 34 pathways and 89 properties are provided in the document

and coded for this data set. For purposes of this analysis, each pathway descriptor was reviewed

and initially sorted into one of the three domains of personal development, relationships, or

68
system maintenance/change. Then, each pathway descriptor was coded based on the categories

established by the sample set.

Table 10.

ACGME CLER Pathways to Excellence Group Data Extractions Coded by Domain

Domain
Personal System
Focus Area Development Relationships Maintenance Total
Patient Safety 6 5 11 22
Health Care Quality 3 4 7 14
Care Transitions 2 8 4 14
Supervision 1 7 6 14
Duty Hours &
Fatigue Mitigation 1 1 8 10
Professionalism 0 3 12 15
Total (%) 13 (14.6) 28 (31.5) 48 (53.9) 89 (100)

Synthesis of Results

The primary aim of this study is to determine the components of the clinical learning

environment. By utilizing a configurative approach and extracting data from qualitative studies

describing the clinical learning environment, the researcher will be able to identify common

themes as described by learners, existing measures, and accreditation standards. A comparison of

these themes to the iterative framework will inform the development of the conceptual

framework. This analysis will demonstrate if the theory-based frameworks developed in higher

education can be utilized in the setting of the hospital, and if an additional dimensions or

considerations need to be developed. Finally, the analysis of these results compared to the

current assessment tools and standards used by the both the ACGME and by researchers who

developed measurement instruments, will allow the researcher to examine if current

69
methodologies are comprehensive and complete. Furthermore, weighting of components within

the CLE will allow the stakeholders to assess the weighting of their instruments to assure

appropriate balance of weighted factors within the CLE.

Thematic synthesis is a well accepted method of exploring differences in heterogeneity

(Popay et al., 2006, p. 14). Studies of student perceptions of the clinical learning environment are

heterogeneous, and have a variety of methodological approaches including qualitative,

quantitative and mixed methods research. Some studies are exploratory, while others report

information about relationships between certain factors and findings or outcomes. As the data is

extracted and coded, the researcher will compared and contrasted ways in which the relationships

were identified and analyzed, methodological differences, differences in population

characteristics, and other factors that emerged from study context and social heterogeneity

factors (Popay et al., 2006, p. 14-15). In Chapter 4, the results of the synthesis and analysis are

presented.

70
 Chapter 4: Analysis of Findings

This dissertation examines and identifies factors contributing to the social aspects of the

clinical learning environment (CLE) in medical education. The objective of this study is to create

a conceptual framework of the CLE, and to how the factors influence clinical learning. Chapter 3

described the methodology utilized to identify, assess, and synthesize the data obtained from the

literature to conduct the thematic synthesis, in addition to the results. This chapter presents the

interpretations of the findings from the results of the three data sources. The data, analysis, and

interpretation of the evidence are organized around the research question and the framework

presented in Chapters 2 and 3.

This chapter begins with an analysis of the qualitative data extracted from the primary

sources and identification of codes, categories, and subcategories for each of the dimensions.

These codes, categories and subcategories are compared to the existing frameworks identified

and discussed in Chapter 2. Next, data extracted from the clinical learning environment (CLE)

measurement tools are presented and added to the categories and themes. Finally, data extracted

from the Accreditation Council for Graduate Medical Education (ACGME) CLER Pathways to

Excellence (2014) document will be presented and added to the categories and themes. Data

from these three sources are then compared and analyzed in aggregate. Interpretation of the data

collected, synthesized, and analyzed will be presented in context of the research question.

Finally, a visual representation of a conceptual framework of the CLE will be presented.

Findings of Evidence

The Learning Dimension

Codes sorted to Moos‘ domain of personal learning and goal development are presented

as a result of the synthesis as the learning dimension (Table 11). A total of 82 codes were

71
assigned to this dimension. Four thematic categories emerged in this dimension: Feedback and

assessment; learning opportunities or patient care opportunities; curriculum and learning

objectives; and personal development. The category of learning opportunities, which also refers

to patient care opportunities in the CLE context, accounts for the highest overall frequency count

(n=32, 39.0%), and the most codes attributed from both the CLE instruments (n=7) and the

ACGME Pathways (n=10) in this dimension. The sample group was by far the most prevalent of

the three data sources contributing to the learning dimension, contributing 54 codes (65.9%) of

the total. The ACGME Pathways data was least representative in this dimension, with two

categories (curriculum/learning objectives and personal development) receiving no codes at all.

The four categories emerging from the synthesis include curriculum and objectives, feedback

and assessment, learning opportunities, and personal development in learning. Within each of

these four categories, subcategories were created based on the patterns of data emerging from the

synthesis.

Unlike the CLE instruments and the ACGME Pathways, the majority of the sample group

data was coded to the category of personal development in learning (n=18, 33.3%), followed by

learning opportunities (n=16, 29.6%). Based on the fact that the sample group data was collected

from qualitative studies of the CLE from the perspective of the learner (student or resident), this

result is somewhat anticipated.

72
Table 11.

Categories, Subcategories, and Frequency Counts of the Learning Dimension.

Data Set
CLE
Categories and Subcategories (n) Sample Instruments ACGME Total

Curriculum and Learning Objectives 9 1 0 10

Clear, structured and reviewed (6)
Patient care duties organized (2)
Lectures and content (1)
Feedback and assessment 11 2 3 16
Structured and honest (7)
Timely (3)
Utilizes an assessment instrument (1)
Learning Opportunities (Patient Care) 16 7 10 33
Relates to workplace (2)
Consistent (3)
Assignments have educational benefit (6)
Authentic learning experiences (5)
Personal development in learning 18 5 0 23
Learning skills (2)
Self-directed learning (8)
Problem orientation (8)

Total 54 15 13 82

Note: Subcategory frequency counts were obtained exclusively from the sample set. Each subcategory frequency
count (n) is shown in parenthesis following the subcategory descriptor. Frequency counts for the instrument set and
the CLER Pathways were based on the established categories.

Curriculum and learning objectives. This category refers to the actual curriculum

provided to the learner. This includes the organization of the curriculum including patient care

assignments, delivery of this information to the student, and formal learning experiences such as

lectures and didactic conferences. This category received the lowest frequency count (n=10) in

this dimension, and the vast majority (n=9) of codes were derived from the sample set. This may

73
speak to the importance of the curriculum from the perspective of the learner, but to a lesser

extent from those who measure the CLE, including researchers creating instruments or the

ACGME.

Clarity of learning objectives was a pervasive theme, noting the need and desire of the

learner to have a clear understanding of what is expected to be learned during a defined rotation

or clinical assignment (Boor, 2009; Cross, Hicks Parle, & Field, 2006; Hagg-Martinell, Hult,

Henriksson & Kiessling, 2014; Henning, Shulruf, Hawken & Pinnock, 2011; Mathers, Parry,

Scully, & Popovic, 2006; Thrush et al., 2007). Organization of learning and the relationship of

the intended learning to workplace clinical assignments was also integrated into this category

(Hagg-Martinell et al., 2014; Henning et al., 2011; Seabrook, 2004).

Feedback and assessment. This category is one that has the potential to cross

dimensions due to the dependency of feedback on the supervisor (people dimension). However,

feedback processes, structure, candor in assessment, and timeliness are all factors of feedback

noted by learners as being essential to learning and development (Boor, 2009; Hagg-Martinell et

al., 2014; Henning et al., 2011; Philibert, 2012; Seabrook, 2004). Accounting for 19.5% of the

coded data, feedback and assessment must be considered in a relational context to the role of

faculty, supervision and communication in the people dimension.

Learning opportunities. The learning opportunities category includes learning specific

to the delivery of patient care, and focused on the consistency of the educational experiences. In

a hospital setting, learning opportunities are anchored to the patients that are being cared for. For

example, if a patient presents to the hospital with pneumonia, the learning may be centered on

the disease pathology related to pneumonia and respiratory disorders. If a student were to be

assigned to all patients with pneumonia, then opportunities to learn about other clinical diseases

74
may be lost opportunities. Thus, the patient is considered part of the learning opportunities. This

consistency of learning was noted in several studies (Hagg-Martinell et al., 2014; Henning et al.,

2011; Thrush et al., 2007). However, consistency of learning opportunities applies more to the

learner than the patient. For example, each student or resident assigned to a specific rotation,

such as orthopaedics, should have a consistent learning experience. Hence, one student should

not be assigned all patients with hand fractures, while another student is assigned only to foot

fractures.

The educational benefit of assignments refers to learning through work experiences

versus being assigned work tasks that do not have learning benefit, and was a predominant factor

in this category (Boor, 2009; Dekeletaere, Kelchtermans, Struyf & De Leyn, 2006; Gordon et al.,

2000; Philibert, 2012; Seabrook, 2004). For example, once a student draws a hundred units of

blood, the learning may cease but the work is still required. Once the student has learned that

skill, then additional experience of the same task may no longer be beneficial from a learning

perspective.

Personal development in learning. The final of the four categories represents 27% of

the codes in this dimension. Personal development of the learner includes individualized learning

skills, self-directed learning, and problem orientation. Similar to the curriculum category,

personal development received no assigned codes from the ACGME data. Individualization

refers to the ability of the student to ask questions, grow as a self-directed learner, and adapt

learning styles and preferences in the clinical setting (Boor, 2009; Dekeletaere et al., 2006;

Gallagher et al., 2012; Hagg-Martinell et al., 2014; Henning et al., 2011; Mathers et al., 2006;

Thrush et al., 2007).

75
The People Dimension.

Moos‘ second domain, relationships, is presented based on the thematic synthesis as the

people dimension. This dimension accounts for the majority of all codes between the three

dimensions (Table 12). Four categories emerged from the synthesis of data assigned to this

dimension: Supervision, faculty support, group cohesion and trust, and communication. A total

of 140 codes were attributed to this dimension, with the sample set being the most prevalent

(n=85, 60.7%), followed by CLE instruments (n=28, 20.0%) and ACGME (n=27, 19.3%). The

people dimension includes the categories of supervision, faculty support, group cohesion and

trust, and communication. As presented in Chapter 2, the people dimension is anchored in

aspects related to the community of practice. As a learning environment where people learn

together, around the common goal of patient care, a strong community of practice is guided by

faculty and held together by cohesion of the group‘s stakeholders. Each of the three data sets

contributed codes to each of the four categories in this dimension.

76
Table 12.

Categories, Subcategories and Frequency Counts for the People Dimension

Data Set

CLE
Categories and Subcategories Sample Instruments ACGME Total
Supervision 13 5 5 23
Quality of supervision (6)
Adequate time (7)
Faculty support 48 12 2 62
Encouragement and role modeling (22)
Feedback (19)
Leadership (4)
Competency (3)
Group cohesion and trust 20 10 16 46
Peer cohesion (7)
Faculty-student cohesion (7)
Nurse-student cohesion (4)
Inclusion and trust (2)
Communication 4 1 4 9
Open and encouraged (4)
Total 85 28 27 140

Note: Subcategory frequency counts were obtained exclusively from the sample set. Each subcategory frequency
count (n) is shown in parenthesis following the subcategory descriptor. Frequency counts for the instrument set and
the CLER Pathways were based on the established categories.

Supervision. Supervision is an important aspect in the clinical setting, as students and

residents require supervision by mandate of accrediting agencies (ACGME, LCME) and also for

the safe provision of patient care. Subcategories emerging from the coded data include the

quality of the supervision and adequate time for supervision. Quality of supervision emerged as a

predominant subcategory, as data referenced supervision being perceived by the learner as

contributing to safety of the patient and development of the learner (Cross et al., 2006; Daelmans

et al., 2004; Dolmans, Wolfhagen, Heineman & Scherpbier, 2008; Philibert, 2012; Thrush et al.,

77
2007). In the clinical setting, quality of supervision is an aspect closely related to the next

category, faculty support. Adequate time relates to the conflicting demands of time on faculty

supervisors who are accountable for their own clinical duties, teaching responsibilities, research,

and often administrative duties; furthermore, the limitation of interruptions was noted as being a

contributor to learning satisfaction in the clinical setting (Cross et al., 2006; Hagg-Martinell et

al., 2014; Thrush et al., 2007)

Faculty support. Faculty support emerged as the most predominant category within the

people domain, accounting for 62 (44.3%) of the frequencies. The vast majority of codes came

from the sample set (n=48) which also accounts for the most allocated codes of all categories

derived from the sample. This predominance of data from the sample set may infer the

importance of faculty from the vantage point of the learner. CLE instruments (n=12) and the

ACGME data set (n=2) contributed to this category in a lesser extent; however, the 12 frequency

counts in the coding from the CLE instruments is also the highest allocation of all CLE

instrument codes.

Faculty support includes subcategories of encouragement and role modeling, leadership,

feedback, and faculty competency. Together with the category of supervision, faculty support

has interrelated relationships with the category of feedback in the learning dimension. Together,

supervision and faculty support are integrally related to the learner receiving feedback that

allows for learning and personal growth. Specific aspects include role modeling (Hagg-Martinell

et al., 2014; Henning et al., 2011; Philibert, 2012; Seabrook, 2004), enthusiasm (Dolmans et al.,

2008; Hagg-Martinell et al., 2014; Mathers et al., 2006; Philibert et al., 2010), and competence as

a teacher (Gallagher, Carr, Weng, & Fudakowski, 2012; Gordon et al., 2000; Hagg-Martinell et

al., 2014; Philibert, 2012; Seabrook, 2004).

78
 Group cohesion and trust. A core component of a community of practice, cohesion of

the group was the second most predominant category in the people dimension (n=46). Several

subunits of the term group were identified in the synthesis. Peer cohesion, or the relationships

between learners as a cogent group (Bernabeo, Holtman, Ginsburg, Rosenbaum & Holmboe,

2011; Cross et al., 2006; Daelmans et al., 2004; Gallagher et al., 2012; Philibert, 2012) and

faculty-student cohesion (Boor, 2009; Cross et al., 2006; Hagg-Martinell et al., 2014; Philibert,

2012; Philibert et al., 2010; Seabrook, 2004; Thrush, 2007) received an equal number of

frequency counts, inferring that both types of relationships are integral to the learning process in

the clinical setting. Cohesion and relationships with nursing was also a common thread

(Bernabaeo et al., 2011; Hagg-Martinell et al., 2014; Philibert, 2012; Seabrook, 2004). Finally,

trust, as an aspect of cohesion, was highlighted as a critical aspect of all groups (Bernabeo et al,

2011; Cross et al., 2006; Henning et al., 2011; Philibert et al., 2010). Of note, students

characterize valuable learning and professional growth as being attributed to acceptance by the

community of practice, together with competent and enthusiastic faculty supervisors (Hagg-

Martinell et al., 2014).

Also, an integral aspect of the community of practice, group cohesion and trust has

relational overlap with the category of responsiveness to change in the change dimension.

Likely, this overlap may speak to the linkages between clinical practice, group dynamics,

continuous improvement, and patient safety efforts.

Communication. The final category in this dimension is communication. Open

communication between staff, faculty and learners was a common pattern emerging from this

dimension (Bernabeo et al., 2011; Hagg-Martinell et al., 2014; Philibert et al., 2010). The ability

79
of the learner to be a proactive communicator was predominantly mentioned in one study

(Gallagher et al., 2012).

The Change Dimension

Moos‘ third domain, system maintenance and system change, is presented based on the

synthesis as the change dimension. The frequencies, categories and subcategories for the change

dimension is provided in Table 13. Four categories emerged from the data assigned to this

dimension: Orderly environment, clear expectations, individualization, and responsiveness to

change. The ACGME data set was the most predominant in this domain (n=47, 51.6%),

especially given that this data set only represents 27% of the entire frequency codes; The 30

codes assigned to this dimension from the sample is the least from that data set (17.8%). This

could be attributed to the focus of the learner on his own personal learning and development, as

opposed to that of the hospital or of the patient. Conversely, the ACGME CLER Pathways is

intended for hospitals to focus on the learning environment as a construct to improve patient care

and learning. The four categories emerging from the data in this dimension include orderly work

environment, clear expectations, responsiveness to change, and individualization. Each of these

categories are bound by thematic elements of the workplace as an aspect of learning. Clear

expectations and responsiveness to change received the highest frequency counts in this

dimension (n=27).

80
Table 13.

Categories, Subcategories and Frequency Counts for the Change Dimension

Data Set
CLE
Categories and Subcategories Sample Instruments ACGME Total
Orderly work environment 8 8 5 21
Well organized (6)
Balanced with learning (2)
Clear Expectations 6 1 20 27
What to learn (3)
What to do (2)
Role clarity (1)
Responsiveness to change 6 1 20 27
Acts on feedback (2)
Responsiveness (2)
Management response (1)
Innovation (1)
Individualization 10 5 2 17
Hierarchy clear (4)
Empowerment (2)
Progressive responsibility (4)

Total 30 15 47 92

Note: Subcategory frequency counts were obtained exclusively from the sample set. Each subcategory frequency
count (n) is shown in parenthesis following the subcategory descriptor. Frequency counts for the instrument set and
the CLER Pathways were based on the established categories.

Orderly work environment. The effectiveness of the hospital setting as a learning

environment was noted by several studies as being improved when there is order, organization

and structure (Deketelaere et al., Gallagher et al., 2012; Hagg-Martinell et al., 2014; Mathers et

al., 2006; Philibert, 2012). The balance of the work setting with learning assignments is also

noted (Deketelaere et al., 2006), consistent with work-learning balance noted by ACGME

(2014). The CLER Pathways document references order of the hospital environment in relation

81
to expectations, processes, procedures, and monitoring of reporting adverse events, safety issues,

and patient issues.

Clear expectations. Closely related to the order of the work environment, clear

expectations is established around the learner knowing what to learn (Cross et al., 2006; Hagg-

Martinell et al., 2014; Philibert, 2012) and what to do (Dolmans et al., 2008; Seabrook, 2004).

Likewise, knowing what to learn is a predominant theme of the CLER Pathways document

(n=20), with many examples cited specific to both the learner and the faculty member knowing

what individual roles and responsibilities are, and understanding of the hospitals priorities, issues

relating to policies such as supervision, common processes, and procedures.

Responsiveness to change. The ability of the organization to respond to feedback, act to

improve processes and patient care delivery, and innovation constitute the aspects of this

category. Receiving the same number of codes as clear expectations, this category is also a

predominant theme arising from the CLER Pathways (n=20). Responsiveness, and acting on

feedback, were frequently referenced in the sample set as areas that the learner found to be a

critical trait of a learning organization (Cross et al., 2006; Gordon et al., 2000; Philibert, 2012;

Philibert et al., 2010; Thrush, 2007). Likewise, the CLER Pathways provide descriptors coded to

this category such as engagement in systems-based challenges, learners involved in making

improvements, a supportive culture, and processes for providing feedback and describing

subsequent outcomes.

There were several codes related to this category that related to the people dimension

categories of group cohesion and communication. For example, communication was noted as an

essential foundational element for change to occur within an organization (Bernabeo et al., 2011;

Hagg-Martinell et al., 2014; Philibert et al., 2010). Likewise, group cohesion and trust had many

82
linkages to the community of practice, and to that community being a proactive driver of change

within the organization (Cross et al., 2006; Hagg-Martinell et al., 2014; Philibert et al., 2010;

Thrush et al., 2007).

Relationships Between Dimensions

Figure 4 provides a visual representation of the three dimensions of the social context of

the CLE: learning, people and change. The categories for each of the three dimensions are

provided in Figure 4. As codes were assigned to each dimension, some data emerged as having a

strong association with codes in other dimensions. As noted earlier in this chapter, alignment

between the learning and people dimension emerged between the categories of faculty support,

supervision, and feedback. Likewise, alignment between the people and change dimensions

emerged between the categories of group cohesion/trust, communication, and responsiveness to

change. While the data analysis was not designed to test inter-relatedness of codes across

dimensions, it is worthwhile to note these relationships between dimensions as a factor that may

inform the inter-relatedness of the dimensions.

83
 Learning and people. Gordon et al. (2000) concluded that students and their supervisors

(faculty) need to have a shared vision and understanding of educational goals in order to

maximize optimal clinical assignments, feedback, and oversight of education. Ergo, the role of

faculty in providing support to the learner is underscored as a factor that crosses over from the

people dimension into the learning dimension. Likewise, Henning et al. (2011) identified

learning opportunities and structured learning assignments, an integral feature of practice-based

learning, to have contingencies on faculty-student relations and the ability of faculty to be able to

properly support the student.

84
 People and change. Group cohesion and communication between participants (people

dimension) showed a possible relationship to responsiveness to change (change dimension).

Hagg-Martinell et al. (2014) concluded that the way the communication and interactions take

place affects the ability to be involved in the workplace, and contributors to the working

environment. For example, students feeling appreciated and included by the faculty encouraged

communication, and communication encourages individuals to be more proactive regarding

changes affecting the learning environment. Likewise, Cross et al. (2006) highlight the

importance of the learner and the group in the workplace setting and the active participation that

is needed to improve productivity, learning and outcomes. Daelmans et al. (2004) further this

aspect of cross-dimensional aspects as their conclusions emphasize the importance of teamwork

and engagement on participation in aspects of improving the learning environment. Finally,

Gallagher et al. (2012) highlight the community of students, faculty, and other healthcare

providers as integral to the broader learning organization, and how that organization responds

and functions.

Communities of practice. The hospital environment is referenced as a community of

practice, and is specifically examined in Gordon et al. (2000), Hagg-Martinell et al. (2014), and

Cross et al. (2006). Hagg-Martinell et al. (2014) specifically focus their research on the clinical

learning environment and acceptance in the workplace community. In this article, an emphasis

on Lave and Wenger‘s three elements of a community of practice underscores the study purpose:

(a) a joint enterprise where members contribute to the community; (b) mutual engagement where

members establish norms and relationships; and (c) a shared repertoire of communal resources

like shared norms and relationships, collaboration and problem-solving, language, tools and

routines (p. 16). Gordon et al. (2000) examine the integration of learners into the workplace

85
community as active learners from the very beginning of each clinical assignment, concluding

the community of practice supports clinical learning, especially in busy hospital settings where

patients have complex medical issues and pressures to discharge patients quickly leave little time

for learning outside of practice. Work-based educational environments are emphasized as

needing to connect service delivery with learning, personal growth, professional responsibility,

communication, relationships, and practice improvement (Cross et al., 2006; Hagg-Martinell et

al., 2014; Gallagher et al., 2012; Deketelaere et al., 2006). All of these interrelated factors

underscore the interconnected nature of the three dimensions, each affecting the social context

individually and collectively.

Physical environment. While the emphasis of this study was on the social context of the

CLE, the factors relating to the physical environment and related resources were mentioned in

several studies. One study in particular specifically examined the role of hygiene factors as

variable in the clinical learning environment (Philibert, 2012). A total of 20 codes and 10 factors

specific to the physical environment (resources) were made and coded into a separate category

(Table 14). In addition to the actual physical space (Hagg-Martinell et al., 2014), learners

mentioned employee benefits such as compensation, time off, the call schedule, and the

rotational assignments (Henning et al., 2011; Philibert et al., 2010; Thrush et al., 2007); space

and resources for learning including computers, the library, and simulation equipment (Gordon et

al., 2000; Hagg-Martinell et al., Henning et al., 2011; 2014; Philibert et al., 2010); and

convenience factors such as parking and food (Henning et al., 2011; Philibert, 2012). Each of

these studies associated these factors related to the physical environment as being related to the

overall satisfaction of the learner. However, none of the studies provided an analysis or data

showing an association or correlation with specific outcomes.

86
Table 14

Codes Identified with the Physical Environment

Descriptor No. Codes Sources

Resources for learning 4 Hagg-Martinell (2014); Philibert (2012); Gordon

(2000)
Information technology 3 Hagg-Martinell (2014); Henning (2011); Philibert
(2012)
Compensation 3 Henning (2011); Philibert (2010); Thrush (2007)

Schedule/Call 3 Thrush (2007)

Physical space 2 Hagg-Martinell (2014); Philibert (2012)

Lockers 1 Henning (2011)

Food 1 Henning (2011)

Library Hours 1 Henning (2011)

Parking 1 Philibert (2012)

Clothes 1 Hagg-Martinell (2014)

Total 20

Reinterpreted Model

The results of the data synthesis from the sample group provides insight to the research

question, how does the clinical learning environment influence clinical learning? Qualitative

data was extracted, summarized and tabulated through frequency counts and thematic grouping,

and then compared to independent data sets consisting of the CLE measurement instrument

scales and the ACGME CLER Pathways.

87
 This data is now assessed in comparison to the interpretive model provided in Chapter 2

(Figure 2). The data from the thematic synthesis and the interpretive model are reassessed and

the resultant conceptual framework is now presented (Figure 5).

The conceptual framework closely resembles the interpretive model introduced in

Chapter 2. The changes made to the interpretive model are primarily related to the moderating

88
factors of the three dimensions and the alignment of those dimensions with various outcomes.

New terminology for the dimensions (learning, people, and change) are used to replace Moos‘

terminology from the interpretive framework.

Data from the thematic synthesis suggests the people dimension has interrelated aspects

with each of the other dimensions, learning and change. However, there was not an apparent

relationship that emerged between the dimensions of learning and change. Thus, the model has

been modified to show a two-way relationship between the dimensions of learning and people,

and between people and change. Data from all three sources inform the legitimacy of

Proposition 4 (P4) supporting all three dimensions as factors influencing the social context.

Aggregate frequency counts are significantly higher for the people dimension, leading one to

conclude that relationships, trust and faculty support are central to the social context as a whole,

but also influential to each of the other two dimensions.

The three data sources also inform the dimensions. The sample set data is derived from

qualitative data from students and residents. The CLE instrument data is derived from existing

measurement tools produced by researchers who are generally teaching faculty in medical

schools. The ACGME CLER Pathways data is produced by an accrediting agency responsible

for assuring educational standards and public trust. The patterns of data emerging from these

three sources help to inform the distribution of the results and potentially the related outcomes.

Students, residents and faculty offer insights to the learning environment based on their

perspectives, which are largely related to their own personal learning and development. Thus, the

dominant dimensions of learning and people appear to be most closely aligned with learning

outcomes. However, the ACGME offers insight from the perspective of public trust and

accountability for safe patient care, thus the dominant dimension of change seems most closely

89
aligned with organizational (work) outcomes, such as patient care outcomes. Given the overlap

between dimensions, it is reasonable to view the people dimension as supporting both learning

outcomes and organizational outcomes. Therefore, Propositions 5, 6, and 7 (P5, P6, and P7) are

supported.

Likewise, all of the studies reviewed in this synthesis approached the learning

environment as the moderating factor that impacts learning outcomes. While this was not a

specific focus of the research analysis, no data or studies reviewed contradicted Proposition 9

(P9), thus providing a reasonable basis for supporting the CLE in totality as a moderating

variable.

A feedback loop has been added to the conceptual framework to show the use of

outcomes to inform the learning environment. Learner satisfaction, learning outcomes, program

outcomes, and organizational outcomes all provide data to inform the learning environment and

support continuous improvement. These feedback loops are noted as being foundational factors

of highly functioning CLEs (Gordon et al., 2000; Philibert et al., 2010)

The physical environment was not a focus of this study, however, data extracted from the

sample set included factors of the physical environment as mentioned earlier in this chapter. The

physical environment factors were associated with improved satisfaction of the learner, and also

as an influence on the social context of the environment (Gordon et al., 2000; Hagg-Martinell et

al., Henning et al., 2011; 2014; Philibert et al., 2010; Thrush et al., 2007). Thus, Propositions 1,

2, and 3 (P1, P2, and P3) are also supported. Learner satisfaction was added to the conceptual

framework as an output of the physical context of the CLE.

90
 Summary of Findings

The purpose of this study was to develop a conceptual framework of the clinical learning

environment, and to assess which of the factors most influence clinical learning. Three sources of

data informed the findings of this research. First, a systematic review of the literature and

thematic synthesis (sample set) provided confirmation of the dimensions of learning, people, and

change to be appropriate in the context of a hospital CLE. Likewise, data from the CLE

measurement instruments and the CLER Pathways further supported the reasonableness of these

three dimensions as the mediating factors of the social context.

The sample set also provided data for the development of categories and subcategories

within each dimension. These categories are similar to those developed by Rentoul and Fraser

(1979) informing the normative frameworks of learning environments in higher education. The

categories and subcategories provide further structure and definition to the components of the

CLE. Finally, the CLE outputs were considered as part of this research. Outcome measures

including learning satisfaction, learning outcomes, and organizational outcomes were identified

and aligned with contexts and dimensions. Chapter 5 will now conclude this research

investigation, as implications from the research findings are explored.

Limitations and Interpretations of Findings

This study has several limitations. First, the systematic review identified 15 articles

meeting inclusion and exclusion criteria. While these articles were rich with qualitative data and

provided a strong basis for the thematic synthesis, additional studies would enhance the depth of

the study by providing more information from broader stakeholders of the CLE. Next, the CLE

instrument analysis was limited to an assessment based on subscales and published validation

articles. A detailed analysis of instrument development and validation was not conducted.

91
Finally, the data extracted from the ACGME CLER Pathways document provides a rich data set

for comparative purposes. However, an extended analysis of other ACGME measures may add

information to this analysis regarding data collected by this entity to assess the CLE.

Systematic reviews and thematic syntheses are generally never conducted independently,

but instead by a study group comprised of a diverse set of researchers. Due to the nature of this

dissertation requirement, a single researcher conducted both the systematic review and the

thematic synthesis. Inherent bias was controlled for in the systematic review process, but it is

more difficult to control for bias when conducting thematic and narrative synthesis of data. A

study group would provide more internal validity to the thematic synthesis.

92
 Chapter 5: Conclusions and Implications for Management Practice

The purpose of this dissertation is to develop a conceptual framework of the clinical

learning environment (CLE) to inform research and practice of medical education. Without clear

definition, the CLE remains an elusive construct and results in variation of perspectives, study

methods, measures, and outcomes. An extensive review of theory, evidence, and research in

medical education provided knowledge that was synthesized and developed to form an

interpretive framework of learning environments. The interpretive framework was explored to

delineate mechanisms and relationships of the components working together to form a learning

environment. A systematic review of the literature served as a foundational data source for an

initial thematic synthesis, in addition to data from existing CLE instruments and the ACGME

CLER Pathways to Excellence. These three data sources together allow for a comprehensive

deconstruction of the CLE, development of themes, and testing of the interpretive model. Data

emerging from the synthesis guides themes, new knowledge, and validation of a conceptual

framework of the CLE.

The integration of knowledge from existing research studies, the scholarly literature, and

publications offer a clearer understanding and framework for conceptualizing the components of

a clinical learning environment in the medical education setting. This final chapter summarizes

the conclusions drawn from the research and presented in Chapter 4, presents implications of the

conclusions for management practice, and suggests areas for further research. Limitations of the

study are presented along with final thoughts and summary observations.

Study Conclusions

Learning environment theory, research, and frameworks developed in other contexts such

as higher education, classroom settings, psychiatric hospitals, and work settings served as a basis

93
for this study. Moos‘ (1968) research in a variety of learning contexts provides a theoretical basis

for which the CLE can be applied. Previous research (Schonrock-Adema et al., 2012) confirms

the lack of theory in medical education learning environment assessments, and attributes that

lack of theory to the variation in measurement instruments and approaches. Furthermore,

Schonrock-Adema et al. (2012) provides a rationale for the use of Moos‘ framework in the

context of medical education. This research builds on that rationale, applies learning

environment theories, creates an interpretive framework, tests that framework, and informs the

development of a conceptual framework of the CLE.

The social sciences have much to offer and contribute to research in medical education.

Utilizing theories of organizational environments, human behavior, and learning allows for a

deeper understanding of phenomena as a basis for development and extension of theories in the

clinical context. This study is one example of how knowledge from other fields, like sociology

and education, can inform research study development and further academic contributions in

fields like medical education.

The CLE is a moderating factor explaining the relationship between learners and learning

outcomes in the clinical setting. The CLE as a comprehensive structure impacts the delivery of

learning, knowledge acquisition, and knowledge application in medical education. The social

context of the CLE is comprised of three interrelated dimensions that serve as mediating factors:

learning, people, and change. These mediating factors not only influence the social context, but

also influence each other. This research substantiates these three dimensions as influencing the

social context of the CLE, and provides descriptions of narrative themes emerging from the data

sources. These descriptive themes are the basis for the development of categories and

subcategories of data.

94
 Although the CLE is shown in the conceptual framework as having several components,

it is important to note that the individual parts of the CLE are not stand-alone units. Each and

every component of the CLE interacts with one or more components, and collectively influences

and composes the CLE in its totality. Likewise, while each component of the CLE is shown to

have alignment or influence to particular outcomes, those alignments are not exclusive. This is

illustrated, by example, as the three dimensions of learning, people, and change show alignment

with either learning or organizational outcomes, while each dimension interacts with the others,

and thus influences outcomes that are not primarily aligned.

The three data sources utilized for analysis in this study inform the results. First, the

systematic literature review and subsequent narrative/thematic synthesis provides insights to the

CLE from the perspective of the student and resident as an engaged participant of the CLE.

Anderson and Walberg (1968) substantiated the learner as a valid reporter of learning

environment assessment. This is an important foundational aspect of this analysis, as students

and residents are the primary individuals for whom clinical learning environment efforts are

targeted to. Thus, anchoring this analysis with qualitative data from the perspective of the student

provides substantive data from a validated source. Data from the qualitative studies in the sample

set serve as the basis for the development of themes used to create the categories and

subcategories in each dimension. Furthermore, this data set provides initial evidence to indicate

learner perceptions align with learning outcomes, and place primary emphasis on the dimensions

of learning and people.

Next, the CLE instruments provide for the second data set, analysis, and thematic

synthesis providing perspectives from medical education researchers. Researchers in medical

education are often the clinical faculty who focus research efforts on clinical learning. It is

95
important to note that clinical faculty were, in a previous time, students and residents themselves.

Thus, their perspectives on the learning environment may be more matured, but remain some

they recall their former roles and approach research from that perspective. The CLE instrument

analysis and synthesis adds value to this study by utilizing a second data set for which results can

be compared. The data from this subset informs the study by indicating alignment of synthesized

themes with the dimensions of learning and people. Collectively, these two data sets (sample set

and CLE instruments) crystallize the impressions from both the learner and the faculty primarily

on the dimension of people, and the perceived importance of the aspects of the community of

practice in the CLE.

Finally, the ACGME CLER Pathways to Excellence provides a third and unique

perspective to this data analysis. Initiated from a different stakeholder group, the ACGME data is

created by individuals further removed from the clinical setting, and who are responsible for

creating standards and regulations supporting the learning environment as an influential factor on

patient care delivery. While many of the individuals contributing to the ACGME data are also

physicians, not all are active clinicians, and many are in career roles working to satisfy the

demands of the public and legislators. Thus, the perspectives of the CLE by the ACGME provide

an important third perspective to the overall data set. The thematic synthesis of this data set

indicates alignment with organizational (work) outcomes, such as patient care outcomes.

Collectively, the three data sets allow for triangulation, validation of data, and integration of

diverse perspectives regarding the components of the CLE.

All three data sets inform the results in a variety of ways. For example, the patterns in the

data used to create categories and subcategories revealed that the three dimensions are not

separate from each other, but interrelated. Learning and people have cross-over relationships

96
focused on feedback, supervision and faculty support. People and change have cross-over

relationships focused on group cohesion, communication, and responsiveness to change. Thus

the people dimension influences both learning and change; however, learning and change were

not found in this analysis to have significant influence on each other. This finding may imply

that the people dimension is the most important of the three, as it resulted in the highest number

of frequency counts and is the only dimension emerging as an influence on both of the others.

This finding is important in the conceptualization of the CLE framework, as it implies that one

dimension may have more importance than another, or may affect another dimension either

positively or negatively. These relationships between dimensions, and related outcomes, are the

basis for considerations by management, considerations by educators, and for further research.

Implications for Management Practice

The hospital is a complex environment integrating work, learning, and patient care in a

single setting. Management and practitioners of medical education include a diverse group of

individuals who are stakeholders of the CLE. Management can include hospital chief executive

officers (CEOs), other hospital administrators or leaders, and medical education leaders such as

the designated institutional official (DIO). Although these are diverse management positions,

they share a common thread of strategic or operational responsibility for various aspects of

hospital operations. In a teaching hospital, those operational structures include the clinical

environment where learning and working are both supporting the primary mission of patient care

delivery.

This study is important to management who are accountable for patient care delivery. The

CLE conceptual framework provides a structure and common language that can be utilized by

managers to delineate the components of the CLE. What is known can be measured. As a

97
manager or leader, measurement is a key factor in assessment and improvement. The CLE

framework identifies and shows directional impact of components on various outcomes. Thus,

management can use this framework to inform decision-making and measurement of specific

components aligned with the outcomes for which they are focusing on improving. For example,

if a hospital president is concerned with clinical care outcomes, an initial area of focus may be in

the change dimension and factors contributing to the category of responsiveness to change.

Likewise, areas that cross over into this dimension from the people dimension include assessing

group cohesiveness and communication between practitioners. Fostering aspects of change,

including responsiveness and innovation, may have the most significant impact on organizational

outcomes such as patient care and clinical quality improvement.

This study is also important in deconstructing the clinical learning environment into the

two contexts of physical space and the social context. As a manager, many resources including

capital, human resources, and strategic efforts are focused on improvements related to the

physical plant and patient care aesthetics. Likewise, management fiduciary responsibility

requires decision-making to support resources such as those referenced in Table 14, such as

learning resources, compensation, and information technology support. As a financial decision-

maker, managers the CLE framework can inform leaders how decisions regarding resources may

impact the overall satisfaction of the CLE. When lack of revenue precludes investment in

physical aspects and resources, management can use the CLE framework to predict potential

impact on overall satisfaction of the learner and subsequent impact on the social context. While

aspects of the physical context are important to satisfaction, consideration of the social context in

the overall learning environment should be kept on the forefront of decision-making.

98
 The CLE framework illustrates how the three mediating factors of learning, people and

change influence the social context. The community of practice anchors the people dimension

and also emerges as the most important of the three dimensions to learning outcomes and

learning satisfaction. Managers can use these results and the CLE framework to formulate

strategies to proactively include students and residents into the community of practice, improve

meaningful relationships between faculty and students, and engage in activities to improve group

cohesion.

Finally, the CLE framework can be used by managers to better understand and appreciate

measures of the learning environment conducted by outside agencies, such as the ACGME.

Results from accreditation reviews, the ACGME CLER program, and related agencies provide

data often carrying the label of the clinical learning environment. The CLE framework provides

managers and leaders with a construct by which they can better appreciate, understand, and

transfer knowledge of these accreditation results into the context of the totality of the CLE.

Implications for Educators

The CLE framework also has important implications for medical educators. Most medical

educators are physician clinical faculty, but this group also includes professional educators,

medical school deans, and other health care professionals engaged in leading educational

curricula. Responsibility for the development, implementation, and execution of the curricula

falls to the clinical faculty and educators. The method and practice of teaching, referred to as

pedagogy, has unique aspects in the clinical setting as opposed to a traditional classroom

learning environment. The CLE framework provides important context to the educational

community in medicine.

99
 Educators can use the CLE framework to guide the development of a shared mental

model to serve as a basis for curricular design, teaching methods, and learner assessment.

Without this shared mental model, faculty opinions and assumptions of CLE aspects will be

varied and inconsistent. Incorporating the categories and subcategories of all three dimensions of

the social context into the curriculum allows faculty to create learning experiences consistent

with the goals they are trying to achieve. Aligning the curricular goals with individual learning

outcomes, or program outcomes, may allow educators to better inform structured teaching and

student experiences.

Educators should also be aware of the impact of hygiene factors on student satisfaction

and learning satisfaction. Often, resources and physical space are controlled by the hospital

management, while the curriculum and learning experiences may be controlled by the medical

school sending the students to the hospital (clinical) setting. When the hospital management

makes determination affecting the physical context and resources, dissatisfaction of lacking

resources can impact the students‘ perception of the learning environment as a whole, regardless

of the quality of the curriculum or the skillfulness of the faculty. In order to create well-

constructed clinical learning experiences, this underscores the importance of meaningful

integration between hospital managers and medical school leaders. In the absence of meaningful

integration and teamwork, unilateral management decisions can be detrimental to otherwise

robust efforts and delivery of clinical curricula.

Implications for Further Research

Finally, this study and the CLE framework informs researchers regarding a theoretical

basis for examining the CLE. The variation in CLE instruments reported in previous studies

(Colbert-Getz et al., 2014; Schonrock-Adema et al., 2012) can now be reassessed given a

100
framework for which existing measurement instruments can be evaluated. While this study

proposes a conceptual framework with defined dimensions and categories describing

components of the CLE, further research is required to validate these categories, determine

weighting, and examine how the interrelated nature of various aspects of the dimensions impact

other dimensions and CLE outcomes. Colbert-Getz et al. (2014) demonstrated significant

variation and lack of robust validity of the CLE instruments examined. This lack of validity of

current measurement instrumentation is a concern of managers and educators seeking to measure

the CLE in order to improve outcomes. The CLE conceptual framework provided in this study

can inform the development of new CLE instruments, or the revision of existing measures. The

use of theory and testing of the interpretive model should provide a more robust basis for the

development of instruments that are more heterogeneous in design.

A basic structure to the CLE and dimensional attributes provide a basis for further

research and examination by scholars. Additional qualitative studies eliciting perceptions of

various stakeholders regarding the CLE based on these dimensions may provide rich data for

further analysis. Additional stakeholders such as the patient could provide an important voice as

a component of the community of practice, and influential to perspectives of the CLE that are not

currently assessed by any of the three data sets used in this study.

The ACGME can also utilize this research to inform their efforts for the Clinical

Learning Environment Review (CLER) initiative. Recognizing the majority of data extracted and

coded from the CLER Pathways to Excellence document was aligned with the change dimension,

this study should prompt conversations and additional insight to the other aspects of the learning

environment important to accreditation. Perhaps the ACGME requirements and standards related

to operational accreditation may place more emphasis on the other two dimensions. However, if

101
the goal is to assess the CLE in its totality, then this study may inform those efforts and allow for

adjustment in data that is collected through the accreditation process.

Summary Observations

The clinical learning environment is comprised of both a physical context and a social

context. Both contexts contribute to the environment created where people learn, work and apply

knowledge. In the hospital setting, this complex and fascinating environment is rich with a broad

array of stakeholders who have both divergent and convergent priorities. Appreciating and

understanding the three dimensions of the social context allows one to think about the CLE in a

way that brings structure and definition to an otherwise vague concept. Learning, people, and

change are separate but interrelated dimensions that affect the social context of learning. The

community of practice provides an anchor in the people dimension, bringing together the

confluence of work, learning, faculty, students, residents, patients, and other members of the care

team. Both learning outcomes and organizational outcomes are influenced by these dimensions.

Although this research is an initial attempt at creating a conceptual framework for the

CLE, what has been learned from this research and subsequent analysis is critically important to

further knowledge within the medical education community. This research provides a basis for a

common narrative, shared mental model, and framework for discussion among scholars and

practitioners to develop further studies that will continue to define and inform exactly what

factors contribute to a CLE, and the significance of each of those factors. Deconstructing the

CLE into components that can be easily identified, potentially measured, and aligned with

outcomes is an important step in improving the learning environment for our physicians of today

and tomorrow.

102
 References

Accreditation Council for Graduate Medical Education, Department of Applications and Data

Analysis. (2014). ACGME data resource book. Retrieved from ACGME website:

http://www.acgme.org/acgmeweb/tabid/259/GraduateMedicalEducation/GraduateMedica

lEducationDataResourceBook.aspx

Accreditation Council for Graduate Medical Education. (2014). CLER Pathways to excellence:

Expectations for an optimal clinical learning environment to achieve safe and high

quality patient care. Retrieved from the ACGME website:

https://www.acgme.org/acgmeweb/Portals/0/PDFs/CLER/CLER_Brochure.pdf

Anderson, G. J., & Walberg, H. J. (1974). Learning environments. In A. Walberg (Ed.),

Evaluating educational performance: A sourcebook of methods, instruments, and

examples (pp. 81-98). Berkeley, CA: McCutchan Publishing Corp.

Asch, D. A., Nicholson, S., Srinivas, S., Herrin, J., & Epstein, A. J. (2009). Evaluating

obstetrical residency programs using patient outcomes. Journal of the American Medical

Association, 302(12), 1277-1283. doi:10.1001/jama.2009.1356.

Association of American Medical Colleges. (n.d.). AAMC statement on the learning

environment. Retrieved from the AAMC website:

https://www.aamc.org/initiatives/learningenvironment/

Association of American Medical Colleges. (2015). What does Medicare have to do with

graduate medical education? Retrieved from:

https://www.aamc.org/advocacy/campaigns_and_coalitions/gmefunding/factsheets/25337

2/medicare-gme.html

103
Barbour, R. S. (2001). Checklists for improving rigour in qualitative research: A case of the tail

wagging the dog? British Medical Journal, 322 (7294), 1115-1117.

doi:10.1136/bmj.322.7294.1115

Baron, R. M. & Kenny, D. A. (1986). The moderator-mediator variable distinction in social

psychological research: Conceptual, strategic, and statistical considerations. Journal of

Personality and Social Psychology, 51(6), 1173-1182. doi:10.1037//0022-

3514.51.6.1173

Benbassat, J. (2013). Undesirable features of the medical learning environment: A narrative

review of the literature. Advances in Health Science Education, 18(3), 527-536.

doi:10.1007/s10459-012-9389-5

Bellini, L., Hartmann, D., & Opas, L. (2010). Beyond must: Supporting the evolving role of the

Designated Institutional Official. Journal of Graduate Medical Education, 2(2), 147-150.

doi.org/10.4300/jgme-d-10-00073.1

Bernabeo, E., Holtman, M., Ginsburg, S., Rosenbaum, J., & Holmboe, E. (2011). Lost in

Transition: The Experience and Impact of Frequent Changes in the Inpatient Learning

Environment. Academic Medicine, 86(5), 591-598.

doi:10.1097/ACM.0b013e318212c2c9

Billet, S. (2001). Learning through work: Workplace affordances and individual engagement.

Journal of Workplace Learning, 13(5), 209-214. doi:10.1108/eum0000000005548

Bloom, B. S. (1964). Stability and change in human characteristics. New York, NY: Wiley

Publishers.

104
Boor, K. (2009). The clinical learning climate (Doctoral dissertation, de Vrije Universiteit

Amsterdam). Retrieved from

http://dspace.ubvu.vu.nl/bitstream/handle/1871/19579/8834.pdf?sequence=5

Boor, K., van der Vleuten, C., Teunissen, P., Scherpbier, A., & Scheele, F. (2011). Development

and analysis of D-RECT, an instrument measuring residents‘ learning climate. Medical

Teacher, 33, 820-827. doi:10.3109/0142159x.2010.541533

Cassar, K. (2004). Development of an instrument to measure the surgical operating theatre

learning environment as perceived by basic surgical trainees. Medical Teacher, 26(3),

260-264. doi:10.1080/0142159042000191975

Chan, D. (2002). Development of the Clinical Learning Environment Inventory: Using the

theoretical framework of learning environment studies to assess nursing students'

perceptions of the hospital as a learning environment. The Journal of Nursing

Education, 41(2), 69-75. doi:10.3928/01484834-20090828-06

Colbert-Getz, J. M., Kim, S., Goode, V. H., Shochet, R. B., & Wright, S. M. (2014). Assessing

medical students‘ and residents‘ perceptions of the learning environment: Exploring

validity evidence for the interpretation of scores from existing tools. Academic

Medicine, 89(12), 1687-1693. doi:10.1097/acm.0000000000000433

Combes, J. R. & Arespacochaga, E. (2012). Lifelong learning: physician competency

development. American Hospital Association Physician Leadership Forum. American

Hospital Association: Chicago, IL. Retrieved September 29, 2013 from:

http://www.ahaphysicianforum.org/files/pdf/physician-competency-development.pdf

105
Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997). Systematic reviews: Synthesis of Best

Evidence for Clinical Decisions. Annals of Internal Medicine, 126(5), 376-380.

doi:10.7326/0003-4819-126-5-199703010-00006

Contu, A., & Willmott, H. (2003). Re-embedding situatedness: The importance of power

relations in learning theory. Organization Science 14(3), 283-296.

doi:10.1287/orsc.14.3.283.15167

Cross, V., Hicks, C., Parle, J., & Field, S. (2006). Perceptions of the learning environment in

higher specialist training of doctors: implications for recruitment and retention. Medical

Education, 40(2), 121-128. doi:10.1111/j.1365-2929.2005.02382.x

Cruzes, D. S., & Dyba, T. (2011). Recommended steps for thematic synthesis in software

engineering. 2011 International Symposium on Emperical Software Engineering and

Measurement. doi:10.1109/esem.2011.36

Daelmans, H. E. M., Hoogenboom, R. J. I., Donker, A. J. M., Scherpbier, A. J. J. A., Stehouwer,

C. D. A., & van der Vleuten, C. P. M. (2004). Effectiveness of clinical rotations as a

learning environment for achieving competences. Medical Teacher, 26(4), 305-312.

doi:10.1080/01421590410001683195

Deketelaere, A., Kelchtermans, G., Struyf, E., & De Leyn, P. (2006). Disentangling clinical

learning experiences: an exploratory study on the dynamic tensions in internship. Medical

Education, 40(9), 908-915. doi:10.1111/j.1365-2929.2006.02551.x

Denyer, D. and Tranfield, D. (2009). Producing a systematic review. In D. Buchanan& A.

Bryman (Eds), The Sage Handbook of Organizational Research Methods (pp. 671-89).

London: Sage Publications Ltd. doi:10.1177/13505076100410030604

106
Denyer, D., Tranfield, D., & van Aken, J. E. (2008). Developing design propositions through

research synthesis. Organization Studies, 29(3), 393-413.

doi:10.1177/0170840607088020

DeYoung J. A. (1977). Classroom climate and class success: A case study at the university

level. The Journal of Educational Research, 70(5), 252-257.

doi:10.1080/00220671.1977.10884999

Dolmans, D. H. J., Wolfhagen, I. H. A., Heineman, E., & Scherpbier, A. J. J. (2008). Factors

adversely affecting student learning in the clinical learning environment: a student

perspective. Education for Health: Change in Learning & Practice, 21(3), 10p. Retrieved

from: http://www.educationforhealth.net/temp/EducHealth21332-5536254_152242.pdf

Dunn, L., & Becker, S. (2013, July 23). 50 Things to know about the hospital industry. Becker‟s

Hospital Review. Retrieved from: http://www.beckershospitalreview.com/hospital-

management-administration/50-things-to-know-about-the-hospital-industry.html

Eden, J., Berwick, D., & Wilensky, G. (2014). Graduate medical education that meets the

nation‟s health needs. Washington, DC: The National Academies Press. Retrieved from

the Institute of Medicine website: http://www.nap.edu/read/18754/chapter/1

Endler, N. S., Hunt, J. M., & Rosenstein, A. J. (1962). An S-R inventory of anxiousness.

Psychological Monographs: General and Applied, 76(17), 1-33. doi:10.1037/h0093817

Fraser, B. J. (2012). Classroom learning environments: Retrospect, context and prospect. Second

international handbook of science education, pp. 1191-1239. New York, NY: Springer.

doi:10.1007/978-1-4020-9041-7_79

107
Fraser, B. J. & Fisher, D. L. (1983). Use of actual and preferred classroom environment scales in

person-environment fit research. Journal of Educational Psychology, 75(2), 303-313.

doi:10.1037/0022-0663.75.2.303

Fraser, B. J., Treagust, D. F., & Dennis, N. C. (1986). Development of an instrument for

assessing classroom psychosocial environments at universities and colleges. Studies in

Higher Education, 11(1), 43-53. doi:10.1080/03075078612331378451

Gallagher, P., Carr, L., Weng, S.-H., & Fudakowski, Z. (2012). Simple truths from medical

students: Perspectives on the quality of clinical learning environments. Medical Teacher,

34(5), e332-e337. doi:10.3109/0142159X.2012.670738

Genn, J. M. (2001). AMEE Medical Education Guide No. 23 (Part 1): Curriculum, environment,

climate, quality and change in medical education—a unifying perspective. Medical

Teacher, 23(4), 337-344. doi:10.1080/01421590120063330

Genn, J. M. (2001). AMEE Medical Education Guide No. 23 (Part 2): Curriculum, environment,

climate, quality and change in medical education–a unifying perspective. Medical

Teacher, 23(5), 445-454. doi:10.3109/01421590109177942

Gerst, M. S., & Moos, R. H. (1972). Social ecology of university student residences. Journal of

Educational Psychology, 63(6), 513-525. doi:10.1037/h0033857

Gordon, J., Hazlett, C., ten Cate, O., Mann, K., Kilminster, S., Prince, K., O‘Driscoll, E., Snell,

L., & Newble, D. (2000). Strategic planning in medical education: enhancing the learning

environment for students in clinical settings. Medical Education, 34(10), 841-850.

doi:10.1046/j.1365-2923.2000.00759.x

Gough, D., Oliver, S., & Thomas, J. (2012). An introduction to systematic reviews. London,

England: Sage Publications. doi:10.1002/cjas.1231

108
Hagg-Martinell, A., Hult, H., Henriksson, P., & Kiessling, A. (2014). Students perceive

healthcare as a valuable learning environment when accepted as a part of the workplace

community. Education for Health, 27(1), 15-23. doi:10.4103/1357-6283.134296

Handley, K., Sturdy, A., Fincham, R., & Clark, T. (2006). Within and beyond communities of

practice: Making sense of learning through participation, identity and practice. Journal of

Management Studies, 43(3), 641-653. doi:10.1111/j.1467-6486.2006.00605.x

Harden, R. M. (2001). The learning environment and the curriculum. Medical Teacher, 23(4),

335-336. doi:10.1080/01421590120063321

Henning, M. A., Shulruf, B., Hawken, S. J., & Pinnock, R. (2011). Changing the learning

environment: the medical student voice. Clinical Teacher, 8(2), 83-87.

doi:10.1111/j.1743-498X.2011.00439.x

Herzberg, F. (1976). One More Time: How Do You Motivate Employees? Job Satisfaction — A

Reader, 17–32. doi:10.1007/978-1-349-02701-9_2

Higgins, J. P., & Green, S. (2008). Cochrane handbook for systematic review of interventions.

West Sussex, England: John Wiley & Sons, Ltd. doi:10.1002/9780470712184

Hodges, B. D., & Kuper, A. (2012). Theory and practice in the design and conduct of graduate

medical education. Academic Medicine, 87(1), 25-33.

doi:10.1097/acm.0b013e318238e069

Holt, K. D., Miller, R. S., Philibert, I., Heard, J., & Nasca, T. J. (2012). Residents' perspectives

on the learning environment: Data from the Accreditation Council for Graduate Medical

Education resident survey. Academic Medicine, 85(3), 512-518.

doi:10.1097/ACM.0b013e3181ccc1db

109
Holt, M. C., & Roff, S. (2004). Development and validation of the Anaesthetic Theatre

Eduational Environment Measure (ATEEM), Medical Teacher, 26(6), 553-558.

doi:10.1080/01421590410001711599

Hooven, K. (2014). Evaluation of instruments developed to measure the clinical learning

environment: An integrative review. Nurse Educator, 39(6), 316-320.

doi:10.1097/nne.0000000000000076

Insel, P. M., & Moos, R. H. (1972). The work environment inventory. Palo Alto, CA: Social

Ecology Laboratory, Stanford University.

Insel, P. M., & Moos, R. H. (1974). Psychological environments: Expanding the scope of human

ecology. American Psychologist, 29(3), 179-188. doi:10.1037/h0035994

Isba, R., & Boor, K. (2011). Creating a learning environment. In T. Dornan, K. Mann, A.

Scherpbier, & J. Spencer (Eds.), Medical education theory and practice (pp. 99-114).

Toronto, Ontario: Churchill Livingstone Elsevier. doi:10.1016B978-0-7020-3522-

7.00007-3

Jenson, H. B., Dorner, D., Hinchey, K., Ankel, F., Goldman, S. & Patow, C. (2009). Integrating

quality improvement and residency education: Insights from the AIAMC National

Initiative about the roles of the designated institutional official and program director.

Academic Medicine, 84(12), 1749-1756. doi:10.1097/acm.0b013e3181bf686f

Kanashiro, J., McAleer, S., & Roff, S. (2006). Assessing the educational environment in the

operating room—a measure of resident perception at one Canadian

institution. Surgery, 139(2), 150-158. doi:10.1016/j.surg.2005.07.005

Katz, D., & Kahn, R. (1966). The social psychology of organizations. New York, NY: Wiley

Publishers.

110
Kohn, L. T., Corrigan J. M., & Donaldson, M. S. (2000). To err is human: building a safer

health system. Washington, DC: National Academy Press.

Lave, J., & Wenger, E. (1991). Situated learning: Legitmate peripheral participation. New York,

NY: Cambridge University Press. doi:10.1017/cbo9780511815355.003

Lewin, K. (1936). Principles of topological psychology. New York, NY: McGraw Publishing.

Liaison Committee for Medical Education. (2015). Medical School Directory. Retrieved from:

http://www.lcme.org/directory.htm

Li, L. C., Grimshaw, J. M., Nielsen, C., Judd, M., Coyte, P. C., & Graham, I. D. (2009). Use of

communities of practice in business and health care sectors: A systematic review.

Implementation Science, 4(27). doi:10.1186/1748-5908-4-27

Lucas, P.J., Baird, J., Arai, L., Law, C., & Roberts, H.M. (2007). Worked examples of alternative

methods for the synthesis of qualitative and quantitative research in systematic reviews.

BMC Medical Research Methodology, 7(4), 1-7. doi:10.1186/1471-2288-7-4

Malling, B., Mortensen, L. S., Scherpbier, A. J., & Ringsted, C. (2010). Educational climate

seems unrelated to leadership skills of clinical consultants responsible of postgraduate

medical education in clinical departments. Medical Education,10(1), 62-69.

doi:10.1186/1472-6920-10-62

Mann, K. V. (2011). Theoretical perspectives in medical education: past experience and future

possibilities. Medical Education, 45(1), 60-68. doi:10.1111/j.1365-2923.2010.03757.x

Mann, K., Dornan, T., & Teunissen, (2012). Perspectives on learning. In T. Dornan, K. Mann, A.

Scherpbier, & J. Spencer (Eds.) Medical education theory and practice (pp. 17-38).

Toronto, Ontario: Churchill Livingstone Elsevier. doi:10.1016/B978-0-7020-3522-

7.00002-4

111
Marshall, R. E. (1978). Measuring the medical school learning environment. Journal of Medical

Education, 53, 98-104. doi:10.1097/00001888-197802000-00003

Mathers, J., Parry, J., Scully, E., & Popovic, C. (2006). A comparison of medical students‘

perceptions of their initial basic clinical training placements in ‗new‘ and established

teaching hospitals. Medical Teacher, 28(3), e80-e89. doi:10.1080/01421590600617392

Mays, N., & Pope, C. (2000). Assessing quality in qualitative research. British Medical Journal,

320 (7226), 50-52. doi:10.1002/9780470750841.ch8

Meade, M. O., & Richardson, W. S. (1997). Selecting and appraising studies for systematic

review. Annals of Internal Medicine, 127(7), 531-537. doi:10.7326/0003-4819-127-7-

199710010-00005

Medicare Payment Advisory Commission (2015, March). Report to the Congress: Medicare

Payment Policy, Chapter 3. Retrieved from

http://www.medpac.gov/documents/reports/chapter-3-hospital-inpatient-and-outpatient-

services-(march-2015-report).pdf?sfvrsn=0

Moos, R. H. (1968). The assessment of the social climates of correctional institutions. Journal of

Research in Crime and Delinquency, 5(2), 174-188. doi:10.1177/002242786800500207

Moos, R. H. (1972). Assessment of the psychosocial environments of community-oriented

psychiatric treatment programs. Journal of Abnormal Psychology, 79(1), 9-18.

doi:10.1037/h0032346

Moos, R. H. (1973). Conceptualizations of human environments. American Psychologist, 28(8),

652-665. doi:10.1037/h0035722

Moos, R. H. (1979). Evaluating educational environments. San Francisco: Jossey-Bass.

112
Moos, R. H., & Houts, P. S. (1968). Assessment of the social atmospheres of psychiatric

wards. Journal of Abnormal Psychology, 73(6), 595-604. doi:10.1037/h0026600

Moos, R. H., & Humphrey, B. (1973). Group environment inventory: Technical report. Palo

Alto, CA: Social Ecology Laboratory, Stanford University.

Müller, F. H., & Louw, J. (2004). Learning environment, motivation and interest: Perspectives

on self-determination theory. South African Journal of Psychology, 34(2), 169-190.

doi:10.1177/008124630403400201

Mulrooney, A. (2005). Development of an instrument to measure the practice vocational training

environment in Ireland. Medical Teacher, 27(4), 338-342.

doi:10.1080/01421590500150809

Mulrow, C. D. (1994). Rationale for systematic reviews. BMJ, 309, 597-599.

doi:10.1136/bmj.309.6954.597

Murray, H. (1938). Explorations in personality. New York, NY: Oxford University Press.

Nasca, T. J., Philibert, I., Brigham, T., & Flynn, T. C. (2012). The next GME accreditation

system – rationale and benefits. The New England Journal of Medicine, 366(11), 1051-

1056. doi:10.1056/nejmsr1200117

Nasca, T. J., Weiss, K. B., Bagian, J. P. (2014). Improving clinical learning environments for

tomorrow‘s physicians. The New England Journal of Medicine, 370(11), 991-993.

doi:10.1056/nejmp1314628

de Oliveira Filho, G. R., Vieira, J. E., & Schonhorst, L. (2005). Psychometric properties of the

Dundee Ready Educational Environment Measure (DREEM) applied to medical

residents. Medical Teacher, 27(4), 343-347. doi:10.1080/01421590500046387

113
Onwuegbuzie, A. J., & Collins, K. M. (2007). A typology of mixed methods sampling designs in

social science research. The Qualitative Report, 12(2), 281-316.

doi:10.1177/1558689807299526

Pace, R. (1969). College and university environment scales, 2nd edition. Princeton, NJ:

Educational Testing Service.

Padmore, J. S., Jaeger, J., Riesenberg, L. A., Karpovich, K. P., Rosenfeld, J.C., & Patow, C. A.

(2009). ―Renters‖ or ―owners‖? Residents‘ perceptions and behaviors regarding error

reduction in teaching hospitals: A literature review. Academic Medicine, 84(12), 1765-

1774. doi:10.1097/acm.0b013e3181bf51c1

Padmore, J. S., Karpovich, K. P., Shaver, M. L., & Richard, K. M. (2010). Fundamental aspects

of academic and employment law applied to graduate medical education programs. In J.

Levine (Ed.), Guide to medical education in the teaching hospital, 4th edition (pp. 149-

168). LaVergne, TN: Lightning Source.

Philibert, I. (2012). Satisfiers and hygiene factors: Residents' perceptions of strengths and

limitations of their learning environment. Journal of Graduate Medical Education, 4(1),

122-127. doi:10.4300/JGME-04-01-33

Philibert, I., Johnston, M. J., Hruska, L., Shapiro, J. A., Friedmann, P., Gardent, P. B., Batalden,

P. B., & Leach, D. (2010). Institutional attributes associated with innovation and

improvement: results of a multisite study. Journal of Graduate Medical Education, 2(2),

306-312. doi:10.4300/JGME-02-02-29

Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., & Britten, N. (2006).

Guidance on the conduct of narrative synthesis in systematic reviews. A product from the

114
 ESRC Methods Program. Retrieved from http://www.sphsu.mrc.ac.uk/research-

programmes/ev/methop/narr.html

Reed, D. A., Cook, D.A., Beckman, T. J., Levine, R. B., Kern, D. E., & Wright, S. M. (2007).

Association between funding and quality of published medical education research.

Journal of the American Medical Association, 298(9), 1002-1009.

doi:10.1001/jama.298.9.1002

Rentoul, A. J., & Fraser, B. J. (1979). Conceptualization of enquiry‐based or open classroom

learning environments. Journal of Curriculum Studies, 11(3), 233-245.

doi:10.1080/0022027790110306

Rice, S. (2010, July 8). Are hospitals deadlier in July? CNN‟s The Empowered Patient by

Elizabeth Cohen. Retrieved from:

http://www.cnn.com/2010/HEALTH/07/07/hospital.deaths.july/

Rieselbach, R., Sundwall, D., Shine, K., Epperly, T., & Crouse, B. (2015, August 12). Breaking

the graduate medical education policy logjam [Blog post]. Retrieved from

http://healthaffairs.org/blog/2015/08/12/breaking-the-graduate-medical-education-policy-

logjam/

Riesenberg, L. A., Rosenbaum, P. F., & Stick, S. (2006). Competencies, essential training, and

resources viewed by designated institutional officials as important to the position in

graduate medical education. Academic Medicine, 81(5), 426-431.

doi:10.1097/01.acm.0000222279.28824.f5

Roff, S. (20015). New resources for measuring educational environment. Medical Teacher,

27(4), 291-293. doi:10.1080/01421590500151047

115
Roff, S., & McAleer, S. (2001). What is educational climate? Medical Teacher, 23(4), 333-334.

doi:10.1080/01421590120063312

Roff, S., McAleer, S., Harden, R.M., Al-Qahtani, M., Ahmed, A. U., Deza, H., & Primparyon, P.

(1997). Development and validation of the Dundee ready education environment measure

(DREEM). Medical Teacher, 19(4), 295-299. doi:10.3109/01421599709034208

Roff, S., McAleer, S., & Skinner, A. (2005). Development and validation of an instrument to

measure the postgraduate clinical learning and teaching educational environment for

hospital-based junior doctors in the UK. Medical Teacher, 27(4), 326-331.

doi:10.1080/01421590500150874

Roff, S., McAleer, S., Harden, R. M., Al-Qahtani, M., Ahmed, A. U., Deza, H., Groenen, G., &

Primparyon, P. (1997). Development and validation of the Dundee ready education

environment measure (DREEM). Medical Teacher, 19(4), 295-299.

doi:10.3109/01421599709034208

Rotem, A., Godwin, P., Du, J. (1995). Learning in hospital settings. Teaching and Learning in

Medicine, 7(4), 211-217. doi:10.1080/10401339509539746

Rousseau, D. M. (Ed.). (2012). The Oxford handbook of evidence-based management. New

York, NY: Oxford University Press. doi:10.1111/peps.12058_1

Saarikoski, M., Isoaho, H., Leino-Kilpi, H., & Warne, T. (2005). Validation of the clinical

learning environment and supervision scale. International Journal of Nursing Education

Scholarship, 2(1). doi:10.2202/1548-923x.1081

Schonrock-Adema, J., Bouwkamp-Timmer, T., van Hell, E. A., & Cohen-Schotanus, J. (2012).

Key elements in assessing the educational environment: Where is the theory? Advances

in Health Science Education, 17(5), 727-742. doi:10.1007/s10459-011-9346-8

116
Schulman, L. S. (1970). Reconstruction of educational research. Review of Educational

Research, 40(3), 371-396. doi:10.2307/1169372

Seabrook, M. A. (2003). Medical teachers‘ concerns about the clinical teaching context. Medical

Education, 37(3), 213-222. doi:10.1046/j.1365-2923.2003.01437.x

Seabrook, M. A. (2004). Clinical students' initial reports of the educational climate in a single

medical school. Medical Education, 38(6), 659-669. doi:10.1111/j.1365-

2929.2004.01823.x

Shochet, R. B., Colbert-Getz, J. M., & Wright, S. M. (2015). The Johns Hopkins Learning

Environment Scale: Measuring medical students‘ perceptions of the process supporting

professional formation. Academic Medicine, 90(6), 810-818.

doi:10.1097/acm.0000000000000706

Sifferlin, A. (2013, March 26). Fewer hours for doctors in training leading to more mistakes.

Time. Retreived from: http://healthland.time.com/2013/03/26/fewer-hours-for-doctors-in-

training-leading-to-more-mistakes/

Soemantri, D., Herrera, C., & Riquelme, A. (2010). Measuring the educational environment in

health professions studies: A systematic review. Medical Teacher, 32(12), 947-952.

doi:10.3109/01421591003686229

Syptak, J. M., Marsland, D. W., & Ulmer, D. (1999). Job satisfaction: putting theory into

practice. Family Practice Management, 6(10), 26-31.

http://www.aafp.org/journals/fpm.html

Teunissen, P. W., Scheele, F., Scherpbier, A. J., van der Vleuten, C. P., Boor, K., van Luijk, S.

J., & van Diemen-Steenvoorde, J. A. (2009). How residents learn: Qualitative evidence

117
 for the pivotal role of clinical activities. Medical Education, 41(8), 763-770.

doi:10.1111/j.1365-2923.2007.02778.x

Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in

systematic reviews. BMC Medical Research Methodology, 8(45). doi:10.1186/1471-

2288-8-45

Thrush, C. R., Hicks, E. K., Tariq, S. G., Johnson, A. M., Clardy, J. A., O'Sullian, P. S., &

Williams, D. K. (2007). Optimal learning environments from the perspective of resident

physicians and associations with accreditation lengths. Academic Medicine, 82(10),

S121-S124. doi:10.1097/acm.0b013e318140658f

Tong, A., Flemming, K., McInnes, E., Oliver, S., Craig, J. (2012). Enhancing transparencey in

reporting of the synthesis of qualitative research: ENTREQ. BMC Medical Research

Methodology, 12(1), 181. doi:10.1186/1471-2288-12-181

Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-

informed management knowledge by means of systematic review. British Journal of

Management, 14, 207-222. doi:10.1111/1467-8551.00375

Traub, R. E., Weiss, J., Fisher, C. W., & Musella, D. (1972). Closure on openness: Describing

and quantifying open education. Interchange, 3(2-3), 69-84. doi:10.1007/bf02137636

Trickett, E. J., & Moos, R. H. (1973). Social environment of junior high and high school

classrooms. Journal of Educational Psychology, 65(1), 93-102. doi:10.1037/h0034823

Truven Health Analytics. (2015, March 2). 100 Top Hospitals: Study Overview, 22nd edition.

Retrieved September 26, 2015 from:

http://100tophospitals.com/Portals/2/assets/TOP%2015313%200315%20100%20Top%2

0Study_web.pdf

118
Walberg, H. J. (1969). Social environment as a mediator of classroom learning.Journal of

Educational Psychology, 60(6 Part 1), 443-448. doi:10.1037/h0028499

Walberg, H. J. (1978). Educational environments and effects: Evaluation and policy. Chicago,

IL: McCutchan.

Wenger, E., McDermott, R., & Snyder, W.M. (2002). Cultivating communities of practice. A

guide to managing knowledge. Cambridge, MA: Harvard Business School Press.

doi:10.5465/amle.2009.41788855

Weiss, K. B., Bagian, J. P., & Nasca, T. J. (2013). The clinical learning environment: the

foundation of graduate medical education. Journal of the American Medical Association,

309(16), 1687-1688. doi:10.1001/jama.2013.1931

Yudkowsky, R., Elliott, R., & Schwartz, A. (2002). Two perspectives on the indicators of quality

in psychiatry residencies: program directors‘ and residents. Academic Medicine, 77(1),

57-64. doi:10.1097/00001888-200201000-00014

119
Appendices

120
Appendix A

Summary of Medical Education Environment Instruments Used to Study the Clinical Learning
Environment

Author(s) Instrument Study No. Items, Sub-Scales (N items)

Population No. Subscales
Boor (2009) Dutch Residency 1278 50 items 1. Supervision
Educational Climate Residents 11 subscales 2. Coaching and assessment
Test (D-RECT) 3. Feedback
4. Teamwork
5. Peer collaboration
6. Professional relations between attending
physicians (faculty)
7. Work adapted to resident‘s competence
8. Attendings‘ role
9. Formal education
10. Role of the specialty tutor
11. Patient sign-out

Cassar (2004) Surgical Theatre 25 Surgical 40 items 1. Perceptions of trainer and training
Educational Residents 4 subscales 2. Perceptions of learning opportunities
Environment Measure 3. Perceptions of the atmosphere in the
(STEEM) operating room
4. Perceptions of supervision, workload and
support (8)

Holt and Roff Anaesthetic Theatre 218 40 items 1. Autonomy

(2004) Educational Anesthesia 5 subscales 2. Perceptions of atmosphere
Environment Measure Residents 3. Workload/supervision/support
(ATEEM) 4. Perceptions of teaching and teachers
5. Learning opportunities and orientation to
learning

Keitz (2003) Learners‘ Perceptions 1775 57 items 1. Learning environment

Survey Residents 4 subscales 2. Faculty and Preceptors
3. Work environment
4. Physical environment

Marshall Medical School 93 First Year 50 items 5. Breadth of interest

(1978) Learning Environment Medical 7 subscales 6. Student interaction
Survey (MSLES) Students 7. Organization (Goal direction)
8. Flexibility (authoritarianism)
9. Meaningful learning experience
10. Emotional climate
11. Nurturance

Mulrooney Practice-Based 48 Residents 37 items 1. The practice job

(2005) Educational 4 subscales 2. GP trainer
Environment Measure 3. Teaching and learning
(PEEM) 4. Interaction with other health care
professionals

Roff (2005) Postgraduate Hospital 97 Residents 40 items 1. Perceptions of role autonomy

Educational 3 subscales 2. Perceptions of teaching
Environment Measure 3. Perceptions of social support
(PHEEM)

121
Roff (1997) Dundee Ready 490 medical 50 items 1. Perceptions of teaching
Education students; 256 5 subscales 2. Perceptions of teachers
Environment Measure nursing 3. Academic self-perception
(DREEM) students 4. Perceptions of atmosphere
5. Social self-perception

Rotem (1995) Survey of Learning in 209 residents 46 items 1. Autonomy

Hospital Settings 8 subscales 2. Supervision
(SLHS) 3. Social Support
4. Workload
5. Role clarity
6. Variety
7. Orientation to learning and teaching
8. Orientation to general practice

Oliveira Filho DREEM for Residents 97 residents 50 items 1. Teachers

(2005) 5 subscales 2. Teaching
3. Academic self perception
4. Atmosphere
5. Social self perception

Kanashiro Operating Room 23 Residents 40 items 1. Teaching and training

(2006) Educational 4 subscales 2. Learning opportunities
Environment Measure 3. Atmosphere
(OREEM) 4. Workload/supervision/support

Note. Adapted from ―Key elements in assessing the educational environment: where is the theory?‖ by J. Sconrock-Adema, T. Bouwkamp-
Timmer, E. van Hell, and J. Cohen-Schotanus, 2012, Advances in Health Science Education, 17, p. 733-734. Copyright 2012 by Springer. Also
adapted from ―Creating a learning environment‖ by R. Isba and K. Boor, 2011, in Medical education theory and practice (T. Dornan, K. Mann,
A. Scherpbier, & J. Spencer, Eds), p. 108-111, Toronto, Ontario: Churchill Livingstone. Copyright 2011 by Elsevier Ltd.

122
 Appendix B

Literature Search Keywords and Boolean Operators

ENVIRONMENT (OR) SETTING (OR) POPULATION (OR) FACTORS (OR)

clinical learning environment clinic medical student* nursing
CLE clerkship resident* simulation
learning environment hospital intern clinic
learning culture practice fellow nursing home
learning climate rotation house officer
learning atmosphere clinical placement HEALTH occupations students
AND AND NOT
educational environment internship house staff
educational culture fellowship
educational climate residency
educational atmosphere medical
clinical environment
clinical culture
workplace learning
learning environment, clinical
education environment
learning organization

123
 Appendix C

MeSH Subject Headings Utilized in PubMed Medline Search

ENVIRONMENT (OR) SETTING (OR) POPULATION (OR) FACTORS (OR)

Workplace Clinical Clerkship Education, Medical, nursing
Learning Internship and Residency Undergraduate simulation
Hospitals, Teaching Students, Medical clinic
AND AND NOT
Schools, Medical nursing home
Education, Medical,
Graduate
Education, Medical
Internship and Residency
Education, Medical,
Continuing
Faculty, Medical
Preceptorship

124
Appendix D

Search String Used to Conduct Systematic Review

((((((((((((((education, medical[MeSH Terms]) OR (internship and residency[MeSH Terms]))

OR medical student[MeSH Terms]) OR medical school[MeSH Terms]) OR preceptorship[MeSH
Terms]) OR faculty, medical[MeSH Terms]) OR medical student*[Title/Abstract]) OR
intern*[Title/Abstract]) OR resident*[Title/Abstract]) OR fellow*[Title/Abstract]) OR house
officer[Title/Abstract]) OR house staff[Title/Abstract]))) AND ((((((((((clinic*[Title/Abstract])
OR clinical clerkship[MeSH Terms]) OR education, graduate medical[MeSH Terms]) OR
resident*[Title/Abstract]) OR (internship and residency[MeSH Terms])) OR hospitals,
teaching[MeSH Terms]) OR clerkship[Title/Abstract]) OR clinical placement[Title/Abstract]))
AND ((((((((((((((((((learning environment[Title/Abstract]) OR learning culture[Title/Abstract])
OR learning atmosphere[Title/Abstract]) OR learning climate[Title/Abstract]) OR educational
environment[Title/Abstract]) OR educational culture[Title/Abstract]) OR educational
atmosphere[Title/Abstract]) OR educational climate[Title/Abstract]) OR clinical
environment[Title/Abstract]) OR clinical culture[Title/Abstract]) OR clinical
atmosphere[Title/Abstract]) OR clinical climate[Title/Abstract]) OR workplace
learning[Title/Abstract]) OR education environment[Title/Abstract]) OR education
culture[Title/Abstract]) OR education atmosphere[Title/Abstract]) OR education
climate[Title/Abstract]) OR learning organization[Title/Abstract]))

125
Appendix E

Medical Education Research Study Quality Instrument (MERSQI)

Domain MERSQI Item Item Score Maximum

Domain Score
Study design 1. Study design 3
Single group cross-sectional or single group post-test only 1
Single group pre-test and post-test 1.5
Nonrandomized, 2 group 2
Randomized controlled trial 3

Sampling 2. No. of institutions cited 3

1 0.5
2 1
>2 1.5
3. Reponse rate, %
Not applicable
< 50 or not reported 0.5
50-74 1
> 75 1.5

Type of data 4. Type of data 3

Assessment by study participant 1
Objective measurement 3

Validity of evaluation 5. Internal structure 3

instrument
Not applicable
Not reported 0
Reported 1
6. Content
Not applicable
Not reported 0
Reported 1
7. Relationships to other variables
Not applicable
Not reported 0
Reported 1

Data analysis 8. Appropriateness of analysis 3

Data analysis inappropriate for study design or type of data 0
Data analysis appropriate for study design or type of data 1
9. Complexity of analysis
Descriptive analysis only 1
Beyond descriptive analysis 2

Outcomes 10. Outcomes 3

Satisfaction, attitudes, perceptions, opinions, general facts 1
Knowledge, skills 1.5
Behaviors 2
Patient/health outcomes 3
Total Score 18

126
 Appendix F

Subject Matter Expert Feedback Summary

Validations:
 The topic is relevant, timely, and of critical importance to practitioners and scholars
 Evidence synthesis is welcome and beneficial to this topic
 Excellent nexus between education and management

Recommendations:
 Be sure to include the ACGME CLER process, or the Pathways to Excellence document,
in the data set for analysis
 Exclude studies from the systematic review that utilize a survey or instrument. Since the
premise of the study is that the instruments are flawed, then the data from studies
utilizing CLE measurement instruments would also be flawed. Figure out a way to
include instrument data but exclude studies utilizing them.
 Stick to qualitative studies of the CLE
 Include literature on workplace and practice-based learning
 Must include organizational environments and organizational culture literature
 Clear specification and rationale of why you are including both medical students and
residents, since they are different populations
 Variability in consensus regarding proposed fourth dimension representative of patients
 Variability in feasibility regarding the a second research question on the role of
leadership in the CLE, and availability of evidence
 Include Wegner‘s work on community of practice

Other comments:
 Literature from Canada and Europe will be important to inform this study
 Recommendation of the study by Colbert-Getz et al. (2014)

127
 Appendix G

Sample Set of Studies Meeting Inclusion and Exclusion Criteria

128
Appendix G.

Studies Meeting Inclusion and Exclusion Criteria

First Author Year Journal Title Study Design Population Location Source MERSQI No. No. Study Objective Key Findings
or Instrument Score Institutions Subjects

Qualitative Studies

Bernabeo 2011 Academic Lost in transition: the Qualitative; Residents United States PubMed 6.5 3 97 To examine the experience Frequent transitions
Medicine experience and Focus Group Medline and impact of frequent contributed to a lack of
impact of frequent transitions on residents ownership and other
changes in the potentially harmful effects
inpatient learning for patient care.
environment

Boor 2009 Thesis The clinical learning Mixed methods Both Netherlands ProQuest 7 N/R N/R Dissertation on the
environment measurement of the CLE

Cross 2006 Medical Perceptions of the Qualitative; Residents United PubMed 6 1 46 To examine perceptions of Workplace learning models
Education learning environment Interviews Kingdom Medline the clinical learning influences professional
in higher specialist environment to inform identity
training of doctors: attributes and dispositions
implications for relevant to specialist practice
recruitment and and recruitment
retention

Daelmans 2004 Medical Effectiveness of Survey Students Netherlands PubMed 7 1 81 To examine the occurance of Conditions for adequate
Teacher clinical rotations as a research Medline supervision, feedback and learning in the clinical
learning environment assessment in clinical setting are poorly met and
for achieving experiences clerkship experiences
competence show huge inter-student
variation

Deketelaere 2006 Medical Disentangling clinical Qualitative; Students Belgium Hand 7.5 2 56 To explore and identify the 5 components constitute
Education learning Focus Group Searching learning experiences in the clinical learning
experiences: an clinical practice and the experience: agenda of the
exploratory study on interactions between the internship, attitude of the
the dynamic characteristics of the learner supervisor, culture of the
tensions in and the training setting setting, the intern's
internship learning attitude, and the
nature of the learning
process.

129
First Author Year Journal Title Study Design Population Location Source MERSQI No. No. Study Objective Key Findings
or Instrument Score Institutions Subjects

Dolmans 2008 Education Factors adversely Survey Students Netherlands PubMed 10.5 1 1425 To investigate factors that Clinical material (patients)
for Health affecting student research Medline may adversely affect student should be diverse and
learning in the learning in the clinical students should engage
clinical learning environment. independently and under
environment: a supervision; continuity of
student perspective teaching/supervision is
important for learning.

Gallagher 2012 Medical Simple truths from Qualitative; Students New Zealand PubMed 7 3 30 To identify key conditions that Four themes identified: (1)
Teacher medical students: Focus Group Medline students value most in clinical Structural factors or the
perspectives on the settings organization of the
quality of clinical assignment; (2)
learning Interpersonal factors or the
environments spectrum of support
ranging from staff to peers;
(3) Intrapersonal factors or
the pro-activeness,
preparedness and
personality of students;
and (4) Vocational
development opportunities

Gordon 2000 Medical Strategic planning in Qualitative; Faculty Australia PubMed 6.5 5 N/R To identify issues raised in Four major issues emerge
Education medical education: Focus Group Medline the current CLE and how they as most likely to improve
enhancing the might be addressed to clinical education: (1)
learning environment promote more effective Integration of the learner;
for students in learning in the clinical setting (2) Equipping learners with
clinical settings survival skills; (3) Better
use of the environment
and resources for learning;
and (4) Expertise in using
information technology to
enhance clinical learning

Hagg- 2014 Education Students perceive Qualitative; Students Sweden PubMed 7 1 406 To identify students' views of Three themes were
Martinell for Health healthcare as a questionnaire Medline aspects of the environment identified:
valuable learning that influences their progress Management/planning/org
environment when toward competence. anizing for learning;
accepted as a part of workplace culture; and
the workplace learning a profession
community

130
First Author Year Journal Title Study Design Population Location Source MERSQI No. No. Study Objective Key Findings
or Instrument Score Institutions Subjects

Henning 2011 Clinical Changing the Qualitative; Students New Zealand Hand 6 1 276 To assess the learning Commonalities included
Teacher learning questionnaire Searching environment in order to plan earlier clinical exposure,
environment: the for the future fewer lectures, more
medical student consistent assessment,
voice and more constructive
relationships. Resources,
clarification of roles and
learning outcomes were
also raised.

Mathers 2006 Medical A comparison of Qualitative; Students United States PubMed 7.5 4 33 To examine and identify Most factors influencing
Teacher medical students' Focus Group Medline factors influencing students' student experience related
perceptions of their learning during initial hospital to orienting the new
intial basic clinical placements and if there are learner. Additionally,
training placements differences between new and clinical demand/workload
in 'new' and established hospitals was identified.
established teaching
hospitals

Philibert 2012 Journal of Satisfiers and Qualitative; Residents United States PubMed 10 N/R N/R To determine common Some program attributes
Graduate hygiene factors: consensus lists Medline dimensions in learner are mentioned only when
Medical residents' perceptions of strengths and absent (hygiene factors),
Education perceptions of opportunities of program to and others are more
strengths and assess if ACGME survey associated with perception
limitations of their captures all areas of quality of learning
learning environment environment; Major
Themes (1) Interaction with
Faculty (2) Clinical Volume
and Variety (3)Supervision
and Autonomy; (4)
Evaluations, and (5)
Resources

Philibert 2010 Journal of Institutional Qualitative; Mixed United States PubMed 8 9 N/R To identify observations on Three themes were
Graduate attributes associated Interviews Medline processes and common identified: a culture of
Medical with innovation and attributes of the LE and to integration and inclusion;
Education improvement: results explore if the current recognition of the value of
of a multisite study accreditation model may resident education to the
present barriers to innovation institution; and use of data
for ongoing change,
improvement and
innovation.

131
First Author Year Journal Title Study Design Population Location Source MERSQI No. No. Study Objective Key Findings
or Instrument Score Institutions Subjects

Seabrook 2004 Medical Qualitative; Students United PubMed 6 1 41 To identify medical student Six main themes emerged:
Education Interviews, Kingdom Medline perceptions of the clinical (1) Teaching/learning
focus groups learning environment early in atmosphere; (2)Course
their clinical exposure cohesion, (3) Doctors as
teachers, (4) Teaching
structure, (5) Student
perspectives, and (6)
Teaching practice

Thrush 2007 Academic Mixed Residents United States PubMed 8 1 392 To determine if there is an Quality of faculty most
Medicine methods; Medline association between program strongly associated with
Open-ended quality and accreditation longer accreditation cycle
questions length lengths. Exposure to
patients, education and the
social environment are the
most prevalent factors.

132
 Appendix H

Clinical Learning Environment Measurement Instruments and Subscales

Coded to Social Dimensions

133
Appendix H

Clinical Learning Environment Measurement Instruments and Subscales Coded to Social Dimensions

Scales Classified to Moos’ Domains

Author(s) Instrument Study No. Items, Personal Development & Relationships System Maintenance Outlier
Population No. Subscales Goal Orientation and System Change

Boor (2009) Dutch Residency 1278 Residents 50 Items Coaching and assessment Supervision (S) Work adapted to
Educational Climate Test 11 Subscales (F) residents‘ competence
(D-RECT) Peer collaboration (GC) (IND)
Feedback (F)
Professional relations
Formal education (C) between faculty (GC)
Patient sign-out (LO) Teamwork (GC)
Faculty members‘ roles
(FS)
Role of the specialty
tutor (FS)

Cassar (2004) Surgical Theatre 25 Surgical 40 Items Perceptions of learning Perceptions of Perceptions of the
Educational Environment Residents 4 Subscales opportunities (LO) supervision, workload atmosphere in the
Measure (STEEM) and support (S) operating room (O)
Perceptions of trainer
and training (FS)
Holt and Roff Anaesthetic Theatre 218 Residents 40 Items Learning opportunities Perceptions of teaching Autonomy (IND)
(2004) Educational Environment 5 Subscales and orientation to learning and teachers (FS)
Measure (ATEEM) (LO) Perceptions of
Supervision/support (S) atmosphere (O)
Kanashiro et al. Operating Room 23 Residents 40 Items Learning opportunities Teaching and training Atmosphere (O)
(2006) Educational Environment 4 Subscales (LO) (FS)
Measure (OREEM)
Supervision/support (S)
Keitz et al. Learners‘ Perceptions 1775 Residents 57 Items Learning environment Faculty and Preceptors Working Environment Physical
(2003) Survey 4 Subscales (LO) (FS) (O) Environment

Marshall (1978) Medical School Learning 93 Medical 50 items Breadth of interest (PD) Student interaction (GC) Flexibility (R)
Environment Survey Students 7 subscales (authoritarianism)
(MSLES) Meaningful learning Emotional climate (GC)
experience (LO) Organization (O)
Nurturance (GC)

134
 Scales Classified to Moos’ Domains

Author(s) Instrument Study No. Items, Personal Development & Relationships System Maintenance Outlier
Population No. Subscales Goal Orientation and System Change

Mulrooney Practice-Based 48 Residents 37 Items Teaching and learning GP trainer (TS) The practice job (O)
(2005) Educational Environment 4 Subscales (LO)
Measure (PEEM) Interaction with other
health care professionals
(GC)
Oliveira Filho DREEM for Residents 97 Residents 50 Items Academic self perception Teachers (TS) Atmosphere (O)
(2005) 5 Subscales (PD)
Teaching (TS)
Social self perception
Roff (2005) Postgraduate Hospital 97 Residents 40 Items N/A Perceptions of teaching Perceptions of role
Educational Environment 3 Subscales (TS) autonomy (IND)
Measure (PHEEM)
Perceptions of social
support (GC)
Roff (1997) Dundee Ready Education 490 medical 50 Items Academic self-perception Perceptions of teaching Perceptions of
Environment Measure students; 256 5 Subscales (PD) (TS) atmosphere (O)
(DREEM) nursing
students Perceptions of teachers
(FS)
Social self-perception
(GC)
Rotem (1995) Survey of Learning in 209 residents 46 Items Orientation to learning Social Support (GC) Autonomy (IND)
Hospital Settings (SLHS) 8 Subscales and teaching (PD)
Supervision (S) Role clarity (CE)
Orientation to general
practice (PD) Variety (O)
Workload (O)

Note. Adapted from “Key elements in assessing the educational environment: where is the theory?” by J. Sconrock-Adema, T. Bouwkamp-Timmer, E. van Hell, and J. Cohen-Schotanus, 2012, Advances
in Health Science Education, 17, p. 733-734. Copyright 2012 by Springer. Also adapted from “Creating a learning environment” by R. Isba and K. Boor, 2011, in Medical education theory and practice
(T. Dornan, K. Mann, A. Scherpbier, & J. Spencer, Eds), p. 108-111, Toronto, Ontario: Churchill Livingstone. Copyright 2011 by Elsevier Ltd. Code abbreviations: Learning Opportunities (LO),
Feedback/Asessment (F), Curriculum/Learning Objectives (C), Personal Development (PD), Supervision (S), Faculty Support (FS), Group Cohesion/Trust (GC), Communication (Co), Orderly Work
Environment (O), Clear Expectations (CE), Responsive to Change (RC), Individualization (IND).

135
 Appendix I

ACGME CLER Pathways to Excellence Coded to Social Dimensions

136
 137

Appendix I

ACGME CLER Pathways to Excellence Coded to Social Dimensions

Pathway Properties Classified to Domains

ACGME CLER Pathway Pathway Descriptor Learning People Change & Adaptation Outlier
Focus Area

Patient Safety PS-1: Reporting of Robust reporting system Knows how to report Faculty members report Knows roles and
Adverse Events as a foundation for patient safety events (LO) (C) responsibilities (CE)
identifying patient safety Residents/fellows report Safety events are centrally
(C) captured (O)

PS-2: Education on Educational activities
Patient Safety creating a shared mental
model
Receive patient safety
education specific to site
(LO)
Receive education on
proactive risk assessment
(LO)
Collaborative program Systems-based challenges
development (C) are presented and system
Faculty are proficient change is discussed (RC)
(FS)

PS-3: Culture of Safety The culture/system of Perception of a supportive

the hospital brings about culture (RC)
change and Emotional support to those
improvement involved in patient safety
events (RC)
Culture of Safety surveys
administered and acted upon
(RC)
PS-4: Resident Feedback and Feedback provided on Team involvement with System processes for
Experience in Patient experiential learning safety event reports (F) safety investigations (C) providing feedback and
Safety Investigations and describing outcomes (O)
Follow-up

PS-5: Clinical Site Participation in patient Hospital monitors individual

Monitoring of Resident safety is vital to learning reporting (RC)
Engagement in Patient and clinical care Hospital uses resident safety
Safety reports to implement
improvements (RC)

137
 138

Pathway Properties Classified to Domains

ACGME CLER Pathway Pathway Descriptor Learning People Change & Adaptation Outlier
Focus Area

PS-6: Clinical Site Faculty participation in Hospital monitors faculty

Monitoring of faculty safety is essential to the reporting (RC)
engagement in patient improvement of care Hospital uses faculty safety
safety reports to implement
improvements (RC)
PS-7: Resident Disclosure of an error to Residents receive
experience in disclosure a patient or family disclosure training (LO)
of safety events member is an important
skill Residents are involved in
disclosure of safety events
to patients and families
(LO)
Health Care HQ-1: Education on Education creates a Residents receive Program is developed Residents and faculty engage
Quality Quality Improvement shared mental model to progressive education and collaboratively with in activities of systems-based
achieve improvement experiential training (LO) team (C) challenges (RC)
Faculty proficiency Residents and faculty are
(FS) familiar with hospital‘s
priorities (CE)
Residents engaged in
systems-based activities
(RC)
HQ-2: Resident Experiential learning is Residents are actively
Engagement in QI necessary for systems- involved in QI activities
Activities based changes (RC)

HQ-3: Residents receive Access to data is Residents receive specialty-

data on quality metrics essential for specific data related to their
improvement patients (IND)

HQ-4: Resident Systems improvement Resident engagement in Hospital monitors individual

engagement in planning requires knowledge of department and hospital involvement (IND)
for QI the entire cycle of QI team committees (C)

138
 139

Pathway Properties Classified to Domains

ACGME CLER Pathway Pathway Descriptor Learning People Change & Adaptation Outlier
Focus Area

HQ-5: Resident and Formal education Residents and fellows Residents and faculty are
faculty education on needed for coordinated receive education on familiar with hospital‘s
reducing health approach to individual identification and priorities (CE)
disparities patient needs reduction of health
disparities (LO)
Residents and faculty
receive training in cultural
competency (LO)
HQ-6: Resident Experiential learning is Resident engagement in
Engagement in Clinical needed to identify and QI activities addressing
Site Initiatives to sustain systems-based health disparities (C)
Address Health changes
Disparities

Care Transitions CT-1: Education on Care Formal education creates Residents participate in Residents and faculty know
Transitions a shared mental model interprofessional the hospital‘s policies and
experiences (C) procedures (CE)
Faculty participate in
interprofessional
experiences (C)
CT-2: Resident Standardization is a Process involves Residents use a common
Engagement in Change prerequisite for safe interprofessional process (CE)
of Duty Handoffs patient care team/staff (C)
Process involves
patients and families (C)
CT-3: Resident and Standardization is a Process involves Residents use a standard
faculty engagement in prerequisite for safe interprofessional process (CE)
patient transfers between patient care team/staff (C)
services and locations
Residents participation
with leadership to
develop strategies (C)
CT-4: Faculty Evaluation through Residents receive Faculty monitor care
Engagement in Assessing direct observation assessment and feedback transfers (S)
Resident Transitions of (F)
Care

139
 140

Pathway Properties Classified to Domains

ACGME CLER Pathway Pathway Descriptor Learning People Change & Adaptation Outlier
Focus Area

CT-5: Residents and Verbal communication Residents and faculty

Faculty Engage in practices communicate directly
Communication Between (Co)
Primary and Consulting
Teams

CT-6: Clinical Site Periodic monitoring is Leadership monitors Leadership involves faculty
Monitoring of Care essential to enhance transitions of care and and PDs to make
Transitions patient care provides feedback (F) improvements (RC)

Supervision S-1: Education on Activities creating a Residents and faculty Clear expectations regarding
Supervision shared mental model for receive education on how supervision and progressive
supervision are needed to provide effective autonomy (CE)
for safe patient care supervision (LO)

S-2: Resident perception Eliciting resident Perception of adequate Supportive culture (RC)
of adequacy of perceptions of supervision (S)
supervision supervision is an
indicator of compliance

S-3: Faculty perception Eliciting faculty Perception of adequate Supportive culture (RC)
of adequacy of resident perceptions of supervision (S)
supervision supervision is an
indicator of compliance

S-4: Roles of clinical Non-physician staff Active role and Clinical staff aware of
staff in resident awareness of supervision involvement in ensuring supervision expectations and
supervision is essential to patient supervision is followed progressive autonomy of
safety (S) residents (CE)
Supportive culture (RC)

140
 141

Pathway Properties Classified to Domains

ACGME CLER Pathway Pathway Descriptor Learning People Change & Adaptation Outlier
Focus Area

S-5: Patients and Patients and family Patients and family

families, GME members must be aware members can identify
supervision of roles/responsibilities faculty and residents by
and access to physicians name (Co)
providing care
Patients and family
members have adequate
contact with faculty in
charge of care (Co)
Patients and family
members have adequate
contact with resident
team (Co)
S-6: Clinical site Organizational Site leadership monitors Site leaders provide data to
monitoring of resident monitoring of supervision and physicians and clinical staff
supervision and workload supervision and workload and provides regarding level of
workload is essential to feedback (S) supervision required of
identifying residents (CE)
vulnerabilities and
assuring patient safety

Duty DF-1: Culture of honesty Candid reporting is Perception of honest

Hours/Fatigue in reporting duty hours required for prevention reporting of duty hours
Management & of harm (GC)
Mitigation
(FMM)

DF-2: Resident and Formal education Resident and faculty

faculty education on activities create a shared awareness of strategies (CE)
fatigue and burnout mental model for
consistent safety

DF-3: Resident Resident perceptions of Resident use of FMM Resident perception of

engagement in fatigue fatigue mitigation strategies (LO) institution‘s culture
management and strategies and support of supporting FMM (RC)
mitigation institution
Resident perception of
program‘s culture supporting
FMM (RC)

141
 142

Pathway Properties Classified to Domains

ACGME CLER Pathway Pathway Descriptor Learning People Change & Adaptation Outlier
Focus Area

DF-4: Faculty Faculty perceptions of Faculty perception of

engagement in fatigue fatigue mitigation institution‘s culture
management and strategies and support of supporting resident fatigue
mitigation institution (RC)
Faculty perception of
institution‘s culture
supporting faculty fatigue
(RC)
Faculty exercise non-
judgmental triggering of
FMM (RC)
Program directors conduct
active surveillance of FMM
strategies (RC)
DF-5: Clinical site Periodic monitoring is Administrative leadership
monitoring of fatigue and essential to identifying monitors for resident and
burnout vulnerabilities and faculty fatigue and burnout
enhancing patient safety and implement mitigation
strategies (RC)
Professionalism PR-1: Resident and Formal education creates Residents and faculty receive
faculty education on a shared mental model education and know
professionalism of professionalism expectations regarding
contributing to high professionalism (CE)
quality patient care
Residents and faculty receive
training on policies and
procedures regarding
documentation in EMR (CE)

142
 143

Pathway Properties Classified to Domains

ACGME CLER Pathway Pathway Descriptor Learning People Change & Adaptation Outlier
Focus Area

PR-2: Resident attitudes Attitudes, beliefs and Resident perception of Resident awareness and use
beliefs and skills related skills regarding environment of of policies and processes for
to professionalism professionalism impacts professionalism reporting mistreatment (CE)
quality and safety of supports honesty,
patient care integrity and respect Faculty member and nurse
(GC) perception of resident
awareness of processes (CE)
Residents follow EMR
policies and guidelines (CE)
Faculty perceive residents
follow EMR policies and
guidelines (CE)
Residents acknowledge
professional responsibility
(CE)
PR-3: Faculty Faculty engagement in Faculty and program Faculty are aware of and use
engagement in training training on directors believe institutional
on professionalism professionalism impacts resident documentation policies/processes (CE)
quality and safety of based on direct
patient care observation (GC) Faculty follow institution‘s
policies, procedures and
Program directors and guidelines for EMR
faculty believe documentation (CE)
copyrighted materials
are unavailable in the Faculty believe residents are
public domain (GC) aware of and use institution‘s
policies for reporting
mistreatment (CE)
PR-4: Clinical site Periodic monitoring of Leadership periodically
monitoring of professionalism is assesses for culture of
professionalism essential to identification professionalism of medical
of vulnerabilities and staff and residents (RC)
implementing change to
enhance patient care Leadership monitors
documentation practices of
resident faculty use of the
EMR (O)

Note: Adapted from ―ACGME CLER Pathways to Excellence‖. Code abbreviations: Learning Opportunities (LO), Feedback/Asessment (F),
Curriculum/Learning Objectives (C), Personal Development (PD), Supervision (S), Faculty Support (FS), Group Cohesion/Trust (GC), Communication (Co),
Orderly Work Environment (O), Clear Expectations (CE), Responsive to Change (RC), Individualization (IND).

143
 Curriculum Vitae
Jamie S. Padmore

PERSONAL INFORMATION:
Home: 711 Edelblut Drive, Silver Spring, MD 20901
Business: 5565 Sterrett Place, 5th Floor, Columbia, MD 21044
University: 3900 Reservoir Rd., NW, Med-Dent NW-110, Washington, DC 20009

EDUCATION:
The Ohio State University
College of Medicine, School of Allied Medical Professions
Bachelor of Science, June 1991

Harvard Medical International

Harvard Macy Institute
Scholar in Health Professions Education, January 2007 – May 2007

The University of Maryland University College

Master of Science, August 2010

The University of Maryland University College

Doctor of Management, December 2015

CERTIFICATION:
Georgetown University
School of Graduate Studies
Certificate in Executive Leadership, June 2003

Medical Education Research Certification

Association of American Medical Colleges (AAMC), 2009

PROFESSIONAL EXPERIENCE:

Associate Dean for Graduate Medical Education & Educational Scholarship

Georgetown University School of Medicine
3900 Reservoir Rd., NW
Washington, DC 20057
October 1, 2011 – Present

Corporate Vice President Academic Affairs

MedStar Health
5565 Sterrett Place, 5th Floor, Columbia MD 21044
July 1, 2011 – Present

144
 Asst. Vice President Academic Affairs
MedStar Health
July 2002 – June 30, 2011

Director, Medical Affairs & GME

Washington Hospital Center
110 Irving St., NW, Washington, DC 20010
February 1996 – July 2002

Director, Medical Education

Akron General Medical Center
400 Wabash Ave., Akron OH 44307
April 1994 – February 1996

Medical Illustrator
Akron City Hospital/Summa Health System
525 E. Market St., Akron OH 44304
September 1991 – April 1994

ACADEMIC RANK

Associate Professor (Biomedical Educator Track)

Dept of General Surgery
Georgetown University Medical Center
October, 2011 - Present

HONORS AND AWARDS:

Alliance of Independent Academic Medical Centers

Innovation in Medical Education award, 2011, for the MedStar Health Teaching Scholars program

Alliance of Independent Academic Medical Centers

Innovation in Medical Education award, 2008 for the AIAMC National Initiative Leadership

PROFESSIONAL SOCIEITIES:

American College of Medical Quality, 2010 – Present

Association of Medical Educators in Europe, 2007 – Present
American College of Healthcare Executives, 2003 – Present
Association of American Medical Colleges Group on Resident Affairs, 1998 – Present
Association of American Medical Colleges Group on Educational Advancement, 2007 – Present
American Health Lawyers Association, 2006 – Present
Association of Program Directors in Internal Medicine, 2005 – Present
International Association of Medical Science Educators, 2011 - Present

145
PUBLIC SERVICE:

Association of American Medical Colleges (AAMC)

 Immediate Past Chair, GME Section; 2012 - Present
 GME Section Chairman (Group on Educational Affairs), 2008 – 2012; elected two terms
 Group on Educational Affairs Steering Committee, 2008 – 2012
 National Duty Hours Task Force co-chair, 2012 - 2013
 National Educator Evaluation Task Force, 2010 – 2013
 Group on Resident Affairs Steering Committee, 2005 – 2008; elected
 Selection Committee and Abstract Reviewer (GRA), 2006, 2007

Alliance of Independent Academic Medical Centers

 Board of Directors, 2007 – 2013
 Secretary-Treasurer, 2011 – 2013
 Executive Committee of the Board, 2009 – 2013
 Committee to Integrate Academics and Quality 2007- Present; Chairman 2009 - 2011
 National Initiative Chair, 2009 – 2011
 Chair, Annual Meeting 2008
 GME Forum Co-Chair, 2005 – 2008

State of Maryland Department of Health and Mental Hygiene, Health Services Cost Review
Commission (HSCRC) Innovations in Graduate Medical Education Working Group, 2015.
Conference Co-Chair, CENTILE International Conference to Promote Resilience, Empathy,
and Well-Being in Healthcare Professionals. October 21, 2015. Washington, DC.
AAMC Reactor Panel for Core Entrustable Professional Activities (CEPAER), 2013
Patient-Centered Outcomes Research Institute (PCORI), Scientific Reviewer for
Round I grant applications, 2012
MedBiquitos Curriculum Inventory Working Group, 2011-Present
National Alliance for Physician Competence, 2008 – 2010

INVITED LECTURES

A. Invited Lectures (last 10 years)

ACGME and the Law. Council of Residency Directors (CORD) in Emergency Medicine
Academic Assembly, Nashville, TN. March 6, 2016. (Confirmed)
Leveraging the potential of Clinical Competency Committees (CCCs) to improve milestone
reporting, clinical competence assessment, and promotional decisions. ACGME Annual
Educational Conference, Washington, DC. February 27, 2016. (Confirmed)
Uniformed Services University of the Health Sciences – National Capital Consortium Course
for Program Directors: Assessing Clinical Competence. Forest Glen NCA Simulation
Center, Silver Spring, MD. Feb. 1-5, 2016 (Confirmed)

146
Is my learner fit? Dimensions of well-being and impairment (Panel, Plenary Session).
AAMC Annual Meeting: Learn, Serve, Lead 2015. Baltimore, MD. November 8, 2015.
Advanced Legal Issues in GME. AAMC GRA Leadership Development Course, Part III.
April, 2015. Austin, TX.
Evaluation, Assessment, Feedback and the Clinical Competence Committee. National
Capital Consortium & Uniformed Services University of the Health Sciences Program
Director Course on Assessing Clinical Competence. Forest Glen NCA Simulation Center,
Silver Spring MD. January 7, 2015.
Building Bridges to Move Mountains: Clinical Learning Leadership (Panel). AAMC Group
on Resident Affairs Annual Meeting, Phoenix, AZ. May 5, 2014.
Clinical Competence Committees & Resident Evaluation. Yale University School of
Medicine & Yale New Haven Hospital, New Haven, CT. April 22, 2014.
Oversight of Clinical Competence and Assessment: Considerations for GMECs. Walter
Reed National Naval Medical Center GMEC, Bethesda, MD. February 6, 2014.
Evaluation, Assessment, Feedback and the Clinical Competence Committee. National
Capital Consortium & Uniformed Services University of the Health Sciences Program
Director Course on Assessing Clinical Competence. Forest Glen NCA Simulation Center,
Silver Spring MD. January 6, 2014.
Resident Performance Issues and the CCC: How Education and Academic Law Support
GME. University of Buffalo School of Medicine, December 17, 2014.
Legal & Academic Strategies for Dealing with Problem Resident Issues. Morehouse
Univeristy School of Medicine, Atlanta GA. November 12, 2014.
Evaluation, Assessment, Feedback and the Clinical Competence Committee. Loyola
University School of Medicine/Trinity Health GME Retreat, Chicago, IL. October 11, 2013.
Developing Faculty for High Functioning Clinical Competence Committees (CCCs).
AAMC-GRA Annual Meeting, Savannah, GA. May 6-7, 2013.
Preparing Students for the New ACGME Milestones. AAMC – Northeast Group on Student
Affairs (NGSA) Annual Meeting, Luncheon Keynote Address, Atlantic City, MD. April 11,
2013.
Advanced Issues in GME Moonlighting. American Health Lawyers Association GME
Webinar Bootcamp Series Part V. January 10, 2013.
Advanced Topics in Academic Law. GRA Leadership Development Program Part II,
Chicago, IL. Sept. 17, 2012
Resident Remediation Strategies: Advanced Topics, Yale University School of Medicine,
May 30, 2012.
Transitioning to Leader: A Primer for Chief Residents. Otolaryngology Chief Resident Boot
Camp, University of Pennsylvania, Philadelphia, PA. May 5, 2012.
Approaches to Problem Resident Issues. Loma Linda University School of Medicine Faculty
Retreat, Palm Springs CA. October 27, 2011.

147
Integrating Quality Improvement and GME: Engaging and Evaluating Residents. Clinical
Assembly of Osteopathic Surgeons, Surgical Educators Seminar, Atlanta, GA. Sept. 17,
2011.
Faculty Development in Quality Improvement: Building Human Capital at Every Level in
Medical Education. AAMC Integrating Quality Conference, Chicago, IL. June 8-9, 2011.
Resident Remediation Strategies: Advanced Topics, Yale University School of Medicine,
May 24, 2011.
Leadership Development for Chief Residents (seminar). Otolaryngology NE Regional Boot
Camp for Chief Residents, University of Pennsylvania, Philadelphia, PA. May 14, 2011.
Balancing Academic and Employment Law: Dealing with the Resident Issues. Morehouse
University School of Medicine, Atlanta, GA. April 5, 2011.
Quality Improvement and GME: The MedStar Health Experience. University of Arkansas
Medical Sciences, Grand Rounds, Little Rock, AR. March 10, 2011.
MedStar Health Teaching Scholars Program. University of Arkansas Medical Sciences,
Teaching Scholars program lecture, Little Rock, AR. March 10, 2011.
Outcomes of the AIAMC National Initiative. ACGME Annual Conference, Nashville, TN.
March 5, 2011.
Integrating Quality Improvement and GME: Resident Strategies. AOCOO-HNS Program
Director Retreat, Orlando, FL. February 12, 2011.
Students or Employees? Balancing academic and employment issues in residency.
Association of Anesthesia Core Program Directors, Washington DC. November 5, 2010.
Practical Approaches to Innovative Education – the Challenges and Triumphs. SiTEL
Innovative Education Conference 2010, Washington, DC. April 30, 2010.
AIAMC National Initiative Update, AIAMC Annual Meeting, New Orleans, LA. March 25,
2010.
Resident Remediation Strategies, Yale University School of Medicine, January 2010.
Resident Remediation Strategies, Dartmouth-Hitchcock Medical Center, November 6, 2009
Resident Remediation Strategies, Southern Illinois University School of Medicine,
Springfield, IL., Oct. 20, 2009
Competence Committees, National Rehabilitation Hospital Faculty Retreat, Washington,
D.C., October 9, 2009
Developing a Teaching Scholars Program, National Rehabilitation Hospital Grand Rounds,
Washington, D.C. September 17, 2009.
Resident Remediation Strategies. American College of Obstetrics and Gynecology, CREOG
Annual Meeting, Washington, D.C. August 7, 2009.
GME and Quality: The MedStar Health Story. Medical Grand Rounds, TriHealth (Bethesda
North/Good Samaritan), Cincinnati, OH. May 20, 2009.

148
Advanced Legal Issues in GME. Kaiser Permanente of Northern California GME Spring
Offsite Retreat. Berkeley, CA. May 6, 2009.
Institutional GME Quality Metrics. Plenary Session - GRA Professional Development
Meeting, Atlanta, GA, April 28, 2009.
Streamlining Resident Remediation and Due Process. American College of Chest Surgeons
Annual Meeting, Austin, TX. March 29, 2009,
Integrating Academics and Quality: Lessons Learned and Next Steps for the AIAMC.
AIAMC Annual Meeting, Tucson, AZ. March 27, 2009
Council of Emergency Medicine Program Directors (CORD) Academic Assembly. Las
Vegas, NV. March 6, 2009.
Smart Legal Approaches to Common GME Dilemmas – Dealing with the Problem Resident.
Atlantic Health System GME Retreat, November 10, 2008.
Medicare‘s Contribution to GME in Teaching Hospitals. American Board of Thoracic
Surgery Executive Committee Retreat, Asheville, NC, September 24, 2008.
GRA Leadership Development Course Part II: Advanced Legal Issues in GME.
AAMC/GRA Leadership Development Course, Chicago IL, September 19, 2008.
The AIAMC National Initiative: Improving Patient Care Through GME, GRA Professional
Development Meeting (Small Group Session), Salt Lake City, UT, May 5, 2008.
Smart Legal Approaches to Common GME Dilemmas, University of Virginia Health
System, April 25, 2008.
GME as a Driver of Clinical Quality and Patient Safety: The AIAMC National Initiative,
presented to the New Jersey Council of Teaching Hospitals, April 24, 2008.
Advanced Legal Issues in GME, Henry Ford Health System, Detroit MI. April 4, 2008.
Unlocking the Secrets to Effective Resident Discipline, Allegheny General Medical Center,
Faculty Retreat at Nemacolin Woodlands Resort, March 10, 2008.
Advanced Legal Issues in GME. Baystate / Tufts Medical Center, Springfield MA. January
17, 2008.
Streamlining Resident Remediation Processes – The MedStar Health Approach, Penn State-
Hershey Medical Center, Hershey, PA. November 16, 2007
Smart Legal Approaches to Common GME Dilemmas, Association of Program Directors in
Internal Medicine, Minneapolis, MN. October 6, 2007
What makes a GREAT residency program?, Georgetown University Hospital Department of
Surgery Educational Retreat, September 7-8, 2007.
Advanced Legal Issues in GME, GRA Leadership Development Course, Memphis, TN.
April 22, 2007.
Qualitative and Legal Benefits of Utilizing a Clinical Competence Committee in Assessing
Resident Performance, GRA Professional Development Meeting, April 22-23, 2007.

149
 Enhancing Your Feedback to Residents, Assoc. of Pulmonary & Critical Care Program
Directors, Marco Island, FL. April 15, 2007.
Unlocking the Secrets to Effective Resident Discipline, New York University School of
Medicine, New York, NY. May 17, 2006.
Enhancing Feedback to Residents Through Competence Committees, Yale University School
of Medicine, New Haven, CT. February 2, 2006.
Selection Strategies for House Staff Recruitment, MedStar Health Emergency Medicine
Faculty Retreat, Washington, DC. October 19, 2005.
Bad Hires Make Bad Fires: Improving Residency Selection Processes, GRA Professional
Development Meeting, Austin, TX. April 24, 2006.
Give the People What They Want: Developing Excellence in Program Directors and
Coordinators, GRA Professional Development Meeting (Small Group Session), Austin, TX.
April 25, 2006.
Legal Issues in GME, GRA Leadership Development Course, New Orleans, LA. April 17,
2005.
Resident Remediation – Perils and Pitfalls: Lessons Learned from both Departmental and
Institutional Perspectives, GRA Professional Development Meeting (Small Group Session),
New Orleans, LA. April 18, 2005.

B. Sessions Chaired or Moderated at National Meetings

Scientific and Clinical Foundations of Resilience: Cultivating Self-Care, Well-Being and

Resilence in our Lives and Organizations (Moderator). CENTILE International Conference
to Promote Resilience, Empathy, and Well-Being in Healthcare Professionals. October 21,
2015. Washington, DC.
Building Bridges to Move Mountains: Clinical Learning Leadership (Panel). AAMC Group
on Resident Affairs Annual Meeting, Phoenix, AZ. May 5, 2014.
Interim Report of the National Duty Hours Task Force. AAMC Annual Meeting. Nov. 3,
2012. San Francisco, CA.
The Alliance of Independent Academic Medical Centers National Initiative: Improving
Patient Care Through GME. AAMC Integrating Quality Conference, Chicago, IL. June 8-
9, 2011.
Tools and Techniques for Integrating Quality and Patient Safety Education Across the
Continuum. AAMC Integrating Quality Conference, Chicago, IL. June 3-4, 2010.
A DIO‘s Guide to Final Summative Assessments, GRA Professional Development Meeting,
Austin, TX, April 27-28, 2010.
Measures that Matter: GME Program and Institutional Outcomes, GRA Professional
Development Meeting, Atlanta, GA, April 28, 2009.
Qualitative Benefits of a Clinical Competence Committee, GRA Professional Development
Meeting, Salt Lake City, UT, May 4-7, 2008.
150
 From Catalyst to Sustained Change: Next Steps, AIAMC Annual Meeting Closing Session,
Amelia Island, FL, March 29, 2008
Non-Standard Programs in GME, Moderator, GRA Professional Development Meeting,
Memphis, TN, April 21-24, 2007.
Smart Legal Approaches to Common GME Dilemmas, Moderator, AAMC Annual Meeting,
Seattle, WA, October 31, 2006.
Advanced Immigration Issues, Group Facilitator, Educational Commission on Foreign Medical
Graduates Annual Conference, September 28, 2006.

UNIVERSITY SERIVCE:

GUMC-MedStar Health Educational Advisory Board (EAB) Co-Chair, 2015 - Present

MedStar Health System Medical Education Committee Co-Chair, 2014 - Present
GUMC Faculty Development Committee, Ex-Officio, 2013 - Present
Georgetown-MedStar Scientific Advisory Board, 2012-Present
Co-Director, Medical Education Research Track (SoM), 2011 - Present
Medical School Admissions Interviewer, 2011- Present
Medical Student Independent Study Project (ISP) Committee, 2012 - Present
Committee on Medical Education (Curriculum Committee), 2008 – Present (Ex-Officio)
LCME Medical Student Subcommittee, 2009-10 (member)
Strategic Planning Committee, Educational Innovation Implementation Design Team, 2009
(member)
MedStar Health Institutional Review Board #3, 2010 - 2014

TEACHING ACTIVITIES:

OMED 408: Research Elective in Medical Education Research (Course Director & Faculty)
3-9 students per year; one month elective in M4 year
40 contact hours per student enrolled

OMED 406: Summer Research Elective (Course Director & Faculty)

15-22 students per 8-week elective
80+ contact hours

OMED 201: Seminar in Medical Education Research Methods (Course Director & Faculty)
6-12 students per year; one week elective
30 contact hours

Faculty, MS-2 summer research program for Pellegrini Endowment, 2010 - present

151
 Independent Study Project (ISP) Advisor for select Georgetown medical students
Course co-Director and Faculty, MedStar Health Teaching Scholars program, 2008 – present
Course Director, Chief Resident Leadership Training program, 2007 – present

SCHOLARSHIP AND RESEARCH:

A. GRANTS
Georgetown University CIRCLE grant, 2010 (co-investigator, Medical Education
Research: A Mini-Course for Medical Students). Amount awarded: $25,000.00,
Mary Furlong, MD - PI

B. PUBLICATIONS

a. Original papers in refereed journals

Gusic M, Amiel J, Baldwin C, Chandran L, Fincher R, Mavis B, O'Sullivan P,

Padmore J, Rose S, Simpson D, Strobel H, Timm C, Viggiano T. (2013). Using the
AAMC Toolbox for Evaluating Educators: You be the Judge. MedEd Portal, April
2013.
Weisman DC, Bashir L, Mehta A, Bhatia L, Levine SM, Mete M, and Padmore JS.
A medical resident post-discharge phone call study. Hospital Practice, 2012, 40(2)
Padmore JS, Jaeger J, Riesenberg LA, Karpovich K, Rosenfeld JA, and Patow C.
Renters or Owners? Resident Perceptions and Behaviors Related to Error Reduction
in Teaching Hospitals: A Literature Review. Academic Medicine, 2009;84(12):1765-
74.
Riesenberg LA, Leitzsch J, Massucci J, Jaeger J, Rosenfeld J, Patow, C, Padmore
JS, and Karpovich K. Resident and Attending Physician Handoffs: A systematic
review of the literature. Academic Medicine, 2009;84(12):1775-87.
Patow CA, Karpovich KP, Riesenberg LA, Jaeger J, Rosenfeld JC, Wittenbreer M,
and Padmore J. Residents‘ Engagement in Quality Improvement: A Systematic
Review of the Literature. Academic Medicine, 2009;84(12):1757-64.
Nagler, A., Andolsek, K., Padmore JS. Unintended Consequences of Resident
Portfolios. Academic Medicine, 2009;84(11):1522-6.

b. Books or chapters in books, publications and other journals

Padmore JS, Richard KM, Karpovich KP, Shaver M. Fundamental aspects of

academic and employment law applied to graduate medical education programs. In:
Levine J, ed. AHME Guide to Medical Education, 4th edition. Irwin, PA:
Association of Hospital Medical Educators; 2010:149-168.

152
 Padmore JS, Richard KM, Filak A. Residents who fail to progress. In: Levine J,
ed. AHME Guide to Medical Education, 4th edition. Irwin, PA: Association of
Hospital Medical Educators; 2010:353-374.
Friedlander RB, Green V, Padmore JS, Richard KM. Legal issues in residency
training. In: Andolsek K and Cefalo R. ed. LIFE Curriculum, 2007:8-35.
Richard KM and Padmore JS. The Duty to Disclose. Health Lawyers News 2007;
11(5):18-20.
Richard KM and Padmore JS. Discharge and Dismissal of Residents in Training.
Health Lawyers News 2007;11(4):34-35.
Richard KM and Padmore JS. Practical Approaches for Academic Due Process
Policies. Health Lawyers News 2007; 11(1):16-17.
Richard KM and Padmore JS. Misconduct in GME. Health Lawyers News 2007;
11(2):24-25.
Richard KM and Padmore JS. Unique Legal Aspects of Residency Training
Programs. Health Lawyers News 2006; 10(10):4.
Richard KM and Padmore JS. Does ―Fair Hearing‖ = ―Due Process‖ in Residency
Programs? Health Lawyers News 2006; 10(12):16-17.

c. Abstracts/Posters

Weissinger, P.A. , Padmore, J.S., Furlong, M. A., & Malekzadeh, S. (June 10, 2013).
Preparing the Next Generation of Medical Educators: A Scholar Track for Medical
Students. Poster presentation at the International Association of Medical Science
Educators (IAMSE), St. Andrews, Scotland.

Reilly, M., Chou, E, Padmore, J.S. An automated email to improve satisfaction and
healing of post-op surgical patients. AAMC MedEd Portal Poster Session, AAMC
Annual Meeting, Nov. 4, 2012. San Francisco, CA.

Zonozi, R.R. & Padmore, J.S. A Medical Student-Led Quality Improvement

Initiative to Ensure Consistent and Optimal Communication in the Daily Review of
Central Line Necessity. Society of Critical Care Medicine‘s 41st Congress, Feb. 6,
2012.

Weisman, D.A., Mete M, & Padmore, J.S. A Medical Resident Post-Discharge

Phone Call Study. AAMC Integrating Quality Meeting, Chicago, IL, June 9-10,
2011.

Padmore, J.S., Choudhary, I., & Cioffredi, L.A. Medical Student Applied Learning
in QI During the Pre-Clinical Years: A Study in Handoff Communication. AAMC
Integrating Quality Meeting, Chicago, IL, June 9-10, 2011.

Padmore, J.S. The Impact of Institutional Due Process Standards on the Timing
and Frequency of Academic Interventions of Residents, Association of Medical

153
Educators in Europe (AMEE), Annual Meeting, Prague Czech Republic, September
1, 2008.
Jamshed, N, Mete M, Padmore J, Sinha S. Identifying Training Needs of Internal
Medicine Faculty in Geriatrics with a Novel Geriatrics Skills Assessment Tool
(GSAT). 2010 Annual Scientific Meeting of the American Geriatrics Society,
Orlando FL, May 12-15, 2010.
Padmore, JS, Orlowski JM. Implementation of a Central Line Training Program in
GME to Reduce Blood Stream Infection Rates. AIAMC Annual Meeting, Amelia
Island FL. March 27-29, 2008.
Jaeger, J, Karpovich KP, Padmore JS, Patow C, Riesenberg LA, Rosenfeld JC. Do
GME Programs Affect Clinical Quality and Patient Safety in Teaching Hospitals?
A Critical Review of the Literature. AIAMC Annual Meeting, Amelia Island FL.
March 27-29, 2008.
Padmore, JS. The Impact of Academic Due Process Standards on the Timing and
Frequency of Academic Interventions of Residents. ACGME-ABMS Conference,
―Physician Competence: From Deconstruction to Reconstruction‖, Rosemont IL.
September 15-16, 2007.

154