Review of Establishing Essential

Competencies for
Program Evaluators
LaBeverly Hooks &
Yolanda Jarrett
Topics
History of Evaluation

How did the field start?

State of competency development

Rationale of Competencies

Why develop guidelines?

Taxonomy of Essential Competencies

What specific guidelines do we follow?

Future Initiatives
History
- Great Society Programs

- Social research techniques

- Different conditions called for innovation

- Organizations, journals, and textbooks

- A field grown from its practice

Competency Development
Typical Professional Process
- Group of experts develop initial list of
categories, models, and competencies
- Review process
- Competencies made public
- Used for training and continuing education
programs
- Periodic reviews to update the
competencies as theory, research, and
practice evolve with time.
Program Evaluations Process
- Disagreement whether a standardized
competency can be all inclusive
- Opinion A: The field cannot be an unified
profession without competencies that work
across different situations
- Opinion B: Identifying competencies is not
possible if the field is truly diverse and
cannot work across different situations
- The taxonomy of essential evaluator
competencies proposed in King, Stevahn,
Ghere, and Minnema (2001)
Rationale for Competencies
- Lack of standards means no or limited licensing/credentialing

- Inability to identify qualified evaluators for a job opportunity

- Lack of clarity in training and education to become an evaluator

- Training and education organizations may use inappropriate curriculum

- Focusing research on theory-based descriptive models versus prescriptive

models to guide effective practice
Rationale for Competencies
Improved Training Enhanced Reflective Practice

- Program structure - Professional and Self Awareness

- Types of Interested Evaluators

- Course instruction and assessment
- New
- Effective evaluations versus competent
- Accidental
evaluators
- Transitioning
- Overlook the development of skills
- Experienced but still learning
- Professional development
Rationale for Competencies
Advancement of Research on Evaluation Professionalization of the Field

- Formation of best/effective practices - Intrinsic benefit

- Areas of research - Definition and Identity

- Role of Competencies
- Generalizes what is important to the field
- Impact of Training
- Gateway to licensure/credentialing
- Keys to Successful Evaluation
- Progress in Descriptive Theories - Program Accreditation
Taxonomy of Essential Competencies
Competency
- Guidelines that identify levels of competence

- Established to identify knowledge, skills, attitudes, and self-regulation

- Framework that guides a profession

Evaluation competencies
- Derived through a consensus of a diverse set of evaluation professionals

- A fairly new process in AEA the last 20 years

- Complicated due to diversity in types of evaluation

Evaluation Competencies (2005)
Initial Competencies
- 4 broad domains, each with a main category, each with specific items

Original Domains (4)

A. Systematic Inquiry

B. Competent Evaluation Practice

C. General Skills for Evaluation Practice

D. Evaluation Professionalism

AEA professionals felt they were too broad and did not specify contents
Evaluation Competencies (2005)
Working Domains (6)
A. Professional Practice

B. Systematic Inquiry

C. Situational Analysis

D. Project Management

E. Reflective Practice

F. Interpersonal Competence
Working Domain specifics
I. Professional Practice
Norms and values of evaluation practice (ethics and standards)

II. Systematic Inquiry

Technical aspects (design, data collection, analysis, interpretation, reporting)

III. Situational Analysis

Analysis based on context and interests

IV. Project Management

Practical needs (budgeting, coordinating resources, supervising procedures)
Working Domain specifics
V. Reflective Practice
Personal assessment of skills and expertise, professional development

VI. Interpersonal Competence

People skills (communication, negotiation, conflict resolution, collaboration, cultural)

Needs Addressed:
Revisions gained additional specificity in domains areas (p.54)

Created more precision within each competency

Multiple vs separated descriptors within categories

Interim step towards widespread validation in the field

Systematic method helps form evaluation questions

Future Initiatives
2 Main Future Initiatives

Need for validation of competencies

Systematic, comprehensive evaluation study

Representative of the diversity in evaluation types

Constructing descriptive rubrics specifying performance proficiency

Self-Assessment

Evaluation tool for training programs to further clarify evaluation practice

Possible future licensing and credentialing tool

Summary
AEA Taxonomy

Fairly new guidelines, standards, and competencies

Created to promote professional development, practice values, and

evaluative skills

Guiding Principles have already been established

Competencies have been created but not universally agreed upon

Still opportunities to validate and create concrete competencies

Useful in helping evaluators and trainers to conduct ethical practice

References
Stevahn, L., King, J.A., Ghere, G., & Minnema, J. (2005). Establishing Essential

Competencies for Program Evaluators. American Journal of Evaluation,

26(1), 43-59.

American Evaluation Association. (2015). Invitation to Provide Feedback on the

AEA Draft Competencies. American Evaluation Association: About Us

Retrieved from http://www.eval.org/p/cm/ld/fid=472