Search

Home Features Leadership Insight Across the Industry

Podcast About

Needs Analysis for Assessment-Based Certificate

Programs
By Julie Patrick
8.25.21

Imagine that you have an excellent idea for an assessment-based certificate

program. You may be the subject matter expert (SME), or a SME has asked for your
help in creating the program. How do you verify the need for your program? You
must go beyond your own opinion or your inner circle of SMEs’ opinions to ensure
your program is structured in a defensible manner.

A proper needs analysis is imperative to the success of an assessment-based

certificate program. Begin with the end in mind. A needs analysis helps identify the
target audience, the gaps in current performance and the content necessary to
support success. The needs analysis results help you create learning objectives
and inform the program's design, including the content you will include and how
you will assess the learning. The needs analysis results can also inform your
marketing strategy, extending your reach to serve more individuals and
organizations.
To determine success, define and align these various perspectives:

What are the benefits of the certificate to organizations

and the profession as a whole?

Organizational Tip: Even if you are developing a program for individuals who
Outcomes serve a broad range of organizations, consider talking with
representatives of these organizations who can describe their
desired results and how your target audience could support
those results.

What is the necessary performance level to achieve the

organizational impact?
Ideal
Performance Tip: The definition of ideal performance in the role will help you
create learning objectives and assessments that are in
alignment.

Who is the target audience that will need to demonstrate

the ideal performance?
Targeted
Learners Tip: You will likely discover that the target audience does not
have the knowledge and skills necessary to support success;
hence, the need for the assessment-based certificate program.

Information about organizational outcomes, ideal performance and targeted

learners provides details regarding the content to include and how to design the
program, including the assessment. Be sure to confirm alignment among these
perspectives as you create the assessment-based certificate program. Confirming
alignment among these perspectives provides validation that your certificate
program has the value it intends. If you do not have clarity, ask more questions to
ensure that the program meets the needs of organizations, roles and individuals.

So, how do you get this information? Collect data from qualified individuals and
stakeholders who are knowledgeable about the intention of the program. Also,
make sure these individuals understand the “use, purpose, scope and stakes of
the certificate program.”1 There are myriad ways to approach data collection, such
as interviews (individual and group), work observation, surveys and extant data.
These methods yield different information, so be intentional with your choice of
approaches. Consider using a combination of these approaches to get the
information you need.

With individual interviews, identify the people you would like to interview and
create questions intended to draw out sufficient information. For example, to
define organizational outcomes, interview leaders within organizations that
hire the participants in the assessment-based certificate program.
 Information about the organizational outcomes helps target content,
activities and assessment approaches that support organizational success.
With work observation, identify who and what you will observe to gather
information about ideal performance. For example, you may select
representatives of the target audience and request time to watch what they
do to achieve success. Not only can an observation inform the learning
objectives and content to include, but it can also reveal important details to
include in the assessment as well. Observation may also help determine
where the target audience is today and where they need to be — the
performance gap.
With group interviews or focus groups, identify a group of people who will
collaborate to offer valuable information. For example, you may gather
representatives of the target learners and their supervisors. These people
may provide opinions about what they need to do on the job after the
program and recommend approaches for the learning opportunity, such as
methods and media that appeal to them and why. They can also provide
input on effective assessment measures so that the assessment matches
the program’s intention. Be sure to keep the group interviews or focus groups
on track, using facilitation techniques that honor participants’ contributions
while adhering to an efficient process.
With surveys, identify the audience from whom you would like more
information and create questions that are easy to answer. For example,
question types may include rating scales to indicate importance and
selected-response questions versus open-ended questions. We are
interested in receiving data that are easy to analyze rather than taking a lot
of time to make sense of written comments. Responses to survey questions
may provide specific information such as demographic details of the target
learners.
With extant data, review material related to the content associated with the
assessment-based certificate program, the organizations that will employ
individuals who take the program, past performance of those in the
positions related to the program and individuals who are doing the work. A
published report from an outside entity would not replace a needs analysis.
Examples include competency models, industry or field statistics, strategic
plans, benchmarking reports, job descriptions, meeting minutes, certificate
recipients’ feedback, and performance management processes and results.

IMPORTANT: Is Your Program High Stakes?

High-stakes programs require a job analysis. As stated in the I.C.E. 1100
Standards for Assessment-Based Certificate Programs Terms and
Definitions, “High stakes are associated with substantial potential for
adverse consequences for the public, clients, patients, etc. Programs may
also be high stakes when participants’ careers hinge on their success in
accomplishing the intended learning outcomes of an assessment-based
certificate program.” A job analysis is “any of several methods used singly
 or in combination to identify the performance domains and associated
tasks, knowledge and/or skills relating to the purpose of the credential and
providing the foundation for program validation. Also known as task or
practice analysis, job-task analysis or role delineation study.” Applicants
for I.C.E. 1100 accreditation for assessment-based certificate programs
would be well served to closely assess their program’s stakes level before
submission.

If conducted, a job analysis is used to specify the domain of testable

content on the assessment blueprint and determine the weighting (i.e.,
how many items) given to each section and/or objective. For lower stakes
programs without a job analysis, the assessment blueprint and weighting
should be derived from the needs analysis. Note that a traditional job
analysis does not typically provide all of the data that a needs analysis
does, and so additional steps will be necessary to determine the
organizational outcomes, ideal performance and targeted learners.
Regardless of whether the exam blueprint is derived from a job analysis or
the needs analysis, it is imperative that the domain of testable content
and the weightings are derived through one of these processes, and not
from simply tradition or the opinions of only a few people.

After collecting the data from the needs analysis, analyze it to develop or confirm
the organizational outcomes, ideal job performance and needs of the target
audience. Create a needs analysis report that outlines the information you
uncovered and the way you obtained it. Include who was involved and how you
involved them. Also, confirm the qualifications of those involved to ensure that you
have consulted individuals and stakeholders that add value to the program. Use
your needs analysis report to develop learning objectives, create the assessment
and determine the content to include to serve the need or fill the gap identified.

What should you do if you already created an assessment-based certificate

program and didn’t complete a needs analysis? In this case, the best course of
action is to use these recommended needs analysis approaches to either confirm
your program's current structure or undergo a restructuring based on the findings.
For example, collect data about desired organizational outcomes from those
organizations for whom your participants work. Ask qualified individuals and
stakeholders questions about ideal performance and compare the responses to
the content in your program. Create a learner profile based on input from qualified
individuals and stakeholders. Compare the learner profile to the characteristics of
those who have experienced your program. Document the results of the needs
analysis and revise your program as necessary. Be prepared to revise the program.
A cursory review labeled as a needs analysis will probably not be sufficient
justification to leave the program unchanged.
Ultimately, the needs analysis should tell the story of why the assessment-based
certificate program was created. It should provide critical information about goals,
performance and learners. A comprehensive needs analysis, relying on input from
qualified individuals and stakeholders, serves as a solid foundation for delivering
a sound program, meeting the needs of individuals and organizations.

Source
1. I.C.E. 1100 Standards for Assessment-Based Certificate Programs

Julie Patrick
JP Learning Associates
Julie Patrick is a talent development consultant with JP Learning Associates. She
has been a trainer, facilitator and instructional designer since 1992. As a former
manager of a corporate university, Patrick was responsible for various training
programs. She holds numerous certifications and designs and implements
learning and development solutions that help clients achieve their desired
business results. Patrick also serves as a facilitator of certificate programs and
has written infolines and templates and tools for the Association for Talent
Development (ATD). She serves as a member of the Assessment-based Certificate
Accreditation Council (ACAC) for the Institute for Credentialing Excellence (I.C.E.).
Patrick has a bachelor's degree in business administration and a master's degree
in organization development.

2001 K. Street, NW t. 202.367.1165

3rd Floor North f. 202.367.2165
Washington, D.C. 20006 info@credentialingexcellence.org

   

credentialingexcellence.org

Login