THE DELPHI TECHNIQUE

Randall B. Dunham, Ph.D.

September 1, 1998

PURPOSE

The purpose of the Delphi technique is to elicit information and judgments from
participants to facilitate problem-solving, planning, and decision-making. It does so
without physically assembling the contributors. Instead, information is exchanged
via mail, FAX, or email. This technique is designed to take advantage of
participants’ creativity as well as the facilitating effects of group involvement and
interaction. It is structured to capitalize on the merits of group problem-solving and
minimize the liabilities of group problem-solving.

REQUIREMENTS

The Delphi technique requires a Coordinator to organize requests for information,

information received, and to be responsible for communication with the
participants. The Delphi technique requires an efficient communication channel to
link the Coordinator with each of the participants. It is common to use the U.S. mail
for this purpose, but FAXes and email can decrease the time required for
completing a Delphi technique. Elapsed time from beginning to end of the process
averages 44 days using the U.S. mail and as little as five days using email.
Nevertheless, the Coordinator’ s job can take substantial time. Recent experience
suggests that coordination of the Delphi technique using email with 20 participants
and the processing of three questionnaires could utilize 30-40 hours of the
Coordinator’ s time.

PROCESS

1. Identify the issue and solicit ideas. For example:

What action could be taken to provide faster response to patient

inquiries between visits?
Prepare and send the first questionnaire, which asks each participant to
engage in individual brainstorming so as to generate as many ideas as
possible for dealing with the issue.

2. Response to first questionnaire. Each participant lists his/her ideas

(Questionnaire #1) in a brief, concise manner and returns the list
anonymously to the Coordinator. These ideas need not be fully developed.
In fact, it is preferable to have each idea expressed in one brief sentence or
phrase. No attempt should be made to evaluate or justify these ideas at this
point in time.

3. Create and send Questionnaire #2. The Coordinator prepares and sends
a second questionnaire to participants that contains all of the ideas sent in
response to the first questionnaire and provides space for participants to
refine each idea, to comment on each idea’ s strengths and weaknesses
for addressing the issue, and to identify new ideas.

4. Response to second questionnaire. Participants anonymously record their

responses to Questionnaire #2 and return them to the Coordinator.

5. Create and send Questionnaire #3. The Coordinator creates and sends a
third questionnaire that summarizes the input from the previous step and
asks for additional clarifications, strengths, weaknesses, and new ideas.

6. Continuation of the process. If desired, the Coordinator performs

iterations of the preceding process until it becomes clear that no new ideas
are emerging and that all strengths, weakness, and opinions have been
identified.

7. Resolution. Resolution may occur in one of two ways.

• If dominant, highly evaluated ideas emerge via consensus, the

exercise is declared finished. The end product is a list of ideas with
their concomitant strengths and weaknesses.
 • The Coordinator conducts a formal assessment of the group’ s
opinions of the merits of the ideas. There are a number of ways to
conduct a formal evaluation. In one method, the Coordinator
prepares a questionnaire that lists all the ideas and asks participants
to rate each one on a scale. For example, a 7-point scale could be
used that ranges from 0 (no potential for dealing with the issue)
through 7 (very high potential for dealing with the issue). If this
approach is used, participants send the rating forms to the
Coordinator, who compiles the results and rank-orders the ideas
based on the evaluations.

A second approach for evaluating the ideas is that which is used in

the Nominal Group Technique for "voting." With this approach, the
Coordinator asks each member to identify the top five ideas and
assign five points to the most promising idea, 4 points to the next
most promising, and 3, 2, and 1 points to the third, fourth, and fifth-
best ideas. These votes are returned to the Coordinator, who tallies
the results and prepares a report. The report notes the rank order of
the ideas based on the total number of points received and indicates
the number of people who voted for each idea.

QUESTIONNAIRE #1

The purpose of this questionnaire is to elicit your ideas regarding the following
issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please engage in individual brainstorming so as to generate as many ideas as

possible for dealing with this issue. Please list each idea in a brief, concise manner
and email your response to me. Your ideas need not be fully developed. In fact, it
is preferable to have each idea expressed in one brief sentence or phrase. No
attempt should be made to evaluate or justify these ideas at this point in time. Your
ideas will be anonymously included in the next questionnaire.
Idea #1:

Idea #2:

Idea #3:

Idea #4:

Idea #5:

Idea #6:

Idea #7:

Idea #8:

Idea #9:

Idea #10:

QUESTIONNAIRE #2
The purpose of this questionnaire is to report all of the ideas sent in response to
the first questionnaire and to solicit new ideas for dealing with the issue:

What action could be taken to provide faster response to patient

inquiries between visits?
Please refine ideas already received by clarifying them where desired and by
listing the strengths and weaknesses you associate with each. Please list any new
ideas at the bottom of the questionnaire and comment on each new idea’ s
strengths and weaknesses for addressing the issue. Your ideas will be
anonymously included in the next questionnaire.
Idea #1
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #2
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #3
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #4
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #5
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #6
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #7
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #8
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #9
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #10
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #11
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #12
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #13
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #14
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #15
• Your clarification (if any):
• Strengths:
• Weaknesses:
My new ideas:
#1
• Strengths:
• Weaknesses:
#2
• Strengths:
• Weaknesses:
#3
• Strengths:
• Weaknesses:

QUESTIONNAIRE #3 (alternative A)
Attached is a listing of all strengths, weaknesses, and clarifications of ideas sent in
response to the second questionnaire dealing with the issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please further refine these ideas by providing additional clarification where desired
and by listing additional strengths and weaknesses you associate with each.
Please list any new ideas at the bottom of the questionnaire and comment on each
new idea’ s strengths and weaknesses for addressing the issue.
Please note that this will be the last questionnaire. After receiving all participants’
responses to this questionnaire, I will provide you with a sheet on which to assign
votes for five ideas you feel will best deal with the issue. All votes will be tallied and
results sent to participants. Your ideas will be anonymously included in the next
report.
Idea #1
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #2
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #3
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #4
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #5
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #6
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #7
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #8
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #9
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #10
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #11
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #12
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #13
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #14
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #15
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #16
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #17
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #18
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #19
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #20
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #21
• Your clarification (if any):
• Strengths:
• Weaknesses:
My new ideas:
#1
• Strengths:
• Weaknesses:
#2
• Strengths:
• Weaknesses:
#3
• Strengths:
• Weaknesses:
ATTACHMENT: RESPONSES TO SECOND QUESTIONNAIRE
Idea #1
• Strengths listed:
• Weaknesses listed:
Idea #2
• Strengths listed:
• Weaknesses listed:
Idea #3
• Strengths listed:
• Weaknesses listed:
Idea #4
• Strengths listed:
• Weaknesses listed:
Idea #5
• Strengths listed:
• Weaknesses listed:
Idea #6
• Strengths listed:
• Weaknesses listed:
Idea #7
• Strengths listed:
• Weaknesses listed:
Idea #8
• Strengths listed:
• Weaknesses listed:
Idea #9
• Strengths listed:
• Weaknesses listed:
Idea #10
• Strengths listed:
• Weaknesses listed:
Idea #11
• Strengths listed:
• Weaknesses listed:

BALLOT
The purpose of this ballot is to solicit votes for the five ideas that best deal with the
issue:

What action could be taken to provide faster response to patient

inquiries between visits?

The following are all of the ideas that have been submitted by participants. Please
identify the top five ideas and assign five points to the most promising idea, 4
points to the next most promising, and 3, 2, and 1 points to the third, fourth, and
fifth-best ideas. Vote for only five ideas. Your ideas will be anonymously included in
the next questionnaire.
Idea #1:
I would give this idea (0, 1, 2, 3, 4, 5) points.
Idea #2:
I would give this idea (0, 1, 2, 3, 4, 5) points.
Idea #3:
I would give this idea (0, 1, 2, 3, 4, 5) points.
Idea #4:
I would give this idea (0, 1, 2, 3, 4, 5) points.
Idea #5:
I would give this idea (0, 1, 2, 3, 4, 5) points.
Idea #6:
I would give this idea (0, 1, 2, 3, 4, 5) points.
Idea #7:
I would give this idea (0, 1, 2, 3, 4, 5) points.
Idea #8:
I would give this idea (0, 1, 2, 3, 4, 5) points.
VOTING RESULTS
The following reports the results of votes cast for ideas submitted on the issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please note the rank order of the ideas based on the total number of points
received. Also indicated are the number of people who voted for each idea.
Idea #6 received 22 points. 10 people voted for this idea.
Idea #13 received 20 points. 14 people voted for this idea.
Idea #2 received 19 points. 8 people voted for this idea.
Idea 18 received 15 points. 10 people voted for this idea.

QUESTIONNAIRE #3 (alternative B)
Attached is a listing of all strengths, weaknesses, and clarifications of ideas sent in
response to the second questionnaire dealing with the issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please further refine these ideas by providing additional clarification where desired
and by listing additional strengths and weaknesses you associate with each.
Please list any new ideas at the bottom of the questionnaire and comment on each
new idea’ s strengths and weaknesses for addressing the issue.
Please note that this will be the last questionnaire. After receiving all participants’
responses to this questionnaire, I will send each of you a rating form on which you
will be asked to assess the merits of all ideas submitted using a 7-point scale. After
those rating forms have been returned, I will compile the results and rank-order the
ideas based on participants’ evaluations. Your ideas will be anonymously
included in the next report.
Idea #1
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #2
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #3
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #4
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #5
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #6
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #7
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #8
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #9
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #10
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #11
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #12
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #13
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #14
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #15
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #16
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #17
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #18
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #19
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #20
• Your clarification (if any):
• Strengths:
• Weaknesses:
Idea #21
• Your clarification (if any):
• Strengths:
• Weaknesses:
My new ideas:
#1
• Strengths:
• Weaknesses:
#2
• Strengths:
• Weaknesses:
#3
• Strengths:
• Weaknesses:

ATTACHMENT: RESPONSES TO SECOND QUESTIONNAIRE

Idea #1
• Strengths listed:
• Weaknesses listed:
Idea #2
• Strengths listed:
• Weaknesses listed:
Idea #3
• Strengths listed:
• Weaknesses listed:
Idea #4
• Strengths listed:
• Weaknesses listed:
Idea #5
• Strengths listed:
• Weaknesses listed:
Idea #6
• Strengths listed:
• Weaknesses listed:
Idea #7
• Strengths listed:
• Weaknesses listed:
Idea #8
• Strengths listed:
• Weaknesses listed:
Idea #9
• Strengths listed:
• Weaknesses listed:
Idea #10
• Strengths listed:
• Weaknesses listed:
Idea #11
• Strengths listed:
• Weaknesses listed:

RATING SHEET
The purpose of this questionnaire is to report strengths, weaknesses, and
clarification of all ideas submitted in response to the third questionnaire for dealing
with the issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please assess the merits of all ideas listed below using a scale that ranges from 0
(no potential for dealing with the issue) through 7 (very high potential for dealing
with the issue). Please email your evaluation to me. After all rating forms have
been returned, I will compile the information, rank-order the ideas based on the
evaluations, and send you the results. Your ideas will be anonymously included in
the next report.
Idea #1:
This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.
Idea #2:
This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.
Idea #3:
This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.
Idea #4:
This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.
Idea #5:
This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.
Idea #6:
This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.
Idea #7:
This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.
Idea #8:
This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.
Idea #9:
This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.

RESULTS
The following reports the results of points that participants assigned to the ideas
submitted on the issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please note the rank order of the ideas based on the total number of points
received.

Idea #6 received 66 points, for an average rating of 3.3

Idea #13 received 56 points, for an average rating of 2.8
Idea #2 received 25 points, for an average rating of 1.25
Idea 18 received 20 points, for an average rating of 1.0

Copyright 1998 Randall B. Dunham;

University of Wisconsin-Madison School of Business Instruction web site
http://instruction.bus.wisc.edu/obdemo/readings/delphi.htm