TECHNOLOGICAL FORECASTING AND SOCIAL CHANGE 3, 427--439 (1972) 427

Subjective Methodology in Forecasting

F R I T Z R. S. D R E S S L E R

Abstract
The state of the art in subjective methodology is summarized? In addition, viewpoints and insights into
the nature of forecasting and subjective methodologies are offered. All are based on the fundamental
idea that subjective data and processes are built into the very nature of forecasting. An examination of
the research in subjective methodologies is made and found to be pursuing a common strategy--a
strategy that parallels the multi-indicator development in objective methodology. This is termed the
"SIG-process" strategy. It is further examined and articulated and found consistent with past and
present research, expanding applications of subjective methodology, and future innovations incor-
porating computer and computer-related technology.

I. The Anatomy of Forecasts

Forecasting as both an art and an emerging science is now recognized as an important
and separate body-of-knowledge with its own methods (underlying strategies) and
techniques (means to exercise strategies) [1]. Forecasting has become sufficiently
systematized to justify international attention by Jantsch [2] and to produce theoretical
papers such as by Wald [3] and reviews such as by Roberts [4].
The available literature suggests that the forecast process can be organized into two
fundamental "abilities":
1. The ability to detect and recognize underlying data patterns; that is, to measure.
2. The ability to utilize data patterns to give a projection of expected events; that
is, to project.
If, for example, the overall question to be addressed is a forecast of an individual's
health status five years into the future, the first logical step is to assess that individual's
health today; that is to measure his health today. Then, having that information, we
can proceed to project his health status five years into the future.
The separation of the forecast process into a measurement and projection step
becomes more distinct as the process becomes more formal. When, for example,
sufficient data are available and can be organized into analytical models, a measurement
model (to determine present status) and a distinct projection model (to determine future
values) are developed. On the other hand, the separation of the forecast process into
two steps becomes less distinct as the subjective content of the overall process increases.
In fact, both measurement and projection may pass completely into a subjective mode
and become indistinguishable and "buried in the intuitive capability" of individuals.
In general, both exploratory and normative forecasting [5] use a measurement and
projection step. However, on the basis of published procedures it is clear that
exploratory forecasting emphasizes the projection step, where as normative forecasting

FRITZ R. S. DRESSLERiS a member of the staff of the University City Science Center, 3508 Market
Street, Philadelphia, Pa. 19104. He has an extensive background in industry and government and as
a consultant in technological forecasting and economic analysis. He is with the University City
Science Center, Philadelphia, Pennsylvania.
i Based on research performed in support of health and behavior projects at the University City
Science Center.
Copyright © 1972 by American Elsevier Publishing Company, Inc.
428 FRITZ R. S. DRESSLEK
emphasizes the measurement step. This is reasonable considering the different require-
ments and aims of each type of forecasting. Exploratory forecasting "explores" into
the future with intuitive or formal projections. The emphasis is into the future using
known data (measurements) as a point of departure. On the other hand, normative
forecasting is oriented to establishing a basis for resource allocations and takes as input
given goals. It searches for the underlying data patterns (relationships) between
allocated resources and generated technology, based, primarily, on past known
or assumed development histories. Projection, if any, is minimized in the rush to
apply highly-refined analytical techniques to define an " o p t i m u m " set of resource
allocations.
In summary, the anatomy of a forecast is well described by Wald as a "cognitive
model in operative use." That is, forecasting brings into play, in some measure, sub-
jective data established, in part, by observations in experiments and in non-experimental
real-life events.

II. Subjective Methodology

As the nature of the item on which a forecast is to be made becomes less deterministic
and the number of its components and extraneous variables increase, the basis for a
forecast becomes increasingly subjective. Following Dalkey [6], subjective capability
can be thought of as a continuum, organized into three areas. At one end are assertions
for which there is a great deal of evidence. This information can be called knowledge.
At the other end is material that has little or no evidential backing--call it speculation.
In between is a wide area of material for which there is some basis for belief. This can
be termed the area of opinion. The dividing lines between these three areas are, of
course, vague. However, the concept of a three-way split guards against a tendency to
dismiss whatever is not knowledge as mere speculation. Given that forecasts have to
be made, it seems desirable to maximize the probability of staying within, at least,
the opinion area of subjective capability. This strategy has been applied throughout
historical times and can be viewed as a simple extrapolation of the adage "two heads
are better than one" to "n expert heads are better than one non-expert head." The
use of a group of experts should improve the probability of being within the opinion
area. Using a group also opens the possibility of obtaining a mutually reinforcing effect
to indeed generate a "group opinion" and control the negative affects of individual
speculations. It should be noted that individual speculations may at times be highly
desired. At other times these speculations may be counterproductive and tend to delay
or confuse the main issues involved.
The press and tempo of technological and cultural changes in contemporary society
have heightened the interest to achieve advances in subjective forecasting techniques.
One result of this heightened interest was a variation of the traditional group discussion
called brainstorming. The essential idea behind brainstorming was to create a dynamic,
no-ideas-barred discussion to breakdown stereotyped thinking and procedures. Often
brainstorming sessions were aimed at forecasting complex events but tended to develop
a "can-you-top-this-idea" free-for-all where subjective capability was suppressed by
shouts for attention and intellectual survival.
Brainstorming helped to expose the shortcomings of face-to-face interaction in
traditional group processes. The recognition of these shortcomings and the desire to
use groups of experts in subjective forecasting led to research to organize the entire
procedure and minimize the negative influences of face-to-face interaction.
SUBJECTIVE METHODOLOGY IN FORECASTING 429
Over the last twenty years, and especially within the last ten years, research has
produced an array of techniques to systematically elicit opinions as the basis for
making forecasts. When examined as a whole, these techniques are seen to have some
elements in common. They deal with expert opinion; therefore, they deal with subjective
data and processes. Also, they use some form of data feedback to produce an iterative
process involving a group of individuals to arrive at a "group consensus" (for example,
the Delphi technique). This being the case, these techniques can be collectively referred
to as Subjective Iterative Group Processes, or simply SIG-processes.
Clearly, the elements of SIG-processes have always been available. However, recent
research has skillfully organized these elements to achieve a common SIG-process
strategy:

1. Maximize the probability of being within the opinion area of subjective capability
(use experts in a group to moderate the influence of individual speculations).
2. Maximize group cooperative phenomena (use iterative procedures incorporating
information feedback).
3. Minimize the recognized negative influences of human interactions (retain some
form of anonymous response).

Two major thrusts to apply SIG-processes have evolved; one features controlled
face-to-face group interaction; the other allows no face-to-face group interaction.
Over the last twenty years, a number of studies [7] have detailed the shortcomings
of traditional group processes involving face-to-face interaction. Among the short-
comings, the most serious were found to be:

1. Undue influence of dominant individuals

2. Group pressure for conformity
3. Unwillingness of individuals to abandon publicly expressed opinions
4. "Semantic derailment" (much discussion in a group has to do with individual
and group interests, not with problem solving).

Given the above shortcomings, one simple approach would be to exclude face-to-
face interaction but still maintain the "group-of-experts rule." Such an approach has
developed. Although its foundations can be traced to work in the early 1950's, its name
--Delphi technique--did not appear in print until 1959 [8]. There is no face-to-face
interaction in the Delphi technique. Instead, formal questionnaires or other formal
communication channels, such as on-line computer communication, are used to convey
information between individuals and thus preserve a double anonymity. The respon-
dents are not known to one another, nor are any estimates associated with any respon-
dents.
The Delphi technique received a great boost in general interest with the publication
of a Rand Corporation Report by Helmer and Gordon [9] in 1964 on a study of
forecasting technological events. This publication was well timed and coincided with
a surge of iDterest in long-range exploratory forecasting, with, of course, a need for
an efficient and systematic technique to elicit expert opinion.
Researchers in subjective methodology are well aware of the shortcomings of
traditional group processes. They are also well aware of the powerful and beneficial
aspects of face-to-face interaction--the essence of the traditional group process. These
benefits include:
430 FRITZ R. S. DP,.ESSLF_~
1. Immediate data feedback
2. Sense of group "presence" and competition
3. Non-verbal communication
4. Clarification of misunderstandings.
The potential value of these benefits suggests that group processes contain the face-
to-face feature, but include procedures to diminish the negative aspects of face-to-face
confrontations. (Such attempts are also seen, for example, in panel discussions in-
corporating time-limits and a controlling moderator). Developments along this line
have been far more informal and difficult to identify than in the Delphi technique.
However, we can take the group process model reported by Delbecq and Van de Ven
[10] as representative of the state of the art using controlled face-to-face interaction.
It is termed the Program Planning Model (PPM) and has had wide application in
business, industry, government, and education. For example, Gustafson [11 ] has used
PPM as a conceptual framework to design a technique to elicit estimates of relationships
between diseases and indicator complexes. In this context PPM is transformed into
separate and unique application termed the Estimate-Talk-Estimate (E-T-E) technique.
Obviously, the ideal SIG-process would be one that contained all the benefits of
a face-to-face interaction while at the same time avoiding all negative affects. The trend
of research suggests that such an ideal is the goal towards which the various techniques
are moving. For example, Gordon reports [12] recent experiments with mini-Delphi
in which only a portion of the anonymity of Delphi is preserved. Turoff has developed
his Delphi conference [13] in which participants interact via an on-line computer using
teletype units or CRT's for message transmission. His idea is to automate the data
transmission and analysis normally used in Delphi and thus secure some of the speed
and communication advantages of direct interaction.
The trend of group-process developments as well as recent experimental and sug-
gested applications argue that SIG-Processes are a means to create effective com-
munications rather than a means to create "correct answers." This generalization is
in part the basis for adopting the collective term, SIG-process. This generalization is
also consistent with the overall movement of future research, a movement that presses
forward under the sheer need to obtain and use a broad range of information concerning
the future. It is not surprising then that SIG-processes have been used to generate ideas,
develop utility functions, collect data, and obtain value judgments--all of which have
no "correct answer."

l'ft. SIG-Processes
As shown, SIG-processes can be organized along two major thrusts, each producing
operational techniques: (1) no face-to-face interaction (e.g., Delphi) and (2) structured
face-to-face interaction (e.g., PPM and E-T-E). This suggests that the degree of
interaction between the individuals making estimates is an important criterion to
characterize SIG-processes. For example, brainstorming2 with its free face-to-face
interaction can be taken to represent maximum group interaction. On the other hand,
Delphi is highly structured and has no face-to-face interaction and thus can be taken
to represent minimum interaction. As a conceptual aid, consider an "interaction
continuum" ranging from maximum to minimum along which all possible SIG-

2 Strictly speaking, this is not a SIG-process. But, it does serve to definemaximumface-to-face

interactionand can be consideredthe limitapproachedby the relativelymorestructuredSIG-proccsses.
SUBJECTIVE METHODOLOGY IN FORECASTING 431
processes can be located. On such a continuum, PPM and E-T-E would be located
somewhere between the two extremes. Also, mini-Delphi could be located somewhere
between PPM/E-T-E and Delphi.
Discussions with researchers in subjective methodology suggest that another charac-
terization of SIG-processes is the amount of "integration" between individuals per-
forming the forecasts and the group requesting the forecasts. Integratioia can be taken
to mean the degree to which the individuals performing forecasts become involved
with the overall goals and requirements of the group requesting the forecasts. This
further suggests using an "integration continuum" to locate SIG-processes. Other
distinguishing characteristics can be given; however, interaction and integration "axes"
should be sufficient to give an elementary mapping of SIG-processes. Such a mapping
is given in Fig. 1.

L INTERACTION d
I- -I
BRAINSTORMING CMAX) BASIC DELPHI(.MIN)

~ E-T -E
Z

/'-'~ _ MI N I ~
k.. ottp,, "+ _'r

DIRECT
I
DELPHI~

INODIRECT
DISC USSIONSIDISCUSSIONS
NO ET,"
+1
I
I

I
FTF~ INTERACTION INTERACTION

FTF~FACE'TO" FACE
Fig. 1. Simplifiedmappingof selectedSIG-processes.

Figure 1 can be viewed as a simplified morphological mapping of SIG-processes,

designed more as an instructional device than as an analysis tool. In this case, the
identified SIG-processes are necessarily "fuzzy clouds" rather than well-defined com-
partments. Nevertheless, the mapping offers a device to view the differences and
similarities of various techniques, apply SIG-processes to various forecast situations,
and also offers a tool for projecting future trends in SIG-processes.
As seen, there is no easy way to characterize SIG-processes. However, as a group,
all are distinguished by a formal attempt to organize subjective forecasting. Indeed,
there is a formal separation between the individuals making forecasts and those
controlling the entire process. That is, SIG-processes can be distinguished by an
appropriate protocol to establish an "invisible discipline," to control the forecasting
process. For example, Delphi can be viewed as a "debate-by-questionnaire" between
two groups:
432 FRITZ R. S. DRESSLER

I. An executive group that has been established to plan and manage the entire
process, and
2. An expert group composed of carefully chosen individuals whose opinions are
to be obtained and refined during the Delphi process.

The basic pattern of an executive group and a distinct expert group is found in all
SIG-processes. Suitability of application is assured via a situation-determined protocol,
thus the groups are combined in a wide variety of ways. However, all protocols seek
simply to create communication between the interacting "units of individual activity"
rather than overlay an impeding bureaucratic procedure.
Do SIG-processes "work?" Which protocol is best for a given situation ? At this
time, both questions have not been fully assessed. But, most researchers agree that
SIG-processes "work" in that they generally produce a narrowing of opinions and
a shifting of the median as the questioning proceeds.
Dalkey [14] recently verified these results in a series of detailed experiments using
Delphi. A large number of UCLA students were given a questionnaire using "almanac-
type" questions with known answers against which the group consensus could be
checked. Admittedly, questions of this sort differ from estimates about the future, but
they afforded an opportunity to observe some of the characteristics of group inter-
actions and behavior under controlled conditions.
Do Dalkey's results reflect the performance of Delphi when the questions under
study relate to the future? No hard answer can be given; however, in Campbell's
short-range forecast [15] of future economic conditions made by Delphi and "normal"
methods, the Delphi forecasts proved to be more accurate.
Ament [16] has compared the 1964 long-range forecasts made under Helmer and
Gordon to actuality and found that of the 22 events forecast to occur by 1970, 15
have occurred, 5 have not, and 2 are uncertain. Again, not a hard answer; but, one
that tends to support the Delphi technique as useful for making exploratory forecasts.
The accuracy question has proven very difficult to evaluate. For example, Ament
found it necessary to convene a special panel to assess if the events discussed in the
Helmer and Gordon Delphi had indeed occurred, and if so, to what degree.
Research has also focused on the protocol of SIG-processes to determine if some
underlying phenomenon is present. For example, Dalkey found that the estimates
obtained by Campbell followed the same lognormal distribution as those given to his
"almanac-type" questions. Martino [17] has analyzed the data obtained in the responses
to the Delphi conducted by North and Pyke at TRW. He found a similar distribution.
These findings support the belief that the Delphi technique is not a chaotic process
but is in fact well-behaved and can be described quantitatively.
What is known about the accuracy and protocol of other SIG-processes ? Gustafson
[18] has performed controlled evaluations of E-T-E and three other techniques con-
sidered applicable to making estimates (measurements) in medical diagnosis. He found
the E-T-E protocol superior to Delphi for his estimating task. Do these findings
invalidate the SIG-processes concept ? The answer seems to be no if the difference in
forecast situations is taken into account.
Gustafson used only eight questions. All dealt with the heights of individuals, male
and female, and were, therefore, all within the scope of the respondenrs daily ex-
periences. Gustafson was definitely making measurement-type estimates in the area
of opinion.
SUBJECTIVE METHODOLOGY IN FORECASTING 433
On the other hand, the majority of Delphi applications deal with full forecasts
(measure and projection) or, in the case of Dalkey, with questions far removed from
the respondent's daily experiences, thus moving into the area of speculation. For
example, Dalkey asked such questions as, "What was the total motor fuel consumption
in millions of gallons in 1965 in Massachusetts?"
One way to view the Dalkey and Gustafson results is to say that each showed the
necessity of matching protocols to the nature of the required estimates.
If an estimate falls within the scope of experience and training of a set of motivated
individuals, use the E-T-E protocol to obtain the best answer. If, on the other hand,
an estimate is remote to the experience and training of a set of individuals or is in
the nature of an exploratory forecast, then use the Delphi protocol to diminish in-
dividual speculations and avoid the risks of direct face-to-face interaction. In this case
the forecast situation is such that there is little or no gain in applying the positive
aspects of face-to-face interaction--so why risk exposure to the negative aspects with
such a meeting.
The accuracy and protocol of SIG-processes have not been fully evaluated. However,
the limited, controlled work that has been performed supports the view that SIG-
processes are a means to create effective communication between individuals to
promote a common strategy for collective problem solving rather than create "correct
answers."

IV. Relationship of SIG-Processes to Other Forecast Methods

SIG-processes allow individuals to be exposed to and hopefully consider viewpoints
and data that they would not be aware of acting strictly as individual forecasters. In
this way SIG-processes have given subjective methodology a parallel development with
objective forecasting methodology. Figure 2 presents an organization of forecast
methodology that helps summarize this viewpoint.
In Fig. 2 the basic division of forecast methods is given along a subjective/objective
dimension. We can consider the subjective methods as those in which the forecast is
carried out "in the heads" of individuals. SIG-processes are shown as the means to
control subjective forecasts so as to consider a number of indicators 3 to guide the
required forecasts and thus achieve improved answers. These indicators are largely
generated during data feedback.
Objective methods are those by which the forecast process can be relatively well
specified. In the extreme, these methods allow the forecast process to be so well specified
that other researchers can reproduce the process and obtain the same forecast. Objective
methods lend themselves to mathematical organization and thus to computer-aided
operations. Objective methods have been further decomposed into two organized
approaches: Classification and formulation. The essential idea in classification is to
define the item of interest relative to predetermined groups. This approach requires
that near-homogeneous groups be determined and that data exist to allow matching
an individual item with a group and measuring and projecting the group variable of
interest. On the other hand, the formulation method assumes consistency in the relation-
ship (formula) between a variable and its indicators (causal variables). Changes in the
indicators are considered to yield average responses in the total population of interest.
3 An indicator is a variable that can be shown to influencethe item of interest. In objective method-
ology it is sometimesreferred to as a causal variable. Thus the term indicator is used here as a generaliza-
tion of causal variables applicable to both subjective and objective methodology.
30
434 FRITZ R.S. DRESSLER

i FORECAST METHODOLOGY i
I
I i
I,UD'ECT"E I
J
MULTI-
I I
ZERO
INDICATOR INDICATOR

'~ORCANIZED JUDGMENT I I
(SlG- PROCESSES) FORMULATION CLASSIFICATION

I I
I I I I
MULTI" ZERO MULTI" ZERO
INDICATOR INDICATOR INDICATOR INDICATOR

Fig. 2. An organization of forecast methodology.

Classification and formulation methods may utilize the same data. But note how
they differ. The formulation method assumes that all items in a group "respond" the
same way to changes in any given indicator, while the classification method makes
allowance for the fact that an indicator may "affect" different items in different ways.
The formulation method assumes that there is no interaction between indicators, while
the classification method is not limited to this assumption.
Figure 2 contains the terms "Multi-Indicator" and "Zero-Indicator." These terms
are used, respectively, to designate a method based on the ability to organize a number
of indicators and a method representing the limiting case of no indicators.
The multi-indicator case is often called causal; referring to the fact that a number
of indicators (causal variables) have been identified and are used in some organized
way in the forecast. The zero-indicator case is often called simple; and uses only the
item of interest (dependent variable); usually, an analysis is carried out to identify
its regularities over time.
Causal methods go beyond the dependent variable to consider indicators which
cause changes in the dependent variable. An attempt is made to (1) relate certain
indicators to the dependent variable; (2) forecast the indicators (simple forecast); and
(3) infer the value of the dependent variable from changes in the indicators. The key
assumptions are that the indicators can themselves be forecast relatively accurately
in comparison to the dependent variable and that established relationships will remain
constant over time.
The range from multi-indicator to zero-indicator, that is, from causal to simple can
be viewed as a "continuum." The exact number of indicators, if any, that should be
SUBJECTIVE METHODOLOGY IN FORECASTING 435
used will depend on the nature of the situation, the available data, and the required
estimates.
Clearly, Fig. 2 serves as an organization of forecast (measurement and projection)
methods in its own right. For example, using Figure 2 we can relate forecast methods
as shown below: 4
Common Terminology
Figure 2 Terminology (State of the Art)
Subjective Multi-Indicator SIG-Processes*
Objective Formulation Multi-Indicator Econometric or Regression
Objective Formulation Zero-Indicator Time Series
Objective Classification Multi-Indicator Configuration
Objective Classification Zero-Indicator Association*
* These names developed during research at the University City Science
Center.

It is difficult to conceive that under any circumstances a single item can be isolated
in the "subjective capability" of individuals; therefore, the subjective/zero-indicator
method is considered impossible to obtain and is, therefore, not given in the above
listing. Similarly, all listed methods are difficult to conceive as isolated or "pure"
methods. In reality, all borrow some support from each other and emerge only as the
main theme in any applied situation. Nevertheless, the listed methods serve to display
an overall approach to forecasting and thus aid in establishing a subjective methodology
distinct from an objective methodology. Note that through the application of SIG-
processes, subjective methodology has developed organized and comprehensive tech-
niques (multi-indicator) similar to objective methodologies.

V. Future Developments in Subjective Methodology

It seems proper that a paper discussing some elements of forecasting should contain
comments as to what the future holds for these elements.
At this time, subjective methodology is seen developing along two parallel paths:
(1) Developments in SIG-process techniques and, (2) a series of reciprocal developments
between SIG-processes and computer-related technology in data transmission, storage
and display, and analysis.
Increasingly, interesting SIG-process variations are being introduced to expand
their scope of applications. For example, Gustafson (utility functions), Williamson
(estimates of disease impairment), Turoff (steel production processes), and Scheele
(models of human and social interaction) have developed graphical schemes as the
basis for information exchange and final outputs in SIG-processes. 5
In the several studies which used graphical methods to generate quantitative data,
it was found that the range of opinions converges in later estimates and that the median
curve shifts, similar to conventional SIG-Processes.
Advances and refinements in protocols approaching the ideal SIG-Process can be
expected. For example, the Institute for the Future 6 has experimented with Mini-
Delphi--sort of a Delphi " o n the way to becoming" an E-T-E process. This protocol
4 Based on review of the literature, a number of common names can be identified for most of the
methods shown in Fig. 2. An attempt was made to pick the most common.
s D. Gustafson, University of Wisconsin, Madison, Wisconsin; J. Williamson, The Johns Hopkins
University School of Hygiene and Public Health, Baltimore, Maryland; M. Turoff, Officeof Emergency
Preparedness, Executive Office of the President, Washington, D.C.; S. Scheele, Social Engineering
Technology, Los Angeles, California.
6 The Institute for the Future, 2725 Sand Hill Road, Menlo Park, California 94025.
436 FRITZ R. S. DRESSLER
has been found to produce the characteristic narrowing of opinions and median shifts
found in the more elaborate Delphi protocol. Continued work by such individuals as
Dalkey, Gustafson, and Helmer may reveal a relationship between accuracy and
protocol and thus advance the state of the art in matching SIG-processes with given
forecast situations.
Historically, the majority of SIG-process developments have been along the first
path. That is, they concerned SIG-processes as such. Recently, however, computer and
computer-related technology has exerted a strong influence on SIG-processes. The
beginnings of this influence can be seen in the simple automation achieved by North
[19], who succeeded in processing his inter-round data automatically. A more extensive
use of technology is seen in the establishment of a data file of experts at the Institute
for the Future. Also planned is a system known as "D-net" which will extend the
utilization of this data file through the use of computers and modern communications
systems, permitting:
1. Matching of skill profiles and specific forecasts, and
2. Rapid exchanges between participants on a routine basis.
The most advanced application of technology was performed by Turoff [20] in the
spring of 1970. His "Delphi Conference" involved 20 individuals scattered across the
country, all tied into a central on-line computer. The whole operation was very similar
to a real-time accounting system--only opinions were recorded instead of accounting
information.
Delphi conference is still in an experimental stage. Based on results to date, the
questions about its future are of a "how-far-can it go" rather than a "does it work"
nature. In fact, Turoff has initiated design of a 2nd-generation Delphi conference that
will expand the present dialogue mode and incorporate cross-impact methods.
The advanced and expanded dialogue mode in future Delphi conferences will allow
increased interaction and integration between respondents and the executive group.
It can be viewed as a definite step towards the ideal SIG-process.
Given Turoff's work to date, it is possible to conceptually extend present ideas to
an on-line SIG-process having the following features [21]:
Extended Dialogue. Participants respond to original questions posed by the
executive group in terms of binary (Yes or No) or scaled voting, as well as make
comments and add new (better) questions. In addition, a number of "information
displays" can be made available on request, with or without monitor editing of
information. That is, the SIG-process can operate under monitor control or it can
enter a "democratic" mode of operation under which respondents engage each other
directly in an "at-arms-length" debate over specific issues. Pictorial as well as script
and numerical displays can be reviewed from on-demand data files. Also, all estimates-
in-progress can be reviewed as to voting, comments, counter-comments, and question
modifications.
Automatic Data-Cycling. A generalized cross-impact method [22] can be in-
corporated into the usual proceedings so that the degree of "mutual compatibility"
of estimates can be given as well as specific estimate information. This service can
also be made available at the respondent level so that individuals can "see" the
impact of changing their estimates when (I) considering only their set of estimates
and (2) considering all estimates made to date. This feature opens the practical
possibility of using two dimensions in estimating: (1) the estimates themselves and
SUBJECTIVE METHODOLOGY IN FORECASTING 437
(2) the "compatibility of estimates." For example, we can estimate (measure) the
health and behavioral descriptors of an individual; then ask what is the "degree
of compatibility" between the measured descriptors. In this case, the "degree of
compatibility" can be interpreted as the probability that an individual can indeed
have this unique set of descriptors.
Multi-Group Control. The executive group can conduct interlaced sessions with
different expert groups. For example, one set of experts can be used to estimate the
timing and feasibility of technological events (exploratory forecasting) while another
set can give the social-economic impact of technological events. A third group can
estimate the required resources (normative forecasting) to achieve selected feasible
and desired technological events.
Steady-State Operation. The great bulk of the planning and preparation pre-
liminary to a typical SIG-process need not be repeated every time forecasts are
required.
Is all this far fetched ? Developments in health delivery systems aimed at bringing
health services to such groups as prison inmates and remote Indian reservations are
approximating the concept of on-line SIG-processes. Also, the work be DeSanti et
al. [23] in Wisconsin is pointing the way to practical on-line SIG-processes on a state-
wide basis. Their success could be the forerunner of even more extensive on-line
applications. What we are already seeing is a combining of maturing subjective and
objective methodologies through the medium of computer- and computer-related-
technology to achieve a "learning system" concept in forecasting. Such a development
would parallel the trend in economic analysis [24], and, indeed, may help bring about
a learning-system stage in economic analysis.
In a learning system, SIG-processes would be inter-laced with objective procedures
in a series of feedback loops that permit parameter and even structural changes within
both methodologies. The system would incorporate emerging technology in com-
munication, data storage and display, and data analysis to achieve a "forecasting
utility" with continuous data collection and data interpretation. The system would
be clearly forecast oriented. That is, a learning-system concept would involve many
of the data elements of the usual management information system but could, in
addition:
1. Examine various data elements to seek new internal relationships, organize these
relationships, and thus create new data and models of real-world events. In other
words, measure real-world events.
2. Explore into the future. That is, project exp.ected values and events.
The specialization of techniques for measurement and projection can be expected
to continue; but, by incorporating advanced technology, they will become increasingly
interwoven, even at a high level of systemization.
In summary, the continuing impact of technology reinforces the viewpoint that the
primary objective of SIG-processes is "... the establishment of a meaningful group
communication structure" [25]. This in turn, it is felt, will lead to a recognition that:
I. The present use of SIG-processes to obtain numerical estimates is but a small
and specialized portion of their total capability. It represents their most developed
form and, because of its quantifiable nature, demonstrates in the most transparent
438 FRITZ R. S. DRESSLER
fashion (at least an interval scale to indicate the narrowing of opinions and
median shifts) the strategy underlying SIG-processes.
2. The concept of "group consensus" should be generalized to include considered
judgment expressed in a verbal mode, jointly developed by the executive and
expert groups. [In numerical estimates, a consensus can be mathematically defined
and developed from the answers supplied by the expert group. However, when
non-quantifiable information such as policy decisions is developed, the executive
and expert groups are drawn together (increased integration) so that it may
become impossible to identify separate outputs, thus the concept of the conven-
tional "expert-group consensus" no longer applies.]
3. "Truth" is more a process than a single event (fact), and requires interpretation.

VI. Summarizing Remarks

Historically, the scientific organization of subjective capability is seen as a series
of fragmented and, to a large part, independent and specialized developments. At least
three main "phases" can be identified:
I. Micro group-process research, performed by people-oriented individuals such as
social and behavioral scientists. "People-parameters" are emphasized. Numerical
values in process outputs are used more as surrogate people-parameters than as
useful data. [See, for example, Ref. 7.]
2. Macro group-process research, performed by systems-oriented analysts who tend
to view the individuals involved as elements in an overall system. System para-
meters are emphasized, focusing mainly on the relationship between the charac-
teristics of output data and system design. [See Refs. 6, I l, and 14.]
3. Combined micro/macro research. This phase is now rapidly developing, aided
primarily, by computer- and computer-related-technology. [See Refs. 13 and 20.]
All three phases of research can be "connected" by viewing them as promoting a
common problem-solving strategy (SIG-process strategy). This viewpoint also reveals
the inevitable "automation" of SIG-processes. That is, it reveals that subjective
methodology has, in its way, become sufficiently organized such that an appropriate
automation (computer-aided) with appropriate hardware is justified in parallel with
past developments in objective methodologies.
We tell ourselves we are in the "Great Objective-Methodology Age." However, the
shortcomings are beginning to show themselves [26], and it is now time for subjective
methodology to have a champion--if only to put some "balance" into the accepted
modes of problem solving. Why not follow the sound philosophy that organized data
and processes be applied as extensively as possible in each forecast situation. Therefore,
in some forecast situations, objective methodologies will be most appropriate and
emerge as the main theme; in others, SIG-processes will be most appropriate and
emerge as the central approach.
Admittedly, there remains the question of which SIG-process to use. The best advice
is to use common sense. Remember, all SIG-processes apply a common strategy to
elicit subjective data. They are, therefore, application techniques and not, as often
assumed, distinct methodologies. The trick is to devise the "best" protocol for a given
forecast situation. (Is measurement or projection emphasized? How much time is
available ? Other resources?) Some protocols and their applications are available in
the literature (basic Delphi and its many variations, various controlled interacting
SUBJECTIVE METHODOLOGY IN FORECASTING 439
n o m i n a l - g r o u p s , such as E-T-E, etc.) a n d are helpful. Also, Section III gives some
guidance.

References
1 A number of sources can be given to outline the methods and techniques of forecasting, for example,
R. Ayres, Technological Forecasting, McGraw-Hill, New York (1969); (ed.), J. Bright, Technological
Forecasting for Industry and Government, Prentice-Hall, Englewood-Cliffs, New Jersey (1969).
2 A book distinguished by its internationalscope, E. Jantsch, TechnologicaIForecasting inPerspective,
OECD Publications, Washington, D.C. (1967).
3 H. Wald, "Forecasting on a Scientific Basis", proceedings of an international summer institute
held in Cuvia, Portugal, September 1966, Centro De Economia E Financas (1967).
4 E. Roberts, Technological Forecasting 1, 113 (1969).
5 For a summary of exploratory and normative forecasting, see, for example, Jantsch or Roberts,
ibid.
6 N. Dalkey, The Delphi Method: An Experimental Study of Group Opinion, The RAND Corporation,
Santa Monica, California (June 1969) (RM 5888-PR).
7 A number of sources are available, for example, S. Asch, "Effects of Group Pressure Upon the
Modification & Distortion of Judgments", Readings in Social Psychology, 3rd ed., Holt, Rinehart,
and Winston, London (1958); H. Kelly and J. Thibaut, "Experimental Studies of Group Problem
Solving and Process", in Handbook ofSocialPsychology, Vol. 11(G. Lindzey, ed.), Addison-Wesley,
Reading, Mass. (1964).
8 0 . Helmer and N. Rescher, On the Epistemology of the lnexact Sciences, The RAND Corporation,
Santa Monica, California, R-353 (February 1960); also in Management Science 6 (1959), pp.
25-52.
9 T. Gordon and O. Helmer, Report on a Long-Range Forecasting Study, The RAND Corporation,
Santa Monica, California, (AD 607-7777) September 1964.
10 A. C. Delbecq, and A. Van De Ven, A Group Process Model for Problem Identification andProgram
Planning, University of Wisconsin, Madison, Wisconsin (October 1970).
11 D. Gustafson, and R. Ludke, Information Manual on Computer-Aided Medical Diagnosis Study
for Ph.vsician Likelihood Ratio Estimators, University of Wisconsin, University General Hospital
Aided Medical Diagnosis Project, Madison, Wisconsin (February, 1970).
12 T. Gordon, The Current Methods o f Future Research, The Institute for the Future, Middletown,
Massachusetts, 1970.
13 M. Turoff, Technological Forecasting 3 (1972, to be published).
14 See Ref. 6, also: S. Brown, S. Cochran, and N. Dalkey, The Delphi Method, H: Structure of Experi-
ments, The RAND Corporation, Santa Monica, California (June 1969) (RM 5951 PR).
15 R. Campbell, Methodological Study o f the Utilization of Experts in Business Forecasting, Ph.D.
dissertation, University of California, Los Angeles, California (September 1966, unpublished).
16 R. Ament, "Comparison of Delphi Forecasting Studies in 1964 and 1969", Futures, March 1970.
17 J. Martino, Technological Forecasting 1, 355-358 (1970).
18 D. Gustafson, R. Shukla, A. Delbecq, and G. Walster, A Comparative Study of Differences in
Subjective Likelihood Estimates Made by Individuals, Interacting Groups, Delphi Groups and Nominal
Groups, The University of Wisconsin, Madison, Wisconsin (January 1971).
19 H. North, and L. Pyke, 1EEESpectrum 6, No. 1 (1969).
20 Turoff, ibid, see also, M. Turoff, "A Synopsis of Innovation by the Delphi Method", paper pre-
sented at the 1970 ORSA meeting held in Detroit, Michigan.
21 F. Dressier, Subjective Methodology, UniversityCity Science Center (internal report), Philadelphia,
Pa. (November 1970).
22 For a review of the literature and the method see, M. Turoff, An Alternative Approach to Cross
lmpact Analyses, copies can be obtained from M. Turoff, Office of Emergency Preparedness,
Office of the Assistant Director for Resource Analysis, Executive Office of the President, Washing-
ton, D.C.
23 V. De Santi et al., Genie, Office of Economic Opportunity, Executive Office of the President,
Washington, D.C. (June 1971).
24 Roberts, ibid.
25 M. Turoff, "Delphi and its Potential Impact on Information Systems", to be given at the Fall
Joint Computer Conference Proceedings (1971).
26 I. R. Hoos, A Critical Review of Systems Analysis: The California Experience (NASA Rpt. No.
CR-61350), prepared by University of California at Berkeley for NASA, Washington, D.C.
(December 1968).