The Critical Incident Technique and Educational Research

Author(s): John E. Corbally, Jr.

Source: Educational Research Bulletin, Vol. 35, No. 3 (Mar. 14, 1956), pp. 57-62
Published by: Taylor & Francis, Ltd.
Stable URL: https://www.jstor.org/stable/1474450
Accessed: 07-06-2020 15:19 UTC

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

Taylor & Francis, Ltd. is collaborating with JSTOR to digitize, preserve and extend access to
Educational Research Bulletin

This content downloaded from 140.213.188.240 on Sun, 07 Jun 2020 15:19:15 UTC
All use subject to https://about.jstor.org/terms
 Educational Research
Bulletin MARCH 14, 1956
f f Vol. XXXV, No. 3

The Critical Incident Technique and

Educational Research
By JOHN E. CORBALLY, JR.

USE of the "critical incident technique" has been increasing

in educational research. This method, originally devel-
oped by industrial psychologists, presents an excellent
means of investigating behavior, particularly the effectiveness
of specific behavior in fulfilling educational purposes.
The writer, who has recently completed a study' utilizing
this technique, felt that certain comments regarding its use in
educational research would be of value-both in evaluating
research using the method and in considering future utilization
of the technique. In order to accomplish these tasks, it seems
necessary to describe the method briefly, to consider its advan-
tages and disadvantages as a research tool, and to indicate
certain refinements which may increase its value for educational
research.
John C. Flanagan is generally recognized as the originator
of the critical incident technique. In an article written in 1949,
he outlined the methodology of the technique and certain re-
quirements for its successful use.2 In brief, the procedure
involves job analysis through a study of the total job rather
than through an investigation of the separate parts of the job.
In practice, this involves having competent observers watch
men on the job and observe the outcomes of their work. When
the outcomes seem especially satisfactory or unsatisfactory to
the observer in terms of the aims of the job, a report is made
of the actual behavior of the worker which led to these results.
From a large number of such reports, it is possible to isolate the
acts which seem to have the most influence on the effectiveness
SIt is expected that a monograph reporting this study, which is entitled "Critical
Elements in School Board and Community Relations," will be published by the College of
Education late in 1957.
' Flanagan, John C. "Critical Requirements: a New Approach to Employee Evalu-
ation," Personnel Psychology, II (Winter, 1949), PP 419-25.
57

This content downloaded from 140.213.188.240 on Sun, 07 Jun 2020 15:19:15 UTC
All use subject to https://about.jstor.org/terms
58 EDUCATIONAL RESEARCH BULLETIN
of the work. Such acts are known as the "critica
the job. The reports of behavior which led to es
factory or unsatisfactory outcomes are known
"critical incidents." The identification of critical elements is
valuable in that this indicates behaviors in which workers should
strive to reach effectiveness due to the significant effect of the
manner in which these elements are performed on the outcomes
of the job.
Those acts which do not become classified as the critical
elements of a job are considered to be "noncritical" for one of
two reasons. Either the performance of these elements varies
little from worker to worker or such performances bear little
relation to the final outcomes of the job. It should be noted
that the first alternative makes it imperative to realize that not
all noncritical elements are inconsequential.
In summary, the method consists of: observation of on-the-
job behavior, evaluation of significant success or lack of success
in meeting the aims of the job, reporting incidents which led to
marked success or failure in meeting the aims of the job, and
treatment of the data in such incidents to isolate and categorize
the critical elements of the job.
An example might well provide an additional summary to
this description of the critical incident technique. In this case,
a school superintendent was the observer and he was observing
his board of education as they performed their varying tasks.
He was observing them with particular emphasis on the manner
in which their behavior influenced school-community relations.
He observed, for example, one incident in which an irate citizen
appeared before the board demanding to know why his social
club was denied permission to use the school gymnasium for a
bazaar. The board members were courteous and tactful as they
carefully explained their written and long-established policy
regarding community use of school buildings. The citizen was
completely satisfied, thanked the board for its time, and compli-
mented the board members on the clarity of their policy.
This was, then, a report of a critical incident with satisfac-
tory results. From this report, the following critical elements
could be isolated:
The board adopts written policy to give consistency to its actions.
The board adheres to its written policy.
Board members are courteous and tactful with visitors at board
meetings.

This content downloaded from 140.213.188.240 on Sun, 07 Jun 2020 15:19:15 UTC
All use subject to https://about.jstor.org/terms
 MARCH 14, 1956 59
From a large number of
ments may be isolated and
insight into the critical ele
description of the metho
report of a study of researc

T HERE
oped inare always
one field for usedangers
in another. Asinoriginally
adaptingcon- a research tool devel-
ceived, this technique was used to study men as they worked
with machines (that is, airplanes, scientific instruments, assem-
bly lines).' In education, men are studied as they work with
men. This introduces a number of variables not present in the
first instance. Consequently, care must be taken to ensure that
the complexity of the job or task studied, particularly with ref-
erence to human interaction, does not invalidate the technique.
Thus, while it seems possible to study effective behavior as
one group (school-board members) works with one other group
(lay citizens), it does not seem logical to assume that the
method could be adapted easily to study a total program such
as school-community relations. (School here would include
teachers, pupils, administrators, and members of the school
board, and community would include individuals, organizations,
neighborhoods, and so forth). The number of variables--
already approaching a danger point in the first example-would
appear to be completely out-of-hand in the second.
Therefore, the use of the critical incident technique in
educational research should be restricted to studies of situations
with limited complexity.
Second, the method assumes that observers can report inci-
dents in which outcomes in terms of the aims of the undertaking
are clearly recognizable. This leads to two distinct problems.
Outcomes in education are often either deferred, unrecogniz-
able, or both. Aims are often unformed, controversial, misun-
derstood, capable of many interpretations, or any combination
of these and other factors.
Therefore, in designing a research project in which the
8 Flanagan, John C. "Critical Requirements for Research Personnel." Pittsburgh:
American Institute for Research, March, 1949. Mimeographed and microfilmed.
'Flanagan, John C. "Job Requirements," Current Trends in Industrial Psychology,
edited by W. Dennis (Pittsburgh: University of Pittsburgh Press, 1949, pp. 32-54). See
also, Gordon, Thomas, "The Use of the Critical Incident Technique in the Construction of
an Evaluation Procedure for Airline Pilots," American Psychologist, IV (July, 1949),
p. 301; and Preston, Harley O., The Development of a Procedure for Evaluating Ofcers
in the United States Air Force (Pittsburgh: American Institute for Research, 1948).

This content downloaded from 140.213.188.240 on Sun, 07 Jun 2020 15:19:15 UTC
All use subject to https://about.jstor.org/terms
60o EDUCATIONAL RESEARCH BULLETIN
critical incident technique is to be used, great care must b
to ensure that the problem is one in which aims and ou
can be recognized by various competent observers wit
validity and reliability.
A third problem in the use of this technique lies
danger that either the researcher, the reader, or both w
fuse the frequency of mention of a critical elemen
"degree of criticalness." At the present time, the use
critical incident technique does not in any way indicate
critical element is more critical than another. Some elements
may occur more often in the course of a job than will others,
but this frequency of occurrence in no way denotes a degree of
criticalness.
Therefore, reports of critical incident studies must stress
that the technique is not designed to discriminate between sev-
eral types of behavior with regard to their criticalness except to
indicate that some behaviors are critical and others are non-
critical.
The method does depend to a great degree on the subjective
judgment of competent observers. To the statistically minded,
this fact may lead to some deprecation of the use of the method
in research. Too often, however, educational research has suf-
fered from the application of one or both of two assumptions.
The first is to assign a high degree of objectivity to anything
that can be brought under statistical treatment. The second is
to hesitate to push into an area with research unless a method
can be devised which at least gives the appearance of complete
objectivity. To be sure, objectivity must be sought to the very
utmost of the ability of the researcher. However, a method
which provides usable and apparently valid results should not
be discarded because it seems to have elements of subjectivity.
Therefore, efforts should be made to improve the method,
but its use should not be discouraged because it seems to possess
elements of subjectivity.
The isolation and categorization of critical elements from
critical incident reports is an extremely difficult task. The exact
meanings of descriptions supplied by many different observers
from many varying locations are often obscure. Ideally, the
method functions best when the research design includes the use
of a team of well-trained observers. Practically, however, this
is often impossible. It becomes necessary, then, for the research

This content downloaded from 140.213.188.240 on Sun, 07 Jun 2020 15:19:15 UTC
All use subject to https://about.jstor.org/terms
 MARCH 14, 1956 61
worker to develop method
erably using some other re
process.
One final caution must be stated. In reporting research
using this method, great care must be taken to make clear the
meaning of such terms as "critical element," "critical incident,"
or "noncritical elements." While the terms offer little con-
fusion after some use, the casual reader may not assign the
proper meanings to the words and may rather easily make false
assumptions.
In spite of some disadvantages, the critical incident tech-
nique has much to offer the researcher in education and other
social-science fields. The technique offers an outstanding method
of studying a task in terms of the behavior of those engaged in
the task. It provides a means of studying a task in the actual
and varying situations where the task must be done day after
day. It provides recommendations which can be utilized imme-
diately by practitioners in the field. The data, which are gath-
ered in terms of critical incidents, provide much insight into the
problems facing individuals as they attempt to perform certain
tasks and provide case-study material for use in training others
to perform these tasks. Also, the data provide many examples
of good practice in the field which are useful for both in-service
and pre-service training. Finally, the findings from critical
incident studies have been used to aid in the development of
evaluation devices to screen those desirous of entering certain
fields of endeavor."
Research in education has placed increased emphasis on
behavior, particularly in the areas of teaching and administra-
tive competency. In the furtherance of investigations of this
type, no method seems to offer more promise than the critical
incident technique.

T HEarisepreceding section presented several problems which

in the use of the critical incident technique in educa-
tional research. In order to meet certain of these problems, the
following suggestions seem appropriate.
The bulk of the reported research in education using this
" Whalen, Richard E., Jr. "Effectiveness of Elected and Appointed School Board
Members." 1953. An unpublished doctoral dissertation on file in the library of Indiana
University. See also, American Institute for Research, "The Development of Tests of
Aptitude and Proficiency," The American Institute Research Note, No. 5, 1951.

This content downloaded from 140.213.188.240 on Sun, 07 Jun 2020 15:19:15 UTC
All use subject to https://about.jstor.org/terms
62 EDUCATIONAL RESEARCH BULLETIN

technique has been done by the individual

usually as part of a doctoral study. In v
arising from the choice of observers and t
observers' reports, it is likely that the tea
vide more fruitful results than can the in
Research projects using this method mi
training of a team of observers as a first
doubt that this approach will cause new pr
in observing a wide number of incidents an
ever, such a method, if properly develo
materially the objectivity of the results.
A second problem involves an extension
presently used, the technique identifies c
does not supply information regarding an
ness. Logic indicates that not all critica
tremely critical." Further study is nece
development of this next step. An hypoth
that this next step can be taken through a
of critical incidents. At present, the resul
be satisfactory or unsatisfactory in term
undertaking. A more careful analysis of t
well provide insight into the criticalness of
found in critical incidents.
A final, and difficult, problem arises in th
a critical incident. In most studies in the
crisis can be identified, but the many signifi
to it are less clear. It is likely that certain
be overlooked in a study because certain s
are overlooked by observers. Here again,
tered in the selection and training of obs
span used for observation.
The critical incident technique appears to
tages for educational research-particular
havior. The method presents several diffic
be taken to ensure that it is used only in
projects. On the other hand, the techniqu
standing advantages for certain types of
and, if certain refinements can be made, t
wide application and provide valuable finding

This content downloaded from 140.213.188.240 on Sun, 07 Jun 2020 15:19:15 UTC
All use subject to https://about.jstor.org/terms