Prediction of mandibular rotation:

An empirical test of clinician performance Dr. Baumrind

Sheldon Baumrind, D.D.S., M.S.,* Edward L. Korn, Ph.D.,**

and Eugene E. West, D.D.S.***
Sun Francisco, Cahf.

An experiment was conducted in an attempt to determine empirically how effective a number of expert clinicians
were at differentiating “backward rotators” from “forward rotators” on the basis of head-film information which
might reasonably have been available to them prior to instituting treatment for the correction of Class II
malocclusion. As a result of a previously reported ongoing study, pre- and posttreatment head films were
available for 188 patients treated in the mixed dentition for the correction of Class II malocclusion and for 50
untreated Class II subjects. These subjects were divided into 14 groups (average size of group, 17; range, 6 to
23) solely on the basis of type of treatment and the clinician from whose clinic the records had originated. From
within each group, we selected the two or three subjects who had exhibited the most extreme backward rotation
and the two or three subjects who had exhibited the most extreme forward rotation of the mandible during the
interval between films. The sole criterion for classification was magnitude of change in the mandibular plane
angle of Downs between the pre- and posttreatment films of each patient. The resulting sample contained 32
backward-rotator subjects and 32 forward-rotator subjects. Five expert judges (mean clinical experience, 28
years) were asked lo identify the backward-rotator subjects by examination of the pretreatment films. The findings
may be summarized as follows: (1) No judge performed significantly better than chance. (2) There was strong
evidence that the judges used a shared, though relatively ineffective, set of rules in making their discriminations
between forward and backward rotators. (3) Statistical analysis of the predictive power of a set of standard
cephalometric measurements which had previously been made for this set of subjects indicated that the
numerical data also failed to identify potential backward rotators at a rate significantly better than chance, We
infer from these findings that the ability of clinicians to identify backward rotators on the basis of information
available at the outset of treatment is poor. Hence, we believe that it is unlikely that such predictions play any
consequential operational role in the planning of successful orthodontic therapy at the present state of the art.

Key words: Growth prediction, mandibular rotation, backward rotation, cephalometric analysis,
clinical judgment

I n 1969, BjCrk’ focused the attention of the location was perceived to be almost entirely of a
orthodontic specialty upon the contribution of mandibu- translational character, by which was meant that, ex-
lar growth rotation to the repositioning of the mandibu- cept for minor local oscillations in position, all points
lar dentition relative to the maxillary dentition and the on the mandible, including the teeth, were believed to
skull during growth and treatment. The dominant theo- displace uniformly with respect to the anterior cranial
retical construct of the specialty theretofore, based base.
largely on the work of Brodie’ and the Broadbent By means of his brilliant use of metallic implants,
group, had held that during growth the mandible trans- Bjiirk3 was able to demonstrate that the apparent rela-
located “downward and forward” relative to the an- tive constancy of the inclination of the lower border of
terior cranial base. This downward and forward trans- the mandible masks the existence of systematic rota-
tions of the deep structures of the mandibular corpu~.~
To varying degrees in different subjects, according to
From the Craniofacial Research Instrumentation Laboratory, Department of BjGrk, developmental changes carry the inner struc-
Growth and Development, School of Dentistry, University of California, San tures of the corpus either upward anteriorly and down-
Francisco.
This study was supported by National Institutes of Health-National Institute ward posteriorly (which has come to be called “for-
of Dental Research Grant DE03598. ward rotation”) or, alternatively, downward anteriorly
*Professor Growth and Development/Radiology/Orthopaedic Surgery. and upward posteriorly (which has come to be called
**Assistant Professor of Biomathematics, School of Medicine, University of
California, Los Angeles. “backward rotation”). The distinction between rota-
***Professor of Orthodontics, Department of Growth and Development. tional displacement and translational displacement is
371
372 Baumrind, Kern, and West Am. J. Orthod.
~Vovember 1984

A. Pure Translation

&Rotation of 20”Around o Center at 0

!
C .Rototion of 2O”Around a Center of 0’ ). Combined A 8 S
-

Fig. 1. Different types of translocation on a two-dimensional surface. Within each subfigure, the inter-
secting vertical and horizontal lines define a two-dimensional frame of reference with respect to which
displacements of points can be measured. This frame of reference may be thought of as, for example, a
cephalometric system with SN as its x axis and with its origin at sella. The rectangles of each subfigure
might then represent the positions of either the maxilla or the mandible at two time points. A, Pure
translation. Note that all points (represented by A, 8, and C) displace equally in the X direction and
equally in the Y direction. B, Rotation of 20” around a center at 0. Note that each point (A, 13, and C)
displaces differently in X and Y from every other point. C, Rotation of 20” around a center at 0. Note that
even though the angular rotation is the same as in subfigure B, the points A, /3, and C have displaced
differently from their displacement in that subfigure. D, Combined rotation and translation. This sub-
figure was created by combining the operations of subfigures A and B. Note that the displacements of
points A, 8, and C are different from those in any of the preceding subfigures. Despite the fact that the
subfigure was generated by translating first and then by rotating about point 0, there is an infinite
number of ways in which the timepoint 1 and timepoint 2 rectangles can be moved back into conso-
nance with each other. In general, the rectangles could be counter-rotated 20” around any point in the
plane and then translated in X and Y, but it is important to note that, even with the magnitude of rotation
held constant (at 20” in this case), the magnitudes of the translations in X and Y required to move the
rectangles back into consonance differ according to the location of the point about which the rotation
is performed, The implications of this fact to the interpretation of mandibular rotation will be considered
more fully in a future article

important because in rotational translocations each
point on the body displaces along a different vector
from all other points, rather than along the same vector
as is the case with a pure translation (Fig. 1). It follows
that when a mandibular rotation of a given magnitude
and direction occurs during treatment, different teeth
are affected differently, whereas if mandibular dis-
placements are translational in nature, all the lower
teeth will displace similarly.
The focus of this article is not upon theoretical
considerations but, rather, upon the manner in which
the concept of differential tooth displacement as a
function of rotational displacement of the mandible is
used in practice. Even before the publication of Bjork’s
1969 paper,’ some clinicians had pointed to a some-
what cruder rotation of the mandible, the effects of
which needed to be considered in the planning of treat-
ment. This was the so-called “opening of mandibular
VOiUrn~ 86 Prediction of mandibular rotation 373
Numhr 5

Table I. Description of the sample

CASE CHARACTERISTICS
“STUDY” CHARACTERISTICS
Case “True Class II
ID’ State” $ Se’ Age MPA-D GognSN Severity Overbite Overjet

Study #7205-Control a F 8yr lmo 29 2 32.9 -2.4 8.3 7.5

I---
19 Cases from which b F 8yr 7mo 33.2 35.7 0.7 7.0 11.0
2 Forward Rotators and C B 9yr 3mo 25.7 32.8 1.8 3.0 3.4
3 Backward Rotators d 0 8yr 5mo 39.4 45.6 -1.0 -30 8.2
were selected e 0 7yr 6mo 31.6 39.0 PO.1 0.4 3.1

Study #7302 - Control a F F 1Oyr 2mo 15.3 29 3 2.5 7.0 5.2

22 Cases from which b F M 8yr 5mo 22 2 33 8 13 1.6 78
2 Forward Rotators and C B F 7yr 6mo 18.2 28.3 2.3 3.7 3.7
2 Backward Rotators d 0 M 9yr Omo 19 1 22.6 2.4 66 6.6
were selected

Study #7503 - Control a F M 8yr Omo 32.5 38.2 -2.7 1.8 6.4
9 Cases from which b F M 9yr 6mo 28.5 33.7 -1.1 5.2 4.3
3 Forward Rotators and C F M 8yr Omo 26.1 32.4 0.7 0.4 6.3
2 Backward Rotators d 6 M 7yr Ilmo 28.3 36.4 --0.1 -1 8 19
were selected e 0 M 8yr Omo 32 9 35 2 0.1 3.8 8.9

Study #7202-Cervical a F M 8yr 5mo 27.1 34.7 1.2 1.4 2.1

6 Cases from which b F M Syrllmo 34.5 35.4 2.1 3.6 4.8
2 Forward Rotators and C I3 F 6yr 7mo 26.5 36.1 -- 1.0 04 5.1
2 Backward Rotators d B F IOyr Omo 18.2 28 3 ~ 1.4 5.0 7.6
were selected
-
Study #7301 -Cervical a F M 15~ Omo 21 6 30.1 ~0.6 5.5 4.9
14 Cases from which b F M 12~ Omo 29.8 33.0 0.1 3.5 7.1
2 Forward Rotators and C 0 F IOy 3mo 27.4 36.9 -0.2 1.8 7.0
2 Backward Rotators d f3 F 12~ Omo 26.9 34.9 -1.0 8.3 117
were selected
-
#7304 - Cervical a F F N/A 16.6 30.7 0.7 4.0 8.5
13 Cases from which b F F llyr Imo 27.3 32.6 -0.8 4.8 4.1
2 Forward Rotators and C B F 1Oyr Omo 29 3 29.8 - 1.7 45 4.6
2 Backward Rotators d 6 F 9yr lmo 34.2 31.4 -2.3 61 8.5
were selected

Study #7406 - Cervical a F F 9yr 4mo 27.7 38.4 - 2.1 -5.4 9.6
22 Cases from which b F M 8yr 9mo 24.1 30.0 - 1.6 3.8 9.9
3 Forward Rotators and C F F 9yr 6mo 24.0 32.2 0.3 ~ 1.0 7.1
3 Backward Rotators d B M 8yr 9mo 25.6 31.4 -2.2 5.4 12.6
were selected e B M 8yr 6mo 24.3 35.7 - 1.2 6.3 6.1
f 0 F 9yr 5mo 24 5 35 0 -0.4 1.1 9.1
-
Study #7407 - Cervical a F F 9yr Omo 30 4 38.2 -3.3 1.7 8.9
19 Cases from which b F F Ilyr 9mo 26.6 33.2 -0.7 2.4 10.0
3 Forward Rotators and C F M llyr 7mo 26.9 33.6 1.2 2.1 2.5
3 Backward Rotators d El M 12yr Omo 25.5 32.6 ~ 1.3 4.5 8.9
were selected e B F IOyr 9mo 25.7 29.3 -0.7 5.3 IO 7
f B F 1Oyr 4mo 21.5 31.1 ~ 1.4 6.1 5.9

*Recoded and reordered for this publication.

$Measured direction of rotation between time points. (Mean of four independent measurements.)

plane angle ’ ’ around a fulcrum located near the con-
dyle, which Schudy,j Root,6 and others believed would
occur if therapeutic forces were employed in such a way
as to extrude or evulse the posterior dentition in either
jaw. According to this formulation, extrusion of the
upper molars by the use of cervical traction or of the
lower molars through the use of heavy Class II elastic
forces (or, for that matter, the extrusion of any teeth
through the use of intermaxillary “vertical” elastics)
would tend to cause a “backward rotation” (that is, an
 Am. J. Orthod.
374 Baurnrind. Korn, und West
,November 1984

Table I (Cont’d)

CASE CHARACTERISTICS
‘STUDY” CHARACTERISTIC5
Case “True
I.D.’ State”* Sex Age MPA-D Gogn SN iL:$ Overbite Overjet

Study #7105 - High Pull a F F 8yr Imo 29.3 38.8 m-2 3 0.4 10.8
20 Cases from which b F F IOyr Imo 31.3 37.3 0.3 1.0 10.9
3 Forward Rotators and C F F 7yr 9mo 20.5 36.4 3.1 2.2 9.8
3 Backward Rotators d B F 7yr 4mo 38.0 48.4 3.4 -0 9 10.0
were selected e B F 8yr 4mo 24.4 37.9 0.8 m~3.6 10 8
f B F 7yr IOmo 28.0 35.8 2.0 15 10 8
Study #7306- High Pull F F 8yr 6mo 28.9 36.8 -18 22 92
17 Cases from which F F 7yr Imo 30.1 32.8 1.8 0.4 8.6
2 Forward Rotators and B M 8yr Omo 32.9 38.6 1.7 4.0 10 1
2 Backward Rotators B F 8yr 7mo 39.5 47.1 08 08 11 7
were selected
__-
Study #7501 -High Pull F M IOyr Omo 30.4 42.9 2.7 4.0 9.8
16 Cases from which F M 8yr Omo 27.5 41.5 ~ 1.3 32 12.0
2 Forward Rotators and B M 12yr 3mo 32.8 44.1 1.3 4.1 9.0
2 Backward Rotators B F 8yr 2mo 24.5 36.7 2.0 50 7.6
were selected

Study #7204-Activator a F M 10yr 2mo 33.6 41.9 2.6 7.3 8.8

20 Cases from which b F M 9yr Ilmo 20.9 25.6 3.7 3.4 8.6
2 Forward Rotators and C B F 9yr IOmo 33.7 35.6 1.1 6.7 7.3
2 Backward Rotators d B M IOyr 6mo 30.9 38.6 23 4.8 77
were selected

Study #7305-Activator a F F IOy 8mo 31.5 37.3 -0.4 3.5 10.4

18 Cases from which a F F 8yr 9mo 28.5 38.7 -0.9 3.5 12.8
2 Forward Rotators and C B M 9yr 6mo 32.0 39.9 1.0 43 8.0
2 Backward Rotators d 6 M 1Oyr 3mo 22.5 31 8 4.7 89 112
were selected

Study #7403 - Activator a F F 1Oyr 3mo 31.0 41.4 -0.6 4.8 6.9
23 Cases from which b F F Ilyr 2mo 26.6 34.1 0.4 1.2 10.7
2 Forward Rotators and C B F 8yr 7mo 31.2 36.5 -0.4 5.0 126
2 Backward Rotators d B M Ilyr 5mo 27.1 33.4 -69 7.5 10.2
were selected

*Recoded and reordered for this publication.

$Measured direction of rotation between time points.

increase in mandibular plane angle). The effect of such
a displacement would be to move the lower teeth
“downward and backward” relative to the upper teeth,
making it more difficult to correct the Class II malrela-
tionship.
In some sense, the ideas of Bjork on the one hand
and of Schudy, Root, and others on the other hand are
by now intertwined in the clinician’s mind. It is now
generally assumed to be the case that “backward
rotators,” however defined, are more difficult to treat
and require a different mechanotherapeutic approach. It
therefore becomes important to identify prior to treat-
ment those subjects who are candidates for “backward
rotation” during therapy. Various signs have been pre-
sented which would lead the clinician to anticipate
backward rotation. Among these are the AFH/PFH
“proportionality” of Isaacson, Bjork’s descriptions of
mandibular conformation, Schudy’? focus on vertical
development, and the generally held doctrine that
“high-angle cases tend to open. ” The general question
that this article attempts to answer, by means of an
empirical experiment, is “How well can expert clini-
cians identify and classify potential backward rotators
on the basis of information available to them before
treatment is begun? ’ ’
MATERIALS AND METHOD
This study is based on the examination of records
and data from a previously reported study of the ef-
fects of mixed-dentition Class II treatment.*-I3 The
data were obtained by the UCSF method of head-film
analysis .I4
 Prediction of mandihdar rotation 375

Within our previously reported data and record Table II. Qualifications of the judges*
base, a sample of 238 subjects had been identified.
These cases had been drawn from a larger original Judge I.D.
sample of 303 subjects on the basis of criteria discussed 1 2 3 4 5
in an earlier article.‘O These 238 cases include records
for 74 cervical, 53 high-pull, 61 activator, and 50 un- Excluswe Practice 33 24 35 32 19
treated Class II control patients. They are subdivided University Instructor 30 11 35 22 18
into 14 groups, which we call “studies, ” on the basis
of their dates of acquisition by our research group. ABO Member 28 13 35 24 11
Within each study, all cases are from the office of Angle Society Member 28 15 26 28 10
the same clinician or investigator and were treated by
the same therapeutic method. The sample sizes in the Published Paoers 11 4 6- 2
several studies vary from 6 to 23 (mean, 17). Within
each group the two or three cases that showed the
“All time intervals are in years
greatest forward rotation and the two or three cases that
showed the greatest backward rotation of the mandible
during the treatment or observation period, measured as
the change in the mandibular plane angle of Downs of the UCSF Craniofacial Research Package from co-
MPA(D), were drawn for use in the present experi- ordinate data previously stored in our data base.‘”
ment. Cases were drawn retrospectively, solely on the The conventional measures were facial plane angle,
basis of the magnitude of rotation. Some representative angle of convexity, AB plane angle, mandibular plane
measurements for the sample thus selected are listed in angle of Downs, Y axis angle, occlusal plane angle of
Table I. Downs, interincisor angle, Ul to AP (Ang), Ul to NA
Five expert judges were asked to evaluate the 64 (mm), Ul to NA (Ang), Ll to NB (mm), Ll to NB
pretreatment films using any and all techniques that (Ang), occlusal plane to SN angle, SNA angle, SNB
they considered appropriate. Each of these judges had angle, ANB angle, GOGN: SN angle, pogonion to NB
practiced orthodontics exclusively for at least 23 years, (mm), and lower 1 to mandibular plane angle. All of
was a diplomate of the American Board of Orthodon- these measures were computed as previously defined.‘j
tics, and had served as an instructor or professor of The measures not usually made on head films by
orthodontics. Table II summarizes the qualifications of clinicians were gonial angle, ramus length, mandibular
the expert judges. body length, total face height, upper face height, lower
One judge at a time examined the four to six films face height, and condyle-SN distance. The definitions
from each study. His sole task was to identify, by any of these measures may be seen in Fig. 3 and its associ-
techniques he considered appropriate, which cases in ated legend. A typical set of values from a representa-
that group had rotated backward and which had rotated tive study may be seen in Fig. 4, A and B. A typical
forward. The judges were specifically nor requested to scoring sheet may be seen in Fig. 4, C. Finally, the
rank order the cases in terms of magnitude of change. judges were provided with information as to how many
The original (pretreatment) head films of the four to of the four to six cases being evaluated in each study
six cases in the study were mounted on a bank of view had actually shown increases in MPA(D) and how
boxes sufficiently large so that all films from each study many had shown decreases in MPA(D) as measured on
could be examined at the same time. A typical set of the timepoint 2 film.
four films from a representative study may be seen in To our knowledge, little quantitative research has
Fig. 2. The judges were permitted to reposition the been done on the impact of treatment upon mandibular
films in any order or orientation they desired and to rotation. Isaacson and associates16 have noted that, on
make any measurements they wished. To simplify their average, treated cases show a common tendency to
measurement task, they were also provided with the backward rotation (which could be measured by in-
pretreatment numerical values for a set of conventional crease in MPA(D)) while untreated cases show a ten-
head-film measures frequently made by clinicians for dency toward forward rotation (which could be mea-
use in treatment planning. They were also provided sured as decrease in MPA(D)). Our own observations
with values for some additional measures not usually on the full sample from which these cases are drawn
made on head films but which we thought might possi- confirm these findings. lo* i1 These tendencies might be
bly be of aid in the decision-making task. These mea- expected to have important impacts on the conclusions
sures had been computed automatically by the software that our expert judges were asked to draw, since clearly

376 B aumrind, Korn, and West
Fig. 2. A typical set of four pretreatment
films from Study 7304, coded randomly
appliance wear must have been uneven within each
study. For this reason, we considered it desirable that
the judges have some way of estimating the degree to
which the subjects adhered to the prescribed regimen of
appliance wear. However, no direct quantitative mea-
sure of diligence in appliance wear is available in our
study (or in any other clinical study of treatment effect
with removable appliances of which we are aware). As
an estimate of appliance wear, we therefore substituted a
measure of treatment efficiency in modifying the major
measure toward which treatment was directed-to wit,
the achievement of correction of the Class II malrela-
tionship. Hence, we asked the judges to reassess their
first conclusions by re-examining the films, given two
additional pieces of data for each case. These were (1)
the magnitude of Class II correction achieved (as de-
Am. J. Orrhod.
.Vwemher 1984
head films from a single study. The four pretreatmeni head
as they were when presented to the judges.
fined below) and (2) the elapsed time between films.
We reasoned that the judge could use .this additional
information to estimate the diligence of appliance wear.
Such an estimate would obviously be far from a perfect
measure, but it might well be helpful within the indi-
vidual studies.
PRIOR ESTIMATES BY THE EXPERTS OF THEIR
OWN PREDICTIVE EFFICIENCY
Orthodontists differ in their estimates of their own
skill in judging development on the basis of examina-
tion of pretreatment films. For example, Ricketts and
his associates17xl8 believe that the pattern of change
in mandibular orientation and confirmation can be
discerned by their method of arcial growth estima-
tion, with or without specialized computer simulation.
Volume 86 Prediction of mandibular rotation 377
Number 5

Johnstonrg and Harris and associateszOhave devised

templates for the rough prediction of future growth
from head films, although their claims concerning the
therapeutic utility of these devices are considerably
more restrained than those of Ricketts and co-workers.
BjSrk I* 3 believes that clinically useful predictions can
sometimes be made from head films using a knowledge
base derived from subtle empirical observation of the
developmental patterns of bone remodeling and tooth
eruption.
It might be expected that there would be some rela-
tionship between the strength of an orthodontist’s belief
in his ability to predict and his rate of success in pre-
dicting, although it is by no means clear in which di-
rection such a belief would act. The present judges
varied in the strength of their beliefs in their powers
of prediction; none of the five judges believed that
he could judge correctly all the time, but four of the five
were quite sure their ability to identify backward
rotators was considerably better than chance.

STATISTICAL DESIGN Fig. 3. Additional measures. In addition to a set of standard

The purpose of the study was to determine how cephalometric measurements made on the pretreatment head
film for each case, the judges were supplied with values for
well experts can distinguish outcomes as to forward
seven measurements not ordinarily made by most clinicians,
and backward mandibular rotation on the basis of in- These additional measurements and their definitions are as fol-
formation available on pretreatment films. Within each lows: 7, Gonial angle-The included angle condyle to gonion
of 14 groups of cases (each group being called a to menton. 2, Aamus length-The distance from condyle to
gonion. 3, Mandibular body length-The distance from gonion
“study”), the most extreme forward- and the most ex-
to menton. 4, Total face height-The distance from nasion to
treme backward-rotating cases (determined retrospec- menton. 5, Upper face height-The distance from nasion
tively) were collected so as to form a subsample of to ANS. 6, lower face height-The distance from ANS to
extreme cases. Five experts were then asked to identify menton. 7, Condyle-sella distance-The distance from sella to
the backward rotators within each subsample on the condyle.
sole basis of information available in the pretreatment
films and the knowledge of the number of forward- and
backward-rotating cases within the subsample.
Some considerations in this study design were as rotators. This was done in order to approximate the
follows: clinical situation of an experienced clinician who has
1. The experts were not required to rank the obser- some knowledge of the prior probability of the per-
vations within each study subsample by magnitude or centages of forward and backward rotators present in
to estimate the actual magnitude of rotation. This was the general patient population. Had we not given the
done because the usual clinical task when one is ac- judges prior knowledge of the number of forward
tually treating patients is not to rank order or to estimate rotators and the number of backward rotators within
magnitude of change but, rather, to identify extreme each subsample, the task of estimating outcome as be-
cases. An additional consideration in not requiring rank tween forward and backward rotators would have been
ordering was the fact that the differences in magnitude rendered considerably more difficult.
among the two or three forward rotators within each 3. The experts were given the films in 14 separate
subsample (or, alternatively, the differences among the packets (one study subsample to a packet) because we
two or three htrckward rotators within each subsample) did not want to confound differences in type of treat-
were usually quite small and were not of clinical conse- ment and in quality of films in the different studies
quence. with the differences in forward-rotating and backward-
2. Within each of the 14 subsamples, the experts rotating conformation which we were attempting to
were told how many of the four to six cases supplied identify.
were forward rotators and how many were backward 4. In order to preserve the independence among the
 Am. J. Orthod.
378 Baumrind, Korn, and West November 1984

Fig. 4. Facsimiles of representative numerical materials. A, The set of head-film values supplied to
each judge to aid him in his first evaluation of the four head films shown in Fig. 2. B, The additional
head-film values supplied to each judge to aid him in his second evaluation of the four head films shown
in Fig. 2. C, The completed score sheet of a single judge for both the first and second evaluations of all
the head films in the experiment.

Table III. Probability of successful classification of mate as to the probable diligence with which the
backward rotators by chance alone* appliance was worn.
It should be apparent from the design that this study
Number of Percent of was not intended in any way to test the question of
backward rotators backward rotators One Sided whether or not (and to what extent) rotations of the
correctly classified correctly classified P Values mandible actually occur in growth and/or treatment.
Rather, it tested the degree to which such rotations
17 53% 43 (assuming that they do occur) can be predicted by
18 56910 .27 skilled clinicians from physical records available to
19 59% .14 them at a single timepoint prior to treatment-that is to
20 63% 07 say, under the conditions which usually obtain in clini-
21 66% ,025 cal practice.
22 69% .0083
ANALYSIS
23 72% .0023
24 75% .0005 In analyzing the data as to the effectiveness of the
25 78% .oooi judges’ estimates, we originally asked the following
questions:
*When there are actually a total of 32 backward rotators out of a total 1. To what extent are the judges successful in
of 64 cases. identifying the correct cases as backward
rotators? Are the observed differences from
chance, if any, sufficiently great to be statisti-
judgments of the several judges, only one judge at a cally significant?
time was permitted to examine the films. 2. To what extent are the different judges in
5. Each expert examined the films from each study agreement in their estimates of different cases,
twice: once without data as to the duration of treatment irrespective of whether or not their estimates are
and the magnitude of Class II correction and a second correct?
time with such data available. The additional data were 3. Is there information in the numerical analyses
provided to the judge during the second examination in which would, if used optimally, improve the
order to allow him to make a somewhat realistic esti- performance of the judges? (In other words, are
Volume 86 Prediction of mandibular rotation 379
Numbrr 5

Table IV. Concordance among the estimates of the judges

FIRST ESTIMATION SECOND ESTIMATION

Study Number and “True
yD:e TOTAL Judge No. Total Judge No.
Treatment Type State”*
0 F 12345 B F 12345

7407 - Cervical a F 4 1 BBB B 3 2 BB B

b F 0 5 23 BB
C F 4 1 BBBB 4 1 BBBB
d B 2 3 BB 3 2 BBB

r
f B 323 2 BB B BB 42 31 B B
7105 - High Pull a F 2 3 BB 3 1 -BBB
b F 23 B 0 4 0 B-BBB
C F 0 5 13 -B
d B 5 0 BBBBB 2 2 B- B
e B 5 0 BBBBB
f B 1 4 B 22 B-

7306 - High Pull a F 23 B B 32 BB 0

b F 0 5 2 3 B B
C B 4 1 BBBB 3 2 BB B
d B 4 1 BBBB 2 3 BB
+
7501 -- High Pull a F 4 1 BB B 3 1 B-BB
b F 2 3 B B 2 2 B- f3
C B 4 1 BB BB 13 - I3
d B 0 5 2 2 -B B

7204 - Actwator
t a F 5 0 BBBBB 5 0 BBBBB
b F 23 BB 1 4 B
C B 1 4 B 1 4 I3
d B 2 3 BB 32 BB B

7305 - Acttvator a F 1 4 B 4 1 BBBB

b F 1 4 B 1 4 B
C B 4 1 BBBB 3 2 0 BB
d B 4 1 BB BB 23 B B
i
7403 - Actwator a F 3 2 BBB 2 2 -BB
b F 23 BB 2 2 -0 0

: B 41 41 B BBB B 222 2 B- B B
L
*Recoded and reordered for this publication.
*Measured direction of rotation between time points

the numerical analyses better than chance in
ways different from the conclusions of the
judges?)
The first question can be answered for the present
design by means of the Mantel-Haenszel statistic.21 In
the present sample we have 14 subsamples from as
many studies. Of these subsamples, 9 have two forward
and two backward rotators, 3 have three forward and
three backward rotators, 1 has two backward and three
forward rotators, and 1 has three backward and two
forward rotators. The last two subsamples are the only
ones that contain unequal numbers of forward and
backward rotators. Taken together, these two untreated
studies are, as it were, “mirror images” of each other.
Since the two unmatched subsamples represented stud-
ies within which the same treatment (that is, no treat-
ment) had been delivered, it is possible to say that, for
each of the four treatment types, the number of forward
and backward rotators is precisely matched.
The analysis focused on the efficacy of the experts
in identifying the backward rotators in the sample.
There were 32 such cases altogether. According to the
Mantel-Haenszel statistic, one would expect 16. I of
these films to be correctly classified by chance alone.
For this particular study design, that number is very
close to exactly 50% of the backward rotator cases, but
in general it need not be so. The statistical significance
of a given discrimination of the films into two groups
depends only on the total number of backward rotators
correctly identified and not on the specific number cor-
380 Baumrind, Korn, and West Am. J. Orthod.
November 1984

Table IV (Cont’d)

FIRST ESTIMATION SECOND ESTIMATION

Study Number and “True
Treatment Type %” t State”* Total Judge No. Total Judge No.
B F 1234 5 B F 12345

7205 -Control a F 2 3 B B 4 1 BBB B

b F 5 0 BBBBB 3 2 BBB
C B 14 B 14 B
d B 5 0 BBBBB 5 0 BBBBB
e B 2 3 B B 2 3 B B

7302 -Control F 4 1 BBBB 1 4 B

z F 23 BB 4 1 BBBB
C B 14 B 23 B B
d B 3 2 BBB 32 BB B

7503 -Control F 4 1 BBBB 2 3 B

b” F 1 4 B 1 4 B
C F 0 5 2 3 B B
d B 2 3 B B 2 3 B B
e B 3 2 B B B 3 2 BBB
7202 - Cervical a F 2 3 BB 3 2 BBB
b F 3 2 BBB 14 B
C B 4 1 BBBB 3 2 B BB
d B 1 4 El 4 1 B

7301 - Cervical a F 3 2 BBB 3 2 BBB

b F 3 2 BB B 32 B BB
C B 4 1 BBB B 14 B
d B 0 5 14 B

7304 - Cervical a F 2 3 BB 1 B
b F 3 2 BBB 3 B B
C B 1 4 B 3 BB
d B 4 1 BBBB 2 B BB

7406 - Cervical a F 5 0 BBBBB 2 B-B

b F 1 4 B 1 B-B B
: BF 32 32 BBB 2 -BB
B B 0 B-BBB
e B 1 4 B 4 -
f B 3 2 BB B 3 - B

*Recoded and reordered for this publication.

*Measured direction of rotation between time points.

rectly identified within each study. For our particular
study design, the one-sided p values for a given number
of correctly classified “backward-rotator” films are
given in Table III. For example (from Table III), an
expert who classifies 22 of the 32 backward rotators
correctly (69%), will be doing 19% = 69% - 50%
better than would be expected by chance (50% =
16.1/32). The statistical significance of this expert’s
classification would be p < 0.01 (actually, p =
0.0083). We see that, for this experiment, an ex-
pert will have to classify at least 66% of the backward-
rotator cases correctly to be statistically significant at
the p < 0.05 level.
RESULTS
The raw values for the first categorizations by the
five judges (made without information as to the elapsed
time between films and the magnitude of Class II cor-
rection) are shown in Table IV. The first column of this
table states the “true” condition for each case, as a
backward or forward rotator. This value has been de-
termined by calculations of the change in MPA(D) be-
tween two films taken approximately 2 years apart. The
landmarks used in these calculations have been derived
by averaging independent replicate estimates by four
judges using a previously described method.14 The re-
maining columns of the table list the individual as-
Volume 86 Prediction of mandibular rotation 381
Numkr 5

Table V. Number and percent of backward rotators correctly identified

Estimation Judge 1 Judge 2 Judge 3 Judge4 Judge 5

Fwst 17 (53%) 15 (47%) 15 (47%) 17 (53%) 19 (59%)

Second 17 (53%) Incomplete 11 (34%) 16 (509/o) 14 (44%)

sessments by the judges, and the concordances among
judges for both the first and second estimations are
listed in the remaining columns of the table.
With these data at hand, we may proceed to answer,
in order, the three questions posed in the preceding
section on analysis.
Question 1. To what degree were the
judges successful in identifying the correct cases
as backward rotators?
The answer to this question may be obtained by a
simple ennumerative tabulation of the data in Table IV.
The results of such an operation, in terms of number
and percent of successes for each judge, are shown in
Table V.
It may be seen in this table that the most successful
judge (Judge 5) correctly identified 19 of the 32 back-
ward rotators, a success rate of 59%. Consulting Table
III, we observed that the probability that such a success
rate could have been achieved by chance alone is 0.14.
From these data, we conclude that none of the experts
performed at a level that was statistically better than
chance alone.
Question 2. To what extent did the different judges
agree in their estimates for different cases,
irrespective of whether or not their estimates
were correct?
We believe this question to be an important one
because it would be valuable to know the degree to
which the judges acted according to some common set
of rules, even though those rules were not particularly
effective. (A common set of rules that were correct
would, of course, have led the judges to identify back-
ward rotators at a rate better than chance, which we
have just demonstrated did not happen. A common set
of rules that were not correct would have led to a better
than chance agreement among the judges as to the rota-
tional status of each subject. However, the status thus
agreed upon would likely as not have been incorrect.)
The answer to this question may again be obtained
by tabulating the data of Table IV. Table VI summa-
rizes the degree of consensus among the experts as to
Table VI. Number of cases in which specified
numbers of judges agreed on the rotational status of
the subject (first estimation only)
Number of
Judges Agreeing
the rotational status of the subjects. Data from the first
estimation only are reported since judge performance
did not improve during the second estimation.
We observe in Table VI that in twelve cases (I9%),
all five judges agreed as to expected outcome. (Table
IV shows that they were right in 6 of these cases and
wrong in the other six cases.) In 26 cases (41%) four of
the five judges agreed, while in the remaining 26 cases
(41%), three judges predicted one outcome while the
other two judges predicted the other outcome. Al-
though agreement is far from complete, it is much bet-
ter than would have been expected if the five judges had
merely been randomly assigning cases to the forward or
backward categories by chance. In that situation, we
would have expected that the judges would have agreed
completely in only 4.3 cases (7%) while in far the
majority of the cases (39.5, or 62%), they would have
split their decision 3 to 2. The observed agreement is
significantly better than chance at the p < 0.01 level
(details available from the author?). Hence, we con-
clude that, at least to some extent, the judges operate on
the basis of some common set of rules but that these
rules, as used, are not particularly effective.
Question 3. Was there information in the numerical
analyses which would, if utilized optimally, have
improved the performance of the judges?
The remaining question raised earlier was whether
or not there had been information in the numerical data
382 Baurnrind. Km-n, and West Am. J. Orthod.
.Novrmbrr 1984

Table VII. Individual numerical measures which orthodontists. To avoid bias, the variables to be kept
classified backward rotators as well as or better than were chosen independently of the knowledge of the
any of the judges outcomes for the subset of 64 cases used in this experi-
ment. Since each variable dropped was highly corre-
Backward Rotators lated with at least one variable kept, it is reasonable to
Correctly Classified expect that selection of a different subset of variables
Measure would have led to very similar results for the operations
and
Association* No. % P Value* reported in the remainder of this article.
A univariate analysis was then conducted to test
how well each of the 13 retained variables, taken in-
Mandibular Body Length + 20 63% .07
dividually, could be used to identify the backward
ANBAngle 20 63% .07
rotators in the sample. First, for each variable, the
Class II Correction + 20 63% .07
cases with the highest values of the variable in each
Inter-incisal Angle + 19 59% .14
study were predicted to be backward rotators. These
Condyle - SN Distance + 19 59% 14
predictions were then compared to the true outcomes to
Upper Face Height i 19 59% .14
see how well they actually predicted backward rotators.
* + Higher values for the measure predict backward rotation. Then, for the same variable, the cases with the iowesr
- Lower values for the measure predict backward rotation. values of that variable in each study were predicted to
*One-sided. be backward rotators and the truth value of this order-
ing was determined. If the high values of the variable
presented to the judges which might have enabled them classified more backward rotators correctly than did the
to perform better if utilized optimally. In fact, the low values, we denoted the association between that
judges made relatively little use of the numerical variable and the outcome as positive. If the low values
data-somewhat less than had been anticipated by at predicted better than the high values, then we denoted
least one of the authors (S. B.). The reasons for this the association as negative. This procedure was fol-
finding remain unclear. Contributing factors may have lowed for all 13 variables.
been that these judges, precisely because they are such As might have been anticipated, none of the 13
seasoned clinicians, may have come to rely more heav- measures, taken individually, was powerful enough to
ily on their perceptual gestalt of the x-ray film than on distinguish between forward rotations and backward ro-
abstracted sets of numbers. 23 Alternatively, it is possi- tations at a statistically significant level. A number of
ble that the mode of presentation of the numerical val- the measures taken individually did, however, identify
ues (Fig. 2, A) was so densely packed as to distract and backward rotators in this purticular .sample as well
turn off practitioners of a craft more oriented to analog as or better than did any one of the five judges. The
and verbal data presentation than to numerical data. most effective measures and their p values are listed in
We therefore sought to find out how successfully Table VII.
the 32 backward rotators in our subsample of 64 cases These p values have not been adjusted for the mul-
could be identified by computational procedures alone, tiple statistical tests that were performed. In the present
given only the listing of numerical values for the 28 situation, since there are 26 simultaneous tests (13 for
measures of Fig. 2, A. Since the information contained 10~: values and 13 for high values), a conservative pro-
in these measures is highly redundant, we considered it cedure would be to declare the predictive ability of any
desirable at the outset to reduce the number of variables single test to be statistically significant at the p < 0.05
analyzed. Had we retained all 28 variables, the chance level only if its observed one-sided p value was less
of the occurrence of spurious findings of statistical sig- than 0.05/26 = 0.0019. This criterion would require
nificance merely because of the number of statistical any given individual test to classify 75% of the back-
tests being performed would have become unacceptably ward rotators correctly in order to be categorized as
large. statistically significant (see Table III). By this standard,
Therefore, a correlation matrix was created for the the predictive power of none of the variables (as dem-
28 numerical variables for all 238 cases in the primary onstrated in Table VII) is greater than might reasonably
sample. Variables were dropped, one at a time, until be acounted for by chance alone.
the correlations between the remaining variables all had The findings of this experiment may be summarized
absolute values less than 0.58. In choosing which vari- as follows:
ables were to be dropped, preference for retention was 1. Our expectation had been that the expert judges
given to those variables currently used most widely by would be less than perfectly effective in identifying

potential backward rotators using the data available but
that the judges would certainly perform considerably
better than chance. The findings reported in Tables IV
and V establish quite conclusively that, with this set of
test films, this group of judges performed no better than
chance as far as identifying backward rotators is con-
cerned.
2. The question had been raised as to whether the
degree of consensus among the judges was greater than
might have been expected by chance alone. The pres-
ence of a high order of agreement among the judges
would imply (whether or not the judgments were valid)
that there existed some operationally common set of
criteria. The findings reported in Table VI establish that
such a common set of criteria did exist among these
judges
3. Our expectation was that the numerical values
for the standard measures of the Downs, Steiner,
Tweed, and Reidel analyses would be less effective in
classifying backward rotators than were the expert
judges. A comparison of Tables V and VII reveals that
a number of the standard measures were, when used
individually, more effective in classifying backward
rotators within this purtirular sample than were any of
the judges. However, none of these measures, taken
individually, predicted backward rotation at a statisti-
cally significant level. There remained a possibility that
the information contained in the set of measures taken
together might have predicted backward rotation at a
statistically significant level. To test for this possibility,
we performed a conditional logistic regressionz4 of the
13 measures on the binary outcome variable represent-
ing whether the case was a backward rotator. This
analysis found the best linear combination of the 13
measures for predicting the outcome, given the con-
straints of the experimental design. For the present
sample, this prediction had a probability of chance oc-
currence greater than 0.1 (in fact, the value was approx-
imately 0.4). Hence, we conclude that the numerical
data from all measures taken together did not make it
possible to identify potential backward rotators at a rate
better than chance.
DISCUSSION
The intent of this study had been to analogize as
closely as possible the conditions under which a clini-
cian would actually be called upon to identify a candi-
date for treatment whose potential for backward rota-
tion during therapy is extreme. To this end, we sought
to make available to the judge information sources as
similar as possible to those which he would use in
performing the same act of classification under clinical
conditions. The information provided to the judge in-
Prediction of mandibular rotation 383
cluded: (1) a high quality copy of the pretreatment head
film, (2) a table of measurements derived from the film,
(3) information as to the type of therapy that was per-
formed, and (4) a knowledge of the total number of
backward rotators in the four- to six-film subsample to
which each particular film belonged.
It seems quite clear that the judges did not succeed
in using the information sources in a manner that would
have been clinically effective. The judges failed to
classify backward rotators better than they would have
by chance. The numerical values of some of the
measures did slightly better than the judges in this
sample, but not beyond the limits of chance, particu-
larly when one considers the large number of tests
performed.
In order to determine what extrapolations from this
experiment to the clinical situation are appropriate,
several questions need be addressed.
Question 1. Was change in the mandibular plane
angle of Downs a proper measure of backward
rotation?
Reply. The studies of Bjork’ and of Bjork and
Skiellerz5 have sharpened the perception that remodel-
ing changes on the lower border of the mandible mask a
considerable portion of the total rotation of the mandi-
ble which occurs during growth and treatment. Cer-
tainly, it would have been better had we been able to
use implant material, but such material was not avail-
able to us at the time this study was conducted. In the
absence of implant cases, we used in our sample only
the most extreme 64 cases in our total experimental
sample of 238. We interpret the studies of Bjork and
Skieller to imply that the magnitudes are probably
markedly understated using MPA(D) as a measure of
rotation but that the directions of rotation (forward or
backward) are highly likely to be correctly stated.
Question 2. Was change in MPA(D) a proper
classifying variable for collecting the test sample?
Alternatively, should we instead have identified
extreme cases on the basis of some different
posttreatment variable (for example, molar
extrusion) or on some single antecedent variable
(for example, original value for MPA(D) or for
GOGN : SN)?
Reply. Since the question here concerned the ability
to predict direction of rotation, it seems evident that the
most powerful classifying variable to be used was the
best available measure of mandibular rotation itself.
However, it might indeed be quite interesting to clas-
sify on the basis of other variables for the purpose of
answering other questions.
384 Baumrind, Korrf, und West

Question 3. Would a different group of judges, rotation of the mandible during treatment. It may
operating on the basis of a different set of biologic further be inferred that the reason for the therapeutic
constructs, have performed in a superior manner? success of skilled clinicians in the absence of a prior
Reply. As we have indicated in Table II, the aca- knowledge of their patients’ potential for aberrant or
demic and clinical credentials of our set of judges are excessive mandibular rotation is the skilled clinician’s
excellent. However, it is always possible that some ability to make in-course corrections to his treatment
other groups of judges could have performed more ef- plan as the problems arise during treatment. Indeed. the
fectively. To this end, we are anxious to conduct repli- scientific study of the manner in which skilled clini-
cations of this experiment, using other groups of qual- cians make such in-course corrections to their treatment
ified judges who desire to participate. plans is perhaps the most important (and difticult) un-
approached task of clinical research in orthodontics to-
Question 4: Did the inclusion of treated subjects
day.
in the sample confound the task of identifying We note in closing that the question as to whether
patients with backward rotation potential? or not more effective predictions can be made on the
Would it, therefore, have been a cleaner basis of study of serial (that is. longitudinal) head film
experiment had we restricted the sample sets is unaddressed in this study. We also note that
to untreated subjects? the sample used in this study is available for re-
Reply. It is true that most previous studies on man- examination by other expert judges using new and
dibular rotation have been restricted to the examination nonstandard assessment techniques with the under-
of changes in untreated subjects, but the ultimate point standing that the results of their trials would be reported
of all clinical research in orthodontics must surely be to in subsequent publications.
draw valid conclusions about the probable results of
treatment itself. Since the goal of this study was to test REFERENCES
our effectiveness in predicting treatment outcome, it I. Bjork A: Prediction of mandibular growth rotation. Abt J
OKTHOII 55: 585-599, 1969.
seemed reasonable that we test our skill on a sample of
2. Brodie AC: On the growth of the human head from the thud
subjects for whom the actual outcome of treatment was month to the eighth year of life. Am J Anat 68: 209.262. 1941.
already known by direct measurement. For this reason, 3. Bjork A: The use of metallic implants in the study of facial
we used a sample containing both treated and untreated growth in children: method and application. Am J Phys An-
subjects while grouping the cases into studies according thropol 29: 243-254, 1968.
4. Bjork A. Skieller V: Normal and abnormal growth of the man-
to the type of treatment that had actually been adminis-
dible: a synthesis of longitudinal cephalometric implant studies
tered over a period of 25 years. Eur J Orthod 5: l-45, 1983.
5. Schudy FF: The control of vertical overbite in clinical orthodon-
CONCLUSIONS tics. Angle Orthod 38: 19-39, 1968.
On the basis of this empirical study, we consider 6. Root T: Written handout, available from authors of present
article.
the following inferences to be appropriate:
7 lsaacson JR, Isaacson RJ, Speidel TM, Worms FW: Extreme
1. The information that can be extracted from pre- variation in vertical facial growth and associated variation in
treatment lateral skull head films at our present level of skeletal and dental relations. Angle Orthod 41: 219-229, 1971.
understanding appears to be insufficient to permit clini- 8 Baumrind S, Molthen R, West EE, Miller DM: Mandibular
cally useful predictions to be made as to the likelihood plane changes during maxillary retraction. Part I. Atvt J OKTHOU
74: 32, 1978.
of subsequent backward rotation of the mandible during
9 Baumrind S, Molthen R, West EE, Miller DM: Mandibular
treatment. Thus, even though it might be desirable on plane changes during maxillary retraction. Part 2. AM J ORTHOU
theoretical grounds to know each patient’s potential for 74:603, 1978. (Addenda, AM J ORTHOD 75: 86, 1979.)
backward rotation at the outset of treatment, it appears 10 Baumrind S, Kom EL, Molthen R, West EE: Changes in facial
that this information is, in practice, unavailable for use dimensions associated with the use of forces to retract the
maxilla. AM J ORTHOD 80: 17, 1981.
in treatment planning at that time.
II Baumrind S, Kom EL: Patterns of change in mandibular and
2. The positive side of this demonstration of our facial shape associated with the use of forces to retract the
apparent inability to make clinically useful pretreat- maxilla. AM J ORTHOD SO: 31, 1981.
ment predictions concerning backward rotation should 12 Baumrind S, Kom EL, lsaacson RJ, West EE, Molthen R:
not be overlooked. The five judges involved in this Qualitative analysis of the orthodontic and orthopedic effects of
maxillary traction. AM J ORTHOD 84: 384-398, 1983.
study are all recognized as outstandingly successful
13 Baumrind S, Kom EL, Isaacson RJ, West EE, Molthen R: Su-
clinical therapists. From this fact, we may deduce that perimpositional assessment of treatment-associated changes in
it is possible for clinicians to treat effectively in the the tempomandibular joint and the mandibular symphysis. AM J
absence of prior knowledge as to the likely direction of ORTHOD 84: 443-465, 1983.
Volume 86 Prediction of mandibular rotation 385
Number 5

14. Baumrind S, Miller DM: Computer-aided head film analysis, the 22. Kom EL, Baumrind S: Inter-rater agreement among orthodon-
University of California, San Francisco method. AM J ORTHOD tists in a blocked experiment. Stat Med (in press).
78: 41, 1980. 23. Baumrind S, Miller DM, Molthen R: The reliability of head film
1.5. Baumrind S, Fran& R: The reliability of head film mea- measurements. 3. Tracing superimposition. AM J ORTHOD 70:
surements. 2. Conventional angular and linear measures. AM J 617-644, 1976.
ORTHOD 7%: 41, 1980. 24. Breslow NE, Day NE: Statistical methods in cancer research.
16. Isaacson RJ, Zapfel RJ, Worms FW, Bevis RR, Speidel TM: Vol. 1. The analysis of case-control studies, Geneva, 1980,
Some effects of mandibular growth on the dental occlusion and International Agency for Research on Cancer, chap. 7.
profile. Angle Orthod 47: 97- 106, 1977. 25. Bjork A, Skieller V: Facial development and tooth eruption: an
17. Ricketts RM: A principle of arcial growth of the mandible. implant study at the age of puberty. AM J ORTHOD 52: 339-383,
Angle Orthod 42: 368386, 1972. 1972.
18. Schulhof R, Bagha L: A statistical evaluation of the Ricketts and
Johnston growth-forecasting methods. AM J ORTHOD 67: 258.
Reprint requests to.
216, 1915.
Dr. Sheldon Baumrind
19. Johnston L: A simplified approach to prediction. AM J ORTHOD
Craniofacial Research Instrumentation Laboratory
67: 253-251, 1975.
Department of Growth and Development
20. Harris JE, Johnston L, Moyers RE: A cephalometric template:
University of California San Francisco
its construction and clinical significance. AM J ORTHOD 49:
San Francisco, Calif. 94143
249-263, 1963.
21. Fleiss JL: Statistical methods for rates and proportions, ed. 2,
New York. 1981, John Wiley & Sons.