Scientometrics, Vol. 8.

Nos 1--2 (1985) 117-136

ON THE DEGREE OF INTERDISCIPLINARITY OF

RESEARCH PROGRAMS: A QUANTITATIVE ASSESSMENT
S. AMIR+

Environmental Protection Service

P.O. Box 6158 Jerusalem (Israel)

(Received June 18, 1984)

It is widely maintained that the study of policy alternatives, particularly if they are
associated with introducing new tehcnologies that may engender vast social and environ-
mental repercussions, ought to be interdisciplinary. There is, however, much confusion in
the literature as to what exactly is meant by the term interdisciplinary. In the present paper,
we quantitatively assess the extent of interdisciplinarity of studies and of research programs.
First, we propose several working definitions of the concept of interdisciplinarity. Second,
we consider the construction of indicators that quantify these definitions. Third, as an
example, we examine whether or not a given policy oriented research program is truly
interdisciplinary.

Introduction

It has become common knowledge that interdisciplinary research is resource intensive

and in need of careful nurture and management, a -3 Consequently, science policymakers
face an intricate question; i.e., how to evaluate the relative contributions to scientific
progress of interdisciplinarity vs. that o f mere higher costs. Immediately, one wonders
whether common disciplinary studies may not contribute more than their inter-
disciplinary counterparts to achieving the desired impacts, had both been funded to
the same extent. This question, and similar ones, clarify a need, which is not merely
epistemological bt, t applicative as well, to grasp the notion of interdisciplinarity
quantitatively.
The term "interdisciplinary" may mean different things to various writers. This is
evident in many writings, where authors fail to define rigorously what they mean by
this adjective. Certainly, it is superficial to say that interdisciplinary is the opposite of
disciplinary; i.e., that interdisciplinary and adisciplinary are synonymous. One can
envisage many adisciplinary approaches to scientific research; interdisciplinary and
multidisciplinary are but two of the most important.
A distinctive characteristic of an interdisciplinary research is its integratedness.
Following Hattery, 4 we can define interdisciplinary research as an integrative research
+Present address: Department of Applied Physics and Mathematics, Soreq Research Center,
Yavne, Israel.

Scien tometrics 8 {1985) 117

S. AMIR: INTERDISCIPLINARITYOF RESEARCHPROGRAM

process which takes place among researches with different disciplinary backgrounds. It
stands in contrast to multidisciplinary research which does not accommodate a
continuous process of integrative communications among different disciplines.
Cutler s mentions the existence of a common goal to which scientists work collabora-
tively as an additional distinctive characteristic of interdisciplinary research.
Darvas and Haraszthy 6 mention many kinds of adisciplinary approaches to research.
Their Classification may be better clarified i f one is ready to bear in mind the image
of a scientific tree similarly to the well known taxonomic trees formed in biological
classifications. The term "disciplines" is used to denote different tree branches which,
nevertheless, belong to the same tier of the tree. A tier is a term that can be conceived
only loosely because our tree is also an evolutionary tree, and it does not have well
defined tiers. Sometimes the terms "scientific fields", "subfields", and "subdisciplines"
replace the term "scientific disciplines". This should not raise any confusion. It only
indicates that, as of now, a well established, unanimously agreed to scientific tree
does not exist. Writers may agree on parts of the tree rather than on its entirety.
UNESCO nomenclature for fields of science and technology7 ranks scientific branches
into fields, disciplines, and subdisciplines. The word "speciality" is also very popular
recently.8 ,9
We claim that when we are concerned with the distinction between disciplines
and their diverse elusterings, a reference is to be made to examination of possible
mappings of our scientific tree. Novel points may be located in active meristematic
regions or become the result of modified branchings of the tree as a whole or of its
parts.
According to this view, a scientific branch is interdisciplinary if it is an interactive
union of parts of well defined disciplines, fields or subdisciplines. In this definition,
two points should be made clear. First, the resulting branch is not a mere sum of
various branches. It results from intensive interactions between preexisting branches,
and it should include more than can be reached by their simple addition. Second, the
parts being incorporated into a new interdisciplinary branch can originate within
different tiers of the tree because, at any given moment, the tree's various branches
are not developed to the same extent.
An interdisciplinary branch has its own identity: its material and its conceptual
existence on our scientific tree. If decomposed, its original components cannot be
recovered intact because the remainder, the fruit of intensive interactions, does not
have an independent existence. The remainder is not a proper branch and cannot be
mapped on our scientific tree. Sometimes, this notion of interdisciplinarity has been
termed transdisciplinarity e.g., by Rossini et ~1.1o
In contrast, a scientific- branch is multidisciplinary ff it is an exact sum of preexistin
disciplines, fields, or subdisciplines. A multidisciplinary branch does not have its own

1[8 Scientometrics 8 (1985)

 S. AMIR: INTERDISCIPLINARITYOF RESEARCHPROGRAM

identity. Its existence is only conceptual and totally dependent on the branches
incorporated to form it. Since nothing new has been formed, these branches can be
recovered completely on decomposing a multidisciplinary branch.
Other terms like "pluridisciplinary" and "cross disciplinary" can be defined along
similar lines, but we are not concerned with these alternatives to disciplinary research
in the present study. They can be subsumed under the term "multidisciplinary" in
indication of the plurality of ways to combine different disciplines.
In practice, when we want to discern a disciplinary research program either from
a multidisciplinary research program or from an interdisciplinary research program,
we face huge difficulties. These difficulties have lead Darvas and Haraszthy 6 not only
to subsume diverse clusterings of different disciplines under the multidisciplinary
category, but also to adopt an everyday usage, as they claim, and obviate the existence
of distinctions between interdisciplinary and multidisciplinary research. Building on
their foundation, the present study shows that additional stelbs can be taken to
elucidate the distinction between multidisciplinary and interdisciplinary research. In
so doing, we obviously bear in mind the inherent difficulty of measuring, or even of
evaluating, the extent to which disciplines do really interact when they form an
interdisciplinary study.
If conceptually difficult to evaluate, a direct statistical estimation of the interac-
tions between different disciplines is out of question. The indicators proposed
in the present study must be partial;2 s they are related to the extent of interaction
between disciplines, but by no means can they identically outline that interaction.
Furthermore, they vary in their degree of partiality; i.e., in the extent to which they
faithfully trace interactions between disciplines. The reason for not concentrating
on the least partial indicators is practical. In general, less partial indicators require for
their elucidation more extensive and better refined information. Too often, however,
this kind of information is not available at all, or it is prohibitively expensive to get.
The present study is statistical in nature. We define several partial indicators of the
degree of interdisciplinarity of studies and of research programs. Later we experiment
with these indicators by computing their values for a given research program. Inten-
tionally, we have avoided following an alternative approach to assess the degree of
interdisciplinarity of research programs. Participating scientists and research coordinators
have not been interviewed on their assessment of the extent of interaction in their
own studies. Our results may fred a research program not to be interdisciplinary,
while participating scientists would claim they had interacted quite intensively. This
is said not to undervalue the subjective evaluation, whether it has been elicited from
peer reviews or from layman perceptions. We merely claim that the statistical and the
behavioral approaches are two alternatatives to tackle similar questions. At present,
they differ so markedly that it is methodologically unsound to intermingle them

Scientometrics 8 (1985} 119

S. AMIR: INTERDISCIPLINARITYOF RESEARCHPROGRAM

intentionally. This state of affairs will prevail so long as doubts exist whether or not
those directly involved in a presumably interdisciplinary study can completely separate
the objective from the subjective; i.e., reality from what ought to be. Our study
constitutes a tentative and preliminary exercise trying to answer a positive question,
i.e., whether or not a given research program is interdisciplinary.

Methods

For the present study, our sample is the research program of Israel's Environ-
mental Protection Service (IEPS). The IEPS was established in 1973 as an environ-
mentally minded coordinating body. It began to finance environmental studies in
1974. Since then it has supported 155 studies for a total sum of IS 813 000 at 1975
prices (approximately equivalent to ~1300 000 at 1975 prices).
The original sample of 155 studies includes 94 studies whose files contain well
documented information on their scientists. 157 scientists have participated in carrying
out these 94 studies. 107 of the 155 studies initiated have been completed up to now.
Of the studies completed, 53 include what is normally called a final report. These are
further divided into 21 studies whose final reports did not contain any cited references
and 32 studies whose final reports contain more than one cited reference. The number
of references surveyed altogether is 577. These studies, the scientists performing them,
and the references cited in them constitute our sample in the present study.
The scientific field of scientists is determined according to UNESCO proposed
nomenclature for fields of science and technology7 applied to the field in which they
have achieved their highest degree. Only rarely does a scientist's present field of study
differ markedly from the field in which his highest degree has been achieved. Our
sample includes only one clear case like this. In addition, our sample includes a few
borderline cases by which we mean, e.g., that chemists by training have reoriented
themselves to close fields like chemical technology or vice versa.
By analyzing their content, studies have been mapped into their corresponding
scientific field, except that by assumption all studies concerned share in the environ-
mental field. For this analysis, we have used, again, UNESCO's nomenclature. In
addition, we have relied heavily on our own classificatory criteria for assessing the
main concerns of each study. These criteria have been derived from several other
classifications: functional, pollutant-wise, and medium-wise, that we have already
imposed on the same body of data in other circumstances. 11- a 3 and for other reasons.
In mapping studies into scientific fields, we have taken advantage of our intimate
familiarity with the IEPS research program. Unfortunately, the same procedure cannot
be applied to a study of the sample of citations. In principle, however, there are many

120 Scien to me trics 8 (1985)

 S. AMIR: INTERDISCIPLINARITYOF RESEARCH PROGRAM

possible ways to guide us in forming the distribution of references to their scientific

fields. One may associate references with journals and map well classified journals
into scientific fields. Alternatively, one may map titles of articles into scientific fields,
basically by referring to their key words. More delicate, but certainly more cumber-
some, mappings are based on content analysis. In the present study, the title of each
article has been analyzed and mapped according to key words.
Our analysis proceeds gradually, from partial to more comprehensive indicators. The
main reason for presenting several descriptors is that there exists nothing like the
best indicator: less partial, more informative descriptors embody more refined data,
but high quality data are harder or more expensive to get. In general, a decision has
to be made as to how much one is ready to invest to improve one's analysis and
make it more accurate; however, this decision cannot be made with no regard whatsoever
to the quality of the data available in any given case.

Analysis

The distribution o f studies

The simplest indicators are derived from distributions of the studies themselves.
We consider two such distributions. In the first, studies are jointly distributed according
to the number of scientists per study and according to the number of their scientific
fields. In the second, studies are jointly distributed according to the number of
references cited in a study and according to the number of their scientific fields.
Scientist distribution indicators. Table 1 presents the distribution of 94 studies as
a function of s, the number of scientists per study, and of b, the number of their
scientific fields. Indicators calculated from this table are functions of s and b.
Consider the distribution of studies in Table 1. A necessary, but not sufficient,
condition for a study to be called interdisciplinary when judged relative to the number
of scientific fields of its scientists is that its scientific team be multJdisciplinary.
Referring to Table 1, the coarsest indicator may count the number of studies whose
scientific team is multidisciplinary and determine its proportion (11/94 --- 0.117) in
the whole sample. The resulting figure can be compared with the proportion of
multidisciplinary studies in a known disciplinary program to determine its statistical
significance. If one's criterion claims a disciplinary Program to include no multi-
disciplinary study, our program is interdisciplinary at a significance level of 0.001.
Such an indicator seems, however, to be much too coarse. The figures in Table 1 show
that more than 50% of the studies are performed by one scientists, however, these
"lone wolves" constitute only 33% of the scientists working in the program. The Gini

Scientometrics 8 {1985) 121

S. AMIR: INTERDISCIPLINARITY OF RESEARCH PROGRAM

?" o
,.6 06 o

ta
t:x,

.~' 0 ~

~'~
r.,.) to,, m

,.,t

e'.q ~ em eq oo
z~

e'q e'q e~ ~ I"- o)

t~
O

o) .

e'.l kO ,..-t ~ "~

r II

0)

tt~ eq ~ OO

O
t--,

122 Scientometries 8 (1985)

 S. AMIR: INTERDISCIPLINARITYOF RESEARCH PROGRAM

index, defining the concentration of scientists among studies with respect to team
size, is 0.262. In contrast to this finding, we may calculate from the same table that
about 88% of the studies are performed by disciplinary teams accounting for almost
80% of the scientists. The corresponding Gini index, defining the concentration of
scientists with respect to breadth, is 0.064. Evidently, this implies that team size is
a feature more sought for than field breadth.
More Sophisticated indicators would consider explicitly the extent of multi-
disciplinarity, i.e. the breadth in terms of scientific fields of a given scientific team.
Thus, more weight would be put on a team of four scientists diversified into three
scientific fields than on a team o f four scientists diversified only into two scientific
fields. Thinking along these lines, we may assert that more interdisciplinarity, i.e.,
more intensive interactions, should be associated with a sicentific team of four
scientists classified into three scientific fields than with a scientific team of three
Scientists classified into three scientific fields.
The numeric examples presented above clearly show that any indicator derived
from s and b should satisfy several properties as follows:
(a) It should increase with both ~"and b.
(b) It should be more sensitive to changes in b than to changes in s indicating that
diversity is more valued than mere size.
(c) It should incorporate explicit judgements on the intensity of substitution
between s and b and on proportional increases in both s and b. For example, it may
be felt that as s/b increases, more scientists are required to substitue for a given
decrease in b.
(d) It may be felt that an indicator should assume a zero value if s = b = I, or
even whenever b = 1, irrespective of the value of s.
A possible indicator satisfying the above conditions is:

dO, s) = O xA-- 1) (i)

where ot is a given constant: 4

The degree of interdisciplinarity of a given research program may be determined
as a weighted average of the degree of interdisciplinarity of its constituent studies.
Thus, ifNbs is the number of studies with s scientists classified into b scientific
fields, and if d(b, s) is the degree of interdisciplinarity of a study given its b and s,
the degree of interdisciplinarity o f the whole research program is:

(2)
whereN= Z Nbs.
b,s

Scientometrics 8 (1985) 123

S. AMIR: INTERDISCIPLINARITYOF RESEARCH PROGRAM

There degree of interdisciplinarity of our reseach program calculated from Table 1

is 0.63 a. 1 s
It is easily seen that the value of D(b, s) in any case never exceeds that corres.
ponding to a study conducted by three scientists who are members of the same
scientific field. Such a program reflects the benefits of cooperation, but it hardly can
be called interdisciplinary.
Reference distribution indicators. Consider the group of references cited in the final
reports that constitute our sample. Table 2 presents the distribution of studies as a
function of s, the number of references per study and of b, the number of their
different scientific fields. Table 2 is analogous to Table 1 with references substituting
for scientists.
The figures presented in Table 2 show that 56.2% of the studies include a
heterogenous list of references, while only 11.7% of the studies are performed by
adisciplinary teams. References are less equally distributed than scientists regardless
of whether the size distribution of each variable or its field distribution is concerned.
The appropriate Gini indices, defining the concentration of references amond studies
with respect to s or b are 0.414 and 0.322 respectively. Since these indices are closer
to each other than their counterparts, derived from the distribution of scientists, we
expect references to be more interdisciplinary at least in relation to simple indicators
defined in Eq. (1). One reason for this outcome is, however, the very fact that, on
the average, there are more references than scientists per study.
Several other reasons preclude us from treating the two distributions alike.
Nevertheless, utilizing similar considerations to those set above, we devise new
indicators based on the distribution of articles according to the number of references
per article and the number of their scientific fields. In so doing, we have to recall
that the field distribution of references, m o r e t h a n the field distribution of scientists,
is akin to the distribution of individuals among species in statistical ecology. 16 Not
only that references do not interact with each other, but the breadth of a list of
references is expected to depend on its size. This expectation is borne out in the
present study.
Whereas data in Table 1 clearly show no statistical tendency on the part of larger
scientific teams to be significantly more multidisciplinary, similar data presented in
Table 2 indicate that some relations between the size of the list of references and its
breadth, measured in terms of the number of scientific fields per study, do exist.
From Table 2, it is clear that the average number of fields per study can be
approximated by a logarithmic, or by a fractional power function of the size of the
list of references. The indicators proposed should, therefore, be proportional to the
breadth of a given list and inversely proportional to the logarithm, or to a fractional
power of its size.

124 Scientometrics 8 (1985)

 S. A M I R : I N T E R D I S C I P L I N A R I T Y OF RESEARCH PROGRAM

,4 ,4 ~ ,..,' ,A c;

o~
2

O "~ .=.
z~

"5
r

t"q tt3

~o

Z~

a~
g'/
9.,~ ~
q.~ II

.7:

t'q t'~ r

O
O ~ O O O O

I I I I I

Scien to metrics 8 (1985) 125

S. AMIR: INTERDISCIPLINARITY OF RESEARCH PROGRAM

We define another indicator of interdisciplinarity as follows:

8(b, s) =/3 (b - 1)/~v/s" (3)

where/3 is a constant, xa
Assuming there are Nbs studies with breadth b and size s, the degree of inter,
disciplinarity of the whole program is defined as follows:

A(b, s) = (l/N) b~, s Nbs 8(b, s) (4)

whereN = Z Nbs.
b,s

The value of A(b, s) for the whole program is calculated to be 0.26/3. This value of
interdisciplinarity corresponds to a study having about 59 cited references spread over
two different scientific fields. Such a study hardly can be called interdisciplinary.
Weighted indicators. The indicators so far presented suffer from a common feature.
They disregard, almost entirely, the scientific tree one is assumed to have in one's
mind. Truly, some considerations are being given to,lthe number of different
scientific fields, but the nature of these fields is completely immaterial. However, if
one has a scientific tree in mind, one probably assumes that physics and chemistry
are more closely related to each other than physics to sociology. Moreover, one want.,
to use such an assumption to determine the degree of interdisciplinarity of a given
research program.
For the present study, we have constructed a tentative scientific tree whose
branches have been clustered by adjoining proximate scientific fields according to
UNESCO nomenclature. This specific clustering is not to be unanimously accepted.
It is, however, important to recognize the following point: either one uses a given
conception of a tree with respect to which one determines the degree of inter-
disciplinarity of various research programs, or one uses various possible classifications
of a given research program to construct the statistically fittest scientific tree.
Given a scientific tree, there exist two fundamental approaches to calculating the
degree of interdiseiplinarity of a scientific team (a reference list). The first approach
assigns to each pair of scientific fields a number, equal to the level in the tree to
which one must descend so as to place the pair in the same cluster. The second
approach assigns to each pair of scientific fields a number, equal to the number of
nodes (or internodes) that separate the pair in the tree. In each case, the degree of
interdisciplinarity of a study is later defined as a function of the values assigned to
all pairs.

126 Scientometrics 8 (I 985)

 S. AMIR: INTERDISCIPLINARITYOF RESEARCH PROGRAM

Table 3
A level count matrix presentation of the scientific tree*
(numbers in brackets are UNESCO code numbers)

Scientific field
Scientific field
"(32) (23) (53) (61)
(33) (24) (56) (62)
(31) (25) (59) (12) (63)

Medical (32)
Technological (33) 0/1 2 4 3 4
Agricultural (31)

Life Science (24)

Chemistry (23) 2 0/1 4 3 4
Earth & Space (25)

Economics (53)
Law (56) 4 4 0/1 4 2
Politi~l Science (59)

Mathematics (12) 3 3 4 0/1 4

Psychology (61)
Arts (62) 4 4 2 4 0/1
Sociology (63)

*A level count is one if the UNESCO one digit code number is the same
for two fields. It is zero for any single scientific field.

Table 3 presents the proposed scientific tree in a matrix form. Each entry in the
matrix'defines how close the two respective scientific fields are. Closeness has been
define 6 as the number of tiers which must be descended in order to adjoin two
separate fields into a cluster. Since the numbers derived at are the same independently
of the path taken to connect any two fields, the resulting matrix must be symmetric.
By definition, if the breadth of a team (a list of references) is one, the degree of
its interdisciplinarity must be zero. There are, therefore, 11 studies in Table 1 and 18
studies in Table 2 to be considered for our present exercise of assessing a weighted
index of the degree of interdisciplinarity of the whole program.
For our computations, we use the matrix entries in Table 3. Summing up these
numbers, and dividing the sum by the number of scientists, we derive a weighted
measure of the breadth of each study.

Scientometries 8 (1985) 127

S. AMIR: INTERDISCIPLINARITY OF RESEARCH PROGRAM

Thus, let bij denote the number of levels which must be descended to connect
between any two scientific fields, i and j. For any study k, the new index is defined
as follows:

"Yk = a {(1/Sk) i~,] b k. (5)

i,i

where sk denotes the size of the team of the k'th study. An index of interdisciplinarity
for the whole program is derived from the 7 k 's as follows:

r'= (1/N) k Tk (6)

Calculating this index for our program, we get 0.25 a. This figure should be
compared with the lowest index possible for any multidisciplinary study wkich is
0.5 a.
Similarly, we treat the distribution bf studies with respect to a field classification
of their lists of references. For any study, an index of interdisciplinarity is defined
as follows:

Xk (3 (1/Sk) Z (7)
i,]
i,]

From the Xk's an index of interdiscipthaarity is derived for the whole program by
averaging over individual studies as follows:

A = (l/N) kZ Xk (8)

The value of this index calculated for our program is 1.08/L

Joint field distribution

Consider the classification of scientists that participate in a research program to

their scientific fields and examine the resulting frequency distribution. Several indicato
can be derived from this distribution, and they, by necessity, are similar to indices of
diversity, such as Shannon or Simpson indices, that are widely used in statistical
ecology) 9 The proposed indicators suffer a crucial shortcoming: they better describe

128 Seientometrics 8 (1985)

 S. AMIR: INTERDISCIPLINARITYOF RESEARCHPROGRAM

the degree of multidisciplinarity of a research program than its degree of inter-

disciplinarity. The reason for this shortcoming should be obvious. A field distribution
of scientists tells us nothing as to how and to what extent scientists of different
scientific fields collaborate in undertaking various studies. In other words, a single
field distribution of scientists provides no clue to questions such as how scientists
are distributed among studies, and especially among the scientific fields of individual
studies.
Scientist distributiorL Table 4 provides us with the desired picture. It shows how
scientists are distributed among their scientific superfields and among the scientific
superfields of their studies. The data for this table have been aggregated from a similar
table that assigned scientists to their scientific fields as well as to the scientific fields
of their studies. It was clear that this classification would result in most classes being
empty. We have decided, therefore, to envision a scientific tree and utilize it to
coalesce several scientific fields into superfields. Our consequent tree is explicitly
presented in Table 4 under the different superfield categories of the studies.
Referring to Table 4, two extreme viewpoints may be considered. First, in case
scientific fields of scientists and those of their studies are independent, the figures in
Table 4 would reflect it. Each entry would be approximately equal to the product of
the proportions of the corresponding classes of scientists times the total number of
scientists. If this view is unacceptable, it is interesting to know why and to what
extent observed figures deviate from those expected under the assumption of
independency. A second possible viewpoint may become extremely important under
such conditions; Le., whether or not observed figures deviate from the assumption of
full disciplinarity. In other words, even totally disciplinary research programs may
include some scientists whose scientific fields differ from that assigne d to the program
under consideration. In addition, one would expect some statistical noise: misspecified
variables and other errors in classifying data may lead to observed distributions that
are not restricted to have positive values only on their diagonals. This cannot be taken
to mean that research programs under consideration are adisciplinary.
It seems reasonable to assume that the more a research program becomes inter-
disciplinary, the more it fits a random distribution of the scientific fields involved.
When expected values are calculated from data in Table 4, the resulting value of
X2 is 262.93, The probability of getting it, given the independency assumption, is
much less than 0.001; hence the independency assumption must be rejected. If inter-
disciplinarity is to be measured in terms of independency between scientific superfields
of scientists and those of their studies, the figures in Table 4 suggest the considered
program not to be interdisciplinary.
The degree of interdisciplinarity may, however, be based on various concepts. We
can disregard any statistical expectation and define it as one minus the sum of diagonal

$cientometrics 8 (1985) 1.29

S. AMIR: INTERDISCIPLINARITY OF RESEARCH PROGRAM

eD
r~

oo i/) oo

9 r ,
t"~ r tt3 ~'~ r ',r r t'... ~ , 84
O

e-i
5 tt~

r~

~6 ~6

r-,,~
oo

U
r

.,<
r~ "~

oo
"" O
,4 ,,6

O'.,

,'t::l

t~ u

O
r/)

0)

u
.<

o~

130 Scientometrics 8 (1985)

 S. AMIR: INTERDISCIPLINARITYOF RESEARCHPROGRAM

elements when the latter are expressed in terms of proportions. Thus, if N is the total
number of observations and ni] is the number of observations in the (i, j) class, the
degree of interdisciplinarity is defined as:

A c = (N - Z. n]/)/N (9)
1

The degree of interdisciplinarity recovered from Table 4 is 0.196.

In developing indicators of interdisciplinarity that utilize joint distributions, we
better avoid sliding into extremes. The last indicator obtains a value of one if diagonal
elements are all zero; however, a program whose field distribution is characterized
by an empty diagonal, would not be considered interdisciplinary as much as it would
be regarded amateurish. Any professional program should show natural scientists to
preferentially staff studies in natural science fields. Only deviations from expectations,
either in the directiun of a dominant diagonal or in the direction of a null diagonal,
indicate to what extent a program is merely multidisciplinary or dilettante. Somewhere
in between, a research program may be called interdisciplinary, and its degree of
interc[isciplinarity increases as it moves farther from any extreme.
Ofher indicators of the degree of interdisciplinarity, therefore, take into considera-
tion }he expectation that diagonal elements will reflect some degree of relatedness
between scientific superfields of scientists and those of their studies. One way to take
this point into consideration is by dividing Eq. (9) by one minus the sum of expected
proportions of the diagonal elements. Thus, if ni/(e) is the number of expected
observations in class (i, j), a newly defined indicator of the degree of interdisciplinarity is:

A d = A c / [N - E,n//(e)] / N (lo)

Thus defined, the degree of interdisciplinarity for Table 4 is 0.297.

Considering the fact that the latter indicator assumes the value one when the sum
of o6servations equals the expected sum, we observe quite a strong dependency
betwgen scientific superfields of scientists and scientific superfields of their studies.
Tile last indicator of the degree of interdisciplinarity measures to what extent
observations differ from interdisciplinarity. It has been mentioned that an inter-
disciplinary research program is expected to show some degree of independency
between scientific fields of scientists and those of their studies. The extent to which
this expectation is not materialized is defined as:

A e = Z. [nil - ni] (e)] / [N - .~ nil (e)] (11)

1 1

Seientometrics 8 (1985) 131

S. AMIR: INTERDISCIPLINARITYOF RESEARCHPROGRAM

Thus, the greater A e is, the less interdisciplinary is the considered research program.
Calculating A e and its standard error (see Fleiss 21 for Table 4, we get the following
result:

A e = 0.703 and s.e.(Ae) = 0.0526

Clearly, Ae is statistically significant at a level of p < 0.00 !, which means that the
association between scientific superfields of scientists and those of their studies is by
no means random. It is, however, interesting to note that Ae differ from one as well
(p < 0.001), but this result cannot constitute an eVidence for rejecting multidisciplinari
altogether. Actually, scientists are inclined to study in their own scientific superfields,
and our tentative findings show this tendency to increase as scientific fields get less
specialized. Our yet unpublished data show, expectedly, that technologists diversify
among disciplines within their field of technology more than they tend to
communicate with members of other scientific fields.
Reference distribution. Table 5 presents figures on the distribution of our known
sample of 577 references. As for Table 4, raw figures showing the distribution of
references among their scientific fields and among the scientific fields of the studies
citing them, have been condensed into scientific superfields.
If our research program is interdisciplinary, we expect to find studies in a given
scientific field citing from various scientific fields. Calculating the value of X~, we
get X6 2 = 868.59 (p < 0.001). Thus, the picture is quite clear: the distribution of
refere~lces tends to deviate from that expected if references are cited independently of
the citing studies. Moreover, references tend to adhere to studies in a clear manner.
Their scientific field is markedly associated with the scientific field of the citing
studies.
The dependency of the distribution of references on the distribution of the citing
studies can be assessed as has been done previouslylby several indicators. Calculating
Eq. (9), the corresponding value for Table 5 is 0.061. The value obtained from Eq. (10
for Table 5 is 0.109. The last indicator is derived from Eq. (11). The corresponding
value and its standard error are:

A e = 0.894 and s.e.(Ae) = 0.0337.

The resulting Ae is significant statistically as the probability of getting it by chance

alone is smaller than 0.001. This implies that our research program is more multi-
disciplinary than interdisciplinary, not only as judged from the point of scientist
distribution but also from the point of reference distribution.

132 Scien tometrics 8 (1985)

 S. AMIR: INTERDISCIPLINARITY OF RESEARCH PROGRAM

r ".-' t-'q "..-'

[-,

r r162
Z~

,s:l

5
e-,

',3

Scientometrics 8 (1985} 133

9
S. AMIR: INTERDISCIPLINARITYOF RESEARCHPROGRAM

Discussion

Knowing whether or not a research program is interdisciplinary is important not

merely in itself but also as a matter of policy. The amount of resources available for
any research program is not infinite; however, if policy making research ought to be
interdisciplinary, at a price it can be made so. Naturally, some obvious limitations
make interdisciplinary research programs less attractive for participating scientists than
competing disciplinary research programsJ ,a ~22 -24 These limitations can be overcome
if interdisciplinary research is differentially supported and is carefully protected in a
greenhouse-like environment. To specifically protect it in a university setup, more
resources, special incentives, and modified rewarding systems are needed. 1 ,a,s
Does it pay to channel extra resources in short supply into interdisciplinary
research, or is it more rewarding to save these resources and espouse a disciplinary
policy making research? Aware of this question, we have started the present study
which looks for ways to make a clear distinction between interdisciplinary and
disciplinary research programs. For this aim, we propose several new scientific
indicators and examine their evaluation of the degree of interdisciplinarity of a given
research program. Knowing how to make this distinction and how to quantify it, we
can go ahead to the next stage and associate costs and benefits with research programs
that vary in their degree of interdisciplinarity as well as in their scientific contributions
A quantitative approach is selected because only after other inputs have been properly
controlled for, the benefits of mere interdisciplinarity can be estimated meaningfully.
A study is defined to be interdisciplinary if, in preparing it, scientists of various
scientific fields interact. Our analysis suggests that, despite the fact that no statistically
clear methods exist to assess directly the extent of interaction among scientists, much
room is still available to evaluate the degree of interdisciplinarity of a given research
program indirectly. The simplest indicators are functions of the number of scientists
(references) per study and of the number of their scientific fields. These indicators
are made less partial, the deeper they embody in their structures the differing intensiti,
of interactions that have evolved between scientific fields which vary in their proximit:
More elaborate indicators take into consideration the joint distribution of studies and
of their scientists (references) among scientific fields. They enable us to judge whether
a given research program is really interdisciplinary or is only multidisciplinary; i.e., a
collection of disciplinary studies. The value of this exercise for science policy making
has been claimed enormous, and it clearly justifies more attempts in the direction of
quantitiative assessment.
Analysing statistically some indicators of interdisciplinarity for the IEPS research
program indicates, to a large extent, that the sampled program is merely a multi-
disciplinary one. The picture in its entirety shows that scientists do cooperate, but

134 Scientometrics 8 (1985)

 S. AMIR: INTERDISCIPLINARITY OF RESEARCH PROGRAM

their cooperation is narrowed down to reap some additional benefits of deeper and
deeper specialization. Scientists do not work alone, b u t they prefer to share more
with their likes than with those who pretty much differ in training and in profession.
A quantitative expression of this tendency is delineated through our proposed indicators.

References and notes

/
1. B. W. MAR, W. T. NEWELL, B. O. SAXBERG, Interdisciplinary research in the university
setting, Environmental science and Technology, 10 (1976) 650.
2. R. T. BARTH, R. STECK (Eds), Interdisciplinary Research Groups: Their Management and
Organization, University of Washington, Seattle, 1979.
3. I. L. WHITE, Interdisciplinarity, The Environmental Professional, 1 (1979) 51.
4. L. H. HATTERY, Interdisciplinary research management: research meeds and opportunities,
in: Interdisciplinary Research Groups: Their Management and Organization, R. T. BARTH, R.
STECK (Eds), University of Washington, Seattle, 1979, p, 9-25.
5. R. S. CUTLER, A policy perspective on interdisciplinary research in US universities, in:
Interdisciplinary Research Groups: Their Management and Organization, R. T. BARTH, R.
STECK (Eds), University of Washington, Seattle, 1979, 295 -314.
6. G. DARVAS, A. HARASZTHY, Interdiciplinary Orientation of Research Teams in Six
European Countries, in: Interdisciplinary Research Groups: Their Management and Organiza-
tion, R. T. BARTH, R. STECK (Eds), University of Washington, Seattle, 1979, p. 34-48.
7. UNESCO, Proposed International Standard Nomenclature for Fields of Science and Technol-
ogy, UNESCO/NS/ROU/2J7, 1973.
8. H. G. SMALL, B. C. GRIFFITH, The structure of scientific literatures: identifying and
graphing specialties, Science Studies, 4 (1974) 17.
9. H. G. SMALL, D. CRANE, Specialties and disciplines in science and social science: an
examination of their structure using citation indexes, Scientometrics, 1 (1979) 445.
10. F. A. ROSSINI, A. L. PORTER, P. KELLY, D. E. CHUBIN, On the integration of the
disciplinary components of interdisciplinary research, in~ Interdisciplinary Research Groups:
Their Management and Organization. R. T. BARTH, R. STECK (Eds), University of
Washington, Seattle, 1979, p. 136-158.
11. S. AMIR, Environmental research in Israel in the Early 70's, EnvironmentalProtection Service,
Jerusalem, 1979, Publication no. 79 -06 (Hebrew).
12. S. AMIR, A Preliminary Survey of Studies Financed by the Environmental Protection Service
During the Years 1974-1980, Ms, 1981, (Hebrew).
13. S. AMIR, Environmental Research in Israel: a Proposal for a Novel Organizational Change,
1984, in press.
14. Other possible indicators that obey similar requirements are : ~ b ~ / ~ a ~/b(~-l); ~ b in s,
and so on.
15. The values of D(b, s) ealculatedfor the other indicators axe 0.88 a, 0.68 a, 0.55 ~ respectively.
16. In statistical ecology, the concept of species richness stands for the number of different species
in a given field sample of organisms.17 In ecological field studies, it is common to find that
species richness positively depends on the size, i.e. the number of individuals, of the sample
taken. Simple diversity indices, defined to depend on species richness and on sample size
simultaneously, are increasing functions of species richness but decrease when the number of
individuals increases.
17. R. K. PEET, The measurement of species diversity, Annual Review of Ecology and
Systematics, 5 (1974) 285.

Scien tometrics 8 (1985) 135

9*
S. AMIR: INTERDISCIPLINARITY OF RESEARCH PROGRAM

18. Similarly, we could have defined other indicators, such as #b/x/~,#b/ln s; ~ (b-1)/lns ,and so
on,
19. The Shannon index is defines as -~.Pi logPi' and the Simpson index is Z.Pi(1-Pi), where
1 1
p:=ni/N is the relative frequency of scientists (references) of the i'th field. These indices are
biased estimators of population diversity if derived from finite samples. ~7'2~ We have
calculated the values of an unbiased estimator of the Simpson index, ~.ni(ni-1)/N ( N - l )
getting 0.8 for the scientific field distribution and 0.77 for the reference fiel~ distribution.
20. E. C. PIELOU, An Introduction to Mathematical Ecology, Wiley Interscience, New York,
1969.
21. J. C. FLEISS, Statistical Methods for Rates and Proportions, Wiley Interscience, New York,
1973.
22. F. A. ROSSINI, A. L. PORTER, Frameworks for integrating interdisciplinary research,
Research Policy, 8 (1979)70.
23. R. WASNIOWSKI, Management of interdisciplinary research programs in a university, in:
Interdisciplinary Research Groups: Their Management and Organization, R. T. BARTH, R.
STECK (Eds), University of Washington, Seattle, 1979, p. 266-277.
24. N. LINDAS, Conclusions from the American Society for Public Administration's Assessment of
four interdiseipfinary research management projects, in: Interdisciplinary Research Groups:
Their Management and Organization. R. T. BARTH, R. STECK (Eds), University of
Waslfington, Seattle, 1979, p. 278-294.
25. B. R. MARTIN, J. IRVINE, Assessing basic research: some partial indicators of scientific
progress in radio astronomy, Research Policy, 12 (1983) 61.

136 Scientometrics 8 {1985)