Health Measurement Scales: A practical

guide to their development and use 5th

Edition, (Ebook PDF)
Visit to download the full and correct content document:
https://ebookmass.com/product/health-measurement-scales-a-practical-guide-to-their
-development-and-use-5th-edition-ebook-pdf/
 Preface to the fifth edition

The most noticeable change in the fifth edition is on the cover—the addition of a third
author, Dr John Cairney. The need for one was obvious to us: Streiner has retired three
times (so far), and Norman is getting to the stage where he is contemplating his first.
This book has been successful beyond what we could ever have imagined, and we want
to see it continue and keep up with developments in the field, but, to paraphrase the
film mogul, Samuel Goldwyn, ‘Include us out’. The solution was to involve a younger
colleague, Cairney. A word about why him.
When the Faculty of Health Sciences at McMaster University began 45 years ago, it
pioneered a new form of teaching, called problem-based learning, with an emphasis on
small-group tutorials. These sessions were led by ‘non-expert tutors’, whose role was
to guide the students’ learning, rather than formal teaching. Consequently, knowledge
of the content area wasn’t required, and indeed, Streiner, a clinical psychologist by
training, tutored sessions in cardiology (after spending a weekend learning where the
heart was located). Although both of us were in at the beginning and helped shape the
learning environment (and, in fact, Norman was, until recently, the Assistant Dean
of Educational Research and Development), we never subscribed to the notion of the
non-expert tutor in the courses we taught; we always believed you needed in-depth
content knowledge in order to lead a class of learners. Eventually, the medical pro-
gramme also learned this profound lesson (thereby ensuring that none of us had to
waste our time and theirs tutoring medical students).
We then moved into more profitable educational ventures, related to teaching what
we knew, like measurement. Over the many years we taught measurement theory and
scale development, we had very few co-tutors. One of the rare exceptions was John,
who grew from filling in for us with the odd lecture (and some were very odd) to now
running the course. So we leave the book in good hands.
But, what has changed in the book? In addition to updating all of the sections to
incorporate new research, we have made some major additions and revisions. Another
very noticeable addition is a flow chart. Based on suggestions we received from read-
ers, this is a map of the road to scale construction, as well as a guide of where to find
the relevant information. The chapter on generalizability theory has been pretty well
rewritten to reconcile the approach we developed with other approaches, as well as
to extend the method to more complicated designs involving stratification and unbal-
anced designs. We have also eliminated some of the more obtuse sections to make it
more accessible to readers. The introduction to the chapter on item response theory
has been extensively revised to hopefully better describe the rationale and technique
of calibrating the items. The chapter on ethics has been expanded to deal with more
problem areas that may arise when researchers are developing scales and establish-
ing their psychometric properties. The ‘Methods of administration’ chapter has been
viii PREFACE TO THE FIFTH EDITION

expanded to deal with technologies that were only just appearing on the scene when
the last revision was written: Web-based administration, the use of smart phones, and
how mobile phones are changing what can and cannot be done in surveys. There
is now a section in the ‘Measuring change’ chapter explaining how to determine if
individual patients have improved by a clinically and statistically significant degree; a
technique called the ‘reliable change index’.
This and previous editions have benefitted greatly from the comments we have
received from readers, and we continue to welcome them. Please write to us at:
streiner@mcmaster.ca
norman@mcmaster.ca
cairnej@mcmaster.ca.
D.L.S.
G.R.N.
J.C.
 Contents

1 Introduction to health measurement scales 1

Introduction to measurement 1
A roadmap to the book 3
2 Basic concepts 7
Introduction to basic concepts 7
Searching the literature 7
Critical review 8
Empirical forms of validity 10
The two traditions of assessment 14
Summary 17
3 Devising the items 19
Introduction to devising items 19
The source of items 20
Content validity 25
Generic versus specific scales and the ‘fidelity versus bandwidth’ issue 28
Translation 30
4 Scaling responses 38
Introduction to scaling responses 38
Some basic concepts 38
Categorical judgements 39
Continuous judgements 41
To rate or to rank 64
Multidimensional scaling 65
5 Selecting the items 74
Introduction to selecting items 74
Interpretability 74
Face validity 79
Frequency of endorsement and discrimination 80
Homogeneity of the items 81
Multifactor inventories 91
When homogeneity does not matter 92
Putting it all together 94
x CONTENTS

6 Biases in responding 100

Introduction to biases in responding 100
The differing perspectives 100
Answering questions: the cognitive requirements 101
Optimizing and satisficing 104
Social desirability and faking good 106
Deviation and faking bad 111
Minimizing biased responding 111
Yea-saying or acquiescence 115
End-aversion, positive skew, and halo 115
Framing 118
Biases related to the measurement of change 119
Reconciling the two positions 121
Proxy reporting 121
Testing the items 122
7 From items to scales 131
Introduction to from items to scales 131
Weighting the items 131
Missing items 134
Multiplicative composite scores 135
Transforming the final score 138
Age and sex norms 143
Establishing cut points 145
Receiver operating characteristic curves 149
Summary 156
8 Reliability 159
Introduction to reliability 159
Basic concepts 159
Philosophical implications 161
Terminology 164
Defining reliability 164
Other considerations in calculating the reliability
of a test: measuring consistency or absolute agreement 167
The observer nested within subject 169
Multiple observations 170
Other types of reliability 171
Different forms of the reliability coefficient 172
Kappa coefficient versus the ICC 178
 CONTENTS xi

The method of Bland and Altman 179

Issues of interpretation 180
Improving reliability 185
Standard error of the reliability coefficient and sample size 187
Reliability generalization 192
Summary 196
9 Generalizability theory 200
Introduction to generalizability theory 200
Generalizability theory fundamentals 202
An example 204
The first step—the ANOVA 204
Step 2—from ANOVA to G coefficients 207
Step 3—from G study to D study 212
ANOVA for statisticians and ANOVA for psychometricians 212
Confidence intervals for G coefficients 214
Getting the computer to do it for you 214
Some common examples 215
Uses and abuses of G theory 224
Summary 225
10 Validity 227
Introduction to validity 227
Why assess validity? 227
Reliability and validity 228
A history of the ‘types’ of validity 229
Content validation 232
Criterion validation 233
Construct validation 235
Responsiveness and sensitivity to change 244
Validity and ‘types of indices’ 244
Biases in validity assessment 245
Validity generalization 250
Summary 250
11 Measuring change 254
Introduction to measuring change 254
The goal of measurement of change 254
Why not measure change directly? 255
Measures of association—reliability and sensitivity to change 256
Difficulties with change scores in experimental designs 261
xii CONTENTS

Change scores and quasi-experimental designs 262

Measuring change using multiple observations: growth curves 264
How much change is enough? 268
Summary 269
12 Item response theory 273
Introduction to item response theory 273
Problems with classical test theory 273
The introduction of item response theory 275
A note about terminology 275
Item calibration 276
The one-parameter model 280
The two- and three-parameter models 282
Polytomous models 284
Item information 286
Item fit 287
Person fit 289
Differential item functioning 289
Unidimensionality and local independence 290
Test information and the standard error of measurement 294
Equating tests 295
Sample size 296
Mokken scaling 296
Advantages 297
Disadvantages 299
Computer programs 300
13 Methods of administration 304
Introduction to methods of administration 304
Face-to-face interviews 304
Telephone questionnaires 307
Mailed questionnaires 312
The necessity of persistence 317
Computerized administration 319
Using e-mail and the Web 322
Personal data assistants and smart phones 328
From administration to content: the impact
of technology on scale construction 329
Reporting response rates 331
14 Ethical considerations 340
Introduction to ethical considerations 340
 CONTENTS xiii

Informed consent 341

Freedom of consent 344
Confidentiality 345
Consequential validation 346
Summary 347
15 Reporting test results 349
Introduction to reporting test results 349
Standards for educational and psychological testing 350
The STARD initiative 352
GRRAS 354
Summary 354

Appendices
Appendix A Where to find tests 357
Appendix B A (very) brief introduction to factor analysis 375
Author Index 381
Subject Index 391
 Chapter 1

Introduction to health
measurement scales

Introduction to measurement
The act of measurement is an essential component of scientific research, whether in
the natural, social, or health sciences. Until recently, however, discussions regard-
ing issues of measurement were noticeably absent in the deliberations of clinical
researchers. Certainly, measurement played as essential a role in research in the
health sciences as those in other scientific disciplines. However, measurement in the
laboratory disciplines presented no inherent difficulty. Like other natural sciences,
measurement was a fundamental part of the discipline and was approached through
the development of appropriate instrumentation. Subjective judgement played a
minor role in the measurement process; therefore, any issue of reproducibility or val-
idity was amenable to a technological solution. It should be mentioned, however, that
expensive equipment does not, of itself, eliminate measurement errors.
Conversely, clinical researchers were acutely aware of the fallibility of human judge-
ment as evidenced by the errors involved in processes such as radiological diagnosis
(Garland 1959; Yerushalmy 1955). Fortunately, the research problems approached
by many clinical researchers—cardiologists, epidemiologists, and the like—frequently
did not depend on subjective assessment. Trials of therapeutic regimens focused on
the prolongation of life and the prevention or management of such life-threatening
conditions as heart disease, stroke, or cancer. In these circumstances, the measure-
ment is reasonably straightforward. ‘Objective’ criteria, based on laboratory or tissue
diagnosis where possible, can be used to decide whether a patient has the disease, and
warrants inclusion in the study. The investigator then waits an appropriate period of
time and counts those who did or did not survive—and the criteria for death are rea-
sonably well established, even though the exact cause of death may be a little more
difficult.
In the past few decades, the situation in clinical research has become more com-
plex. The effects of new drugs or surgical procedures on quantity of life are likely to be
marginal indeed. Conversely, there is an increased awareness of the impact of health
and healthcare on the quality of human life. Therapeutic efforts in many disciplines
of medicine—psychiatry, respirology, rheumatology, oncology—and other health
professions—nursing, physiotherapy, occupational therapy—are directed equally if
not primarily to the improvement of quality, not quantity of life. If the efforts of
these disciplines are to be placed on a sound scientific basis, methods must be devised
to measure what was previously thought to be unmeasurable and to assess in a
2 INTRODUCTION TO HEALTH MEASUREMENT SCALES

reproducible and valid fashion those subjective states, which cannot be converted into
the position of a needle on a dial.
The need for reliable and valid measures was clearly demonstrated by Marshall et al.
(2000). After examining 300 randomized controlled trials in schizophrenia, they found
that the studies were nearly 40 per cent more likely to report that treatment was effect-
ive when they used unpublished scales rather than ones with peer-reviewed evidence
of validity; and in non-drug studies, one-third of the claims of treatment superiority
would not have been made if the studies had used published scales.
The challenge is not as formidable as it may seem. Psychologists and educators have
been grappling with the issue for many years, dating back to the European attempts
at the turn of the twentieth century to assess individual differences in intelligence
(Galton, cited in Allen and Yen 1979; Stern 1979). Since that time, particularly since
the 1930s, much has been accomplished so that a sound methodology for the devel-
opment and application of tools to assess subjective states now exists. Unfortunately,
much of this literature is virtually unknown to most clinical researchers. Health sci-
ence libraries do not routinely catalogue Psychometrica or The British Journal of
Statistical Psychology. Nor should they—the language would be incomprehensible to
most readers, and the problems of seemingly little relevance.
Similarly, the textbooks on the subject are directed at educational or psychological
audiences. The former is concerned with measures of achievement applicable to
classroom situations, and the latter is focused primarily on personality or aptitude
measures, again with no apparent direct relevance. In general, textbooks in these dis-
ciplines are directed to the development of achievement, intelligence, or personality
tests.
By contrast, researchers in health sciences are frequently faced with the desire to
measure something that has not been approached previously—arthritic pain, return
to function of post-myocardial infarction patients, speech difficulties of aphasic stroke
patients, or clinical competence of junior medical students. The difficulties and ques-
tions that arose in developing such instruments range from straightforward (e.g. How
many boxes do I put on the response?) to complex (e.g. How do I establish whether the
instrument is measuring what I intend to measure?). Nevertheless, to a large degree,
the answers are known, although they are frequently difficult to access.
The intent of this book is to introduce researchers in health sciences to these con-
cepts of measurement. It is not an introductory textbook, in that we do not confine
ourselves to a discussion of introductory principles and methods; rather, we attempt
to make the book as current and comprehensive as possible. The book does not
delve as heavily into mathematics as many books in the field; such side trips may
provide some intellectual rewards for those who are inclined, but frequently at the
expense of losing the majority of readers. Similarly, we emphasize applications, rather
than theory so that some theoretical subjects (such as Thurstone’s law of compara-
tive judgement), which are of historical interest but little practical importance, are
omitted. Nevertheless, we spend considerable time in the explanation of the concepts
underlying the current approaches to measurement. One other departure from cur-
rent books is that our focus is on those attributes of interest to researchers in health
sciences—subjective states, attitudes, response to illness, etc.—rather than the topics
 A ROADMAP TO THE BOOK 3

such as personality or achievement familiar to readers in education and psychology.

As a result, our examples are drawn from the literature in health sciences.
Finally, some understanding of certain selected topics in statistics is necessary to
learn many essential concepts of measurement. In particular, the correlation coefficient
is used in many empirical studies of measurement instruments. The discussion of reli-
ability is based on the methods of repeated measures analysis of variance. Item analysis
and certain approaches to test validity use the methods of factor analysis. It is not by
any means necessary to have detailed knowledge of these methods to understand the
concepts of measurement discussed in this book. Still, it would be useful to have some
conceptual understanding of these techniques. If the reader requires some review of
statistical topics, we have suggested a few appropriate resources in Appendix A.

A roadmap to the book

In this edition, we thought it might be useful to try to provide a roadmap or guide to
summarize the process of scale construction and evaluation (testing). While individ-
ual aspects of this are detailed in the chapters, the overall process can easily be lost in
the detail. Of course in doing so, we run the risk that some readers will jump to the
relevant chapters, and conclude there is not much to this measurement business. That
would be a gross misconception. The roadmap is an oversimplification of a complex
process. However, we do feel it is a valuable heuristic device, which while necessarily
lacking in detail, may nevertheless help to shed some light on the process of meas-
urement. A process, which to those who are new to the field, can often seem like the
darkest of black boxes (see Fig. 1.1).
Our road map begins the same way all research begins—with a question. It may
be derived from the literature or, as is sometimes the case in health research, from
clinical observation. Our question in this case, however, is explicitly connected to a
measurement problem: do we have a measure or scale that can be used in pursuit
of answering our research question? As we will discuss in Chapter 2, there are two
important considerations here: are there really no existing measures that we can use?
This of course can arise when the research question involves measuring a novel con-
cept (or an old one) for which a measure does not currently exist. Our position, always,
is not to bring a new scale into the world unless it is absolutely necessary. However, in
situations where one is truly faced with no other option but to create a new scale,
we begin the process of construction and testing—item generation, item testing, and
re-testing in Fig. 1.1. This is reviewed in detail in Chapters 3 to 6. Chapter 7 covers
the transformation of items to scale. While not explicitly included in this diagram, we
assume that once the items are devised and tested, then a scale will be created by com-
bining items and further tested. It is here we concern ourselves with dimensionality
of the scale (whether our new scale is measuring one thing, or many). We discuss one
common test of dimensionality, factor analysis, in Appendix B.
A far more common occurrence, however, is that we have a scale (or many scales)
that can potentially be used for our research project. Following the top right side of
the roadmap in Fig. 1.1, the important question now is whether we have a scale that
has already been used in similar populations (and for similar purposes) to the one we
4 INTRODUCTION TO HEALTH MEASUREMENT SCALES

Gap identified from

research and/or clinical
practice
Chapter2

Can an existing
No Yes
instrument be
used/modified?
Appendix A

Generate items
Chapters 3 & 4
Has it been
Yes assessed
Use it with
comparable
groups?
Test items
Chapters 4–6

No

Revise items
Pilot study
items with
Do items sample from
require target
Yes revision? population

No

Reliability and
generalizability Do the results
studies No suggest the
Chapters 8 & 9 instrument is
appropriate?

Write it up Validity studies

Chapter 15 Chapter 10
Yes

Fig. 1.1 HMS flowchart.

will be sampling in our study. Appendix A lists many resources for finding existing
scales. For example, there are many scales that measure depression. The selection of
a particular scale will be in part based upon whether there is evidence concerning
its measurement properties specifically in the context of our target population (e.g. a
depression measure for children under the age of 12). Review of the literature is how
we determine this. It is not uncommon at this stage to find that we have ‘near’ hits
(e.g. a scale to measure depression in adolescents, but one that has not been used to
measure depression in children). Here, we must exercise judgement. If we believe that
only minor modifications are required, then we can usually proceed without entering
into a process of item testing. An example here comes from studies of pain. Pain scales
can often be readily adapted for specific pain sites of clinical importance (hands, feet,
 A ROADMAP TO THE BOOK 5

knees, etc.). Absence of a specific scale measuring thumb pain on the right hand for
adults aged 60–65 years, however, is not justification for the creation of a new scale.
There are instances though, where we cannot assume that a scale, which has been
tested in one population, can necessarily be used in a different population without first
testing this assumption. Here we may enter the process of item generation and testing
if our pilot results reveal that important domains have been missed, or if items need to
be substantially modified to be of use with our study population (on our diagram, this
is the link between the right and left sides, or the pathway connecting ‘No’ regarding
group to the item testing loop).
Once we have created a new scale (or revised an existing one for our population),
we are ready to test reliability (Chapters 8 and 9) and validity (Chapter 10). Of course,
if the purpose of the study is not the design and evaluation of a scale, researchers
typically proceed directly to designing a study to answer a research question or test a
hypothesis of association. Following this path, reporting of psychometric properties of
the new or revised scale will likely be limited to internal consistency only, which is the
easiest, but extremely limited, psychometric property to assess (see the section entitled
‘Kuder–Richardson 20 and coefficient α’ in Chapter 5). However, study findings, while
not explicitly stated as such, can nevertheless provide important information on val-
idity. One researcher’s cross-sectional study is another’s construct validation study: it
all depends on how you frame the question. From our earlier example, if the results
of a study using our new scale to measure depression in children shows that girls have
higher levels of depression than boys, or that children who have been bullied are more
likely to be depressed than children who have not, this is evidence of construct valid-
ity (both findings show the scale is producing results in an expected way, given what
we know about depression in children). If the study in question is a measurement
study, then we would be more intentional in our design to measure different aspects
of both reliability and validity. The complexity of the design will depend on the scale
(and the research question). If, for example, we are using a scale that requires multiple
raters to assess a single object of measurement (e.g. raters evaluating play behaviour in
children, tutors rating students), and we are interested in assessing whether these rat-
ings are stable over time, we can devise quite complex factorial designs, as we discuss
in Chapter 9, and use generalizability (G) theory to simultaneously assess different
aspects of reliability. As another example, we may want to look at various aspects
of reliability—inter-rater, test–retest, internal consistency; it then makes sense to do
it with a single G study so we can look at the error resulting from the various fac-
tors. In some cases, a cross-sectional survey, where we compare our new (or revised)
measure to a standard measure by correlating it with other measures hypothesized to
be associated with the construct we are trying to measure, may be all that is needed.
Of course, much has been written on the topic of research design already (e.g. Streiner
and Norman, 2009) and an extended discussion is beyond the scope of this chap-
ter. We do, however, discuss different ways to administer scales (Chapter 13). Choice
of administration method will be determined both by the nature of the scale and
by what aspects of reliability and validity we are interested in assessing. But before
any study is begun, be aware of ethical issues that may arise; these are discussed in
Chapter 14.
6 INTRODUCTION TO HEALTH MEASUREMENT SCALES

Of course, as with all research, our roadmap is iterative. Most of the scales that
have stood the test of time (much like this book) have been revised, re-tested, and
tested again. In pursuit of our evaluation of reliability and validity, we often find that
our scale needs to be tweaked. Sometimes, the required changes are more dramatic.
As our understanding of the construct we are measuring evolves, we often need to
revise scales accordingly. At any rate, it is important to dispel the notion that once a
scale is created, it is good for all eternity. As with the research process in general, the
act of research often generates more questions than answers (thereby also ensuring
our continued employment). When you are ready to write up what you have found,
check Chapter 15 regarding reporting guidelines for studies of reliability and validity.

Further reading
Colton, T.D. (1974). Statistics in medicine. Little Brown, Boston, MA.
Freund, J.E., and Perles, B.M. (2005). Modern elementary statistics (12th edn). Pearson, Upper
Saddle River, NJ.
Huff, D. (1954). How to lie with statistics. W.W. Norton, New York.
Norman, G.R. and Streiner, D.L. (2003). PDQ statistics (3rd edn). PMPH USA, Shelton, CT.
Norman, G.R. and Streiner, D.L. (2014). Biostatistics: The bare essentials (4th edn). PMPH USA,
Shelton, CT.

References
Allen, M.J. and Yen, W.M. (1979). Introduction to measurement theory. Brooks Cole, Monterey,
CA.
Garland, L.H. (1959). Studies on the accuracy of diagnostic procedures. American Journal of
Roentgenology, 82, 25–38.
Marshall, M., Lockwood, A., Bradley, C., Adams, C., Joy, C., and Fenton, M. (2000).
Unpublished rating scales: A major source of bias in randomised controlled trials of
treatments for schizophrenia. British Journal of Psychiatry, 176, 249–52.
Streiner, D.L. and Norman, G.R. (2009). PDQ epidemiology (3rd edn). PMPH USA, Shelton,
CT.
Yerushalmy, J. (1955). Reliability of chest radiography in diagnosis of pulmonary lesions.
American Journal of Surgery, 89, 231–40.
 Chapter 2

Basic concepts

Introduction to basic concepts

One feature of the health sciences literature devoted to measuring subjective states
is the daunting array of available scales. Whether one wishes to measure depression,
pain, or patient satisfaction, it seems that every article published in the field has used a
different approach to the measurement problem. This proliferation impedes research,
since there are significant problems in generalizing from one set of findings to another.
Paradoxically, if you proceed a little further in the search for existing instruments to
assess a particular concept, you may conclude that none of the existing scales is quite
right, so it is appropriate to embark on the development of one more scale to add to
the confusion in the literature. Most researchers tend to magnify the deficiencies of
existing measures and underestimate the effort required to develop an adequate new
measure. Of course, scales do not exist for all applications; if this were so, there would
be little justification for writing this book. Nevertheless, perhaps the most common
error committed by clinical researchers is to dismiss existing scales too lightly, and
embark on the development of a new instrument with an unjustifiably optimistic and
naive expectation that they can do better. As will become evident, the development of
scales to assess subjective attributes is not easy and requires considerable investment
of both mental and fiscal resources. Therefore, a useful first step is to be aware of any
existing scales that might suit the purpose. The next step is to understand and apply
criteria for judging the usefulness of a particular scale. In subsequent chapters, these
will be described in much greater detail for use in developing a scale; however, the
next few pages will serve as an introduction to the topic and a guideline for a critical
literature review.
The discussion that follows is necessarily brief. A much more comprehensive set of
standards, which is widely used for the assessment of standardized tests used in psych-
ology and education, is the manual called Standards for Educational and Psychological
Tests that is published jointly by the American Educational Research Association, the
American Psychological Association, and the National Council on Measurement in
Education (AERA/APA/NCME 1999). Chapter 15 of this book summarizes what scale
developers should report about a scale they have developed, and what users of these
scales should look for (e.g. for reporting in meta-analyses).

Searching the literature

An initial search of the literature to locate scales for measurement of particular
variables might begin with the standard bibliographic sources, particularly Medline.
8 BASIC CONCEPTS

However, depending on the application, one might wish to consider bibliographic

reference systems in other disciplines, particularly PsycINFO for psychological
scales and ERIC (which stands for Educational Resource Information Center:
<http://eric.ed.gov/>) for instruments designed for educational purposes.
In addition to these standard sources, there are a number of compendia of measur-
ing scales, both in book form and as searchable, online resources. These are described
in Appendix A. We might particularly highlight the volume entitled Measuring
Health: A Guide to Rating Scales and Questionnaires (McDowell 2006), which is a crit-
ical review of scales designed to measure a number of characteristics of interest to
researchers in the health sciences, such as pain, illness behaviour, and social support.

Critical review
Having located one or more scales of potential interest, it remains to choose whether
to use one of these existing scales or to proceed to development of a new instru-
ment. In part, this decision can be guided by a judgement of the appropriateness of
the items on the scale, but this should always be supplemented by a critical review of
the evidence in support of the instrument. The particular dimensions of this review
are described in the following sections.

Face and content validity

The terms face validity and content validity are technical descriptions of the judge-
ment that a scale looks reasonable. Face validity simply indicates whether, on the face
of it, the instrument appears to be assessing the desired qualities. The criterion repre-
sents a subjective judgement based on a review of the measure itself by one or more
experts, and rarely are any empirical approaches used. Content validity is a closely
related concept, consisting of a judgement whether the instrument samples all the
relevant or important content or domains. These two forms of validity consist of a
judgement by experts whether the scale appears appropriate for the intended purpose.
Guilford (1954) calls this approach to validation ‘validity by assumption’, meaning the
instrument measures such-and-such because an expert says it does. However, an expli-
cit statement regarding face and content validity, based on some form of review by an
expert panel or alternative methods described later, should be a minimum prerequisite
for acceptance of a measure.
Having said this, there are situations where face and content validity may not be
desirable, and may be consciously avoided. For example, in assessing behaviour such
as child abuse or excessive alcohol consumption, questions like ‘Have you ever hit your
child with a blunt object?’ or ‘Do you frequently drink to excess?’ may have face valid-
ity, but are unlikely to elicit an honest response. Questions designed to assess sensitive
areas are likely to be less obviously related to the underlying attitude or behaviour and
may appear to have poor face validity. It is rare for scales not to satisfy minimal stand-
ards of face and content validity, unless there has been a deliberate attempt from the
outset to avoid straightforward questions.
Nevertheless, all too frequently, researchers dismiss existing measures on the basis
of their own judgements of face validity—they did not like some of the questions, or
 CRITICAL REVIEW 9

the scale was too long, or the responses were not in a preferred format. As we have
indicated, this judgement should comprise only one of several used in arriving at an
overall judgement of usefulness, and should be balanced against the time and cost of
developing a replacement.

Reliability
The concept of reliability is, on the surface, deceptively simple. Before one can obtain
evidence that an instrument is measuring what is intended, it is first necessary to
gather evidence that the scale is measuring something in a reproducible fashion. That
is, a first step in providing evidence of the value of an instrument is to demonstrate
that measurements of individuals on different occasions, or by different observers, or
by similar or parallel tests, produce the same or similar results.
That is the basic idea behind the concept—an index of the extent to which meas-
urements of individuals obtained under different circumstances yield similar results.
However, the concept is refined a bit further in measurement theory. If we were con-
sidering the reliability of, for example, a set of bathroom scales, it might be sufficient
to indicate that the scales are accurate to ±1 kg. From this information, we can easily
judge whether the scales will be adequate to distinguish among adult males (prob-
ably yes) or to assess weight gain of premature infants (probably no), since we have
prior knowledge of the average weight and variation in weight of adults and premature
infants.
Such information is rarely available in the development of subjective scales. Each
scale produces a different measurement from every other. Therefore, to indicate that a
particular scale is accurate to ±3.4 units provides no indication of its value in measur-
ing individuals unless we have some idea about the likely range of scores on the scale.
To circumvent this problem, reliability is usually quoted as a ratio of the variability
between individuals to the total variability in the scores; in other words, the reliability
is a measure of the proportion of the variability in scores, which was due to true dif-
ferences between individuals. Thus, the reliability is expressed as a number between
0 and 1, with 0 indicating no reliability, and 1 indicating perfect reliability.
An important issue in examining the reliability of an instrument is the manner in
which the data were obtained that provided the basis for the calculation of a reliabil-
ity coefficient. First of all, since the reliability involves the ratio of variability between
subjects to total variability, one way to ensure that a test will look good is to conduct
the study on an extremely heterogeneous sample, for example, to measure knowledge
of clinical medicine using samples of first-year, third-year, and fifth-year students.
Examine the sampling procedures carefully, and assure yourself that the sample used
in the reliability study is approximately the same as the sample you wish to study.
Second, there are any number of ways in which reliability measures can be obtained,
and the magnitude of the reliability coefficient will be a direct reflection of the
particular approach used. Some broad definitions are described as follows:
1. Internal consistency. Measures of internal consistency are based on a single admin-
istration of the measure. If the measure has a relatively large number of items
addressing the same underlying dimension, e.g. ‘Are you able to dress yourself?’,
10 BASIC CONCEPTS

‘Are you able to shop for groceries?’, ‘Can you do the sewing?’ as measures of
physical function, then it is reasonable to expect that scores on each item would
be correlated with scores on all other items. This is the idea behind measures of
internal consistency—essentially, they represent the average of the correlations
among all the items in the measure. There are a number of ways to calculate these
correlations, called Cronbach’s alpha, Kuder–Richardson, or split halves, but all
yield similar results. Since the method involves only a single administration of the
test, such coefficients are easy to obtain. However, they do not take into account
any variation from day to day or from observer to observer, and thus lead to an
optimistic interpretation of the true reliability of the test.
2. Other measures of reliability. There are various ways of examining the reproducibil-
ity of a measure administered on different occasions. For example, one might ask
about the degree of agreement between different observers (inter-observer reliabil-
ity); the agreement between observations made by the same rater on two different
occasions (intra-observer reliability); observations on the patient on two occasions
separated by some interval of time (test–retest reliability), and so forth. As a min-
imum, any decision regarding the value of a measure should be based on some
information regarding stability of the instrument. Internal consistency, in its many
guises, is not a sufficient basis upon which to make a reasoned judgement.

One difficulty with the reliability coefficient is that it is simply a number between 0 and
1, and does not lend itself to common-sense interpretations. Various authors have
made different recommendations regarding the minimum accepted level of reliabil-
ity. Certainly, internal consistency should exceed 0.8, and it might be reasonable to
demand stability measures greater than 0.7. Depending on the use of the test, and the
cost of misinterpretation, higher values might be required.
Finally, although there is a natural concern that many instruments in the literature
seem to have too many items to be practical, the reason for the length should be borne
in mind. If we assume that every response has some associated error of measurement,
then by averaging or summing responses over a series of questions, we can reduce this
error. For example, if the original test has a reliability of 0.5, doubling the test will
increase the reliability to 0.67, and quadrupling it will result in a reliability of 0.8. As a
result, we must recognize that there is a very good reason for long tests; brevity is not
necessarily a desirable attribute of a test, and is achieved at some cost (in Chapter 12,
however, we will discuss item response theory, which is a direct a challenge to the idea
longer tests are necessary to maximize reliability).

Empirical forms of validity

Reliability simply assesses that a test is measuring something in a reproducible fashion;
it says nothing about what is being measured. To determine that the test is measur-
ing what was intended requires some evidence of ‘validity’. To demonstrate validity
requires more than peer judgements; empirical evidence must be produced to show
that the tool is measuring what is intended. How is this achieved?
 EMPIRICAL FORMS OF VALIDITY 11

Although there are many approaches to assessing validity, and myriad terms used
to describe these approaches, eventually the situation reduces to two circumstances:
1. Other scales of the same or similar attributes are available. In the situation where
measures already exist, then an obvious approach is to administer the experimen-
tal instrument and one of the existing instruments to a sample of people and see
whether there is a strong correlation between the two. As an example, there are
many scales to measure depression. In developing a new scale, it is straightforward
to administer the new and old instruments to the same sample. This approach is
described by several terms in the literature, including convergent validation, criter-
ion validation, and concurrent validation. The distinction among the terms (and
why we should replace all of them with simply construct validation) will be made
clear in Chapter 10.
Although this method is straightforward, it has two severe limitations. First, if
other measures of the same property already exist, then it is difficult to justify
developing yet another unless it is cheaper or simpler. Of course, many researchers
believe that the new instrument that they are developing is better than the old,
which provides an interesting bit of circular logic. If the new method is better than
the old, why compare it to the old method? And if the relationship between the
new method and the old is less than perfect, which one is at fault?
In fact, the nature of the measurement challenges we are discussing in this book
usually precludes the existence of any conventional ‘gold standard’. Although there
are often measures which have, through history or longevity, acquired criterion
status, a close review usually suggests that they have less than ideal reliability and
validity (sometimes, these measures are better described as ‘fool’s gold’ standards).
Any measurement we are likely to make will have some associated error; as a result,
we should expect that correlations among measures of the same attribute should
fall in the midrange of 0.4–0.8. Any lower correlation suggests that either the reli-
ability of one or the other measure is likely unacceptably low, or that they are
measuring different phenomena.
2. No other measure exists. This situation is the more likely, since it is usually the
justification for developing a scale in the first instance. At first glance, though, we
seem to be confronting an impossible situation. After all, if no measure exists, how
can one possibly acquire data to show that the new measure is indeed measuring
what is intended?
The solution lies in a broad set of approaches labelled construct validity.
We begin by linking the attribute we are measuring to some other attribute
by a hypothesis or construct. Usually, this hypothesis will explore the differ-
ence between two or more populations that would be expected to have differing
amounts of the property assessed by our instrument. We then test this hypo-
thetical construct by applying our instrument to the appropriate samples. If the
expected relationship is found, then the hypothesis and the measure are sound;
conversely, if no relationship is found, the fault may lie with either the measure or
the hypothesis.
Let us clarify this with an example. Suppose that the year is 1920, and a bio-
chemical test of blood sugar has just been devised. Enough is known to hypothesize
12 BASIC CONCEPTS

that diabetics have higher blood sugar values than normal subjects; but no other
test of blood sugar exists. Here are some likely hypotheses, which could be tested
empirically:
0 Individuals diagnosed as diabetic on clinical criteria will have higher blood sugar
on the new test than comparable controls.
0 Dogs whose pancreases are removed will show increasing levels of blood sugar in
the days from surgery until death.
0 Individuals who have sweet-tasting urine will have higher blood sugar than those
who do not.
0 Diabetics injected with insulin extract will show a decrease in blood sugar levels
following the injection.
These hypotheses certainly do not exhaust the number of possibilities, but each can
be put to an experimental test. Further, it is evident that we should not demand a
perfect relationship between blood sugar and the other variable, or even that each and
all relationships are significant. But the weight of the evidence should be in favour of
a positive relationship.
Similar constructs can be developed for almost any instrument, and in the absence
of a concurrent test, some evidence of construct validity should be available. However,
the approach is non-specific, and is unlikely to result in very strong relationships.
Therefore, the burden of evidence in testing construct validity arises not from a single
powerful experiment, but from a series of converging experiments.
Having said all this, a strong caveat is warranted. As we will emphasize in the rele-
vant chapters (Chapter 8 for reliability, Chapter 10 for validity), reliability and validity
are not fixed, immutable properties of scale that, once established, pertain to the scale
in all situations. Rather, they are a function of the scale, the group to which it is admin-
istered, and the circumstances under which it was given. That means that a scale that
may appear reliable and valid when, for example, it is completed by outpatients in
a community clinic, may be useless for inpatients involved in a forensic assessment.
Potential users of a scale must assure themselves that reliability and validity have been
determined in groups similar to the ones with which they want to use it.

Feasibility
For the person wanting to adopt an existing scale for research or clinical purposes, or
someone with masochistic tendencies who wants to develop a new instrument, there
is one final consideration: feasibility. There are a number of aspects of feasibility that
need to be considered: time, cost, scoring, method of administration, intrusiveness,
the consequences of false-positive and false-negative decisions, and so forth.
Time is an issue for both developing a scale and for using it. As will become evident
as you read through this book, you cannot sit down one Friday evening and hope
that by Monday morning you will have a fully developed instrument (although all
too many do seem to have been constructed within this time frame). The process of
writing items, seeing if they work, checking various forms of reliability, and carrying
out even preliminary studies of validity is an extremely time-consuming process. It
 EMPIRICAL FORMS OF VALIDITY 13

can take up to a year to do a decent job, and some people have devoted their entire
careers to developing a single scale (we will forego any judgement whether this reflects
a laudable sense of commitment or a pathological degree of obsessiveness).
Even when an existing scale has been adopted by a researcher, it must be realized
that time is involved on the part of both the researcher and, more importantly, the
respondent. It takes time to score the scale and enter the results into a database.
Although computer-based administration can lessen this load, it means that time must
be spent preparing the scale to be used in this way. For the respondent, the time
involvement is naturally what is required to complete the instrument. Time may not
be an issue for a person lying in a hospital bed recuperating from an operation; he or
she may in fact welcome the distraction. However, it can be an issue for outpatients
or non-patients; they may simply want to go home and get on with their lives. Having
to complete an instrument, especially a long one, may result in anger and frustration,
and lead them to answer in a haphazard manner; a point we will discuss in Chapter 6
when we look at biases in responding.
Cost can also be a problem. If the scale has been copyrighted, it is illegal to just
photocopy it or administer it via computer for research purposes (although, to quote
Hamlet, ‘it is a custom more honor’d in the breach than the observance’). Similarly,
scoring may or may not be an issue. Some scales are scored by simply counting the
number of items that have been endorsed. Others may add a bit of complexity, such
as having different weights for various items or response options, or requiring the
user to reverse score some items (that is, giving Strongly Agree a weight of 1 and
Strongly Disagree a weight of 5 for some items and reversing this for others). If scoring
is done by hand, this can introduce errors (and if there’s a possibility of mistakes, it’s
guaranteed they will occur). Some commercially available scales require computerized
scoring, which adds to both complexity and cost; fortunately, these aren’t used too
often for research purposes.
There are a number of different ways to administer a scale, and each has its own
advantages and disadvantages. The easiest way is to simply hand the instrument to the
respondents and have them answer themselves. However, this assumes they are able
to read the items (raising issues of literacy level and vision) and write the answers (dif-
ficult for patients who may have their arm in a cast or stuck with intravenous tubes).
There is also less control regarding the order in which the questions are read or ensur-
ing that all of the items have been completed. If the scale is administered by a research
assistant, costs are again involved for training and salary; and it precludes giving it to
large groups at once. Computer administration is becoming increasingly more feas-
ible as laptop computers get smaller and more powerful with every generation (which
now seems occur every 2–3 weeks). It has the advantage of obviating the need for
humans to score the responses and enter them into a database, but cost and the ability
to test many people at once are still considerations (see Chapter 13 for an extended
discussion of administration).
You may feel it is important to know the respondents’ income level in order to
describe the socioeconomic level of your sample; or to ask about their sex lives when
you’re looking at quality of life. Bear in mind, though, that some people, especially
those from more conservative cultures, may view these questions as unduly intrusive.
14 BASIC CONCEPTS

They may simply omit these items or, if they are particularly offended, withdraw
completely from the study. It is usually best to err on the side of propriety, and to
emphasize to the respondents that they are free to omit any item they deem offensive.
This is more difficult, though, if an entire scale probes areas that some people may
find offensive.
A scale that places a ‘normal’ person in the abnormal range (a false-positive result)
or misses someone who actually does have some disorder (false-negative result) is not
too much of an issue when the scale is being used solely within the context of research.
It will lead to errors in the findings, but that is not an issue for the individual complet-
ing the scale. It is a major consideration, though, if the results of the scale are used to
make a decision regarding that person—a diagnosis or admission into a programme.
In these circumstances, the issue of erroneous results is a major one, and should be
one of the top criteria determining whether or not a scale should be used.

The two traditions of assessment

Not surprisingly, medicine on the one hand, and psychology and education on the
other, have developed different ways of evaluating people; ways that have influenced
how and why assessment tools are constructed in the first place, and the manner in
which they are interpreted. This has led each camp to ignore the potential contribution
of the other: the physicians contending that the psychometricians do not appreciate
how the results must be used to abet clinical decision-making, and the psychologists
and educators accusing the physicians of ignoring many of the basic principles of test
construction, such as reliability and validity. It has only been within the last two dec-
ades that some rapprochement has been reached; a mutual recognition that we feel
is resulting in clinical instruments that are both psychometrically sound as well as
clinically useful. In this section, we will explore two of these different starting points—
categorical versus dimensional conceptualization and the reduction of measurement
error—and see how they are being merged.

Categorical versus dimensional conceptualization

Medicine traditionally has thought in terms of diagnoses and treatments. In the most
simplistic terms, a patient either has a disorder or does not, and is either prescribed
some form of treatment or is not. Thus, diastolic blood pressure (DBP), which is meas-
ured on a continuum of millimetres of mercury, is often broken down into just two
categories: normotensive (under 90 mmHg diastolic in North America), in which case
nothing needs to be done; and hypertensive (90 mmHg and above), which calls for
some form of intervention.
Test constructors who come from the realm of psychology and education, though,
look to the writings of S. Smith Stevens (1951) as received wisdom. He introduced the
concept of ‘levels of measurement’, which categorizes variables as nominal, ordinal,
interval, or ratio—a concept we will return to in greater depth in Chapter 4. The basic
idea is that the more finely we can measure something, the better; rating an attri-
bute on a scale in which each point is equally spaced from its neighbours is vastly
superior to dividing the attribute into rougher categories with fewer divisions. Thus,
 THE TWO TRADITIONS OF ASSESSMENT 15

Table 2.1 Categorical and dimensional conceptualization

Categorical model Dimensional model

1. Diagnosis requires that multiple criteria,
each with its threshold value, be
satisfied
2. Phenomenon differs qualitatively and
quantitatively at different severities
3. Differences between cases and
non-cases are implicit in the definition
4. Severity is lowest in instances that
minimally satisfy diagnostic criteria
5. One diagnosis often precludes others
Source: data from Devins, G., Psychiatric rating scales, Paper presented at the Clarke Institute of Psychiatry,
Toronto, Ontario, Copyright © 1993.
Occurrence of some features at high
intensities can compensate for
non-occurrence of others
Phenomenon differs only quantitatively at
different severities
Differences between cases and non-cases
are less clearly delineated
Severity is lowest among non-disturbed
individuals
A person can have varying amounts of
different disorders

psychometricians tend to think of attributes as continua, with people falling along the
dimension in terms of how much of the attribute they have.
The implications of these two different ways of thinking have been summarized
by Devins (1993) and are presented in modified form in Table 2.1. In the categor-
ical mode, the diagnosis of, for example, major depression in the fifth edition of the
Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric
Association 2013) requires that the person exhibit at least five of nine symptoms (one
of which, depressed mood or loss of interest/pleasure, must be among the five) and
that none of four exclusion criteria be present. In turn, each of the symptoms, such as
weight change or sleep disturbance has its own minimum criterion for being judged to
be demonstrated (the threshold value). A continuous measure of depression, such as
the Center for Epidemiological Studies Depression scale (CES-D; Radloff 1977), uses
a completely different approach. There are 20 items, each scored 1 through 4, and a
total score over 16 is indicative of depression. No specific item must be endorsed, and
there are numerous ways that a score of 16 can be achieved—a person can have four
items rated 4, or 16 items rated 1, or any combination in between. Thus, for diagnostic
purposes, having many mild symptoms is equivalent to having a few severe ones.
Second, DSM-5 differentiates among various types of depressions according to their
severity and course. A bipolar depression is qualitatively and quantitatively different
from a dysthymic disorder; different sets of criteria must be met to reflect that the
former is not only more severe than the latter, but that it is cyclical in its time course,
whereas dysthymia is not. On the other hand, the CES-D quantifies the depressive
symptomatology, but does not differentiate among the various types. Consequently,
it would reflect that one person’s symptoms may be more extensive than another’s,
irrespective of category.
One implication of this difference is that there is a clear distinction between cases
and non-cases with the categorical approach, but not with the dimensional. In the for-
mer, one either meets the criteria and is a case, or else the criteria are not satisfied, and
16 BASIC CONCEPTS

one is not a case. With the latter, ‘caseness’ is a matter of degree, and there is no clear
dividing line. The use of a cut point on the CES-D is simply a strategy so that it can be
used as a diagnostic tool. The value of 16 was chosen only because, based on empirical
findings, using this score maximized agreement between the CES-D and clinical
diagnosis; the number is not based on any theoretical argument. Furthermore, there
would be less difference seen between two people, one of whom has a score of 15 and
the other 17, than between two other people with scores of 30 and 40, although one is
a ‘case’ and the other not in the first instance, and both would be ‘cases’ in the second.
Another implication is that, since people who do not meet the criteria are said to
be free of the disorder with the categorical approach, differences in severity are seen
only among those who are diagnosed (point 4 in Table 2.1). Quantification of sever-
ity is implicit, with the assumption that those who meet more of the criteria have a
more severe depression than people who satisfy only the minimum number. With the
dimensional approach, even ‘normal’ people may have measurable levels of depressive
symptomatology; a non-depressed person whose sleep is restless would score higher
on the CES-D than a non-depressed person without sleep difficulties.
Last, using the categorical approach, it is difficult (at least within psychiatry) for a
person to have more than one disorder. A diagnosis of major depression, for example,
cannot be made if the patient has a psychotic condition, or shows evidence of delu-
sions or hallucinations. The dimensional approach does permit this; some traits may
be present, albeit in a mild form, even when others coexist.
The limitations of adhering strictly to one or the other of these ways of thinking
are becoming more evident. The categorical mode of thinking is starting to change,
in part due to the expansion of the armamentarium of treatment options. Returning
to the example of hypertension, now patients can be started on salt-restricted diets,
followed by diuretics at higher levels of the DBP, and finally placed on vasodilators.
Consequently, it makes more sense to measure blood pressure on a continuum, and
to titrate the type and amount of treatment to smaller differences than to simply
dichotomize the reading. On the other hand, there are different treatment implica-
tions, depending on whether one has a bipolar or a non-cyclical form of depression.
Simply measuring the severity of symptomatology does not allow for this. One reso-
lution, which will be discussed in Chapter 4, called multidimensional scaling is an
attempt to bridge these two traditions. It permits a variety of attributes to be meas-
ured dimensionally in such a way that the results can be used to both categorize and
determine the extent to which these categories are present.

The reduction of measurement error

Whenever definitive laboratory tests do not exist, physicians rely primarily on the
clinical interview to provide differential diagnoses. The clinician is the person who
both elicits the information and interprets its significance as a sign or symptom
(Dohrenwend and Dohrenwend 1982). Measurement error is reduced through train-
ing, interviewing skills, and especially clinical experience. Thus, older physicians are
often regarded as ‘gold standards’ since they presumably have more experience and
therefore would make fewer errors. (In a different context, though, Caputo (1980,
 SUMMARY 17

p. 370) wrote, ‘They haven’t got seventeen years’ experience, just one year’s experience
repeated seventeen times’.)
In contrast, the psychometric tradition relies on self-reports of patients to usually
close-ended questions. It is assumed that the response to any one question is sub-
ject to error: the person may misinterpret the item, respond in a biased manner, or
make a mistake in transcribing his or her reply to the answer sheet. The effect of
these errors is minimized in a variety of ways. First, each item is screened to deter-
mine if it meets certain criteria. Second, the focus is on the consistency of the answers
across many items, and for the most part, disregarding the responses to the individ-
ual questions. Last, the scale as a whole is checked to see if it meets another set of
criteria.
The ‘medical’ approach is often criticized as placing unwarranted faith in the clinical
skills of the interviewer. Indeed, as was mentioned briefly in Chapter 1, the reliability
(and hence the validity) of the clinical interview leaves much to be desired. Conversely,
psychometrically sound tests may provide reliable and valid data, but do not yield
the rich clinical information and the ways in which patients differ from one another,
which come from talking to them in a conversational manner. These two ‘solitudes’
are starting to merge, especially in psychiatry, as is seen in some of the structured
interviews such as the Diagnostic Interview Schedule (DIS; Robins et al. 1981). The
DIS is derived from the clinical examination used to diagnose psychiatric patients,
but is constructed in such a way that it can be administered by trained lay people,
not just psychiatrists. It also relies on the necessity of answering a number of ques-
tions before an attribute is judged to be present. Thus, elements of both traditions
guided its construction, although purists from both camps will be dissatisfied with the
compromises.

Summary
The criteria we have described are intended as guidelines for reviewing the litera-
ture, and as an introduction to the remainder of this book. We must emphasize that
the research enterprise involved in development of a new method of measurement
requires time and patience. Effort expended to locate an existing measure is justified,
because of the savings if one can be located and the additional insights provided in the
development of a new instrument if none prove satisfactory.

Further reading
American Educational Research Association, American Psychological Association, and
National Council on Measurement in Education (1999). Standards for educational and
psychological testing. American Educational Research Association, Washington, DC.

References
American Educational Research Association, American Psychological Association, and
National Council on Measurement in Education (1999). Standards for educational and
psychological testing. American Psychological Association, Washington, DC.
18 BASIC CONCEPTS

American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders
(5th edn). American Psychiatric Publishing, Arlington VA.
Caputo, P. (1980). Horn of Africa. Holt, Rinehart and Winston, New York.
Devins, G. (1993). Psychiatric rating scales. Paper presented at the Clarke Institute of Psychiatry,
Toronto, Ontario.
Dohrenwend, B.P. and Dohrenwend, B.S. (1982). Perspectives on the past and future of
psychiatric epidemiology. American Journal of Public Health, 72, 1271–9.
Guilford, J.P. (1954). Psychometric methods. McGraw-Hill, New York.
McDowell, I. (2006). Measuring health (3rd edn). Oxford University Press, Oxford.
Radloff, L.S. (1977). The CES-D scale: A self-report depression scale for research in the general
population. Applied Psychological Measurement, 1, 385–401.
Robins, L.N., Helzer, J.E., Crougham, R., and Ratcliff, K.S. (1981). National Institute of Mental
Health Diagnostic Interview Schedule: Its history, characteristics, and validity. Archives of
General Psychiatry, 38, 381–9.
Stevens, S.S. (1951). Mathematics, measurement, and psychophysics. In Handbook of experi-
mental psychology (ed. S.S. Stevens), pp. 1–49. Wiley, New York.
 Chapter 3

Devising the items

Introduction to devising items

The first step in writing a scale or questionnaire is, naturally, devising the items
themselves. This is far from a trivial task, since no amount of statistical manipula-
tion after the fact can compensate for poorly chosen questions; those that are badly
worded, ambiguous, irrelevant, or—even worse—not present. In this chapter, we
explore various sources of items and the strengths and weaknesses of each of them.
The first step is to look at what others have done in the past. Instruments rarely
spring fully grown from the brows of their developers. Rather, they are usually based
on what other people have deemed to be relevant, important, or discriminating.
Wechsler (1958), for example, quite openly discussed the patrimony of the subtests
that were later incorporated into his various IQ tests. Of the 11 subtests that com-
prise the original adult version, at least nine were derived from other widely used
indices. Moreover, the specific items that make up the individual subtests are them-
selves based on older tests. Both the items and the subtests were modified and new
ones added to meet his requirements, but in many cases the changes were relatively
minor. Similarly, the Manifest Anxiety Scale (Taylor 1953) is based in large measure
on one scale from the Minnesota Multiphasic Personality Inventory (MMPI; Hathaway
and McKinley 1951).
The long, and sometimes tortuous, path by which items from one test end up in
others is beautifully described by Goldberg (1971). He wrote that (p. 335):
Items devised around the turn of the century may have worked their way via Woodworth’s
Personal Data Sheet, to Thurstone and Thurstone’s Personality Schedule, hence to
Bernreuter’s Personality Inventory, and later to the Minnesota Multiphasic Personality
Inventory, where they were borrowed for the California Personality Inventory and then
injected into the Omnibus Personality Inventory—only to serve as a source of items for
the new Academic Behavior Inventory.

Angleitner et al. (1986) expanded this to ‘and, we may add, only to be translated and
included in some new German personality inventories’ (p. 66).
There are a number of reasons that items are repeated from previous inventories.
First, it saves work and the necessity of constructing new ones. Second, the items have
usually gone through repeated processes of testing so that they have proven themselves
to be useful and psychometrically sound. Third, there are only a limited number of
ways to ask about a specific problem. If we were trying to tap depressed mood, for
instance, it is difficult to ask about sleep loss in a way that has not been used previously.
Hoary as this tradition may be, there are (at least) three problems in adopting it
uncritically. First, it may result in items that use outdated terminology. For example,
20 DEVISING THE ITEMS

the original version of the MMPI (in use until 1987) contained such quaint terms
such as ‘deportment’, ‘cutting up’, and ‘drop the handkerchief ’. Endorsement of these
items most likely told more about the person’s age than about any aspect of his or
her personality. Perhaps, more importantly, the motivation for developing a new tool
is the investigator’s belief that the previous scales are inadequate for one reason or
another, or do not completely cover the domain under study. We would add further
that in an era where researchers are increasingly under pressure to produce products
that generate revenue, the practice of lifting items from existing scales, especially those
that are in the public domain or otherwise not protected by current patent or copyright
protection, for use in the creation of scales that will be only be available for cost, seems
at best a dubious ethical and legal practice. At this point, new items can come from
five different sources: the patients or subjects themselves, clinical observation, theory,
research, and expert opinion; although naturally the lines between these categories are
not firm.

The source of items

A point often overlooked in scale development is the fact that patients and potential
research subjects are an excellent source of items. Whereas clinicians may be the best
observers of the outward manifestations of a trait or disorder, only those who have
it can report on the more subjective elements. Over the years, a variety of techniques
have been developed, which can elicit these viewpoints in a rigorous and systematic
manner; these procedures are used primarily by ‘qualitative’ researchers and are only
now finding their way into more ‘quantitative’ types of studies. Here, we can touch
only briefly on two of the more relevant techniques; greater detail is provided in texts
such as Taylor and Bogden (1998) and Willms and Johnson (1993).

Focus groups
Willms and Johnson (1993) describe a focus group as (p. 61):
a discussion in which a small group of informants (six to twelve people), guided by a
facilitator, talk freely and spontaneously about themes considered important to the inves-
tigation. The participants are selected from a target group whose opinions and ideas are
of interest to the researcher. Sessions are usually tape recorded and an observer (recorder)
also takes notes on the discussion.

In the area of scale development, the participants would be patients who have the
disorder or subjects representative of those whose opinions would be elicited by the
instrument. At first, their task would not be to generate the specific items, but rather
to suggest general themes that the research team can use to phrase the items them-
selves. Usually, no more than two or three groups would be needed. Once the items
have been written, focus groups can again be used to discuss whether these items
are relevant, clear, unambiguous, written in terms that are understood by potential
respondents, and if all the main themes have been covered. These groups are much
more focused than those during the theme generation stage since there is a strong
externally generated agenda, discussing the items themselves.
Another random document with
no related content on Scribd:
 DANCE ON STILTS AT THE GIRLS’ UNYAGO, NIUCHI

Newala, too, suffers from the distance of its water-supply—at least

the Newala of to-day does; there was once another Newala in a lovely
valley at the foot of the plateau. I visited it and found scarcely a trace
of houses, only a Christian cemetery, with the graves of several
missionaries and their converts, remaining as a monument of its
former glories. But the surroundings are wonderfully beautiful. A
thick grove of splendid mango-trees closes in the weather-worn
crosses and headstones; behind them, combining the useful and the
agreeable, is a whole plantation of lemon-trees covered with ripe
fruit; not the small African kind, but a much larger and also juicier
imported variety, which drops into the hands of the passing traveller,
without calling for any exertion on his part. Old Newala is now under
the jurisdiction of the native pastor, Daudi, at Chingulungulu, who,
as I am on very friendly terms with him, allows me, as a matter of
course, the use of this lemon-grove during my stay at Newala.
 FEET MUTILATED BY THE RAVAGES OF THE “JIGGER”
(Sarcopsylla penetrans)

The water-supply of New Newala is in the bottom of the valley,

some 1,600 feet lower down. The way is not only long and fatiguing,
but the water, when we get it, is thoroughly bad. We are suffering not
only from this, but from the fact that the arrangements at Newala are
nothing short of luxurious. We have a separate kitchen—a hut built
against the boma palisade on the right of the baraza, the interior of
which is not visible from our usual position. Our two cooks were not
long in finding this out, and they consequently do—or rather neglect
to do—what they please. In any case they do not seem to be very
particular about the boiling of our drinking-water—at least I can
attribute to no other cause certain attacks of a dysenteric nature,
from which both Knudsen and I have suffered for some time. If a
man like Omari has to be left unwatched for a moment, he is capable
of anything. Besides this complaint, we are inconvenienced by the
state of our nails, which have become as hard as glass, and crack on
the slightest provocation, and I have the additional infliction of
pimples all over me. As if all this were not enough, we have also, for
the last week been waging war against the jigger, who has found his
Eldorado in the hot sand of the Makonde plateau. Our men are seen
all day long—whenever their chronic colds and the dysentery likewise
raging among them permit—occupied in removing this scourge of
Africa from their feet and trying to prevent the disastrous
consequences of its presence. It is quite common to see natives of
this place with one or two toes missing; many have lost all their toes,
or even the whole front part of the foot, so that a well-formed leg
ends in a shapeless stump. These ravages are caused by the female of
Sarcopsylla penetrans, which bores its way under the skin and there
develops an egg-sac the size of a pea. In all books on the subject, it is
stated that one’s attention is called to the presence of this parasite by
an intolerable itching. This agrees very well with my experience, so
far as the softer parts of the sole, the spaces between and under the
toes, and the side of the foot are concerned, but if the creature
penetrates through the harder parts of the heel or ball of the foot, it
may escape even the most careful search till it has reached maturity.
Then there is no time to be lost, if the horrible ulceration, of which
we see cases by the dozen every day, is to be prevented. It is much
easier, by the way, to discover the insect on the white skin of a
European than on that of a native, on which the dark speck scarcely
shows. The four or five jiggers which, in spite of the fact that I
constantly wore high laced boots, chose my feet to settle in, were
taken out for me by the all-accomplished Knudsen, after which I
thought it advisable to wash out the cavities with corrosive
sublimate. The natives have a different sort of disinfectant—they fill
the hole with scraped roots. In a tiny Makua village on the slope of
the plateau south of Newala, we saw an old woman who had filled all
the spaces under her toe-nails with powdered roots by way of
prophylactic treatment. What will be the result, if any, who can say?
The rest of the many trifling ills which trouble our existence are
really more comic than serious. In the absence of anything else to
smoke, Knudsen and I at last opened a box of cigars procured from
the Indian store-keeper at Lindi, and tried them, with the most
distressing results. Whether they contain opium or some other
narcotic, neither of us can say, but after the tenth puff we were both
“off,” three-quarters stupefied and unspeakably wretched. Slowly we
recovered—and what happened next? Half-an-hour later we were
once more smoking these poisonous concoctions—so insatiable is the
craving for tobacco in the tropics.
Even my present attacks of fever scarcely deserve to be taken
seriously. I have had no less than three here at Newala, all of which
have run their course in an incredibly short time. In the early
afternoon, I am busy with my old natives, asking questions and
making notes. The strong midday coffee has stimulated my spirits to
an extraordinary degree, the brain is active and vigorous, and work
progresses rapidly, while a pleasant warmth pervades the whole
body. Suddenly this gives place to a violent chill, forcing me to put on
my overcoat, though it is only half-past three and the afternoon sun
is at its hottest. Now the brain no longer works with such acuteness
and logical precision; more especially does it fail me in trying to
establish the syntax of the difficult Makua language on which I have
ventured, as if I had not enough to do without it. Under the
circumstances it seems advisable to take my temperature, and I do
so, to save trouble, without leaving my seat, and while going on with
my work. On examination, I find it to be 101·48°. My tutors are
abruptly dismissed and my bed set up in the baraza; a few minutes
later I am in it and treating myself internally with hot water and
lemon-juice.
Three hours later, the thermometer marks nearly 104°, and I make
them carry me back into the tent, bed and all, as I am now perspiring
heavily, and exposure to the cold wind just beginning to blow might
mean a fatal chill. I lie still for a little while, and then find, to my
great relief, that the temperature is not rising, but rather falling. This
is about 7.30 p.m. At 8 p.m. I find, to my unbounded astonishment,
that it has fallen below 98·6°, and I feel perfectly well. I read for an
hour or two, and could very well enjoy a smoke, if I had the
wherewithal—Indian cigars being out of the question.
Having no medical training, I am at a loss to account for this state
of things. It is impossible that these transitory attacks of high fever
should be malarial; it seems more probable that they are due to a
kind of sunstroke. On consulting my note-book, I become more and
more inclined to think this is the case, for these attacks regularly
follow extreme fatigue and long exposure to strong sunshine. They at
least have the advantage of being only short interruptions to my
work, as on the following morning I am always quite fresh and fit.
My treasure of a cook is suffering from an enormous hydrocele which
makes it difficult for him to get up, and Moritz is obliged to keep in
the dark on account of his inflamed eyes. Knudsen’s cook, a raw boy
from somewhere in the bush, knows still less of cooking than Omari;
consequently Nils Knudsen himself has been promoted to the vacant
post. Finding that we had come to the end of our supplies, he began
by sending to Chingulungulu for the four sucking-pigs which we had
bought from Matola and temporarily left in his charge; and when
they came up, neatly packed in a large crate, he callously slaughtered
the biggest of them. The first joint we were thoughtless enough to
entrust for roasting to Knudsen’s mshenzi cook, and it was
consequently uneatable; but we made the rest of the animal into a
jelly which we ate with great relish after weeks of underfeeding,
consuming incredible helpings of it at both midday and evening
meals. The only drawback is a certain want of variety in the tinned
vegetables. Dr. Jäger, to whom the Geographical Commission
entrusted the provisioning of the expeditions—mine as well as his
own—because he had more time on his hands than the rest of us,
seems to have laid in a huge stock of Teltow turnips,[46] an article of
food which is all very well for occasional use, but which quickly palls
when set before one every day; and we seem to have no other tins
left. There is no help for it—we must put up with the turnips; but I
am certain that, once I am home again, I shall not touch them for ten
years to come.
Amid all these minor evils, which, after all, go to make up the
genuine flavour of Africa, there is at least one cheering touch:
Knudsen has, with the dexterity of a skilled mechanic, repaired my 9
× 12 cm. camera, at least so far that I can use it with a little care.
How, in the absence of finger-nails, he was able to accomplish such a
ticklish piece of work, having no tool but a clumsy screw-driver for
taking to pieces and putting together again the complicated
mechanism of the instantaneous shutter, is still a mystery to me; but
he did it successfully. The loss of his finger-nails shows him in a light
contrasting curiously enough with the intelligence evinced by the
above operation; though, after all, it is scarcely surprising after his
ten years’ residence in the bush. One day, at Lindi, he had occasion
to wash a dog, which must have been in need of very thorough
cleansing, for the bottle handed to our friend for the purpose had an
extremely strong smell. Having performed his task in the most
conscientious manner, he perceived with some surprise that the dog
did not appear much the better for it, and was further surprised by
finding his own nails ulcerating away in the course of the next few
days. “How was I to know that carbolic acid has to be diluted?” he
mutters indignantly, from time to time, with a troubled gaze at his
mutilated finger-tips.
 Since we came to Newala we have been making excursions in all
directions through the surrounding country, in accordance with old
habit, and also because the akida Sefu did not get together the tribal
elders from whom I wanted information so speedily as he had
promised. There is, however, no harm done, as, even if seen only
from the outside, the country and people are interesting enough.
The Makonde plateau is like a large rectangular table rounded off
at the corners. Measured from the Indian Ocean to Newala, it is
about seventy-five miles long, and between the Rovuma and the
Lukuledi it averages fifty miles in breadth, so that its superficial area
is about two-thirds of that of the kingdom of Saxony. The surface,
however, is not level, but uniformly inclined from its south-western
edge to the ocean. From the upper edge, on which Newala lies, the
eye ranges for many miles east and north-east, without encountering
any obstacle, over the Makonde bush. It is a green sea, from which
here and there thick clouds of smoke rise, to show that it, too, is
inhabited by men who carry on their tillage like so many other
primitive peoples, by cutting down and burning the bush, and
manuring with the ashes. Even in the radiant light of a tropical day
such a fire is a grand sight.
Much less effective is the impression produced just now by the
great western plain as seen from the edge of the plateau. As often as
time permits, I stroll along this edge, sometimes in one direction,
sometimes in another, in the hope of finding the air clear enough to
let me enjoy the view; but I have always been disappointed.
Wherever one looks, clouds of smoke rise from the burning bush,
and the air is full of smoke and vapour. It is a pity, for under more
favourable circumstances the panorama of the whole country up to
the distant Majeje hills must be truly magnificent. It is of little use
taking photographs now, and an outline sketch gives a very poor idea
of the scenery. In one of these excursions I went out of my way to
make a personal attempt on the Makonde bush. The present edge of
the plateau is the result of a far-reaching process of destruction
through erosion and denudation. The Makonde strata are
everywhere cut into by ravines, which, though short, are hundreds of
yards in depth. In consequence of the loose stratification of these
beds, not only are the walls of these ravines nearly vertical, but their
upper end is closed by an equally steep escarpment, so that the
western edge of the Makonde plateau is hemmed in by a series of
deep, basin-like valleys. In order to get from one side of such a ravine
to the other, I cut my way through the bush with a dozen of my men.
It was a very open part, with more grass than scrub, but even so the
short stretch of less than two hundred yards was very hard work; at
the end of it the men’s calicoes were in rags and they themselves
bleeding from hundreds of scratches, while even our strong khaki
suits had not escaped scatheless.

NATIVE PATH THROUGH THE MAKONDE BUSH, NEAR

MAHUTA

I see increasing reason to believe that the view formed some time
back as to the origin of the Makonde bush is the correct one. I have
no doubt that it is not a natural product, but the result of human
occupation. Those parts of the high country where man—as a very
slight amount of practice enables the eye to perceive at once—has not
yet penetrated with axe and hoe, are still occupied by a splendid
timber forest quite able to sustain a comparison with our mixed
forests in Germany. But wherever man has once built his hut or tilled
his field, this horrible bush springs up. Every phase of this process
may be seen in the course of a couple of hours’ walk along the main
road. From the bush to right or left, one hears the sound of the axe—
not from one spot only, but from several directions at once. A few
steps further on, we can see what is taking place. The brush has been
cut down and piled up in heaps to the height of a yard or more,
between which the trunks of the large trees stand up like the last
pillars of a magnificent ruined building. These, too, present a
melancholy spectacle: the destructive Makonde have ringed them—
cut a broad strip of bark all round to ensure their dying off—and also
piled up pyramids of brush round them. Father and son, mother and
son-in-law, are chopping away perseveringly in the background—too
busy, almost, to look round at the white stranger, who usually excites
so much interest. If you pass by the same place a week later, the piles
of brushwood have disappeared and a thick layer of ashes has taken
the place of the green forest. The large trees stretch their
smouldering trunks and branches in dumb accusation to heaven—if
they have not already fallen and been more or less reduced to ashes,
perhaps only showing as a white stripe on the dark ground.
This work of destruction is carried out by the Makonde alike on the
virgin forest and on the bush which has sprung up on sites already
cultivated and deserted. In the second case they are saved the trouble
of burning the large trees, these being entirely absent in the
secondary bush.
After burning this piece of forest ground and loosening it with the
hoe, the native sows his corn and plants his vegetables. All over the
country, he goes in for bed-culture, which requires, and, in fact,
receives, the most careful attention. Weeds are nowhere tolerated in
the south of German East Africa. The crops may fail on the plains,
where droughts are frequent, but never on the plateau with its
abundant rains and heavy dews. Its fortunate inhabitants even have
the satisfaction of seeing the proud Wayao and Wamakua working
for them as labourers, driven by hunger to serve where they were
accustomed to rule.
But the light, sandy soil is soon exhausted, and would yield no
harvest the second year if cultivated twice running. This fact has
been familiar to the native for ages; consequently he provides in
time, and, while his crop is growing, prepares the next plot with axe
and firebrand. Next year he plants this with his various crops and
lets the first piece lie fallow. For a short time it remains waste and
desolate; then nature steps in to repair the destruction wrought by
man; a thousand new growths spring out of the exhausted soil, and
even the old stumps put forth fresh shoots. Next year the new growth
is up to one’s knees, and in a few years more it is that terrible,
impenetrable bush, which maintains its position till the black
occupier of the land has made the round of all the available sites and
come back to his starting point.
The Makonde are, body and soul, so to speak, one with this bush.
According to my Yao informants, indeed, their name means nothing
else but “bush people.” Their own tradition says that they have been
settled up here for a very long time, but to my surprise they laid great
stress on an original immigration. Their old homes were in the
south-east, near Mikindani and the mouth of the Rovuma, whence
their peaceful forefathers were driven by the continual raids of the
Sakalavas from Madagascar and the warlike Shirazis[47] of the coast,
to take refuge on the almost inaccessible plateau. I have studied
African ethnology for twenty years, but the fact that changes of
population in this apparently quiet and peaceable corner of the earth
could have been occasioned by outside enterprises taking place on
the high seas, was completely new to me. It is, no doubt, however,
correct.
The charming tribal legend of the Makonde—besides informing us
of other interesting matters—explains why they have to live in the
thickest of the bush and a long way from the edge of the plateau,
instead of making their permanent homes beside the purling brooks
and springs of the low country.
“The place where the tribe originated is Mahuta, on the southern
side of the plateau towards the Rovuma, where of old time there was
nothing but thick bush. Out of this bush came a man who never
washed himself or shaved his head, and who ate and drank but little.
He went out and made a human figure from the wood of a tree
growing in the open country, which he took home to his abode in the
bush and there set it upright. In the night this image came to life and
was a woman. The man and woman went down together to the
Rovuma to wash themselves. Here the woman gave birth to a still-
born child. They left that place and passed over the high land into the
valley of the Mbemkuru, where the woman had another child, which
was also born dead. Then they returned to the high bush country of
Mahuta, where the third child was born, which lived and grew up. In
course of time, the couple had many more children, and called
themselves Wamatanda. These were the ancestral stock of the
Makonde, also called Wamakonde,[48] i.e., aborigines. Their
forefather, the man from the bush, gave his children the command to
bury their dead upright, in memory of the mother of their race who
was cut out of wood and awoke to life when standing upright. He also
warned them against settling in the valleys and near large streams,
for sickness and death dwelt there. They were to make it a rule to
have their huts at least an hour’s walk from the nearest watering-
place; then their children would thrive and escape illness.”
The explanation of the name Makonde given by my informants is
somewhat different from that contained in the above legend, which I
extract from a little book (small, but packed with information), by
Pater Adams, entitled Lindi und sein Hinterland. Otherwise, my
results agree exactly with the statements of the legend. Washing?
Hapana—there is no such thing. Why should they do so? As it is, the
supply of water scarcely suffices for cooking and drinking; other
people do not wash, so why should the Makonde distinguish himself
by such needless eccentricity? As for shaving the head, the short,
woolly crop scarcely needs it,[49] so the second ancestral precept is
likewise easy enough to follow. Beyond this, however, there is
nothing ridiculous in the ancestor’s advice. I have obtained from
various local artists a fairly large number of figures carved in wood,
ranging from fifteen to twenty-three inches in height, and
representing women belonging to the great group of the Mavia,
Makonde, and Matambwe tribes. The carving is remarkably well
done and renders the female type with great accuracy, especially the
keloid ornamentation, to be described later on. As to the object and
meaning of their works the sculptors either could or (more probably)
would tell me nothing, and I was forced to content myself with the
scanty information vouchsafed by one man, who said that the figures
were merely intended to represent the nembo—the artificial
deformations of pelele, ear-discs, and keloids. The legend recorded
by Pater Adams places these figures in a new light. They must surely
be more than mere dolls; and we may even venture to assume that
they are—though the majority of present-day Makonde are probably
unaware of the fact—representations of the tribal ancestress.
 The references in the legend to the descent from Mahuta to the
Rovuma, and to a journey across the highlands into the Mbekuru
valley, undoubtedly indicate the previous history of the tribe, the
travels of the ancestral pair typifying the migrations of their
descendants. The descent to the neighbouring Rovuma valley, with
its extraordinary fertility and great abundance of game, is intelligible
at a glance—but the crossing of the Lukuledi depression, the ascent
to the Rondo Plateau and the descent to the Mbemkuru, also lie
within the bounds of probability, for all these districts have exactly
the same character as the extreme south. Now, however, comes a
point of especial interest for our bacteriological age. The primitive
Makonde did not enjoy their lives in the marshy river-valleys.
Disease raged among them, and many died. It was only after they
had returned to their original home near Mahuta, that the health
conditions of these people improved. We are very apt to think of the
African as a stupid person whose ignorance of nature is only equalled
by his fear of it, and who looks on all mishaps as caused by evil
spirits and malignant natural powers. It is much more correct to
assume in this case that the people very early learnt to distinguish
districts infested with malaria from those where it is absent.
This knowledge is crystallized in the
ancestral warning against settling in the
valleys and near the great waters, the
dwelling-places of disease and death. At the
same time, for security against the hostile
Mavia south of the Rovuma, it was enacted
that every settlement must be not less than a
certain distance from the southern edge of the
plateau. Such in fact is their mode of life at the
present day. It is not such a bad one, and
certainly they are both safer and more
comfortable than the Makua, the recent
intruders from the south, who have made USUAL METHOD OF
good their footing on the western edge of the CLOSING HUT-DOOR
plateau, extending over a fairly wide belt of
country. Neither Makua nor Makonde show in their dwellings
anything of the size and comeliness of the Yao houses in the plain,
especially at Masasi, Chingulungulu and Zuza’s. Jumbe Chauro, a
Makonde hamlet not far from Newala, on the road to Mahuta, is the
most important settlement of the tribe I have yet seen, and has fairly
spacious huts. But how slovenly is their construction compared with
the palatial residences of the elephant-hunters living in the plain.
The roofs are still more untidy than in the general run of huts during
the dry season, the walls show here and there the scanty beginnings
or the lamentable remains of the mud plastering, and the interior is a
veritable dog-kennel; dirt, dust and disorder everywhere. A few huts
only show any attempt at division into rooms, and this consists
merely of very roughly-made bamboo partitions. In one point alone
have I noticed any indication of progress—in the method of fastening
the door. Houses all over the south are secured in a simple but
ingenious manner. The door consists of a set of stout pieces of wood
or bamboo, tied with bark-string to two cross-pieces, and moving in
two grooves round one of the door-posts, so as to open inwards. If
the owner wishes to leave home, he takes two logs as thick as a man’s
upper arm and about a yard long. One of these is placed obliquely
against the middle of the door from the inside, so as to form an angle
of from 60° to 75° with the ground. He then places the second piece
horizontally across the first, pressing it downward with all his might.
It is kept in place by two strong posts planted in the ground a few
inches inside the door. This fastening is absolutely safe, but of course
cannot be applied to both doors at once, otherwise how could the
owner leave or enter his house? I have not yet succeeded in finding
out how the back door is fastened.

MAKONDE LOCK AND KEY AT JUMBE CHAURO

 This is the general way of closing a house. The Makonde at Jumbe
Chauro, however, have a much more complicated, solid and original
one. Here, too, the door is as already described, except that there is
only one post on the inside, standing by itself about six inches from
one side of the doorway. Opposite this post is a hole in the wall just
large enough to admit a man’s arm. The door is closed inside by a
large wooden bolt passing through a hole in this post and pressing
with its free end against the door. The other end has three holes into
which fit three pegs running in vertical grooves inside the post. The
door is opened with a wooden key about a foot long, somewhat
curved and sloped off at the butt; the other end has three pegs
corresponding to the holes, in the bolt, so that, when it is thrust
through the hole in the wall and inserted into the rectangular
opening in the post, the pegs can be lifted and the bolt drawn out.[50]

MODE OF INSERTING THE KEY

With no small pride first one householder and then a second

showed me on the spot the action of this greatest invention of the
Makonde Highlands. To both with an admiring exclamation of
“Vizuri sana!” (“Very fine!”). I expressed the wish to take back these
marvels with me to Ulaya, to show the Wazungu what clever fellows
the Makonde are. Scarcely five minutes after my return to camp at
Newala, the two men came up sweating under the weight of two
heavy logs which they laid down at my feet, handing over at the same
time the keys of the fallen fortress. Arguing, logically enough, that if
the key was wanted, the lock would be wanted with it, they had taken
their axes and chopped down the posts—as it never occurred to them
to dig them out of the ground and so bring them intact. Thus I have
two badly damaged specimens, and the owners, instead of praise,
come in for a blowing-up.
The Makua huts in the environs of Newala are especially
miserable; their more than slovenly construction reminds one of the
temporary erections of the Makua at Hatia’s, though the people here
have not been concerned in a war. It must therefore be due to
congenital idleness, or else to the absence of a powerful chief. Even
the baraza at Mlipa’s, a short hour’s walk south-east of Newala,
shares in this general neglect. While public buildings in this country
are usually looked after more or less carefully, this is in evident
danger of being blown over by the first strong easterly gale. The only
attractive object in this whole district is the grave of the late chief
Mlipa. I visited it in the morning, while the sun was still trying with
partial success to break through the rolling mists, and the circular
grove of tall euphorbias, which, with a broken pot, is all that marks
the old king’s resting-place, impressed one with a touch of pathos.
Even my very materially-minded carriers seemed to feel something
of the sort, for instead of their usual ribald songs, they chanted
solemnly, as we marched on through the dense green of the Makonde
bush:—
 “We shall arrive with the great master; we stand in a row and have
no fear about getting our food and our money from the Serkali (the
Government). We are not afraid; we are going along with the great
master, the lion; we are going down to the coast and back.”
With regard to the characteristic features of the various tribes here
on the western edge of the plateau, I can arrive at no other
conclusion than the one already come to in the plain, viz., that it is
impossible for anyone but a trained anthropologist to assign any
given individual at once to his proper tribe. In fact, I think that even
an anthropological specialist, after the most careful examination,
might find it a difficult task to decide. The whole congeries of peoples
collected in the region bounded on the west by the great Central
African rift, Tanganyika and Nyasa, and on the east by the Indian
Ocean, are closely related to each other—some of their languages are
only distinguished from one another as dialects of the same speech,
and no doubt all the tribes present the same shape of skull and
structure of skeleton. Thus, surely, there can be no very striking
differences in outward appearance.
 Even did such exist, I should have no time
to concern myself with them, for day after day,
I have to see or hear, as the case may be—in
any case to grasp and record—an
extraordinary number of ethnographic
phenomena. I am almost disposed to think it
fortunate that some departments of inquiry, at
least, are barred by external circumstances.
Chief among these is the subject of iron-
working. We are apt to think of Africa as a
country where iron ore is everywhere, so to
speak, to be picked up by the roadside, and
where it would be quite surprising if the
inhabitants had not learnt to smelt the
material ready to their hand. In fact, the
knowledge of this art ranges all over the
continent, from the Kabyles in the north to the
Kafirs in the south. Here between the Rovuma
and the Lukuledi the conditions are not so
favourable. According to the statements of the
Makonde, neither ironstone nor any other
form of iron ore is known to them. They have
not therefore advanced to the art of smelting
the metal, but have hitherto bought all their
THE ANCESTRESS OF
THE MAKONDE
iron implements from neighbouring tribes.
Even in the plain the inhabitants are not much
better off. Only one man now living is said to
understand the art of smelting iron. This old fundi lives close to
Huwe, that isolated, steep-sided block of granite which rises out of
the green solitude between Masasi and Chingulungulu, and whose
jagged and splintered top meets the traveller’s eye everywhere. While
still at Masasi I wished to see this man at work, but was told that,
frightened by the rising, he had retired across the Rovuma, though
he would soon return. All subsequent inquiries as to whether the
fundi had come back met with the genuine African answer, “Bado”
(“Not yet”).
 BRAZIER

Some consolation was afforded me by a brassfounder, whom I

came across in the bush near Akundonde’s. This man is the favourite
of women, and therefore no doubt of the gods; he welds the glittering
brass rods purchased at the coast into those massive, heavy rings
which, on the wrists and ankles of the local fair ones, continually give
me fresh food for admiration. Like every decent master-craftsman he
had all his tools with him, consisting of a pair of bellows, three
crucibles and a hammer—nothing more, apparently. He was quite
willing to show his skill, and in a twinkling had fixed his bellows on
the ground. They are simply two goat-skins, taken off whole, the four
legs being closed by knots, while the upper opening, intended to
admit the air, is kept stretched by two pieces of wood. At the lower
end of the skin a smaller opening is left into which a wooden tube is
stuck. The fundi has quickly borrowed a heap of wood-embers from
the nearest hut; he then fixes the free ends of the two tubes into an
earthen pipe, and clamps them to the ground by means of a bent
piece of wood. Now he fills one of his small clay crucibles, the dross
on which shows that they have been long in use, with the yellow
material, places it in the midst of the embers, which, at present are
only faintly glimmering, and begins his work. In quick alternation
the smith’s two hands move up and down with the open ends of the
bellows; as he raises his hand he holds the slit wide open, so as to let
the air enter the skin bag unhindered. In pressing it down he closes
the bag, and the air puffs through the bamboo tube and clay pipe into
the fire, which quickly burns up. The smith, however, does not keep
on with this work, but beckons to another man, who relieves him at
the bellows, while he takes some more tools out of a large skin pouch
carried on his back. I look on in wonder as, with a smooth round
stick about the thickness of a finger, he bores a few vertical holes into
the clean sand of the soil. This should not be difficult, yet the man
seems to be taking great pains over it. Then he fastens down to the
ground, with a couple of wooden clamps, a neat little trough made by
splitting a joint of bamboo in half, so that the ends are closed by the
two knots. At last the yellow metal has attained the right consistency,
and the fundi lifts the crucible from the fire by means of two sticks
split at the end to serve as tongs. A short swift turn to the left—a
tilting of the crucible—and the molten brass, hissing and giving forth
clouds of smoke, flows first into the bamboo mould and then into the
holes in the ground.
The technique of this backwoods craftsman may not be very far
advanced, but it cannot be denied that he knows how to obtain an
adequate result by the simplest means. The ladies of highest rank in
this country—that is to say, those who can afford it, wear two kinds
of these massive brass rings, one cylindrical, the other semicircular
in section. The latter are cast in the most ingenious way in the
bamboo mould, the former in the circular hole in the sand. It is quite
a simple matter for the fundi to fit these bars to the limbs of his fair
customers; with a few light strokes of his hammer he bends the
pliable brass round arm or ankle without further inconvenience to
the wearer.
SHAPING THE POT

SMOOTHING WITH MAIZE-COB

CUTTING THE EDGE

 FINISHING THE BOTTOM

LAST SMOOTHING BEFORE

BURNING

FIRING THE BRUSH-PILE

 LIGHTING THE FARTHER SIDE OF
THE PILE

TURNING THE RED-HOT VESSEL

NYASA WOMAN MAKING POTS AT MASASI

Pottery is an art which must always and everywhere excite the
interest of the student, just because it is so intimately connected with
the development of human culture, and because its relics are one of
the principal factors in the reconstruction of our own condition in
prehistoric times. I shall always remember with pleasure the two or
three afternoons at Masasi when Salim Matola’s mother, a slightly-
built, graceful, pleasant-looking woman, explained to me with
touching patience, by means of concrete illustrations, the ceramic art
of her people. The only implements for this primitive process were a
lump of clay in her left hand, and in the right a calabash containing
the following valuables: the fragment of a maize-cob stripped of all
its grains, a smooth, oval pebble, about the size of a pigeon’s egg, a
few chips of gourd-shell, a bamboo splinter about the length of one’s
hand, a small shell, and a bunch of some herb resembling spinach.
 Nothing more. The woman scraped with the
shell a round, shallow hole in the soft, fine
sand of the soil, and, when an active young
girl had filled the calabash with water for her,
she began to knead the clay. As if by magic it
gradually assumed the shape of a rough but
already well-shaped vessel, which only wanted
a little touching up with the instruments
before mentioned. I looked out with the
MAKUA WOMAN closest attention for any indication of the use
MAKING A POT. of the potter’s wheel, in however rudimentary
SHOWS THE a form, but no—hapana (there is none). The
BEGINNINGS OF THE embryo pot stood firmly in its little
POTTER’S WHEEL
depression, and the woman walked round it in
a stooping posture, whether she was removing
small stones or similar foreign bodies with the maize-cob, smoothing
the inner or outer surface with the splinter of bamboo, or later, after
letting it dry for a day, pricking in the ornamentation with a pointed
bit of gourd-shell, or working out the bottom, or cutting the edge
with a sharp bamboo knife, or giving the last touches to the finished
vessel. This occupation of the women is infinitely toilsome, but it is
without doubt an accurate reproduction of the process in use among
our ancestors of the Neolithic and Bronze ages.
There is no doubt that the invention of pottery, an item in human
progress whose importance cannot be over-estimated, is due to
women. Rough, coarse and unfeeling, the men of the horde range
over the countryside. When the united cunning of the hunters has
succeeded in killing the game; not one of them thinks of carrying
home the spoil. A bright fire, kindled by a vigorous wielding of the
drill, is crackling beside them; the animal has been cleaned and cut
up secundum artem, and, after a slight singeing, will soon disappear
under their sharp teeth; no one all this time giving a single thought
to wife or child.
To what shifts, on the other hand, the primitive wife, and still more
the primitive mother, was put! Not even prehistoric stomachs could
endure an unvarying diet of raw food. Something or other suggested
the beneficial effect of hot water on the majority of approved but
indigestible dishes. Perhaps a neighbour had tried holding the hard
roots or tubers over the fire in a calabash filled with water—or maybe
an ostrich-egg-shell, or a hastily improvised vessel of bark. They
became much softer and more palatable than they had previously
been; but, unfortunately, the vessel could not stand the fire and got
charred on the outside. That can be remedied, thought our
ancestress, and plastered a layer of wet clay round a similar vessel.
This is an improvement; the cooking utensil remains uninjured, but
the heat of the fire has shrunk it, so that it is loose in its shell. The
next step is to detach it, so, with a firm grip and a jerk, shell and
kernel are separated, and pottery is invented. Perhaps, however, the
discovery which led to an intelligent use of the burnt-clay shell, was
made in a slightly different way. Ostrich-eggs and calabashes are not
to be found in every part of the world, but everywhere mankind has
arrived at the art of making baskets out of pliant materials, such as
bark, bast, strips of palm-leaf, supple twigs, etc. Our inventor has no
water-tight vessel provided by nature. “Never mind, let us line the
basket with clay.” This answers the purpose, but alas! the basket gets
burnt over the blazing fire, the woman watches the process of
cooking with increasing uneasiness, fearing a leak, but no leak
appears. The food, done to a turn, is eaten with peculiar relish; and
the cooking-vessel is examined, half in curiosity, half in satisfaction
at the result. The plastic clay is now hard as stone, and at the same
time looks exceedingly well, for the neat plaiting of the burnt basket
is traced all over it in a pretty pattern. Thus, simultaneously with
pottery, its ornamentation was invented.
Primitive woman has another claim to respect. It was the man,
roving abroad, who invented the art of producing fire at will, but the
woman, unable to imitate him in this, has been a Vestal from the
earliest times. Nothing gives so much trouble as the keeping alight of
the smouldering brand, and, above all, when all the men are absent
from the camp. Heavy rain-clouds gather, already the first large
drops are falling, the first gusts of the storm rage over the plain. The
little flame, a greater anxiety to the woman than her own children,
flickers unsteadily in the blast. What is to be done? A sudden thought
occurs to her, and in an instant she has constructed a primitive hut
out of strips of bark, to protect the flame against rain and wind.
This, or something very like it, was the way in which the principle
of the house was discovered; and even the most hardened misogynist
cannot fairly refuse a woman the credit of it. The protection of the
hearth-fire from the weather is the germ from which the human
dwelling was evolved. Men had little, if any share, in this forward
step, and that only at a late stage. Even at the present day, the
plastering of the housewall with clay and the manufacture of pottery
are exclusively the women’s business. These are two very significant
survivals. Our European kitchen-garden, too, is originally a woman’s
invention, and the hoe, the primitive instrument of agriculture, is,
characteristically enough, still used in this department. But the
noblest achievement which we owe to the other sex is unquestionably
the art of cookery. Roasting alone—the oldest process—is one for
which men took the hint (a very obvious one) from nature. It must
have been suggested by the scorched carcase of some animal
overtaken by the destructive forest-fires. But boiling—the process of
improving organic substances by the help of water heated to boiling-
point—is a much later discovery. It is so recent that it has not even
yet penetrated to all parts of the world. The Polynesians understand
how to steam food, that is, to cook it, neatly wrapped in leaves, in a
hole in the earth between hot stones, the air being excluded, and
(sometimes) a few drops of water sprinkled on the stones; but they
do not understand boiling.
To come back from this digression, we find that the slender Nyasa
woman has, after once more carefully examining the finished pot,
put it aside in the shade to dry. On the following day she sends me
word by her son, Salim Matola, who is always on hand, that she is
going to do the burning, and, on coming out of my house, I find her
already hard at work. She has spread on the ground a layer of very
dry sticks, about as thick as one’s thumb, has laid the pot (now of a
yellowish-grey colour) on them, and is piling brushwood round it.
My faithful Pesa mbili, the mnyampara, who has been standing by,
most obligingly, with a lighted stick, now hands it to her. Both of
them, blowing steadily, light the pile on the lee side, and, when the
flame begins to catch, on the weather side also. Soon the whole is in a
blaze, but the dry fuel is quickly consumed and the fire dies down, so
that we see the red-hot vessel rising from the ashes. The woman
turns it continually with a long stick, sometimes one way and
sometimes another, so that it may be evenly heated all over. In
twenty minutes she rolls it out of the ash-heap, takes up the bundle
of spinach, which has been lying for two days in a jar of water, and
sprinkles the red-hot clay with it. The places where the drops fall are
marked by black spots on the uniform reddish-brown surface. With a
sigh of relief, and with visible satisfaction, the woman rises to an
erect position; she is standing just in a line between me and the fire,
from which a cloud of smoke is just rising: I press the ball of my
camera, the shutter clicks—the apotheosis is achieved! Like a
priestess, representative of her inventive sex, the graceful woman
stands: at her feet the hearth-fire she has given us beside her the
invention she has devised for us, in the background the home she has
built for us.
At Newala, also, I have had the manufacture of pottery carried on
in my presence. Technically the process is better than that already
described, for here we find the beginnings of the potter’s wheel,
which does not seem to exist in the plains; at least I have seen
nothing of the sort. The artist, a frightfully stupid Makua woman, did
not make a depression in the ground to receive the pot she was about
to shape, but used instead a large potsherd. Otherwise, she went to
work in much the same way as Salim’s mother, except that she saved
herself the trouble of walking round and round her work by squatting
at her ease and letting the pot and potsherd rotate round her; this is
surely the first step towards a machine. But it does not follow that
the pot was improved by the process. It is true that it was beautifully
rounded and presented a very creditable appearance when finished,
but the numerous large and small vessels which I have seen, and, in
part, collected, in the “less advanced” districts, are no less so. We
moderns imagine that instruments of precision are necessary to
produce excellent results. Go to the prehistoric collections of our
museums and look at the pots, urns and bowls of our ancestors in the
dim ages of the past, and you will at once perceive your error.
MAKING LONGITUDINAL CUT IN
BARK

DRAWING THE BARK OFF THE LOG

REMOVING THE OUTER BARK

 BEATING THE BARK

WORKING THE BARK-CLOTH AFTER BEATING, TO MAKE IT

SOFT

MANUFACTURE OF BARK-CLOTH AT NEWALA

To-day, nearly the whole population of German East Africa is
clothed in imported calico. This was not always the case; even now in
some parts of the north dressed skins are still the prevailing wear,
and in the north-western districts—east and north of Lake
Tanganyika—lies a zone where bark-cloth has not yet been
superseded. Probably not many generations have passed since such
bark fabrics and kilts of skins were the only clothing even in the
south. Even to-day, large quantities of this bright-red or drab
material are still to be found; but if we wish to see it, we must look in
the granaries and on the drying stages inside the native huts, where
it serves less ambitious uses as wrappings for those seeds and fruits
which require to be packed with special care. The salt produced at
Masasi, too, is packed for transport to a distance in large sheets of
bark-cloth. Wherever I found it in any degree possible, I studied the
process of making this cloth. The native requisitioned for the
purpose arrived, carrying a log between two and three yards long and
as thick as his thigh, and nothing else except a curiously-shaped
mallet and the usual long, sharp and pointed knife which all men and
boys wear in a belt at their backs without a sheath—horribile dictu!
[51]
Silently he squats down before me, and with two rapid cuts has
drawn a couple of circles round the log some two yards apart, and
slits the bark lengthwise between them with the point of his knife.
With evident care, he then scrapes off the outer rind all round the
log, so that in a quarter of an hour the inner red layer of the bark
shows up brightly-coloured between the two untouched ends. With
some trouble and much caution, he now loosens the bark at one end,
and opens the cylinder. He then stands up, takes hold of the free
edge with both hands, and turning it inside out, slowly but steadily
pulls it off in one piece. Now comes the troublesome work of
scraping all superfluous particles of outer bark from the outside of
the long, narrow piece of material, while the inner side is carefully
scrutinised for defective spots. At last it is ready for beating. Having
signalled to a friend, who immediately places a bowl of water beside
him, the artificer damps his sheet of bark all over, seizes his mallet,
lays one end of the stuff on the smoothest spot of the log, and
hammers away slowly but continuously. “Very simple!” I think to
myself. “Why, I could do that, too!”—but I am forced to change my
opinions a little later on; for the beating is quite an art, if the fabric is
not to be beaten to pieces. To prevent the breaking of the fibres, the
stuff is several times folded across, so as to interpose several
thicknesses between the mallet and the block. At last the required
state is reached, and the fundi seizes the sheet, still folded, by both
ends, and wrings it out, or calls an assistant to take one end while he
holds the other. The cloth produced in this way is not nearly so fine
and uniform in texture as the famous Uganda bark-cloth, but it is
quite soft, and, above all, cheap.
Now, too, I examine the mallet. My craftsman has been using the
simpler but better form of this implement, a conical block of some
hard wood, its base—the striking surface—being scored across and
across with more or less deeply-cut grooves, and the handle stuck
into a hole in the middle. The other and earlier form of mallet is
shaped in the same way, but the head is fastened by an ingenious
network of bark strips into the split bamboo serving as a handle. The
observation so often made, that ancient customs persist longest in
connection with religious ceremonies and in the life of children, here
finds confirmation. As we shall soon see, bark-cloth is still worn
during the unyago,[52] having been prepared with special solemn
ceremonies; and many a mother, if she has no other garment handy,
will still put her little one into a kilt of bark-cloth, which, after all,
looks better, besides being more in keeping with its African
surroundings, than the ridiculous bit of print from Ulaya.
MAKUA WOMEN