Concepts, Operationalization, Professor Jillian E.

Snider
CJBS 250

and Measurement John Jay College of Criminal Justice

 Outline of
Chapter
 Introduction
 Conceptions and
Concepts
 Operationalization
Choices
 Criteria for
Measurement Quality
 Composite Measures
 Summary
 Learning Objectives

• Understand the role of concepts as summary

devices for bringing together observations and
experiences that have something in common
• Explain how concepts are mental images that do
not exist in the real world
• Describe how operationalization specifies
concrete empirical procedures for measuring
variables
• Recognize that operationalization begins with
study design but continues through the duration
of research
• Explain why measurement categories must be
mutually exclusive and exhaustive
Learning Objectives
• Distinguish different levels of measurement
and the properties of different levels
• Understand precision, reliability, and
validity as dimensions of measurement
quality
• Summarize how creating specific, reliable
measures may not reflect the complexity of
the concepts we seek to study
• Understand how multiple measures of a
concept can improve reliability and validity
• Describe composite measures and explain
their advantages
Introduction
• It is essential to specify exactly what we mean,
and do not mean, by the terms we use
• This is the first step in the measurement process
• Because measurement is difficult and imprecise,
researchers try to describe the measurement
process explicitly
• We want to move from vague ideas of what we
want to study to actually being able to recognize
and measure it in the real world
• Otherwise, we will be unable to communicate the
relevance of our idea and findings to an audience
 Conceptions and Concepts
• Clarifying abstract mental images is an
essential first step in measurement
• Conception: mental image we have about
something
• Concepts: words, phrases, or symbols in
language that are used to represent
mental images in communication
• Examples: gender, punishment,
chivalry, delinquency, poverty,
intelligence, racism, sexism, assault,
deviance, income
 Understanding Conception and Concepts
• Crime represents our conception about certain kinds of behavior
• Individuals have different conception and may think of different kinds of behavior when they
hear the word crime
• Police Officers may include simple possession of marijuana among their conception of crime
• Residents (civilians) of states where recreational marijuana use is legal would not include simple
possession of marijuana among their conception of crime
• Conceptions are subjective and cannot be communicated directly
• Therefore, we use the words and symbols of language as a way to communicate our conceptions and
the things we observe that are related to those conceptions
• Crime as a concept is abstract, meaning that in the English language, this label
represents mental images of illegal acts
• Our mental images of crime may be based on real witnessed events or what we see in movie or
on television
• When we talk about “crime” without being more specific, we are talking about an
abstraction
• An abstraction is the quality of dealing with ideas rather than actual events
 Three Classes that Scientists
Measure
• To better understand conceptions, concepts,
and measurement, consider Kaplan’s (1964)
discussion of 3 classes of things that scientists
measure:
1. Direct observables: those things or
qualities we can observe directly (color,
shape)
2. Indirect observables: require relatively
more subtle, complex, or indirect
observations for things that cannot be
observed directly (reports, court
transcripts, criminal records)
3. Constructs: theoretical creations; cannot
be observed directly or indirectly; like
concept
Conceptualization
• Conceptualization is the process by which we
specify precisely what we mean when we use
particular terms
• It is the mental process of making fuzzy and
imprecise notions, such as concepts, more
specific and precise
• Results in a set of indicators of what we have in
mind
• Indicates a presence or absence of the concept
we are studying
• Example: Violent crime = offender uses force (or
threatens to use force) against a victim
 Indicators and Dimensions
• The end product of the conceptualization
process is the specification of a set of
indicators of what we have in mind
• Dimension: specifiable aspect of a
concept
• Example: “Crime seriousness” – can be
subdivided into dimensions
• Dimension – victim harm
• Indicators – physical harm,
economic loss, psychological
consequences
• Specification leads to a deeper
understanding
Confusion Over Definitions and Reality
• Concepts are abstract and only
mental creations
• The terms we use to describe them
do not have real and concrete
meanings
• Example: “Crime seriousness”
• Reification: the process of
regarding as real things that are not
• The representation of factual
assertions that are referred to by
other assertions
 Logicians and scientists have found it
useful to distinguish 3 kinds of
definitions:
1.Real: inherently subjective
2.Conceptual: the working definition of
a concept or term
Creating • Provides focus to our observations
Conceptualization 3.Operational: a statement specifying
what operations should be performed
Order to measure a concept
• Spells out precisely how the concept
will be measured
The specifications of concepts in
scientific inquiry depends on conceptual
and operational definitions
 Progression
of
Measurement
Steps
Operationalization
Choices
• Operationalization:
the process of
developing operational
definitions
• Moves us closer to
measurement
• Requires us to
determine what might
work as a data-
collection method
Measurement as “Scoring”

• Measurement: assigning numbers

or labels to units of analysis in
order to represent the conceptual
properties
• Make observations, and assign
scores to them
• Difficult in Criminal Justice
research because basic concepts
are not perfectly definable
Exhaustive and Exclusive Measurement
• Every variable should have
two important qualities:
1. Exhaustive: you should be
able to classify every
observation in terms of one
of the attributes composing
the variable
2. Mutually exclusive: you
must be able to classify
every observation in terms
of one and only one
attribute
 Levels of Measurement

• Nominal: offer names of labels for

characteristics
• Examples: race, gender, state of
residence
• Ordinal: attributes can be logically rank-
ordered
• Examples: Opinions, occupational status
• Interval: meaningful distance between
attributes
• Example: IQ
• Ratio: has a true zero point
• Example: # of prior arrests, length of
prison sentence, income
Implications of Levels of Measurement
• Certain analytic techniques have
Levels of Measurement
requirements
• Ratio levels are the highest level,
followed by Interval, Ordinal, and
Nominal
• Ratio can also be treated as Nominal,
Ordinal, or Interval
• You cannot covert a lower Level of
Measurement to a higher one
• Therefore, seek the highest Level
of Measurement possible
Criteria for Measurement
Quality
• The key standards for
measurement quality are
reliability and validity
• Measurements can be made
with varying degrees of
precision
• Common sense dictates that the
more precise, the better
• However, you do not necessarily
need complete precision
Reliability
• Reliability is a matter of whether a
particular measurement technique,
repeatedly applied to the same object,
would yield the same result each time
• Consistency in results
• Problem: even if the same result is
retrieved, it may be incorrect every
time
• Reliability does not ensure accuracy
• Observer’s subjectivity might come
into play
 Methods of Dealing with Reliability Issues

Test-retest method: make the same

measurement more than once – should
expect the same response both times

Inter-rater reliability: compare

measurements from different raters;
verify initial measurements

Split-half method: make more than one

measure of any concept; see if each
measures the concept differently
Validity
• Validity is the extent to which an
empirical measure adequately
reflects the meaning of the
concept under consideration
• Are you really measuring what
you say you are measuring?
• Demonstrating validity is more
difficult than demonstrating
reliability
 Face validity: on its face, does it seem valid? Does
it jibe with our common agreements and mental
images?

Criterion-related validity: compares a measure to

some external criterion

Methods of
Dealing with Construct validity: whether your variables related
to each other in the logically expected direction

Validity Issues
Content validity: does the measure cover the
range of meanings included in the concepts?

Multiple measures: alternative measures

Relationship between Reliability and Validity

• Validity is more important

to a research study
• Reliability cannot tell us if
we are measuring the
correct concept, only if we
are measuring something
consistently
 Composite Measures
• Allows us to combine individual measures to
produce more valid and reliable indicators
• Reasons for using Composite Measures in Criminal
Justice Research:
• The researcher is often unable to develop
single indicators of complex concepts
• We may wish to use a rather refined ordinal
measure of a variable, arranging cases in
several ordinal categories from very low to
very high on a variable such as degree of
parental supervision
• Indexes and scales are efficient devices for
data analysis
Typologies
• Researchers combine variables in different ways
to produce different composite measures
• The simplest of these is a typology, also called a
“taxonomy”
• Typologies are produced by the intersection of
two or more variables to create a set of
categories or types
• Example: Loeber, et al. (1991) Typology of
Change in Juvenile Offending
• Examined boys who reported different types of
offending at different times
• These two measures, delinquency at Time One and
delinquency at Time Two, formed the typology
 Index of Disorder
• “What is disorder, and what isn’t?” (Skogan (1990)
• Distinguish between physical presence and social
perception
• Physical disorder: abandoned buildings, garbage
and litter, graffiti, junk in vacant lots
• Social disorder: groups of loiterers, drug use and
sales, vandalism, gang activity, public drinking,
street harassment
• Example: Perception of disorder - some may view
public drinking as disorderly, while residents of
New Orleans or Las Vegas, may consider public
drinking to be acceptable
 Benefits of Indexes
• A composite index is a more valid measure
than a single question
• Computing and averaging across all items
in a category create more variation than
we could obtain in any single item
• Two indexes are more parsimonious than
nine individual variables
• Parsimony is a concept in which an
explanation of a situation or thing is
created with the fewest assumptions
• The “simplest” theory
• Data analysis and interpretation can be
more efficient
 Summary
• Concepts are mental images we use as summary devices for
brining together observations and experiences that seem to
have something in common
• Our concepts do not exist in the real world, so they cannot be
measured directly
• In operationalization, we specify concrete empirical procedures
that will result in measurements of variables
• Operationalization begins in study design and continues
throughout the research project, including the analysis of data
• Categories in a measure must be mutually exclusive and
exhaustive
• Higher levels of measurement specify categories that have
ranked order or more complex numerical properties
 Summary
• A given variable can sometimes be measured at different levels of
measurement
• The most appropriate level of measurement used depends on the purpose
of the measurement
• Precision refers to the exactness of the measure used in an observation of
description of an attribute
• Reliability and validity are criteria for measurement quality
• Valid measure are truly indicators of underlying concepts
• Reliable measures are consistent
• The creation of specific, reliable measures often seems to diminish the
richness of meaning our general concepts have.
• A good solution is to use multiple measures, each of which taps different
aspects of the concept
• Composite measures, formed by combining two or more variables, are often
more valid measures of complex criminal justice topics