CHAPTER 8 PART 1: SURVEYS

https://quizlet.com/_5rhdtv

Introduction
● Asking people questions or probing for relative positions is the most common data
collection method in social science
○ Surveys are often used for this purpose in criminal justice research
● A survey is a data collection method that
○ Applies a standard instrument
○ In a systematic way
○ To make measurements

Survey Research: Counting Crime

● As a standalone techniques, surveys are well-suited to…
○ Descriptive research
○ Application research
● Surveys are also most appropriate for studies that have individuals as the unit of
analysis
○ The individual who answers a survey is called the respondent
● Surveys work well when we’re counting crime
○ Victimization surveys
○ Self-report surveys

Other Survey Research Topics

● Perceptions and attitudes:
○ To learn how people feel about crime and criminal justice policy
■ Respondent perceptions - not full-fledged reality
■ Results affected by respondent willingness to accurately share
perceptions with researchers
● Targeted victim surveys
○ Used to evaluate policy innovations and program success

Item Design
● Both questions and statements make suitable survey items
● Types of items
○ Close-ended
■ Respondent selects an answer from a list of choices
■ (ex; do you think crime is a problem in your community? YES NO
○ Open-ended
■ Respondent is asked to provide their own answer
■ (Ex; what sort of things would make you think that crime is a problem in a
community? __________ )
Guidelines for Items
● Make items clear: avoid ambiguous questions; do not ask “double-barreled” questions
○ “The Department of Corrections should stop releasing inmates for weekend
release and concentrate on rehabilitating criminals”
● Short items are best - respondents like to read and answer a question quickly
● Avoid negative items: leads to misinterpretation
● Avoid biased items and terms: phrasing can encourage a certain answer

Questionnaire Construction
● Questionnaire format influences your results!
● Ordering may affect the answers given
○ Estimate the effect of question order
○ Begin with most interesting questions
○ End with duller, demographic data
■ Do the opposite for in-person surveys

Contingency Formatting
● Contingency items:
○ Relevant only to some respondents
○ Are answered only if necessary based on the previous response
○ Ex; “have you ever used illegal narcotics?”
■ Yes
■ No
● If yes, proceed to next item
● If no, skip next item
■ “Please list the illegal narcotics you have used.”

Matrix Formatting
● Same set of attributes used for multiple items
● Key advantages
○ Uses space efficiently
○ Easier for respondents
○ Increasing response compatibility across options

Interview Surveys
● Researchers sometimes have interviewers ask items orally and record respondents’
answers
○ Face-to-face interviews
○ Telephone surveys
● What are some general rules for in-person interviewing?

Self Administered Questionnaires

 ● Home-delivered
○ Researcher delivers questionnaire to home of respondent, explains the study,
then comes back later to retrieve the survey
● Mailed (sent and returned) Survey
○ Researchers must reduce the trouble it takes to return a questionnaire
● Computer Based Self-Administration
○ Enhances perceptions of confidentiality which enhances the accuracy of
self-reporting

Response Rates
● If a researcher asked you to participate in a survey, would you agree to do it?
○ In person
○ By phone
○ By mail
○ By the internet
● Response rate refers to the number of people participating in a survey divided by the
number selected for the sample

Weaknesses of Survey Research

● Superficial coverage of complex topics
○ Provide a limited and partial view into the respondent’s reality
○ Detracts from the validity of the research
● Some population might be hard to contact

Strengths of Survey Research

● Surveys tend to be high on reliability
○ Standardization is respondent responses;
■ Data collection benefit
■ Data analysis benefit
● One of the biggest strengths of surveys is the ability to tabulate experiences

CHAPTER 8 PART 2: SAMPLING

Introduction
● Sampling
○ The process of selecting cases for observation from a population
● Why sample?
○ Not possible to collect data from all cases
○ Not necessary to collect data from all cases
● 2 fundamental approaches to sampling
○ Probability sampling
○ Non-probability sampling
The Logic of Probability Sampling
● Probability sampling allows researchers to generalize from observed to unobserved
cases
○ Gives all the cases in the population an equal chance of being selected
● What is the logic behind selecting a sample?
○ To represent a larger population of cases
○ To generalize from observed sample cases to unobserved cases from the
population

Challenges to Representatives
● Bias
○ That cases selected are not “Typical” or “Representative” of the populations from
which they have been chosen
● Equal Probability of Selection Method (EPSEM)
○ A sample is representative if its aggregate characteristics closely match the
population’s aggregate characteristics

Probability Theory
● Sample element:
○ Who or what we are studying (Ex student)
● Population
○ Whole group we seek to generalize to (ex criminology program students)
● Sample statistic:
○ The summary description of a given variable in the sample (ex average of
program satisfaction for sampled criminology students)
● Population Parameter:
○ The actual or assumed value for a given variable in the whole population (Ex
program satisfaction for all criminology students)

Random Sampling
● Probability sampling uses statistical probability to generate a sample that is
representative of the population
○ What is the importance of random selection?
● A sampling frame is necessary in probability sampling
○ This is an exact (or quasi-) list of all the elements in a population
● How to calculate the ideal size of a sample

Simple Random Sampling

● Process
○ Each element in a sampling frame is assigned a number
○ A necessary sample size is determined
 ○ A random number generation is then used to select which elements will be
included in your sample

Systematic Sampling
● Process is the same as simple random sampling with a slight variation
○ There is a pattern to the overall selection of elements
■ Every “n”th element is selected
● The first element is selected randomly, with every “n” element selected in the list from
that first element
● If your list categories elements in an organized way you should not use systematic
sampling
○ Your sample will be biased

Stratified Sampling
● Is not an alternative to random or systematic sampling
○ It’s a modification to their use
● Stratified sampling is a method for obtaining a greater degree of representatives in your
sample
● The variables you choose to depend on
○ What variables you have information on
○ Which are particularly important for the research questions

Nonprobability Sampling
● There are situations when it is impossible to select a probability sample
● Non Probability sampling is that in which the probability that an element will be included
in the sample is not known
○ Findings cannot be reliably generalized to larger population

Purposive Sampling
● Selecting a sample on the basis of your judgement and the purpose of the study
● Sometimes selecting a sample requires our own knowledge of the population, its
elements, and the nature of our research aims
● Purposive sampling is ideal if members of a population are easily identified, but
compiling a list of all of them would be nearly impossible

Convenience Sampling
● Relying on persons available to complete your study with no or minimal limitations on the
characteristics of the types of persons participating
● A convenience sample is made up of people who are easy to reach
○ Online participants
○ Student participants
● Convenience sampling is a matter of taking what you can get and is the least desirable
type of sample
Snowball Sampling
● Involves…
○ Identifying a single subject or small number of subjects
○ Asking the subjects to identify others like them who might be willing to participate
in the study
● Snowball samples are essentially variations on purposive samples and samples of
available subjects

CHAPTER 10 PART 1: QUALITATIVE INTERVIEWS

https://quizlet.com/_5ru2e0

Introduction
● Allows researchers to study more complex processes or the “hows” involving human
perspective
● A qualitative interview…
○ Is an interaction between an interviewer and a respondent
○ The interviewer has a general plan, including the topics to be covered, but not
necessarily a specific set of questions that must be asked in a particular order
● Can be thought of as a purposeful conversation

Qualitative Interviewing
● Can be used in combination with other methods or as the sole way of gathering data in
criminal justice studies
● Allows the researcher to…
○ Gather firsthand accounts of participants’ impressions and their lived experiences
○ Give participants “voice”
○ Understand, to the best of our abilities, participants’ perspectives

Key Features of Qualitative Interviewing

● Richness and meaning
○ What’s the difference between “thin” and “thick” description?
● Critical realist perspective
○ A philosophical view that reality exists, but knowledge is constructed through
multiple meanings
● Build your interview on a platform
○ Familiarize yourself with research on the same and similar topics through a
comprehensive review

Kinds of Qualitative Interviews

● Interview schedule
○ The structure of the interview
 ○ How you plan to go about asking questions to your participants
● There are 3 general types of interview schedules…
○ Structured
○ Semi-structured
○ Unstructured
● How does adopting an inductive or deductive approach affect your choice of interview
schedule?

Structured and Semi-Structured Interviews

● Structured interviews:
○ Completely standardize the questions being asked across all participants
● Semi structured interviews:
○ Also use standardized questions but include unscheduled probes to enhance the
depth of the response
● Why not use semi-structured interviewing all the time?

Burgess, Hartman, Ressler, and Douglas (1986)

● Method
○ Study focused on analyzing crime scene evidence in conjunction with the details
gleaned about these murderers
○ Semi-structured interviews with 36 sexual killers
■ Was not designed to examine motivation, but yielded rich descriptive data
about what moved these men to kill
■ Interviews included questions about the offender, victim, offense, and
crime scene
○ Was the approach to inquiry inductive or deductive in this research?
● Key findings
○ Consistent behaviours across childhood, adolescence and adulthood included
■ Violent daydreaming
■ Compulsive masturbation
■ Isolation
○ The central role or daydreaming and fantasy in these men’s lives is crucial to
what motivated them to kill
○ Based on the data, the researchers produce a five phase motivational model of
sexual murder
■ Ineffective social environment
■ Child and adolescent formative events
■ Patterned responses to events
■ Resultant actions towards others
■ Personal reactions via a mental feedback filter
● Implications
○ Enhanced understanding of these murderers may enhance law enforcement
efforts at
 ■ Detecting other similar offenders
■ Apprehending other similar offenders
○ Help clinicians attempting to intervene in such behaviours by identifying factors
that play a key role in motivating the behaviour
■ In this case, violent fantasizing

Unstructured Interviews
● Unstructured interviews are the most open style of interviewing
○ When there is a lack of information about a group’s experiences, in-depth,
unstructured interviews can yield key insights
○ The main drawback is that your interviews are less directly comparable to each
other
● Two main approaches:
○ Conversations
■ An informal “chat” where conversation flows organically
○ Interview guide
■ Includes a list of topical areas that you want to cover in the conversation

Gaining Access to Participants

● Establish your role relative to the group you’re interested in studying
○ Are you an insider or an outsider?
■ Your level of “insiderness” depends on how close you are to the group of
study
○ What are the advantages and disadvantages to being an insider?
● Criminal justice researchers are most commonly outsiders - having no existing
connection to the group of interest
● Compensation for access and participation is commonly used to encourage participation
○ Particularly useful in studies where the researcher is an outsider

Gaining Access to Organizations

● Usually, gaining access to participants affiliated with a formal organization requires a
formal request and approval
● The best strategy in gaining access to virtually any formal criminal justice organization is
to use a four step process
○ Sponsor
○ Letter
○ Phone call
○ Meeting

Gaining Access to Subcultures

● Gaining access to subcultures in criminal justice requires tactics that are somewhat
different from those required for formal organizations
 ● The basic principle of using a sponsor to gain initial access operates in much the same
way
○ People whose job involves working with offenders
○ Offenders who are connected with other criminals
● A different approach for gaining access to subcultures is to hang around places where
members of the group you are interested in hang out

Conducting Qualitative Interviews

● Rapport building
○ Refers to the quality of your “connection” and communication with your
participants
○ Rapport can be developed by:
■ Beginning an interview with non-threatening conversation
■ Finding areas of common interest and discussing those
■ Adjusting your communication style to compliments the participant’s
● Reflexivity
○ Refers generally to your subjectivity in the research process
○ Related to the critical realist perspective and the idea that reality is composed of
multiple perspectives
○ Interviewers should engage in self-reflexivity to encourage awareness of the
influence we have on the research process
■ How do your feelings influence your relationships with participants?
■ How do your feelings influence the research process more generally?

CHAPTER 10 PART 2: FOCUS GROUPS

Interview Schedule
● Recall, an interview schedule is how you plan to go about asking questions to your
participants
○ What are the three interview schedule types?
■ Topical areas of focus should be drafted before drafting questions
● How you frame interview questions affects how participants respond
○ What are main questions?
○ Specific problems to avoid
■ Double barreled questions
■ Complex questions
● Overly lengthy
● Difficulty of wording
■ Affective wording in questions

Probes
● Prompt participants to elaborate on responses by filling in more detail and depth
● Probes can be:
 ○ Built in
○ Spontaneous
● Probe types include:
○ Attention probe (ex lean in)
○ Continuation probe (ex nod)
○ Clarification probe (ex ask the respondent to clarify or ask a follow-up question)

Focus Group Interviews

● Involves a directed discussion with a small group of people
● A defining characteristic of the focus group method is:
○ Attention to group dynamics to explore how opinions are produced, expressed,
and changed
● Examples of use in criminology include:
○ Professionals working to intercept human trafficking
○ Media and public attitudes towards crime and the justice system
● What’s the difference between a natural and an artificial group?

Conducting Focus Group Interviews

● The focus group method requires not only being a good interviewer, but also having to
navigate interactions among participants
● Groupthink:
○ Occurs when participants minimize conflict by reaching consensus
○ Diminishes access to alternative viewpoints
● Dominant group members:
○ Persons who take over the group
○ Silences participants who are more naturally quiet

Douglas and Cuskelly (2012)

● Aim was to investigate how police in Queensland, Australia, determined that an
individual has an intellectual disability
○ In Australia, if “it becomes apparent” that a person is intellectually disabled,
police must suspend questioning until support is available
● Interested in the capacity of police to recognize than an individual requires additional
supports:
○ Correct identification ensures that the appropriate protections are immediately
put in place

Method
● Participants were seated in a circle to:
○ Maximize face-to-face contact
○ Emphasize equality
○ Allow interaction to occur in a conversational manner
● Two main questions were used to frame the discussion:
 ○ “When you had an occasion to deal with a person with an intellectual disability
during your work, how did you recognize that the person had an intellectual
disability?”
○ “If you came across a person with an intellectual disability how would you
recognize, if no one told you, that this was a person with a disability?”

Implications
● The statements across the 3 focus groups revealed that the main indicators of
intellectual disability highlighted by officers were:
○ Appearance
○ Language difficulties
○ Problems with comprehension
○ Inappropriate behaviour for age
● The findings draw attention to the very “nonclinical” way in which clinical assessments
are being determined in the field
● The researchers suggest the adoption of a clinical screening instrument to better identify
vulnerable persons

Recording Data
● The process of writing our interviews verbatim is called transcription
○ But you want more than just a written record of what your participants said
● Memoing is a technique that involves writing about your research process from
beginning to end
● There are 3 types of memos:
1. Operational
a. Steps you took in each stage of your research
2. Coding
a. Documenting the process of coding your data
3. Analytic
a. Record of how you explored relationships in your data

Data Management and Reduction

● Data reduction involves putting aside information that seems irrelevant
● Coding involves the organization of raw data into conceptual categories
○ Each code is a category into which a piece of data is placed
● Coding takes place in stages:
○ Open
■ Involves “chunking” data in an exploratory fashion
○ Selective
■ Looking for patterns in the reduced version of your data

Data Analysis and Making Claims

● Analyzing data is the point at which researchers make and support claims about findings
 ● There are 3 different ways of presenting qualitative data:
1. Description
a. Letting the data speak for themselves with minimal commentary
2. Analysis
a. Systematically expanding beyond a purely descriptive account based on
observed cases
3. Interpretation
a. Moving beyond what can be explained with a degree of certainty to that which is
more speculative

Quality and Rigor

● Enhancing quality and rigor in qualitative research is all about being thorough in your
research process…
○ An audit trail is a chronicle of the research process and enhances confidence in
the quality of research
● Validity and reliability can be enhanced in qualitative research through…
○ Member checks
■ When participants read drafts of products to verify its accuracy
○ Using multiple coders
■ Allows an inter-rater reliability statistic to be generated

CHAPTER 11: FIELD OBSERVATION

https://quizlet.com/_5rv16h

Introduction
● Field research rencompasses two different methods of obtaining data
○ Direct observation
○ Asking questions
● Most commonly associated with qualitative research but can yield quantitative data
○ Provides the opportunity to produce highly detailed, “rich” descriptions
○ Saturation occurs when sampling more data will not lead to the production of new
information
● Qualitative field research is often a theory of hypothesis generating activity
○ What does this approach to inquiry look like?

Field Research Topics

● Especially appropriate for topics best understood in their natural setting
● Ethnography
○ Focused on detailed and accurate description rather than explanation
● Gives comprehensive perspective
○ Go directly to phenomenon
○ Observe phenomenon as completely as possible
Rosenbaum’s Shoplifting Study
● Key difficulties in observational studies of crime
○ Offenses are relatively rare and unpredictable
○ Unless concealed, field observers are likely to deter criminals from offending
○ Danger to fieldworkers
● Key methods details of the study
○ Employed a team of RAs to observe a sample of shoppers in a particular
department store
○ Sampling strategy was to have an RA discreetly follow every 20th shopper who
came into the store
○ Reliability of RAs was vetted through fake shoplifters

Roles of the Observer

● Complete participant
○ Participate fully; true identity and purpose are not known to subjects
● Participant as observer
○ Make known your position as researcher and participate in the observed activity
● Observer as participant
○ Make known your position as researcher; do not actually participate in the activity
● Complete observer
○ Observe without engaging with that being observed or becoming a participant

Sampling in Field Research

● Controlled probability sampling used rarely, non-probability sampling is common
● Have to consider unique factors relative to other research…
○ Weather
○ Time
● Bear in mind 2 stages of sampling:
○ To what extent are the situations available for observation representative of the
general phenomena you wish to describe and explain?
○ Are your actual observations within those total situations representative of all
observations?

Recording Observations
● Consider a variety of forms for making observations
○ Note taking
○ Tape recording
○ Photographs
○ Video recording
● Field notes
○ Observations are recorded as written notes, often in a field journal; first take
sketchy notes and then rewrite your notes in detail
● Structures observations
 ○ Observers mark close-ended forms, which produce numeric measures

Strengths and Weaknesses

● The advantages and disadvantages of field research can be considered in terms of
○ Validity
■ Whether observations are representative of the true phenomenon under
study
○ Reliability
■ Whether observations are dependable and consistent
○ Generalizability
■ Whether specific research findings from the sample apply to a broader
population from which the sample was drawn

CHAPTER 12 PART 1: CONTENT ANALYSIS

https://quizlet.com/_5rvs80

Introduction
● The methods of focus this week do not require direct interaction with research subjects
● Data from agency records
○ Agencies collect a vast amount of crime and criminal justice data
● Secondary analysis
○ Analyzing data previously collected
● Content analysis
○ Researchers examine a class of social artifacts (Ex written, visual, verbal
materials)
● What’s the difference between obtrusive and unobtrusive measurement?

Content Analysis
● Content analysis involves the systematic study of messages and the meaning those
messages convey
○ Who says what, to whom, why, how, and with what effect?
● What is content analysis best suited for…
○ Quantitative or qualitative research?
○ An inductive or deductive approach?
● Basic process…
○ Decide on operational definitions of key variables
○ Decide what to watch, read, or listen to, and the time frame
○ Analyze collected data

Coding in Content Analysis

● Coding is how data is measured in content analysis
● Coding is conducted in multiple stages including…
○ Open coding
 ■ Going over content in its totality and looking for overarching themes
○ Selective coding
■ Involves looking for themes in the reduced versions of your data
● Choices of content to analyze…
○ Manifest content
■ Visible, surface content that is directly observable
○ Latent content
■ Underlying content meaning that is not directly observable
● Key reminders for strong coding
○ Your operational definition of variables determines the quality of your coding
■ The attributes of these variables must be mutually exclusive and
exhaustive
○ Pre-test (pilot) your coding scheme
○ Assess coding reliability
■ Test-retest method
■ Inter-coder (inter-rater) reliability method

Violence in Video Games

● Thompson and Haninger (2001) examined the contents of “E” rates video games by the
Entertainment Software Rating Board
● Children’s exposure to violence, substance use, and sexual messages in media are a
source of public health concern
○ But content in video games commonly played by children had not been quantified
● Sampled 55 of 600 E-rated games
○ Experienced gamer played for 90 minutes or until game reached natural
conclusion
■ Play was video recorded
○ Videos were coded for:
■ Number of violent incidents, number of deaths, drugs/alcohol/tobacco,
profanity, sexual behaviour, weapon use, explicit music, and more

Cowan & O’Brien (1990) Introduction

● Explored the content of “slasher” films
○ What portion of the violence is directed toward and committed by women vs men
○ What are the relative survival rates of women vs men
○ What is the juxtaposition of sex and violence for male vs female victims
○ What are the differences between male and female survivors vs non-survivors?
● Cultural assumption regarding slasher films
○ Violence primarily directed toward women
○ Violence frequently occurs in relation to erotic scenes
● These cultural assumptions are empirically testable through content analysis

Method
 ● Catalogs from from live local video outlets were examined
○ Movies that appeared in 4-5 catalogs in the “horror” category (excluding films
depicting violence by nonhuman forces) formed the population
○ Of these, 56 films were selected and assigned to 5 coders
● Material coded included…
○ Provocative clothing
○ Nudity
○ Use of sexual and/or obscene language
○ Promiscuity
○ Demographics
● All codes were observed on a present/absent basis
○ Overall inter-rater reliability, was 87% agreement, with percent agreements
ranging from 75% to 100%

Results
● Key results included that
○ Women were not more likely to be victims of slashers than males
○ But, sxiness was paired with non-survival of female victims
● When non-surviving females were compared to surviving females, consistent differences
were found
○ Non-surviving females were more likely to:
■ Wear revealing and provocative clothing
■ Appear nude
■ Use sexual language
■ Be shown engaging in sexual activity at the time of the slashing
● Slasher films reinforce the idea that sexuality is costly; more specifically, sexuality is
costly for females

Social Science and Humanities Undergraduate Research Internship (SSHURI)

● The faculty supervisor will assist the student in gaining useful academic research skills
through research experience
● Internship details
○ Valued at $4000 (paid out in July and August)
○ Student is expected to put in 250 hours
● Application are usually due in late March and must include a brief letter of intent (1-2)
pages) outlining the nature and scope of the project, including;
○ What is your research question?
○ Is there existing literature on the topic and what does it say?
○ Why is this a topic you would like to study and why is it important?
○ How would you use content analysis to answer the research question?
○ Why is content analysis an appropriate method for this research?

CHAPTER 12 PART 2: AGENCY RECORDS AND SECONDARY ANALYSIS

Introduction
● The methods of focus this week do not require direct interaction with research subjects
● Data from agency records
○ Agencies collect a vast amount of crime and criminal justice data
● Secondary analysis
○ Analyzing data previously collected
● Even if we do not personally interact with participants to collect their data, the original
collection of data can still be obtrusive

Secondary Analysis
● Secondary data refers to data collected by other researchers
● A meta-analysis is a statistical procedure that combines the results of multiple studies to
assess the findings collectively
● Advantages
○ Cheaper and faster
○ Benefit from quality researchers’ skills
○ Access international data easily
● Disadvantages
○ Operational definitions may vary in ways that affect validity

Cao (2014) Secondary Data Analysis

● No nationally representative study had been conducted about differential confidence in
the police between aboriginal people and non aboriginal peoples in Canada
● Used in the 2009 General Social Survey of Canada to examine the influence of
aboriginal status on confidence in the police
● Cao’s literature review highlights a number of factors aboriginal persons were more likely
to experience that might negatively impact perceptions of police

Method
● Data for the research are from the General Social Survey
● ___________ variable
○ Confidence in police
■ Law enforcement
■ Responsiveness
■ Approachability
■ Crime reduction
■ Neighbourhood safety
■ Fair treatment
● __________ variable
○ Aboriginal status
■ 815 (4.2%)
■ Age
 ■ Education
■ Gender
■ Marital status
■ Trust in others
■ Experiences with crime

Results
● The results reveal that aboriginal people have a significantly lower level of confidence in
police than other Canadians
● The findings are not surprising given the long history of racism and the strained
relationship between the police and aboriginal persons in Canada
● The findings help identify
○ The details of why aboriginal persons are unhappy with the police services they
receive
○ Tangible steps that can be taken to address these concerns

Agency Records
● Commonly used in descriptive, exploratory and applied studies
● Much of the data produced by criminal justice agencies is intended to describe or
characterize something
● Key types of government agency records
○ Published statistics
○ Non public statistics

Published Statistics
● Most government agencies routinely collect and publish aggregate data to provide a
general overview of the agency and its activities
● A challenge or working with published statistics it that they are typically only available in
summary format
○ Data cannot be used to analyze the individuals from which or about whom
information was originally collected
● Summary data is useful in addressing questions about aggregated patterns or trends
○ But, not useful for providing nuanced details researchers might be interested in

Nonpublic Agency Records

● Agencies produce a wealth of data which is not routinely released to the public
● Many criminal justice agencies will make non-public data available to researchers upon
an appropriately channeled request
● While potentially valuable, researchers want to have a clear understanding of how
agency records were produced
○ What operational definitions were used?
○ How was the data collected?
○ Who collected the data?
Reliability and Validity Problems
● Understanding the details of how agency records are produced is the best guard against
reliability and validity problems
○ Try to get access to a detailed written description of the method outlining
procedures and instruments used
● Other key factors to keep in mind
○ The extent to which agency records are a social construction
○ Agency records are typically not designed for research and are meant to track
people rather than patterns
○ Clerical errors increase with volume

CHAPTER 13: EVALUATION RESEARCH AND PROBLEM ANALYSIS

https://quizlet.com/_5s1nqx

Introduction
● Applied criminal justice research focuses on assessing actual programs
○ In development or existing
○ In terms of expectations and outcomes
● Two key types of applied studies
○ Problem analysis
■ Helps policy makers analyze alternative actions, choose among them and
formulated routine practices for implementing policy to achieve desired
goals
○ Program evaluation
■ Evaluate the actual effects of policies that have been put in place
● Are policies being implemented as planned?
● Are policies achieving their intended goals?

Evidence Based Policy

● Social policies should be founded on evidence
○ Further, policies adopted should be those with the most supportive evidence
● When criminal justice agencies do not follow this principle, they are potentially
○ Not as effective as they could be
○ Wasteful of resources
● Developing evidence based policy requires two basic steps
○ An awareness and comprehension of existing research evidence
○ An understanding of how to judge the quality and scientific believability of that
evidence

The Policy Process

● Policy demands
○ Support for some new course of action of opposition to some existing policy
 ● Policy agenda
○ Consideration of ultimate goals and strategies for achievement
○ Resource allocation
○ Draft standard course of action
● Policy outputs
○ Are the means to achieve desired policy goals
● Policy impacts
○ Assess if the policy action achieving the goal it was intended to achieve

Linked to the Process to Evaluation

● An evaluation is just the use of social science research methods to systematically
assess the effectiveness of social intervention programs
○ Identifying a problem
○ Conceptualizing a solution
○ Implementing a process that enacts that solution
○ Assessing whether the solution resolved the problem

Problem Analysis
● Differs from program evaluation with respect to where each takes place in the policy
process
○ Used to help conceptualize and design alternative courses of action and choose
among them
● Problem analysis involves 4 key steps
○ Define specific problems
○ Conduct analysis to understand causes
○ Search for multiple solutions to bring about lasting reductions in problems
○ Evaluated the success of these activities

Problem Analysis and Scientific Realism

● Traditional research is based on generalizable understandings of cause and effect
relationships
● Scientific realism suggests that it is misguided to believe that a single understanding of a
relationship can be applied universally
○ Local contexts produce additional variables that may impact upon the original
relationship
● In relation to problem analysis, adopting a scientific realist approach would mean
expecting that
○ Similar interventions can naturally be expected to have different outcomes in
different contexts

Program Evaluation
● Involves collecting information about a program to make necessary decisions about the
program
 ● Key types of program evaluations
○ Process based evaluation
■ Geared toward understanding how a program or policy works
● How does it produce the results that it gets?
● Employee training, customer/client interaction, complaints
○ Outcome based evaluation
■ Geared toward understanding if a program or policy meets the goals it
was intended to meet
● Does it produce the results that it was expected to?
● Outcomes, operationalization

Getting Started
● Learning policy goals is a key first step in doing evaluation research
● Program evaluation is equatable with hypothesis testing
○ Policies are like if-then statements
■ Intervention (X) will produce outcome (Y)
● Process evaluations focus on whether programs are carried out
according to plans
● Outcome evaluations evaluate whether specified goals are
attained
● Goal statements then need to be formulated into empirically testable questions

Problem Formulation and Measurement

● Conducting evaluation research requires operationalizing, observing, and recognizing
what is under study
○ General program objectives need to be clarified into empirically testable
statements
○ Different stakeholders often have different goals for a policy
● For an evaluation question to be answerable, it must be possible to identify some
evidence
○ That can realistically be obtained
○ That will be credible
● In the language of methods
○ The policy is the independent variable
○ The outcome is the dependent variable

Welsh, Farrington and Taheri (2015)

● CCTV has become a highly popular method to prevent crime in public space worldwide
○ Despite the dominance of CCTV in policy, questions have been raised about its
effectiveness
● The authors conducted a meta-analysis of 41 studies that assessed the effectiveness of
CCTVs at reducing crime
● Results that averaged the findings from all the studies revealed
 ○ An overall 16% reduction in crime
● But, where the CCTV system was located affected the results
○ Reduced crime by 51% in parking lots
○ Did not significantly reduce crime in city centers, public housing or public transit
● But the type of crime affected the results
○ Did not significantly reduce violent crime (3% overall reduction)
○ Significantly reduced vehicle crime by 26%
● Operational definitions matter

CHAPTER 14: INTERPRETING BASIC SOCIAL SCIENCE STATISTICS

https://quizlet.com/_5s2h69

Quantitative Methods
● Emphasize the numerical analysis of collected data
○ A main objective of quantitative research is to classify features, count them, and
construct statistical models in an attempt to explain what is observed
● Key characteristics of quantitative research include
○ Clearly defined research questions to which objective answers are sought
○ Study is carefully designed before data is collected
○ Data is usually gathered using structured research instruments
○ Results are based on larger sample sizes that are representative of the
population
○ Study can usually be replicated or repeated, given its high reliability
● In general, the strength of quantitative research lies in its ability to
○ Recognize and isolate specific variables
○ Identify relationships of correlation and causality between variables

Means and Standard Deviations

● Are both specific types of descriptive statistics
○ Descriptive statistics summarize data
■ They are a data reduction technique that involves using a single statistic
to represent a distribution (group of scores)
● The mean is the arithmetic average of the scores
○ Calculated by adding all the scores and dividing by the total number of scores
○ All means are accompanied with a standard deviation
● The standard deviation is a single number that tells us the variability, or spread, of a
distribution

P Values
● Inferential statistics use a random sample of data taken from a population to describe
and make inferences about the population
 ○ A key objective of inferential statistics is to describe relationships between
variables
● Relationships between variables are characterized in one of two ways in inferential
statistical analyses..
○ Significant = there is evidence to suggest that there is a correlation or causal
relationship between two variables
● The determination of statistical significance in all the tests we are talking about today is
based on the p value
○ Tells us how likely it is to get results like the ones we have gotten in our research
if we worked with the assumption that we did not expect to see an effect of our
independent variable
○ P values range from 1-1.0
○ P values of less than .05 are generally accepted as indicating a relationship
between variables is statistically significant
T-test 1
● Test purpose
○ Used when you want to compare the means of two groups for a dependent
variable
● What levels of measurement must variables included in the analysis have?
○ IV = dichotomous
○ DV = interval/ratio
● What is an example of a question we might analyze using a t-test?
○ Do students’ test scores vary by sex (male vs female)?
○ Does police foot patrol (practices vc non-practised) affect rates of violent crime?

T-test 2
● Ratcliffe, Taniguchi, Groff and Wood (2011)
○ An independent samples t test was conducted to evaluate the equivalence in
long-term violent crime levels between treatment (foot patrol) and control (no foot
control) groups
○ Based on the 3 years of data used to create the study areas, an independent
samples t test indicated no statistically significant difference between treatment
(mean = 32.41, SD = 14.20) and control groups (mean = 31.95, SD = 13.96, SD
= 13.96), t(118) = .18 p = .86.
● What do the results mean?

Analysis of Variance (ANOVA) 1

● Test purpose
○ Used when you want to compare the means of three or more groups for a
dependent variable
● What levels of measurement must variables included in the analysis have?
○ IV = categorical (3 or more groups, nominal or ordinal)
○ DV = interval/ratio
 ● What is an example of a question we might analyze using an ANOVA?
○ Does income vary by level or education completed (less than secondary school;
secondary school; post-secondary school)
○ Does victim presence for an apology (ambiguous; surrogate; direct) affect
offender perceptions of accountability for harm caused

ANOVA 2
● Saulnier and Sivasubramaniam (2015)
○ There was a significant main effect of victim presence on accountability
● What do the results mean?

Correlations 1
● Test purpose
○ Is a measure of the strength and direction of an association that exists between
two variables
● What levels of measurement must variables included in the analysis have?
○ Variable 1= interval/ratio
○ Variable 2 = interval/ratio
● There is an important exception to this measurement level rule
○ Variables that have not been or cannot be measured at the interval/ratio level can
be dummy coded
○ Dummy coding refers to the process of coding a categorical variable into a
dichotomous variable
● The Pearson r value tells us
○ The direction of a correlation
■ Positive or negative
○ The strength or a correlation
■ Small, medium, or large
● What is an example of a question we might analyze using correlation
○ Is offence severity associated with length of prison sentence?
○ Is length of prison sentence associated with offering a guilty pleas?

Regression
● Test purpose
○ Regression is the next step up after correlation
○ Used when we want to predict the value of a variable based on the value of
another variable
■ The variable we want to predict is the dependent variable
■ The variable we are using to predict the dependent variable’s value is the
independent variable
● What levels of measurement must variables included in the analysis have?
○ The same as correlation
■ interval/ratio
 ■ Dummy coded
● Two types of regression
○ Linear regression
■ The dependent variable is truly measured at the interval/ratio level
○ Logistic regression
■ The dependent variable is dichotomously coded

CHAPTER 15: METHODOLOGY, METHODS, AND VALUES

https://quizlet.com/_5s2us1

Overview
● What are the general debates relevant to the use of methods in social science research
○ Quantitative or qualitative?
○ What’s the difference between being critical and thinking critically?
○ To have, or not to have, values?
● How might you use the content of this class moving forward?
● What are ways in which the quality of this course could continue to improve?

Quantitative or Qualitative?
● The key distinguishing feature is the focus on numerical data
○ The numerical descriptions of things and their relationships is the focus of
quantitative research methods
○ Qualitative research methods emphasize subjective interpretation, holistic
perspectives, looking at local contexts, and producing a greater depth of
understanding
● Selecting a quantitative or qualitative approach should be based on
○ Your research question
○ Your research purpose
○ The data or data sources you have access to
● Quantitative research is typically considered to be the more “scientific” approach to doing
social science while qualitative research has been relegated to the realm of
pseudo-science
● Qualitative criminological research is less commonly published
○ Less than 11% of articles in top tier journals in the discipline employ qualitative
methods

Go Go Qualitative
● Tewksbury (2013) argues that qualitative methods are about gaining more valid
understanding of
○ The social aspects of how crime occurs and how the agents, structures and
processes of responding to crime operate in culturally-grounded contexts
 ● Qualitative methods provide a depth of understanding that is not possible through the
use of quantitative, statistically-based investigations
● While less generalizable, qualitative research offers a more in depth understanding of
social issues and the contexts in which they occur

Becker (1966): Whose side are we on?

● A recurrent theme in this course has been the importance of objective research
○ Sampling techniques
○ Material (survey items and interview questions) construction
○ Reflexivity
● Social scientists confront a false dilemma
○ Do we strive to do neutral research that is value free?
○ Do we express our values and adopt a commitment to a position
● Why is this dilemma false?
● The question is not whether we should take sides, because we inevitably will!
○ The question is: whose side are we on?
● Becker considers two types of research situations where researchers will inevitably take
a side
○ Political
■ There is acknowledged tension between subordinates and authorities
■ Subordinates have some degree of organization
○ Apolitical
■ There is not a defined power struggle between subordinates and
authorities
● Whatever point of view we take, our task is to ensure that our research meets the
standards of good scientific work
○ We do this by using sound methods
○ Impartial techniques that improve the reliability of and validity of our research