Research Design

(Part Two)

Sivakami M
 Unit of analysis
• The what or whom being studied.

• In social science research, the most

typical units of analysis are an
“individual” although there are times
groups or organizations become unit
of analysis.

• Units of analysis in a study are

usually also the units of observation.
 Example (1)

• "Educated (12 or more years of

education) women had fewer mean
children ever born (mean CEB=2.15)
than uneducated women (mean
CEB=4.71) in India.” (NFHS-3).

• What is the unit of analysis???

 Example (2)

• Suppose your aim is to discover whether

students with good study habits received
better grades than students with poor
study habits in School of Health Systems
studies.

• What is unit of analysis?

 Individuals
• Any type of individual may be unit of analysis
– Students
– Faculty members
– Auto drivers
– Single parents
– Children
– Women
– Men

• These individuals are used to make

generalizations about the population they
belong to
 Example of Diarrhea study
• The presentation yesterday

– Diarrhea in Dharavi Slum

– Farmers suicide

• Who is the unit of analysis??

 Groups
• Interested in characteristics that belong to
one group which is considered as one single
entity
• Example: Criminals behaviors
– To understand the members of a criminal gang to
learn about criminals
• Who is the unit of analysis???

– To understand all the criminal gangs in a city to

learn the differences
• between big gangs and small gangs
• Slum and non-slum gangs
– Who is the unit of analysis
 Household consumption
• To understand the dynamics of
household food consumption

– Who is the unit of analaysis

– To understand the intra household

allocation of food consumption

– Who is the unit of analysis

 Organizations
• Formal social organisations

• Study on corporations

• Study on academic departments,

supermarkets, colleges
 Social Artifacts
• Any product of social beings or their
behaviour can also be unit of analysis

– Books, poems, paintings, automobiles,

buildings, songs, pottery, jokes etc

– Interested to how the gender issues are

taught in a class
– Interested to know the paintings from
different countries towards working
population ”
 Faulty reasoning about unit of
analysis
• Ecological fallacy
– Making errors while drawing conclusions
about individuals based solely on the
observation of groups

• Reductionism
• Seeing and explaining complex phenomena in terms
of a single, narrow concept or sent of concepts
– Sociologist
– Demographers
– Economist
The time dimension
 Two principal options
• Researchers have two options to deal
with the issue of time in the research
design

– Cross sectional studies

– Longitudinal studies
 Cross Sectional Study (1)

• A study based on based on observations

presenting a single point in time.

– Exploratory and descriptive studies are

often cross-sectional.

– Retrospective information can be

collected.

• Example: census, NFHS, RCH etc.

 Cross Sectional Study (2)

• Explanatory cross-sectional studies have an

inherent problem.

– Although their conclusions are based on observations

made at only one time, typically they aim at
understanding causal processes that occur over time.

• Example: causal relationship between working

status and fertility based on NFHS-3.
 Longitudinal study
• Longitudinal study is designed to permit
observations of the same phenomenon over
an extended period.

• Longitudinal studies can be more difficult for

quantitative studies such as large-scale
surveys. Nevertheless, they are often the best
way to study change over time. There are
three types of longitudinal studies:
– Trend studies
– Cohort studies
– Panel studies
 Trend Studies

• A trend study is a type of longitudinal study

in which a given characteristic of some
population is monitored over time.

– Example:

• Indian census showed the increasing trend in literacy rate.

• NFHS also shows the same picture, even though different samples
were interviewed at each point.

• Examines change over period of time.

 Cohort Studies (1)

• A study in which some specific

subpopulation, or cohort, is studied
over time, although data may be
collected from different members in
each set of observations.

– Cohort is an age group, such as

people born during the war, people
who got married in 1947, and so
forth.
 Cohort Studies (2)

• Example:
– if you are interested to study the attitudes
of the cohort in the age group of 15-20
years in Iraq towards US .

• 2004 –invasion of Iraq

– A sample of people 15-20 years old might be

surveyed in 2006,
– Another sample of those 25-30 years old in 2016,
– And another sample of those 35-40 years old in
2026.
 Panel Studies

• It is similar to trend and cohort studies but

panel study examines the same set of people
each time.

• The data are collected from the same set of

people at several points in time.
 Example
• Study on parental work participation
and its effects on children

– Many panel studies in developed countries

– Understanding the impact of cancer

treatment
 Disadvantage of the Longitudinal Studies

• Longitudinal studies have advantages over cross-

sectional studies in providing information, that is, in
describing processes over time.

– But this advantage often comes at a heavy cost in both time and
money, especially in large scale survey.

– Panel attrition

• Some of the respondents studied in the first wave of the

survey might not participate in later waves.
• Those who are not interested to participate, if they have
typical characteristics then it will distort the findings.
• Mortality
Modes of observation
 Experiments

• Experiments are done often in natural sciences

• Very often it is done in social sciences

– An experiment is a mode of observation that enables

researchers to probe the causal relationships.
 Three principles of Designing an Experiment or
your research

1. Replication

2. Local control

3. Randomization
 The classical experiment

Three major pairs of components:

1. Independent and dependent variables

2. Pre testing and post testing

3. Experimental and control groups

 Independent and dependent variables
• An experiment examines the effect of an
independent variable on dependent variable.

• Independent variable takes the form of an

experimental stimulus, which is either
present or absent.

– Example: (1) smoking and lung cancer.

– Effect of IEC campaign (regarding small family
norm) on adoption of contraception.
 Pretesting and posttesting

• Subjects are measured in terms of a dependent

variable (pretesting), exposed to a stimulus
representing an independent variable, and then
remeasured in terms of the dependent variable.

• Any difference between first and last measurements

on the dependent variable are then attributed to the
independent variable.

– Effect of IEC (regarding small family norm) on adoption of

contraception.
 Experimental and control groups

• Experimental group to which a stimulus has been

administered ( IEC materials has been distributed).

• Control which does not receive the experimental

stimulus (IEC materials has not been distributed).
• For example, residents of village A received the IEC
materials (regarding advantages of small family
norm) and village B has not received.
 Preexperimental research designs
 Preexperimental research designs, not to
recommend to use because they don't meet the
scientific standards of experimental designs.
 One shot case study : Researcher measures a
single group of subjects on a dependent variable
following the administration of some experimental
stimulus. Lacking of pretest.
 One-group pretest-posttest design: Pretest for
experimental group but lacks a control group.
 Static-group comparison: Based on experimental
and control groups but has no pretests.
 Quasi-experimental design

• Quasi-experiments are distinguished from

“true” experiments primarily by the lack of
random assignment of subjects to an
experimental and a control group.
 Natural Experiments

• For example, hurricane has struck a particular town.

Some residents of the town suffer severe financial
damages, and others escape relative lightly. What, we
might ask are the behavioral consequences of suffering a
natural disaster?
• Are those who suffer most more likely to take precautions
against future disasters than are those who suffer least?
• Then we can conduct the study. We might question them
regarding their precautions before the hurricane and the
ones they are currently taking, comparing the people who
suffered greatly from the hurricane with those who
suffered relatively little.
 Issues

• For example, residents of village A received

the IEC materials (regarding advantages of
small family norm) and village B has not
received.

• Can you tell some issues related to such

type of design.
 Validity Issues in Experimental Research
• Internal validity: It refers to the possibility that
the conclusions drawn from experimental
results may not accurately reflect what has
gone on in the experiment itself.

• The threat of internal invalidity is present

whenever anything other than the experimental
stimulus can affect the dependent variable.
 Sources of internal validity
1. History: (Historical events may occur that will confound
the experimental results.)

2. Maturation: (People are continuing growing and

changing, and such changes can affect the results of
the experiment.)

3. Testing: (Testing and retesting influences people’s

behaviour, thereby confounding the experimental
results.)

4. Instrumentation: (If we use different measures of the

dependent variable in the pretest and posttest, say
different questionnaires.)

5. Statistical regression: Choosing wrong sample and

wrong tests
6. Selection bias
7. Experimental mortality (Drop out): It will create biases in
the experiment and control group.
10. Compensation: Control groups are often deprived of
something. For example, hospital staff might feel sorry for
control group patients and give them extra “tender loving
care.” In such a situation, the control group is no longer a
genuine control group.
11. Compensatory rivalry: Suppose an experimental math
program is the experimental stimulus; the control group
may work harder than before on their math in an attempt to
beat the “special” experimental subjects.
12. Demoralization: In educational experiments, demoralized
control-group subjects may stop studying or get angry.
 Sources of external validity
• External validity: Refers to the possibility that
conclusions drawn from experimental results may
not be generalizable to the “real” world.

• For example, residents of village A received the IEC

materials (regarding advantages of small family norm) and
village B has not received.

• If we distribute the IEC materials in the general population

then can we get expected results.

• There should not be any interaction between the testing and

the stimulus.