What and how are we searching in the jungle of research?

No matter what, let’s get to the point through research!

- Abdi-Khalil Edriss

RESEARCH AND SAMPLE SURVEY

I- Research

1.1. What is Research?

Research is a process of assessing relationships among a set of variables by

using different methodologies relevant to a particular problem. Particularly
in sciences, it is a mechanism of collecting, classifying, analyzing,
discussing, summarizing and drawing valid conclusions about a population
based upon the information contained in a sample of measurements from
that particular population.

Alternatively, it is a systematic way of asking questions and obtaining

knowledge or information on those questions, thus improving our
fundamental understanding of whatever phenomena are involved, including
the discovery or elucidation of general principles and laws (referred to as
basic research) or enabling us find solutions to a practical problem (referred
as applied research).

There are several definitions given for research by different writers. Some
of these definitions are the following:
Research refers the application of the scientific method in the study of
problems. It is a systematic attempt to obtain answers to questions about
events through the application of scientific procedures.

~1~
 Research is a scientific and systematic search for pertinent information on
a specific topic.
Research is a systematic inquiry aimed at providing information to solve
problems.

Research is a systematic process of collecting and analyzing information

for some purpose.

One can easily pick the terms scientific and systematic from the above
definitions. Research is scientific because it is just a search for knowledge
through recognized methods of data collection, analysis, and interpretation.
It is systematic because it uses some steps or procedures in doing this.

Generally, the purpose of research is to discover answers to questions

through the application of scientific procedures.

1.2. Briefly describe Types of Research

Research can be classified in different ways: basic and applied; qualitative and
quantitative; conceptual and empirical; etc. however, the following three main
types of research are discussed briefly as follows:

Basic and Applied Research

i. Basic Research (also called pure or fundamental research): It has a

more general orientation, adding to the existing body of knowledge in
the discipline. It does not necessarily provide results of immediate,
practical use, although such possibility is not ruled out. It may consist
of a research to develop and/or improve on theories, techniques and
measurements. It is aimed to solve perplexing questions of theoretical
nature that have little direct impact on action, performance, or policy
decisions.

ii. Applied Research (also called action research): The purpose of applied
research is to solve an immediate, practical problem. It is oriented to a
specific problem. It has a practical problem solving emphasis. It
emerges out of a general problem which faces a society as whole.

Generally, basic and applied researches are differentiated not by their complexity
or value, but by their goals or purposes. The primary purpose of basic research is
the extension of knowledge; the purpose of applied research is the solution of an
immediate practical problem.

~2~
 Quantitative and Qualitative Research

i. Quantitative Research: Research based on the measurement of quantity

or amount.

ii. Qualitative Research: Research concerned with qualitative

phenomenon.

Conceptual Vs Empirical Research

i. Conceptual Research: related to abstract ideas or theories. Commonly

used by philosophers and thinkers.

ii. Empirical Research: relies on experience or observation alone. It is data

based research coming up with conclusions which are capable of being
verified by observation or experiment.

Note that recognition of the differences among the kinds of research is important
because the kind of research selected will determine the conduct of research. The
kind of research undertaken will also influence mobilization of financial and public
support for that research.

1.3. Describe briefly how applied research will help us answer

research questions or evaluate research hypothesis.

Applied research can be defined as research directed at solving an assigned

task for the purpose of contributing to the solution of a specific problem. It
is directed at solving specific, precisely formulated problems and is
expected to make a practical contribution towards changing policy or
shaping the course of events.

Applied research will therefore help to answer a research question or to

evaluate research hypothesis by giving the research a practical direction for
reaching the objective of the study. It will give the researcher a focused
approach in carrying out the research since it is mainly concerned with
gathering knowledge that may eventually improve the present situation.
Thus the conceptualization process of research is critical as data would be
collected, cleansed and analyzed. This rigorous process or work is intended
to make the research effort scientific with minimized error, and it is very
important in distinguishing valuable from non-valuable research works.

Applied research will also assist the researcher to apply techniques learnt
from different disciplines to solve the researcher’s problems. Different
skills, methodologies and designs may be employed in trying to evaluate
~3~
 the research hypothesis using knowledge acquired from different fields of
study (Booth, 1995). Therefore, applied research will be useful in
formulating the problem, that is, focusing on theory, choosing or
constructing a model to represent the system under study and deriving a
solution from the model, which are the basic steps in applied research.

Applied research is generally geared to answering questions of immediate

importance and it contributes to the theory testing process. It will help the
researcher with necessary background and setting to put the research
question in proper context. It will also provide a logical lead-in to a clear
concise statement of the problem as well as helping the researcher to have a
clear picture as to whether confirm or disconfirm the hypothesis or the
expected results.

Eventually it will assist us to approach our research questions in a manner

that incorporates reality. It will help us to come up with conclusions and
recommendations that can be implemented. This is so that there can be a
change or improvement in the current situation. In other words, applied
research will assist us to gather information that will not be used for the
sake of clarification and understanding but that will lead to tangible
changes. Not only does it help the researcher, but also all the concerned
stakeholders to decide on the best course of action to make in terms of
decision making. These decisions will be made based on the information
that has been gathered from the applied research. In addition applied
research can also help a researcher to evaluate whether or not certain
abstract principles that a researcher has come up with can be applied
realistically. It does this by revealing the possible limitations of those
abstract principles.

1.4. What are characteristics of a good research?

The purpose of the research, or the problem involved, should be clearly
defined and sharply delineated in terms as unambiguous as possible

The research procedures used should be described in sufficient detail to

permit another researcher to repeat the research

The procedural design of the research should be carefully planned to yield

results that are as objective as possible

The researcher should report, with complete frankness, flaws in procedural

design and estimate their effort up on the findings

~4~
 Analysis of the data should be sufficiently adequate to reveal its
significance, and the methods of analysis used should be appropriate

Conclusions should be confined to those justified by the data of the

research and limited to those for which the data provide an adequate basis

Greater confidence in the research is warranted if the researcher is

experienced, has good reputation in research, and is a person of integrity.

II. Monitoring and Evaluation

1.5. What are monitoring and evaluation?

The body of social science known as evaluative research is the systematic

collection of information on the design, implementation and effect of projects
on targeted populations (Rossi and Freeman, 1997). Ideally, the process is
divided into an ongoing monitoring system and periodic evaluations with some
special studies designed to answer specific questions about the project.
Monitoring and evaluation are distinct, though related efforts, with different
overall objectives, and, therefore, require differently designed systems.

Monitoring is concerned primarily with the ongoing collection and review

of information on project implementation, coverage and use. By collecting
information on a regular basis throughout the life of a project, a monitoring
system can be used to assess the quality of the project inputs and services,
the timeliness of service delivery, and the extent to which actual
implementation coincides with the project’s implementation plan.

Evaluation seeks to measure project effects, that is, whether and to what
extent the project’s inputs and services are improving the quality of
people’s lives. Evaluations provide information on the changes in the
behavior and conditions of targeted communities and individuals (Rossi and
Freeman, 1997) by assessing the effectiveness of the project in attaining its
originally stated intermediate and overall objectives. As with project
monitoring, however, evaluations may also reveal unexpected findings,
both positive and negative, which can be used to alter and improve project
design and implementation.

Generally, following an initial baseline survey, one or two midterm evaluations

take place in the mid to late stages of a project and an endpoint evaluation is
conducted upon project completion or at the end of a funding cycle. Ideally, there
should be an efficient ongoing monitoring system in place from the start of the
project.

~5~
Table 1.1: Assessment, monitoring and evaluation

ASSESSMENT
Definition Purpose

The appraisal of the situation prior to an Provide a basis for decisions

intervention. Involves not only determining on priority needs and
what the situation is, but also the background optimal programme
of why it is that way and what opportunities response;
and risks might affect efforts to change the Provide a baseline reference
situation. for future monitoring and
evaluation activities.
Assessment is generally considered a type of
evaluation, called an ex ante evaluation.

MONINOTRING

A proves of tracking or measuring what is Provide accountability for

happening. implementation according to
programme plans;
This includes two kinds of monitoring: Improve programme
Performance monitoring entails measuring implementation
progress in relation to an implementation Trigger rapid adaptation of
plan for an intervention (i.e., programmes, programme response
activities, strategies, policies) and specific particularly in crises or
objectives; unstable contexts
Situation monitoring entails measuring Feed into evaluation;
change in a condition or a set of Provide information for
conditions or lack thereof (e.g., changes in advocacy for changing
the situation of children and women in the policies or programmes
broader country context). (particularly situation
monitoring)

EVALUATION

A process that tries to determine as Improve programmes

systematically and objectively as possible the relevance, methods
worth or significance of an intervention outcomes;
policy. The appraisal of worth or significance Learning, particularly lessons
is guided by reference to defined criteria such that can be generalized to
as relevance, efficiency, effectiveness, impact other programmes or
and sustainability. situations;
Provide accountability for
An evaluation should provide information that programme results.
is credible and useful, enabling the
incorporation of lessons learned into the

~6~
 decision-making process of programme
partners and donors.

Note, the word “objectively” is used to

indicate the need to achieve a balanced
analysis, recognizing bias and reconciling
perspectives of different stakeholders (all
those interested in and affected by
programmes, including beneficiaries as
primary stakeholders) by using different
sources and methods.

RELATIONSHIP BETWEEN MONITORING AND EVALUATION

Monitoring and evaluation are complementary.

Without monitoring, evaluation cannot be done well.

Monitoring is necessary but not sufficient for evaluation. (Monitoring
facilitates evaluation, but evaluation uses additional sources of information)
An evaluation of a programme will often lead to changes in programmes plans.
This may mean changing or modifying data collection for monitoring purpose.
Source: UNICEF, Programme Policy and Procedures Manual, 2001

Table 1.2: Complementarily of monitoring and evaluation

Programme monitoring Programme evaluation

Frequency Periodic, regular Episodic

Main action Keeping track Appraisal
Basic purpose Improve progress in imp Improve relevance,
effectiveness, impact,
future programming
Horizon Short term Long term, beyond scope
of specific programme
Focus in Inputs, process, outputs Also outcomes and
programme cycle impact
References for Work plans, performance Programme objectives
comparison targets and reference and strategy, programme
indicators (this may performance targets as
include early warning well as more widely
indicators for problem accepted benchmarks and
areas) standards such as human
rights, organizational
policy

~7~
 Information Routine or sentinel Same, plus specific
sources systems, field surveys, studies
observation, progress
reports, rapid assessments
Undertaken by Programme mangers, Same, if approach is
community workers, participatory, plus
primary stakeholders, external evaluators
supervisors, funding
organizations
Reporting to Programme managers, Same, if approach is
community workers, participatory, plus
primary stakeholders, policy-makers, wider
supervisors, and funding range of external
organizations stakeholders
Source: UNICEF, Programme Policy and Procedures Manual, 2001

III. Classification of Research

1.6. What are the classifications of research?

Research is generally also classified in three types. Namely, Experimental

research (biology, chemistry, etc. experiments in labs), Quasi-experimental
(laboratory + field researches such as crop hybrid) and Observational research
(observing socio-economic problems through questionnaires and interviews).

STAGES OF RESEARCH

1.7. What are the stages of economic research?

The stages of economic research are: theorizing, dealing with data, theory and
statistical criterion and iterative stages in model selection.

Stage One – Theorizing

Proposed
Theory Mathematical
Models

~8~
POINTS TO PONDER
Tie up proposed theory with a mathematical model, which quantifies and assigns
numbers to interpret the theory. Without quantification of the theory it is usually
impossible to obtain results and interpret them in order to draw sound conclusions.

Stage Two – Dealing with Data

Cleaning of data
and
Pre-testing Collection of
Appropriate
data

Estimation of Parameters
of the model

POINTS TO PONDER
Note that a good or clean data set can tell the true story of the population. It is
therefore important to follow scientific data collection and analysis methods in order
to make good inferences about a population.

Stage Three – Theory and Statistical Criteria

Reject theory
Accept theory Evaluation of the model on if
if compatible the basis of economic and incompatible
with data statistical criteria with data

~9~
POINTS TO PONDER
Make sure you link-think between theory and practice in order to forecast or predict
the outcomes.

Stage Four – Iterative stages in model selection

Assume
functional
form

Find the best

estimate of
assumed model

Assess whether the model helps to

describe the dependent variable

Is the assumed
model
appropriate?

No Yes – stop
develop Continue
New model interpretation
and
discussions

~ 10 ~
SUMMARY OF BUILDING BLOCKS OF SCIENTIFIC RESEARCH

1 2
Observation Identification
of problem area

Refinement of theory 3
(pure research) Theoretical framework
Or
or Network of associations
implementation
(applied research)

4
Hypotheses

9
Interpretation of
data
5
Construct concepts
Operational definitions

8
Data 6
Analysis Research Design

7
Data
Collection

~ 11 ~
IV. Basic Building Blocks

1.8. What are the elements of a sampling problem?

It is important to understand the main elements and measurements of sampling

components. Some are –

i. Population is a set of items or measurements of interest.

ii. Sample is any subset of items selected from a population.
iii. Parameter is any quantifiable characteristic or element of a population.
iv. Sampling units are elements, or non-overlapping collections of elements
from the population that cover the entire population, for example,
provinces, regions, states, districts, villages, etc.
v. A frame is a comprehensive list of sampling units.
vi. A sample statistic (not statistics) is any quantifiable characteristic of a
sample (e.g., mean, proportion, coefficients, covariance, etc.) that may be
used to make inferences about the population parameter.

1.9. How do we create a list of sampling units or a frame?

In order to create a list of sampling units or a frame, it is essential to

o Use listings of larger population units such as states, provinces, towns,

districts, villages or census enumeration areas.
o Contact community leaders, religious and political organizations for
current lists.
o If the population under study is small, then carry out a quick census.
o If these are not possible, estimate the population or at least the number
of households in each of the communities, and employ the suggested
sampling techniques such as probability proportional to size and cluster
sampling. (Refer to chapter three for more.)

~ 12 ~
EXAMPLE - SAMPLING FRAME –

Study
Population

Region Region Region

1 2 3

District District District

1 2 3

Rural Dev Rural Dev

1 3

Rural Dev
2

Enumerated Enumerated
Area 1 Area 3

Enumerated
Area 2

Household Household
1 3

Household
2

~ 13 ~
1.10. What are some of the causes of biased sample?

Some of the pre-cautionary measures that we should take in order to obtain

unbiased and consistent parameter estimates are -

o Do not use samples chosen at will by the interviewer, field supervisor,

coordinator, or director.
o Do not choose samples exclusively from particular groups, such as
children coming to clinic, playground, in certain areas or people you
know.
o Do not restrict your sample to families or households living in easily
accessible households, such as those close to a main road or near a
village center. Note that households living in less accessible areas may
have more information. We should go and visit if the households are
behind a mountain or hills or across a river.
o Do not omit households where no one is at home the first time called.
Revisit later.

1.11. How do we avoid (or control) bias through data collection

methods?

There are various types of biases, or systematic errors, which can affect the
outcome of a research or an evaluation, and lead to erroneous conclusions. If
the scientific collection methods are followed properly, biased could be
reduced or avoided altogether. While it usually impossible to eliminate bias
all together, bias can be minimized when information is collected in exactly
the same way for participants and a control group, both before and after
project activity. To do so, data collectors must ensure that the administration
of the data collection instrument is well thought out and implemented. The
following table summarizes specific measures that can be taken to minimize
the likelihood of bias.

Table 1.3: Minimizing bias through data collection methods

Types of How to minimize bias through
bias Description of Bias data collection stages
Interviewer Baseline and post- Train interviewers thoroughly
bias project data or Standardize the interview
information about protocol
participants and Use objective and closed-
controls by ended questions
interviewers in If there is more than one data
different ways. collector (or team of data
~ 14 ~
 Observed collectors), each collector or
differences are team should gather
thus due to information from different
differential areas, both in the baseline and
measurement and in the subsequent evaluation
recording and not surveys, to minimize the
to project effects effects of individual’s
differences in skill or method.
If possible, the same data
collectors should conduct both
pre- and post-surveys.
For longitudinal surveys, the
same data collector (or team)
should collect information for
the same individuals
throughout the duration of the
evaluation. If this is not
possible, carefully train
replacements to continue data
collection in an identical
manner.
Instrument Measurement Standardize measurement
or errors are due to instruments and procedures
measurement instruments that Calibrate instruments (such as
bias are not identical weighing scales) frequently
for participants
and controls; thus
differences are due
to differing
measurement and
not to project
effects
Recall bias The participants Train interviewers thoroughly
and controls on how to probe for
remember and information and how to help
report information respondents remember past
in different ways events
due to their Use specific and meaningful
different exposures reference or recall period
either to the
intervention or to a
particular outcome
Time or Data on Standardize the time of day or
seasonal bias participants and season of data collection so
controls are information on treatment
~ 15 ~
 collected at groups and controls is
different times of collected during the same time
the day or different period
seasons. These
effects are due to
time or seasonal
effects rather than
the project
activities

1.12. Identify a control or reference group.

An evaluation that seeks to attribute change to project activities requires

the use of a control (or comparison) group. Individuals (or communities) in
a control group must be identical, or as similar as possible, to the group of
project participants. To establish such a control group, evaluators must
identify and control for confounding factors.

1.13. What is a confounding factor?

A confounding factor is any factor that is associated with both the result
being measured and with the delivery of inputs responsible for the result. If
not identically represented in participant and comparison groups,
confounding factors can lead to misinterpretation of project effects.

EXAMPLES
For examples some potential confounding factors in “nutrition projects” are –
income, land, livestock holdings, land type, family size, education, ethnicity,
religion, time availability, etc. to affect the real anthropometric measurements
(rather than the health issue itself!!)

1.14. How can we control or eliminate confounding factors in an

evaluation of a project?

In an evaluation, the method of selecting interviewees or participants and

controls can control or eliminate confounding. Randomization is the most
effective method to control for all confounders, known or unknown.

Another way is by matching (selecting participants and control groups so

that potential confounding factors are equally distributed in both groups).
Furthermore, control during statistical analysis (requiring data to be
collected on potential confounders) can be examined as alternatives.

~ 16 ~
V. Research Ethics
It is appropriate at this juncture to briefly discuss some ethical issues involved in
conducting reliable research.

1.15. What is ethics in research?

Ethics in research refers to a code of conduct or expected societal norm of

behavior while conducting research. Ethical conduct applies to the
organization and the members that sponsor the research, the researchers
who undertake the research, and the respondents who provide them with the
necessary information or data.

The observance of ethics begins with the person conducting the research;
who should do so in good faith, pay attention to what the results indicate,
and surrendering the ego, as well as, pursue research objectives rather than
self-interest.

Ethical conduct should also be reflected in the behavior of the researchers

who conduct the investigation, the participants who provide the data, the
analysts who provide the results, and entire research team that presents the
interpretation of the results and suggests alternative solution(s). Thus,
ethical behavior pervades each step of the research process --- in data
collection, data analysis, reporting, and dissemination of information on the
Internet. How the subjects are treated and how confidential information is
safeguarded, are all guided by research ethics.

1.16. What is ethics in data collection?

Ethics in data collection pertains to those who sponsor the research, those
who collect the data, and those who provide the information or answers to
research questions.

i. SPONSOR: The sponsor should ask for the study to be done to

achieve the objectives of its project, and not for any other self-
serving reasons. The sponsor should respect the confidentiality of
the data obtained by the researcher, and not ask for the individual or
group responses to be disclosed to them, or ask to see the
questionnaires. However, the sponsor is entitled to ask or see the
questionnaires for monitoring the study purposes and to ensure no
fraudulent data are accepted.

~ 17 ~
 Once the report is submitted, they should be open-minded in
accepting the results and the recommendations presented by the
researchers.

ii. RESEARCHER: Some critical ethics that the researchers must

follow include –

a. Treating the information given by the respondent as strictly

confidential and guarding his or her privacy is one of the
primary responsibilities of the researcher. If the confidentiality
part of the survey has been communicated to the organizational
executives before the survey starts, then, should any director or
another top executive desire to take a look at the completed
questionnaires, the prior understanding can be brought to their
attention. Also, report on data for a subgroup, say, fewer than 10
individuals, should be dealt with tactfully to preserve
confidentiality. The data can be combined with others, or treated
in another unidentifiable manner. It is difficult to sanitize reports
to protect sources and still preserve the richness of details of the
study. An acceptable alternative has to be found, since
preserving confidentiality is the fundamental goal.

b. Personal or seemingly intrusive information should not be

solicited, and if it is absolutely necessary for the project, it
should be sought with high sensitivity to the respondent with
specific reasons given to them for seeking it.

c. Whatever the nature of the data collection method, self-esteem

and self-respect of the subjects should never be violated.

d. No one should be forced to respond to the survey, and if

someone does not want to participate, the individual’s desire
should be respected. Informed consent of the subjects should be
the goal of the researcher. This holds true even when data are
collected through mechanical means, like recording interviews,
videotaping and the like.

e. Non-participant observers should be as non-intrusive as

possible. In qualitative studies, personal values could easily bias
the data. It is necessary for the researcher to make explicit his or
her assumptions, expectations, and biases, so that informed
decisions regarding the quality of the data can be made by the
supervisor.

~ 18 ~
 f. Subjects should never be exposed to situations where they could
be subject to physical or mental harm. The researcher should
take personal responsibility for their safety.

g. There should be absolutely no misrepresentation or distortion in

reporting the data collected during the study.

iii. RESPONDENTS: Ethical behavior of respondents (or

interviewees) concerns –

The subject, having chosen the choice to participate in a

study, should cooperate fully in the tasks ahead, like
responding to a survey or taking part in an experiment.
The respondent also has an obligation to be truthful and
honest in the responses. Misrepresentation or giving
information, knowing it to be untrue, should be avoided.

Here is an example that gives insight into why ethics in research is important.

By Dr. Sarah Ann White

College of Medicine, University of Malawi

Suppose a young girl was brought to a pediatric department with severe malaria and was
only treated with an anti-pyretic. She would be at increased risk of death. The doctor
responsible would be guilty of a medical offence.
In medical research the use of an inappropriate study design, or the use of statistical
methods that are not suitable may be a greater offence, since it may result in inferior
medical policy being adopted and implemented, thereby exposing patients’ to increased
risks of mortality or morbidity.

A competent scientific researcher knows that the appropriate use and interpretation of
Statistics are essential for research to be well-founded; just as it is important for a doctor
to correctly diagnose an illness and deliver the relevant medical care. S/he may not be
well versed in many statistical theories and techniques, but s/he knows the value and
importance of obtaining statistical guidance when designing and conducting his/her study.
In order for a policy–maker in the Health sector to be able to read and appropriately
interpret research publications s/he needs to be able to identify potential weaknesses or
limitations in study design and data analysis. Likewise those involved in ethical review
and funding decisions for research proposals need to be able to identify the existence of
potential weaknesses in a study design.

Being mathematically oriented, Statistics are at times avoided, misunderstood, under-

valued or abused. Factors contributing to these practices range from deficiencies in
training received, through ignorance and neglect, to deliberate distortion, suppression or
falsification of data. Like corruption such practices are not restricted to any geographic
location, profession or type of institution.

~ 19 ~
========================================================
MENTAL GYMNASTICS
CHAPTER ONE
=======================================================

1. Distinguish between the following of pairs of terms.

a) Research, Search and Review
b) Monitoring and Evaluation
c) Population and Sample
d) Parameter and Sample frame
e) Confounding factor and Control group

2. True, False or Uncertain. Support your answer.

a) A sample statistic is any subset of items selected from a population.
b) Ethics in research can be compromised based on the sponsoring organization.
c) Randomization can eliminate confounding factors.
d) Sampling units are comprehensive list of elements.
e) Monitoring without evaluation is useless.

3. Why research? Elaborate.

4. Explain three main types of research.
5. How do we create a list of sampling units or a frame?
6. What is ethics in research? Explain its importance.
7. How do you control bias through data collection methods? Give examples.
8. Give a good example of sample frame.
9. Discuss building blocks of scientific research, and their importance.
10. Explain how theory and statistical criterion are related in scientific research.

~ 20 ~