Instructional Materials for

LANGUAGE EDUCATION
RESEARCH
ENRE 30013

Compiled by

EDELYN H. DAGÑALAN
Faculty Member
Department of English, Foreign Languages, and Linguistics
College of Arts and Letters

EMEJIDIO C. GEPILA JR.

Faculty Member
Department of Elementary and Secondary Education
College of Education

Polytechnic University of the Philippines

 Chapter1
Strengthening of Research Orientation

A nation's overall capacity depends considerably on its

research. Universities, as centers of knowledge production
and generation, play a critical role in the national research.
This paper provides a discussion through the literature about
universities' roles in building national research capacity,
government quality assurance initiatives to enhance
university research, institutional measures to build research
capacity, and the impact of these expectations on
academics. Many universities have changed their brief to
include research and the production of knowledge as key to
their institutional directions. As a result, university research has
become highly competitive in a nation's capacity to deliver
knowledge in the world market. Given the prominent role it
plays in the overall national research efforts, university
research is an indicator of performance and educational
growth. In the last two decades, the desire to hold higher
education accountable and the need to receive value for
money have sparked government efforts to evaluate the
research performance of their universities and academics in
many countries. The results of these evaluations form the basis
of the resource allocation decision of government funding
bodies. Changes like these prevail and pose great challenges
to higher education institutions and academics, particularly
the role research plays in assessing their performances.
Government funding, and international ranking and status
drive higher education institutions to strive for research
excellence. Institutional pursuit of a strong research capacity
has led to managerial efforts to encourage and support
research. Incentive systems have also been established to
2
reward research. Research capacity enhancement measures
have had considerable impact on academics. Academics
are expected to conduct research and produce research
publications. More research is required to understand the
changing roles of academics and their work habits that may
lead to building research capacity.

Universities are under pressure to increase external

research funding, and some agencies offer programs to
expand research capacity in certain kinds of institutions.
However, conflicts within faculty roles and other aspects of
university operations influence the effectiveness of particular
strategies for increasing research activity (Huenneke et al.,
2017).

CONCEPTS OF RESEARCH

During the last two decades and more, the socio-

business environment of the world has witnessed dramatic
changes in its nature and scope. Emerging from a historically
economic role, the business organization has evolved in
response to the social and political stigma of national public
policy, explosive growth in technology, price struggle,
competition of new entrants, frequent change in customers
taste and desires etc. These factors and more have created
new knowledge needed for the manager and new problems
for social/managerial decision making.

In such a complex and uncertain environment, business

managers (or social scientists) need to understand how to
identify information and to find out the emerging problem to
make prompt and effective decision making. The studies of
research methods provide the knowledge and skills that the
managers/social scientists need to solve the problems and

3
meet the challenges of a fast-paced decision-making
environment.

In recent years, exploring the research output

particularly in the areas of humanities (management studies)
and social sciences (economics, commerce, history,
sociology, etc.) have been treated as an increasing concern
among the researchers. Management is said to be in
practice with the dawn of civilization. As a discipline, it is
hardly a century old. Starting off as a scientific approach to
the organization of things, it borrowed heavily from disciplines
like economics, sociology, psychology, mathematics and
statistics. Now it is a distinct body of knowledge having the
potential for theory and practice. Though management in
general is said to encompass the aspects like planning,
organizing, staffing, directing and controlling, the discipline
has spread itself to the specialized areas recently. Thus there
are management concepts as applied to finance,
production, human resources, information technology, rural
management, operations and marketing. Each of these
areas have grown to be a major discipline for research and
teaching. In fact, there are number of institutions situated
across our countries which are dealing with such issues but still
lots of work have to be done.

The pre-occupation with methodology is largely

peculiar to any discipline of research, particularly in humanity
and social science research in India. The first staging post in
the life of a young academician is to produce some
evidence of creativity. Not only can the route to this be
taught, but also attempts to do so take away from the spirit of
independent research. On the other hand, the young
researchers need to be fitted out with the basic tools of
analysis. Before analyzing the definitions of research, it is
worthwhile to discuss some basic concepts related to
decision-making.

4
  What is a research?

Research is a diligent enquiry and careful investigation

for new knowledge through systematic, scientific and
analytical approach in any branch of knowledge. Constant
search and research are the guiding factors of research
which helps to discover new facts (Kumar, 2002). The search
for new knowledge also helps to accept or reject or modify
existing fact or knowledge that is already available in the
existing literature.

For a common man, the word research implies a

detailed study of a subject especially in order to discover
(new) information or to some it implies in search of a new
finding. Whereas, the Oxford Advanced Learners Dictionary
defines the word as, ‘a careful investigation or inquiry
especially through search for new facts in any branch of
knowledge’. The Random House Dictionary of the English
language defines ‘research’ as diligent and systematic inquiry
or investigation into a subject in order to discover or revise
facts, theories, applications etc.

The definition of research gets its real meaning with the

words of Clifford Woody, who defines it as ‘research is a
careful inquiry or examination in seeking facts or principles, a
diligent investigation to ascertain something’.

John W. Best in his contribution titled ‘Research in

Education’ opines research as ‘the systematic and objective
analysis and recording of controlled observations that may
lead to development of generalizations, principles, or
theories, resulting in prediction and possibility ultimate control
of events’.

P.M. Cook defines research as ‘it is an honest,

exhaustive, intelligent searching for facts and their meaning

5
or implications with reference to given problem. It is the
process of arriving at dependable solutions to problems
through the planned and systematic collection analysis and
interpretation of data’. The best research is that which is
reliable verifiable and exhaustive so that it provides
information in which we have confidence.

M. H. Gopal defines as ‘it is essentially a systematic

inquiry seeking facts through objective verifiable methods in
order to discover the relationship among them and to
discover from them broad principles or laws’.

K. V. Rao visualizes the concept as ‘research is an

intensive and purposeful search for knowledge and
understanding of social and physical phenomena. It is a
method for the discovery of true values in a scientific way. It is
not merely an accumulation of knowledge but a critical and
scientific analysis of social facts and formulation of
generalizations as a basis of action and foresight’.

The encyclopedia of Social Sciences (Mc-Million)

defines research as, ‘Research refers to a critical and
exhaustive investigation of experimentation having as its aim
the revision of accepted conclusions in the light of new
discovered facts’.

Whereas, the word ‘methodology’ is the combination

of two words ‘method’ which implies a particular way of
doing something plus ‘logus’ the Latin word which implies
‘study’ thus, ‘methodology’ implies ‘a systematic way of
studying something’.

The research method may be defined as all those

methods and techniques that are used for conducting the
research or search. Hence, it refers to the behavior and
instrument used in selecting and conducting research

6
techniques and research design. Some experts, who are
continuously associated with research methodology since a
long, have categorized the research methods in three
categories. They are:

First category is one where those methods are included

which relates to the collection of data or information. In the
second category, different statistical techniques are used for
establishing relationships between variables. Whereas, the
third category consists of those methods which are used to
evaluate the accuracy of the results obtained.

Whereas, Dhondyal (1994) defines research

methodology as ‘a procedure designed to the extent to
which it is planned and evaluated before conducting the
inquiry and the extent to which the method for making
decisions is evaluated’. According to C.R. Kothary (1990),
research methodology is, ‘when we talk of research
methodology we not only talk of the research methods but
also considers the logic behind the methods we use in the
context of our research study and explains why we are using
a particular method or technique and why we are not using
others so that research results are capable of being
evaluated either by the researcher himself or by others’.

There exist large numbers of definitions in research

literature defining the term research methodology. All the
definitions are defined differently by different experts based
on their need and requirements of their study. Few properties
can be summarized as below:

(1) Research is a scientific process.

(2) It leads towards the solution of an existing problem.

7
 (3) It directs towards the development of new principles
of theories or modifies the existing literatures that will be
helpful in predicting future occurrences.

(4) It is based upon observable experiences or

empirical evidence.

(5) The process of search for new knowledge demands

accurate observations and descriptions of the fact(s).

(6) It involves gathering new data for new purposes.

(7) It is characterized by patience and unhurried

activities.

(8) It is carefully recorded and reported.

(9) It is the task of experts or specialists in the related

discipline of the study.

(10) It is an intellectual task.

From the above derived definitions and more other

such definitions that already exist in the literature, research
methodology may be summarized as a way to systematically
solve the research problem. Research methodology
constitutes research methods used in context of research
study and explanation of using of a particular method or
techniques and the why other available techniques are not
used. So that research results are capable of being
evaluated either by the researcher himself or by other persons
who have keen interest in the study area. Hence, research is
a systematic approach consisting of enunciating the
problem, formulating hypothesis, collecting the facts in
relation to the problem, and reaching certain conclusions,
either in the form of solution towards the concerned problem
or in certain generalizations for some theoretical formulation.
8
  Objective of Research

Like that of every discussion, research is also having

some basic objectives. The purpose of research is to discover
answers to questions through the applications of scientific
procedures. Young (1988) has analyzed some important
objectives of research as follows:

1. No two human beings behave equally. Human

psychology is the most complicated matter to predict
in the society. There are always fluctuations found in
their behaviour. Hence, in order to maintain equilibrium
and stability, it requires understanding the human
behaviour. Thus, research helps to explore and
understand human behaviour and their social life.
2. Research helps in portraying accuracy in the
characteristic of a particular individual, situation or a
group in the society / organization and leads to
designing the strategies of development.
3. Test of human beings for a particular good is always
changing. Thus, human nature is to search for new
things. Research helps to extend, correct or verify
knowledge that is either in hidden or not discovered,
4. Two kinds of explanations for unexplained social
phenomena,
5. It helps in predicting the frequency with which a certain
thing occurs or with which it is associated with
something else.
6. To provide new insights into organized society and its
social structures.
7. To test or challenge existing theories and revise them in
the light of new evidence(s). Here existing theories are
either tested or new theories are developed or are
modified.
8. To understand the organizational culture, social life,
social environment, decision making process, etc., and

9
 there by to gain a greater measure of control on other
human behaviour in the organizational and socially
context.

 Features of Research:

Every research has its specific features or

characteristics. Some important features are cauterized
below:

1. Research gathers or collects new knowledge or data

from either primary or firsthand sources and secondary
or already existing sources.
2. The process of research is systematic and accurate.
3. Research secures experts or skilled personnel for any
investigation.
4. Research is characterized as logical and objective,
applying every possible test to verify the data collected
and the process employed.
5. There is no question of biasness in research. Research
eliminates personal feelings and references from the
activity. It is the process of investigating new
knowledge.
6. Research endeavors to organize data in quantities
terms as far as possible.
7. Research in some cases where the research scope is
broader is time consuming.
8. Research redlines courage and hard work, hence, it
requires patience.
9. The process of research is highly purposive and pin
pointed. It generally deals with a significant problem
which demands a solution.
10. The process of research usually involves, a pre-
determined step, formulation of one or more
hypothesis, research design, methods of data or

10
 information collection and presentation of data and is
carefully recoded order reported.

 Factors Require for a Scientific/Good Research:

A good research follows the standards of the scientific

methods. Hence a research team while executing any
research work should consider following factors carefully.

1. Purpose clearly defined:

The problem or the decision to be made should be

clearly defined by the researcher. The decision statement
should include its scope, limitations and the summery
(meaning) of all words /terms that are significant to the
research. The researcher should be clear about the purpose
of his/her study (research) to match with the mission of his/her
organization for whom the research is meant for.

2. Should be based on ethics:

Ethics is defined as a system of moral principles or rules

of behaviour. While going for study, proper care should be
taken at designing the research procedure i.e., the issues
related to study. Safeguards against causing mental or
physical harm to participants should be considered. It should
not hamper religious and sensitive norms while conducting
research and interacting with the respondents. For example,
if the researcher has keen interest to know the behaviour of a
particular group of people who are affected by HIV+. Now
while interacting with these people, the researcher should ask
questions very carefully in a simpler manner without causing
any harm to their sentiments.

3. Research design/methodologies should clear and planned:

11
 The research should also have clear understanding
about the objective of the study. In other words, the nature of
data required, the behaviour of the respondents, the
procedure of data collection, sample coverage, methods of
data collection, models to be used, computer software to be
implemented etc., should be explained clearly.

4. The Researcher should be a skilled expert:

The person involved in the research should be an

expert in the concerned field of study. He/She should be
clear about the procedure, objectives, methodologies and
techniques of research. They should be capable enough to
handle the respondents. The behaviour of the researcher
should be calm and saver. Necessary care should be taken in
case where the study is based on some sensitive issues. The
researcher should have a clear command on the sample
area, the language, the mature of the respondents. The most
important requirement is that they should have enough
patience and keen interest in the study.

5. Unbiased analysis of results:

There should be no biased decision after getting the

results of study. No question of sympathy, kindness,
relationships, etc., should be involved in the research process.
No political interferences should be there in the process of
analysis of the study.

For example Mr. X is doing some research work (let it be

a Ph. D) under the guidance of Mr.Y and they are relative.
Now, if they do the research work in normal process it may
take several years and it may have some value addition in
the shape of outcome of the study. But, since Mr. X is Mr. Y’s
relative, the basic objective of such research may be only
getting a degree. Thus they will try to finish the work very early

12
without doing a proper work. In such cases Mr. X may be
awarded the degree but the result may not have any value
addition to the existing literature.

6. The conclusions should be justified:

The conclusion should be specific and to the point as

per the purpose of study. The objective(s) derived and
hypotheses formulated at the beginning of the study and
conclusion drawn at the end of study shows the gap
between “What was to be done and what you performed”.
This helps in gap analysis. The exact findings should be
represented without biasness. The validity of the conclusions
should be specified.

13
7. Scope for further research:

The research procedures used should be described in

sufficient details and there should be ample scope for the
other researcher for further research, if necessary. The
limitations of the study should be clearly mentioned. Problem
identification is the top most tasks in the process of decision-
making. As it is a known fact that, solution of one problem,
gives birth to a number of related other problems, thus, the
researcher should clearly mention about the limitations of the
study. The limitation may be in the form of limited or small
sample, less budget, less time span, parameters that have
been identified are less in quantity, the study is narrow in
scope, methodology chosen may be purposive, etc.

 Types of Research:

Researches are of various types. It is basically

categorized on the nature of work. Professor R.
Panneerselvam has classified some important types of
research as follows:

1. Exploratory Research:

Exploratory research is the fundamental one and is of

an initial research type which the researcher carried out at
the initial stage of study. In other words, here the researcher
analyses the data and explores the possibility of obtaining as
many relationships as possible between different variables
without knowing their end-applications. A general study is
conducted without having any specific objective(s) of study.
Here a number of hypotheses are developed for an existing
problem.

There are three sources from which exploratory

research develops in the mind of a researcher. They are

14
survey of literature, experience survey and study of existing
case(s).

a. Survey of Literature:

The very beginning phase of searching a problem is the

survey of literature. Survey of literature is the study, involves
collection of available literatures in the required area of
research where the researcher has limited experience, and
critical examination and comparison of the above materials is
needed for better understanding of the concept. This
research also helps the researchers to update with the past
data, data sources and results, which are useful for related
researcher in future. Some examples of survey of literature are
searching a problem from available journals, books,
periodicals, reports published by different government and
non-government agencies etc.

b. Experience Survey:

Experience survey gathers the experiences of skilled

specialists in terms of their skill, expertise and knowledge.
Hence this survey is the survey of experiences of
experts/specialists in a particular field which acts as a data
base for future research. These expertises have been
developed over a period of time or through some other
related studies. Some of the examples of the experience
survey are materials planning, demand forecasting, man
power planning, union / state government budget analysis,
investment decisions analyzing the employment and
unemployment situation etc.

c. Study of case(s):
This is a special kind of study oriented to have insight
into research topics. This kind of study is mainly case study
oriented. It lays emphasis on reviewing the available real story
cases in the literature. Based on the available real case
15
histories, a researcher can proceed in his/her own research
study.

MAJOR STEPS IN RESEARCH

 Problems of Decision-Making:

For a common man, decision-making problem requires

a choice among alternative course of action so as to
achieve the objective(s). The word ‘problem’, in a general
meaning, reflects that something has gone wrong. There are
numbers of alternative course of action among which choice
has to made are often called in management discipline as
‘management strategy’ and in economics as ‘economic
decision-making’. Presently, the business applications of
economics and commerce are attracting more and more
intellectuals to its fold. No wonder, the queen of social
sciences has emerged as the heart of business sciences.
Economics and commerce applies its traditional tools and
concepts for the better and efficient management of
business. Hence, the tools of applied economics and
commerce are basically concerned with the application of
economics in management decision-making. There is no
doubt that it enables the new age managers- the managers
of 21st century business enterprises to perform their role of
convincing and executing complex strategies.

While managers have considerable information about

the outcome for many decisions, they must frequently make
decisions in situations in which the outcome of a decision
cannot be known in advance. A manager may decide, for
example, to invest in a new product development or to
increase the level of technology or to change the price of a
product and/or any other similar alternatives to increase
profits and sales. Even after studying hundreds of technical

16
reports, numbers of annual reports of different companies,
available case studies in the literature etc., a manager may
still not be in a position with confidence to choose the
appropriate alternative(s). Managerial decision making leads
to problems like: which products are to be produced?; what
price is to be charged?; what quantity of the products are to
be produced?; what should be the promotional
expenditures?, how much would be the investment
expenditures?, etc. Thus, decision making is a process of
selection from a set of alternative courses of action which is
thought to fulfill the objectives of the decision problem more
satisfactorily than others.

Similarly, a social science researcher at the beginning

of a research projects not only in a position to decide which
subject area to choose (whether to choose consumer
behaviour by analyzing household consumption pattern and
household budget or to analyze the effectiveness of various
government policies that are meant for removing poverty,
unemployment, etc., and/or number of such related
alternatives like dowry deaths, women empowerment,
women crime, sex ratio etc.), but also what should be the
subject area, what should be the methodology, what is the
amount of budget available for research, how much time is
available with the researcher and so on.

It is a widely accepted fact that business decision-

making process has become increasingly complicated due
to the over growing complexities in the business world more
particularly soon after the adoption of the concept of
globalization by major economics of the globe. Thus, it
requires that the decision maker (s) has to follow some basic
rules and processes which will help them to make efficient
decisions under conditions of risk, uncertainty and
complexities. Hence, decision-making problem may be
defined as ‘a task of identifying the alternative courses of

17
action among a number of alternatives available of
achieving given objective(s), then to analyze all the
alternatives by collecting required information and at the end
choosing the best alternative to achieve the desired
objective in most efficient way’.

The elements of decision making process may be:

 The decision maker himself/herself

 Identification of the decision problem
 Identification of the source or the environment in which
the problem exists and need solution
 Selecting some alternative courses of action among
the feasible alternatives and
 The final selection of the choice of the alternatives

 Steps of Decision-Making Process:

The researcher (pronounced for decision-maker who
may be an executive or research team) while going for
decision-making always gets confused, thinking from where
to start in order to identify the alternative. Systematic inquiry
of the problem is a common and fundamental threat. The
process of systematic inquiry requires careful planning of an
orderly investigation. But there is no hard and fast rule that the
decision maker should follow all the steps in a sequence that
is derived below rather he/she can choose few important
ones based on their convenience and requirements of the
study. Following are some fundamental steps which may help
the decision maker to get the path of searching the
alternatives.

1. Analyzing the objectives:

At the initial stage of going for identification of a
problem the decision maker has to understand completely

18
the mission, vision and the objective of the concerned
organization i.e., he/she has to first understand what their
organization wants from them (in case of business problem).
For instance, if the decision-maker is associated with a private
enterprise then he/she has to understand that the motto of
their organization is to maximize profit or sale and/or both for
achieving rapid growth. On the other hand, a public
enterprise not only based on profit maximization but also is
based on welfare criteria. Where as in case of social science
research the research has to analyse what is the basic
purpose of research? Who is the funding agency and what it
wants? What the associated persons or institutions expect?,
and a number of such related problems.

2. Defining the problem:

The second step, after the objective(s) been identified,
is of defining or identifying the problem. Problem
identification is the top most tasks in the process of decision-
making. It justifies the adage that ‘a problem well defined is a
problem half solved’. This adage emphasizes that an orderly
definition of the problem (which ultimately leads to decision-
making) gives a sense of direction to the existing problem.

3. Identifying possible alternative solutions:

The third step in the process of identifying the
alternative course of action is of specifying few alternative(s)
among the numbers of alternatives selected at the initial
stage of research (as discussed in point 1 above). But due to
a number of reasons the decision maker cannot choose all
the alternatives that is possible to solve the problem rather
he/she has to limit the horizon and should concentrate on
some limited alternatives. This requires considering the
variables that may put more impact on the problem
identified. In such a way, relationship among the variables
and with the problems has to be established. Here, various

19
hypotheses can be formulated which will at the end serve as
alternative solution of the existing problem.

The ripple effects of the 2008 Global economic

meltdown had begun to hurt the Rupees 2,556 crore of
Narottam Paper Ltd. Like all other business houses in India,
Narottam Paper Ltd., was also finding the growing tough. The
general trend of soaring prices and contraction in demand
had started affecting the sale of the company products. Its
customers were also focusing on correcting their inventory
positions (using existing stocks of materials to keep production
lines and marketing activities rolling). Consequently, they
were not buying much from the company. Even the investors
did not like when they saw- Narottam Paper Ltd., stock fell
from Rs. 63.50/- on 1 January, 2008 to a low of Rs. 12.08/- on
18 January 2009. The company was in the midst of Economic
crisis.

In such a chronic situation of the company, Mr.

ParthaSarathi, Managing Director of the company, realized
that some strong means must be taken to extricate the
company from its present crisis. To this end, Mr. ParthaSarathi
held several brainstorming sessions with the company’s
Research and Development team and finally, identified the
focus areas on which decision is needed. The identified areas
are:

1. Necessity of changing company’s strategic planning

policies
2. Managing working capital flows
3. Cutting various costs
4. Paying attention to employee productivity
5. Searching new market potentials

Thus from the above case study it is clear that the

company in wrongly in the trap of economic recession. To

20
combat this recession, the company had identified five
alternatives. Solution of each of the five alternatives may help
the company to achieve its desire objective(s).

4. Evaluating alternative course of action:

Now, among these alternatives, some alternative
requires long time to execute, some may be expensive for the
decision-maker to execute, some may go beyond the limit of
the decision-maker and the one which satisfies all the criteria
of the decision maker. Thus, the decision-maker has to collect
relevant data or information on each possible alternative
those satisfy all the requirements of the decision-makers,
hence, are finalized to solve the existing problem. Data are
regarded as the basic input to any decision-making process
in both business studies and social science. The processing of
data gives statistics of importance of the study. Some data
may be available in form of reports published by various
government departments, annual reports of the industries,
thesis/research publications in various libraries of
institutes/universities, etc., are called as secondary source of
data. Whereas, in some case the decision-maker has to
collect the fresh information with the help of various tools of
data collection like questionnaire, door-to-door interview,
focal group discussion, field study, etc., which are called as
primary source of data. The collected data are then coded
and entered in computer for execution of the result. Proper
tools and techniques of optimization are used for
classification and analysis. Techniques such as regression
analysis, multivariate analysis and operation research
techniques like linear programming, simulation techniques,
etc., are used to obtain a desirable solution. This solution may
be one that helps the decision-maker to achieve the desired
objective. Now, with this process the decision maker gets
optimal solutions to all the possible alternatives that he/she
has been identified in the initial stage of research.

21
5. Validation of the results:
After the course of action is finalized, now it is time to
execute the action in reality i.e., called as decision-making.
The process of validation of results ensures the credibility of
the results. This requires constant monitoring so as to achieve
the desired result. For this, the decision maker has to consult
with the experienced professionals in the related area of
study, consultants who are closely associated with such
studies, academicians who are carrying on research in the
related area(s), etc.

PURPOSES OF RESEARCH

Language and Information Sciences

In Language and Information Sciences, our central aim

is the multifaceted study of linguistic activity, which can be
said to form the core of human intellectual activity. From the
approaches of the humanities and social sciences that treat
language as a human activity embedded in culture and
society, to that of the natural sciences and engineering that
investigates its relationship with the human brain or
computers, our scope spans multiple viewpoints and range of
languages. In this way, we seek to realise the mission of the
Graduate School of Arts and Sciences - that of
internationality and interdisciplinarity - in the context of
'language', with the goal to develop future international
leaders not only in academia but also in wider socie

Interdisciplinary Cultural Studies

In Interdisciplinary Cultural Studies, specialist staff across

Cultural Representation, Cultural Anthropology, and
Comparative Literature and Culture share their experiences
and knowledge of their respective fields, in order to propose
effective approaches for analysing the increasingly
globalising cultural phenomena that transcend states and

22
societies, and the cultural activities that intersect genres. A
vast range of subjects are studied, such as traditional rites
and folk art, global art and its pluralistic expression, the
interaction between media and technology with art, the
forms and structures of semiotics, and the movement and
exchange between cultures, Methods of analysis also vary
widely, including literary criticism, fieldwork, and descriptive
analysis of of online networks, and we are continually striving
to develop new theoretical frameworks. Through such
educational approaches, our goal is to form a cultural vessel
that is open to and responsive to the dynamically changing
paradigms of contemporary society, as well as to equip
future generations with the internationality and
interdisciplinarity to play leading roles in various fields in
education, academia and in wider society.

Area Studies

In Area Studies, our goal in research is to deepen our

inquiry into the various regions and specialist disciplines of our
staff, as well as develop an approach that intersects regions
and disciplines in pursuit of the universal that lies beyond the
particular. Our goal in education is to foster future
generations with not only a rich knowledge and
understanding of individual cultures but also the perceptive
ability to place their own research in a global context. In
doing so, we seek to find a path of multicultural coexistence
that transcends differences in language, religion, thought
and customs.

Advanced Social and International Studies

In Advanced Social and International Studies, which

consists of the two programs of International Relations and
Interdisciplinary Social Sciences, our goal is to unravel various
phenomena associated with globalisation from a social
sciences perspective. In the International Relations program,

23
we frame the study of international relations that integrates
international politics, international economics, international
relations and law, and the history of international relations. In
the Interdisciplinary Social Sciences program, we aim to
elucidate contemporary social phenomena with a
comprehensive and cross-disciplinary approach that studies
the foundational domains of social sciences - namely, law,
politics, economics, and society. Based on this philosophy, our
goal is to foster specialist researchers ready for the new age
in order to return the fruits of our research to wider society. At
the same time, we also aim to produce future international
leaders in spheres as diverse as international organizations,
public service, NGOs, and private think tanks.

Multi-Disciplinary Sciences: Life Sciences

In Life Sciences, we aim to investigate biological

phenomena at various levels, ranging from genes to
organisms, from the comprehensive and integrated view of
'life dynamics'. Transcending the traditional arts-sciences
barrier, the interdisciplinary nature of our research and
education lies at the forefront of current research on areas of
the life sciences, and includes the disciplines of cell biology,
biochemistry, biophysics, molecular biology, sports science,
neuroscience, psychology, and pedagogy. The focus of our
research ranges from the microscopic building blocks of life
such as DNA, proteins and cells, to the structures, growth, and
functions of tissues, organs and the entire human body,
including its psychology. As well as deepening insights into
their own specialist fields, our researchers also aim to further
their cross-disciplinary perspectives by mutual exchange and
inspiration, in order to establish a new form of the life
sciences. In terms of education, we also aim to equip
students with the thorough knowledge and methodological
tools needed for contributing to their individual fields, such
that they can understand the human body both as a
culmination of its molecules, cells and tissues, but also from
24
the macroscopic perspective of the whole field of the life
sciences.

Multi-Disciplinary Sciences: General Systems Studies

In General Systems Studies, we recognise the

importance of having an integrated approach in
understanding and tackling the complex problems of the
modern world - an approach that focuses not only on the
separate analysis of individual constituents, but also on their
interrelationships and how they make up the whole. Such a
perspective forms the crux of system theory, which we seek to
utilise in our education and research in order to lead an
analysis of complex phenomena at all levels. The range of
topics we consider is extensive, and includes natural systems
such as space, the earth, ecology, and biological systems;
man-made systems such as information or engineering
systems; and combined complex systems such as urban or
regional systems. In this program, we aim to develop
graduates with an interdisciplinary and comprehensive
perspective, who can think in terms of systems thinking, to
contribute to all fields across education, research, and wider
society.

Multi-Disciplinary Sciences: Basic Science

In Basic Sciences, those who specialize in the natural

sciences aim to study the hierarchy of material structures,
from elementary particles, atomic nuclei, atoms and
molecules to various higher-order structures, in a longitudinal
approach that herald 'from quarks to intelligent materials' as
its slogan. At the same time, those who specialize in the
history and philosophy of science aim to investigate the role
of science in contemporary society. The goal of the program
as a whole is to create a system for mutual stimulation and
inspiration. With such research purposes as the background,
our educational aim is the development of future leaders and

25
problem-solvers, who have an awareness of the role of
science in society and the coexistence of humans and life
with the environment, so as to contribute to the
advancement of our current frontiers in science and
technology.

CHARACTERISTICS OF A GOOD RESEARCH

Research is a procedure by which we attempt to find

systematically and with the support of demonstrable fact. This
will eventually lead to the answer to a question or the
resolution of a problem. Research has eight distinct
characteristics:

 Research originates with a question or problem in the mind

of the researcher;
 Research requires a clear goal;
 Research requires a specific plan or procedure;
 Research usually divides the principal problem into more
manageable subproblems;
 Research is guided by the specific research problem,
questions or hypothesis;
 Research accepts certain critical assumptions;
 Research requires the collection and interpretation of data
in an attempt to resolve the problem that initiated the
research; and
 Research is, by its nature, cyclical or more exactly helical.
Figure 1.2 clearly lists out the 8 research characteristics.

Exercises:

Analyze the Argument

26
1. Students locate, identify and examine the assumptions
implicit in an article. Identify the author's thesis. Outline the
theoretical framework used to account for the results.
2. Students examine the experimental design, data, and
interpretation of the data in a research paper for adequacy
and consistency.

SLOs: 3, 4 Level: Bachelor’s (Step 1), Master’s (Steps 1 & 2)

Compare Internet and Database Searches

Given an appropriately narrow research topic from the

instructor, students perform a search on the Internet and also
in an academic database. Students will demonstrate their
search strategies and search results, then compare the
findings and discuss the differences between these search
tools with respect to content and search strategy (Burkhardt et
al., 2010, p. 82, 84).

SLOs: 2, 4 Level: Bachelor’s, Master’s

27
28
 Chapter 2
Getting Ideas about Suitable Research Projects

RESEARCH TOPICS OF SPECIAL PROJECTS AND THESES FROM

DIFFERENT UNIVERSITIES AND COLLEGES

Research Paper Topics on Medicine

 Is life-sustaining therapy needed when it’s futile?

 The role of placebo treatment
 How to avoid animal testing?
 Pros and contras of medical marijuana
 Is cow milk healthy?
 Is being a vegetarian useful for child’s health?
 How obesity affects our health?
 Vaccines for kids: their usefulness or damage
 Should prescription drugs be advertised directly to
consumers?
 Do doctors turn their patients into drug addicts?

Research Paper Topics on Education

 How can standardized tests improve education?

 Does college graduates make more money?
 Should education be cheaper?
 How will modern technologies change the way of
teaching in the future?
 The creation of particular learning methods for blind
children
 Social networking and school
 Metal detectors at schools
 The effects of contemporary teaching methods
 The role of technology in lesson planning

29
 How to manage bullies and take actions against
bullying at education institutions?

Research Paper Topics on Environment

 How to reduce global warming?

 How to stop wasting paper and save trees?
 Can overpopulation be managed?
 Should more films about environmental issues be
produced?
 Human impacts on forests
 Underground effects of earthquakes
 How to elaborate optimal adaptation of buildings
threatened by hurricanes?
 Is it possible to predict hurricane impacts?
 Is nuclear power safe for humans?
 How dangerous is GMO food?

Research Paper Topics on Entertainment and Sport

 Are social networks good for our society?

 Do violent video games make kids angry and cruel?
 Is it necessary to forbid using animals for
entertainment?
 Do beauty contests set the non-achievable beauty
standards?
 Are newspapers going to be replaced by online
sources of information?
 How gaming consoles influence the youth?
 Should women be allowed to compete against men?
 What television programs should be banned?
 How tv shows impose fake moral standards?
 Can the use of social media, such as Facebook, lower
teens’ self-esteem?

Research Paper Topics on Politics

30
 Should the drinking age be lowered?
 Should adults have the right to carry a concealed
handgun?
 More gun control laws should be enacted
 How can the international community prevent Iran
from developing nuclear weapons?
 How can ethnic killings be stopped?
 Current prospect for peace between Israel and the
Palestinians
 What world would be like without wars?
 How to avoid workforce reduction?
 Should the death penalty be allowed?
 Is socialism possible?

Research Paper Topics on Psychology

 Is autism a disease or a development idiosyncrasy?

 How to forecast and shape behavioral patterns?
 How to manage child violence?
 How to deal with a mental breakdown?
 The impact of classical music on the work of brain
 How insomnia affects our health?
 How bad dreams influence our mood?
 Is stress really harmful?
 How depression impacts the immune system?
 Intellectually gifted people: how is it possible?

Research Paper Topics on Science and Technologies

 How is light being used to treat cancer and other

diseases?
 What is the current evidence that Mars has had water
and maybe life?
 Can nanomedicine potentially extend the human
lifespan?

31
 What is the future of computing and artificial
intelligence?
 The role cryogenics may play in future.
 Can alternative energy effectively replace fossil fuels?
 Is it beneficial for wild animals to have interactions with
people?
 What evidence do we have that CMB is the result of
the big bang?
 How will self-driving cars change the way people live?
 Can using system like bitcoin help protect identity
theft?

Research Paper Topics on Culture

 A new comprehension of past events

 The origin of racial discrimination
 The roots of antisemitism
 The impact of advertisements and commercials on
modern art
 The most remarkable cultural achievements of the 20th
century and their influence on our society
 Cultural revolutions throughout history
 How pop culture trends influence youth?
 Should pregnant celebrities be exposed on magazine’s
covers?
 Why was Greek cultural influence so important for the
ancient Mediterranean world?
 Why was the Victorian period a time of cultural
change?

Research Paper Topics on Math

 The influence of algorithms

 Is it possible to build a winning monopoly strategy?
 Why is 'x' the unknown?
 How math changed the world?

32
 What's solution to the McDonald's math problem?
 How do math geniuses understand extremely hard
math concepts so quickly?
 Should high school math contests be banned?
 What is the relationship between music and math?
 Are math formulas ever used in real life?
 What are some of the most confusing math problems
ever?

Research Paper Topics on Business

 How do dirty business tactics work?

 Can business be started without money?
 Notorious business leaders
 Entrepreneurship and family business
 Ethical decision making in everyday work situations
 What are the most effective strategies for promoting a
small business?
 Is it worth it to expand the business into a new region or
country?
 How to build a successful startup
 The role of international business and sustainable
development
 The impact of climate change on international business
strategies

Business

Cheap labor U.S. companies that move factories to

undeveloped nations barely pay employees enough to live
on. Is it unethical to pay cheap wages or are companies
doing those workers a favor?

Fitness programs Should companies allow employees to

exercise on work time?

33
Mining What safety measures made the recent Canadian
mining accident a noncasualty event, while recent U.S.
mining accidents have been fatal?

Oil companies Do oil companies make big bucks while the

rest of us pay over $3 a gallon?

Wage gap Women still earn only 75 cents for every $1 a man
earns. Explain why.

Wages There’s a minimum wage, but should there also be a

maximum wage/salary a person can earn?

Workaholics Do Americans work too hard? Does working

more actually reduce productivity? Is a 40-hour workweek
too much? Should there be a mandatory cap on the number
of hours a person can work? Should there be changes in
employment laws to give Americans more relaxation time?

Education

Athletes in high schools Should shoe companies be able to

give away free shoes and equipment to high school athletes?

Censorship Should parents censor textbooks and other

literature for children in schools?

College athletics Should college athletes be paid?

Electronic information preservation Does the Information Age

mean we are losing important historical information?

Exams Exams often do little more than measure a person’s

ability to take exams. Should exams be outlawed in favor of
another form of assessment?

34
Flag Should children be required to say the Pledge of
Allegiance in schools?

Gap year Should teens in the U.S. adopt the British custom of
taking a “gap year” between high school and college?

Grades In some European schools, fewer than 10% of students

get A’s. Is there grade inflation in the U.S.? Why so many A’s
for Americans?

No Child Left Behind Act Is it working?

School lunches Should government impose restrictions on

what kinds of foods can be served in school cafeterias?

School uniforms Pros and cons of school uniforms.

Single-sex schools Do children learn better in boys-only and

girls-only schools?

Environment

Climate change Is global warming a hoax? Is it being

exaggerated?

Coal Should the use of coal be subjected to stricter

environmental regulations than other fuels?

DDT Should DDT be reapproved for use in the wake of recent

bedbug outbreaks across the country?

Wind energy Is wind energy really that cheap? Is it effective?

Is it practical?

Government Policy

35
Airport security Should the government use invasive pat-
downs and body scans to ensure passenger safety or are
there better methods?

al-Qaida Has U.S. policy actually spread terrorism rather than

contain it? Will it get better or worse? Why and how?

Arming pilots Good idea?

Bridges, roads, waterways Why the government gets a bad

report card on America’s infrastructure.

Health care crisis Most developed nations have universal

health coverage. Why doesn’t the U.S., the wealthiest nation,
have it?

Homeland Security Are we safer since the creation of this

department?

Immigration Should illegal immigrants be made legal citizens?

Internet regulation Should the federal government be

allowed to regulate information on the Internet?

Iraq War Is America winning or losing the war? What is the

measurement of success? Have the benefits outweighed the
costs?

Iraq War vets Are they being cheated on medical benefits?

Lead poisoning Should the U.S. stop importing Chinese-

manufactured toys?

Marijuana legalization Should the federal government

legalize the use of marijuana?

36
Military Should the U.S. have mandatory military conscription?
For whom?

Moms Should stay-at-home moms get a salary from the

government?

Obesity and weight loss Should thin people have to pay

Medicare and other health costs for the health problems of
obese people? Should obese people have higher premiums?

Prescription medicines Should there be a national database

to track controlled substances (i.e., oxycodone) or should it
be a state issue?

Privacy Should the government be allowed to wiretap

without permission?

School lunches Should government impose restrictions on

what kinds of foods can be served in school cafeterias?

Sex offenders Once they leave prison, are laws about where
they may live and be employed unfair?

Smoking bans Should the federal government pass a

nationwide indoor smoking ban?

Health

Fast food Are we taking it too far by blaming fast-food

restaurants for obesity? When is it individual responsibility and
when is it appropriate to place blame?

Fitness programs Should companies allow employees to

exercise on work time?

Food Steroids. Antibiotics. Sprays. Are food manufacturers

killing us?

37
Health care crisis Most developed nations have universal
health coverage. Why doesn’t the U.S., the wealthiest nation,
have it?

Lead poisoning Should the U.S. stop importing Chinese-

manufactured toys?

Malpractice How can we balance the need to lower the cost

of malpractice insurance with the fact that physician
malpractice is one of the leading causes of death?

Models Should there be a minimum weight limit?

Noise pollution How much is too much noise? What, if

anything, should we do to curb it?

Obesity and weight loss Should thin people have to pay

Medicare and other health costs for the health problems of
obese people? Should obese people have higher premiums?

Prescription medicines Should there be a national database

to track controlled substances (i.e., oxycodone) or should it
be a state issue?

School lunches Should government impose restrictions on

what kinds of foods can be served in school cafeterias?

Smoking bans Should the federal government pass a

nationwide indoor smoking ban?

Social media Can excessive use of social media contribute to

addictive behaviors (drugs, tobacco, alcohol) or mental
health issues?

Steroids Should they be legalized?

Vaccines Should parents avoid vaccinating their children?

38
Human Behavior

Nonverbal communication How do men and women

communicate differently using body language, and why
does it matter (in dating, the workplace, social circles)?

Road rage Why do normally patient people become

impatient behind the wheel?

Social anxiety How is it different from shyness? And, are we a

society of anxiety?

Suicide bombers What kind of person becomes a suicide

bomber?

Terrorism Can terrorism ever be justified?

Legal Issues

Airplane accidents Who is responsible? Should families of

victims be entitled to compensation?

Bullying laws Should the state or federal government put laws

into place to prevent bullying?

Civil disobedience Is breaking the law for a cause justifiable?

Civil unions If civil couples have the same rights/privileges as

married people, why get married?

Holocaust Should denial of the Holocaust be illegal?

Infidelity In some states, it is illegal to cheat on a spouse.

Should we prosecute cheaters? Is a law that’s not enforced
really a law?

39
Juvenile offenders Should juvenile offenders be tried and
punished as adults?

Organ sale Should we legalize the sale of human organs?

Parents Should parents be held responsible when their

children break laws?

Sex offenders Once they leave prison, are laws about where
they may live and be employed unfair?

Statutory rape Recently, a 17-year-old boy was sentenced to

10 years in prison for having consensual oral sex with a 15-
year-old girl. Are statutory rape laws patronizing to girls and
discriminatory to boys?

Media and Freedom of Speech

Advertising Should certain kinds of ads be banned in the

interest of health/morality/annoyance – alcohol, cigarettes,
prescription meds, etc.?

Art A few years ago, an artist was criticized for depicting the
Virgin Mary with elephant dung. When is art not really art?

Censorship Should parents censor textbooks and other

literature for children in schools?

Church arson Hate crime?

Flag Should children be required to say the Pledge of

Allegiance in schools?

Journalism Should newspaper reporters be required to reveal

their sources?

40
Media Does the media, both print and broadcast, report
fairly? Does it ever cross the line between reporting the news
and creating the news?

Paparazzi What, if any, limitations should be applied to the

paparazzi?

Pornography Parental filters on the Internet. Does censorship

actually increase curiosity and use of pornography?

Public figures Does the public have a right to know about a

public figure’s private life?

Racial bias in media Does news coverage favor whites?

Religion

Church arson Hate crime?

Religion Is there one true religion?

Women in the clergy Should women be priests, pastors,

ministers and rabbis?

Social Issues

Abstinence programs Do they work?

Alcohol Should the U.S. lower the drinking age to 18?

Animal rights Should it be illegal to use animals for sports and

entertainment?

Beauty contests Do they serve any purpose in society?

41
Campus sexual assault Are sexual assault incidences on
college campuses on the rise or has awareness just
increased?

Child soldiers Why and how children are used for war

Civil unions If civil couples have the same rights/privileges as

married people, why get married?

Cyber crime What are the latest ways to steal identity and
money?

Divorce Should we reform laws to make it harder to get a

divorce?

Felons and voting Should convicted felons have the right to

vote?

Feminism Is feminism still relevant? How do young women

view feminism?

Gay marriage Should there be a constitutional amendment

that allows gays and lesbians to legally marry?

Generations How four different generations in the workplace

can work together.

Infidelity In some states, it is illegal to cheat on a spouse.

Should we prosecute cheaters? Is a law that’s not enforced
really a law?

Internet and children Are children smarter (or more socialized)

because of the Internet?

Intersexuals What, if anything, should be done medically and

socially?

42
Literacy Are Americans “literate”?

Models Should there be a minimum weight limit?

Muslim Americans What is life for them like in the U.S. after
September 11, 2001?

Organ sale Should we legalize the sale of human organs?

Privacy What medical information should be confidential?

Who, if anybody, should have access to medical records?

Sex offenders Once they leave prison, are laws about where
they may live and be employed unfair?

Spanking Should it be outlawed?

Sports parents What are the effects on children whose

parents push them in sports?

Statutory rape Recently, a 17-year-old boy was sentenced to

10 years in prison for having consensual oral sex with a 15-
year-old girl. Are statutory rape laws patronizing to girls and
discriminatory to boys?

Title IX Has it helped women’s sports? Has it harmed men’s

sports?

Tough love Does parental “tough love” really work?

Toys Do certain children’s toys create social or emotional

problems?

Wages There’s a minimum wage, but should there also be a

maximum wage/salary a person can earn?

43
Wage gap Women still earn only 75 cents for every $1 a man
earns. Explain why.

Workaholics Do Americans work too hard? Does working

more actually reduce productivity? Is a 40-hour work week
too much? Should there be a mandatory cap on the number
of hours a person can work? Should there be changes in
employment laws to give Americans more relaxation time?

Working mothers What differences, if any, are there in

children who are raised by stay-at-home moms and working
moms? Does society today still discriminate against working
mothers who wish to have flexible work schedules?

World

Middle East Why so many conflicts? Are there solutions?

Overpopulation What would happen globally if the demand

for natural resources becomes greater than the supply?

AREAS OF RESEARCH

BUSINESS ADMINISTRATION

 Global business management

 Knowledge management
 Strategic management studies (financial, human
resource, and other specialization fields)
 Management by culture
 Business and work values/ethics
 Case studies on medium and small scale industries
 Academe-industry linkages
 Micro-macro financing
 Policy-oriented research leading to improvements in
Philippine business and economic sector

44
  Industrial technology
 New paradigm in business and education
 Curriculum studies integrating technology
 Agri-business
 Entrepreneurship

EDUCATION

 Technological inputs to teacher education

 Policy-oriented researches focused on quality and
excellence, relevance and responsiveness, and equity in
higher education
 Multidisciplinary teacher education research such as:
o Curriculum change
o School administration and supervision studies
o Achievement and performance studies
o Teaching competencies
o School and community resources
o Leadership
o Axiological issues
o Human developmental studies
o Peace advocacy
o Education and globalization
o Information technology in education
o Human kinesthetic
o Family, life, culture, sports, and the arts
o Basic researches on mathematics, pure and applied
mathematics
o Health-related researches
o Improvement of classroom performance of Filipino
teachers
o Utilization of games and songs in Filipino
o Production of indigenous materials in teaching
Filipino
o Techniques and strategies in Filipino teaching
o Advancement of science and mathematics

45
 o Use of scientific principles in responding to the
development needs of the country

PUBLIC ADMINISTRATION

 Multidisciplinary research on agriculture and economy

 Peace development studies with direct application to
Philippine situation
 Policy-oriented research
 Multidisciplinary research on health and health-related
disciplines leading to better quality of life for Filipinos and
the delivery of basic health services to the rural areas
 Economic development (e.g., poverty alleviation,
sustainable development issues and concerns,
globalization issues, infrastructure development)
 Human development (e.g., GAD issues, access to basic
needs, labor, migration, and related issues)
 Human resources management
 Civil society and good governance (human rights,
access to social justice, legal rights, NGO services, graft
and corruption, local governance, local initiatives,
political reforms)
 International relations issues
 Primary and secondary data analysis from a data bank
 Establishment of quality indicators for the public
administration program
 Program curricular assessment studies
 Comparative study on curricular offerings of other
schools, both public and private, related to public
administration

APPLIED STATISTICS

 Policy research
 Use of scientific principles in responding to the
development needs of the country in the area of

46
 statistics
 Population and demographic issues
 Poverty alleviation, globalization, labor, migration,
human resource development, and related issues
 Environment-related researches, census, etc.
 Data bank issues

COMMUNICATION

 Family, health, and education communication

 Concepts and implications of communication laws,
policies, and media ethics
 Mass media communication and management
 Networking, broadcasting, and journalism
 Electronic media, electronic gadgets, cable and
technical communication
 Interpersonal, intrapersonal and human-robot
communication
 Cross-cultural and intercultural communication
 Dynamics of community, neighborhood, team and
political communication
 Institutional, governmental, and nongovernmental
communication campaigns
 Ethnic, religious, gay-lesbian subculture, and emerging
communication perspectives
 Influences of pornography, sexually explicit
online/print/audio materials and underground
publications
 Communication strategies in conflict management,
institutional planning and development
 Women-children advocacies, community
environmental conservation, and animal
welfare/protection
 Communication in feature, documentary,
experimental, short, and independent films

47
  Communication in electronic arts, fine arts, folk arts,
and the performing arts
 Advertising, market competition, and public relations
communication
 Assessment/evaluation of publication and productivity
of communication research scholars

INDUSTRIAL ENGINEERING AND MANAGEMENT

 Environment-related issues
 Relevance of curriculum to needs of industry
 Risk management practices
 Development projects
 Policy research with social or economic impact
 Multidisciplinary research on the improvement of
engineering designs and concepts
 Engineering and technology development researches

LIBRARY AND INFORMATION SCIENCE

 Researches on material resources, acquisitions, and

holding of schools
 Efficiency and effectiveness of education, e.g.,
organizational structures and characteristics,
evaluation of administrative policies and effectiveness,
organizational efficiency
 Policy-oriented research leading to the improvement of
library information science
 Use of scientific techniques in the formation of learning
resource centers

PSYCHOLOGY

 Peace and development studies with direct

application to the Philippine situation, e.g.,
o Gender and development

48
 o Child studies, e.g., child labor, street children,
children’s rights, children in conflict with the law
o Drug abuse, and antidrug abuse advocacy issues
o Criminal behavior
 Behavioral researches employing experimental
research methods/designs
 Psychosocial, emotional, and religious interactions
 Human growth and development studies (child,
adolescent, adult and elderly behavior)
 Multidisciplinary research on health and health-related
interests (e.g., sexuality and reproductive health)
 Marriage, family, and sexuality studies
 Abnormal behavior and psychotherapeutic
approaches
 Human resource management issues, e.g.
o Organizational behavior
o Change management
o Work-related stressors and coping mechanisms
o Quality of work-life balance, work-family balance
 Development and validation of psychological tests and
curriculum-based assessment
 Multiple Intelligences
o Intelligence quotient and other mental abilities
o Emotional intelligence
o Moral intelligence
o Spiritual intelligence
 Evaluating interventions, guidance and counselling
programs, and psychological services in school
o School discipline
o Substance abuse prevention programs
o Crisis prevention and management
o Social skills training
o Psychological assessment and evaluation
 Special education
o Gifted education
o Learning disabilities

49
  Home-school-community collaboration

ECONOMICS

 Policy analysis and impact assessment/ studies on:

o Economic development
o Poverty alleviation, e.g., food security, land reform,
etc.
o Sustainable development issues and concerns,
resource conservation
o Globalization issues, import liberalization, labor
export, global competitiveness
 Small and medium scale enterprise, cooperatives
 Infrastructure development
 International relations issues relating with ASEAN
neighbor, development cooperation
 Policy research on industrial technology
 Product development

ENGLISH LANGUAGE TEACHING

 Descriptive studies
o Basic phonological and grammatical descriptions
 Theoretical studies
o Syntax (voice/focus system, subject, topic)
o Semantics (particularly developed domains in
different language communities)
 Sociolinguistic studies
o Variation studies
o Creole studies
o Language acceptance and standardization
o Varying roles for “smaller” languages
o Ethnolinguistic vitality
o Language maintenance and shift discourse
o Pragmatics
 Applied studies

50
 o Cultural learning styles and approaches to formal,
nonformal and informal education
o Questioning styles
o Comparing studies of teacher-participant
interaction
o Comparative studies of formal vs. nonformal
education for young people
o Materials production when grammatical structures
and vocabulary are limited
 Language testing
o Development and validation of achievement and
proficiency tests
o Methods used in language testing
o Content of language tests
o Impact of teaching on testing
o Method of test analyses
 Language teaching
o Content area instruction (teacher preparation for
ESL instruction across disciplines)
o Strategies for pre-service/in-service teacher training
programs
o Learner autonomy
o Validation of a language teaching approach

INFORMATION TECHNOLOGY

 Policy research on IT
 Technological research for the advancement of
Philippine IT
 Programming and software development
 Globalization issues and technological development,
e.g., technology transfer, impact of IT, e-commerce,
etc.

ENGINEERING

51
  Engineering Analysis
o Adaptive control and simulation and optimization
of computers
o Microelectronics
o Engineering economics and cost-benefit analysis
o Ergonomics or human factor engineering
o Information and communication systems
 Structural and Infrastructural engineering
o Bridge performance models
o Constructability analysis
o Seismic assessment of existing built structures
o Modeling and mathematical programming of
structural members
o Development of structural design software
o Structural reliability and performance analysis
o Cost-benefit analysis or modeling of structural
system
 Electronics and Communications Engineering and
Computer Engineering
o Microelectronics design
o Integrated circuits and fiber optics
o Robotics and nanotechnology
o Industrial and power electronics
o Artificial intelligence
o Microprocessors and architecture

Exercise:

 A week before the review, ask students to email you

two to five key terms or theories that they feel they
need to brush up on. Take all that data and compress it
until you have a solid working list of what students want
to review most.
 In class, provide students with visual access to the list (I
found writing all the terms on a chalkboard to be most
effective). Instruct the class to have their notes out in

52
 front of them, with a pad of paper or blank Word
document at their fingertips, and encourage them to
take notes as the review is in progress.
 A trinket of sorts (I highly recommend a plush ball), used
as a “microphone,” helps to give students equal
opportunity to direct the review without putting
individuals on the spot too aggressively. The rules are
simple: she or he who holds the “mic” can pick one
term from the list and using their notes, can offer up
what they already know about the term or concept,
what they are unsure of, or what they need more
elaboration on.
 Actively listen to the speaker and give them some
positive cues if they seem unsure; it’s okay to help them
along the way, but important to step back and let this
review remain student centered. Once the speaker has
said their piece, open the floor to the rest of the class
for questions or additional comments. If you find that
the discussion has taken a departure from the right
direction, re-center the class and provide further
elaboration if need be.

Chapter 3:
Planning the
Research Project

BASIC FORMAT (PUP) AND STYLE OF THE RESEARCH

53
Manuscript Preparation

In the preparation of the manuscript, care shall be

taken to ensure that the following specifications are followed:

Paper and Printing Specifications

Print the final copy with a laser printer using only one
side of a standard-sized PUPGS bond paper (8.5 x 11 inches).

Font Specifications
Use a 12-point, Arial font throughout the thesis or
dissertation, except to compress a large table (see Tables
and Figures on page 14).

Spacing
Double-space the text; single-space long quotations,
table and figure captions, and similar special materials (e.g.,
table legend). Type triple-spaced from the top margin,
centered, boldfaced, and in ALL CAPS all headings of the
preliminary section (except the Copyright Page and Title
Page) and all chapter titles.

Margins
Leave 1.5 inches for the left margin to allow for binding
and trimming and 1 inch for the top, bottom, and right
margins.

Page Numbering
Paginate the preliminaries by using lower case Roman
numerals, centered, flush with bottom margin.

Use Arabic numerals to paginate the text,

references/works cited, and appendices. Number all the
pages consecutively starting at number 2 on the second
page of the first chapter. The first page of each chapter,
though counted, shall not be numbered.

54
 Type the page number in the upper right-hand corner
of the paper with either a portrait layout or a landscape
layout with visuals (i.e., tables and figures).

Other Typing Instructions

1. The impression on the typed/printed copies shall be
black in color.

2. Symbols, characters, or markings not found on the

computer keyboard shall be handwritten using India
ink.

3. Corrections, interlineations, and crossing out of letters

or words shall not be permitted in any of the copies of
the manuscript intended for final submission. Erasures, if
made, shall be neatly carried out in all copies.

Preliminaries
The preliminaries include the copyright page, title
page, certification-and-approval sheet, certification of
originality, acknowledgments, abstract, table of contents, list
of tables, and list of figures.

Copyright Page
Include a Copyright Page in each final copy to show
that the research has been copyrighted and put it before the
Title Page. (See Appendix 9: Copyright Page format and text).

Apply for the copyrighting of the manuscript at the

Copyright Division of the National Library. Submit two extra
copies (other than the number of copies required by the PUP
Graduate School) to the National Library, and pay the
copyright and documentation fees. All copies of final
manuscript to be submitted to the GS Office shall bear a

55
proof that the paper has been copyrighted by the National
Library.

Cover and Title Pages

Follow the instructions on the attached sample-form
pages. Note which items are in ALL CAPS and which are not.
Note the approximate size of the print (see Appendix 7:
Cover Page Format; and Appendix 8: Title Page Format).

Certification-and-Approval Sheet
Prepare a Certification-and-Approval Sheet. Refer to
the attached sample for the wording and format that shall be
followed faithfully. Include this sheet in each final copy of the
research.
The CERTIFICATION is an attestation by the
Thesis/Dissertation Evaluation Committee that the paper has
been examined and recommended for oral examination.
Type triple-spaced from the top margin, centered,
boldfaced, and in ALL CAPS the heading CERTIFICATION.
The APPROVAL contains the formal approval or
commendation of the Panel on Oral Examination (three
members for a thesis and five members for a dissertation,
both headed by a chair). (See Appendix 10: Certification-
and-Approval Format.)
Acknowledgments
Type triple-spaced from the top margin, centered,
boldfaced and in ALL CAPS the heading
ACKNOWLEDGMENTS. Double-space the text that begins
three single spaces from the heading.

Certification of Originality
Type triple-spaced from the top margin, centered,
boldfaced, and in ALL CAPS the heading CERTIFICATION OF
ORIGINALITY. Double-space the text that begins three
spaces from the heading.

56
 All copies to be submitted shall contain an original
Certification of Originality bearing the actual (not
photocopied) signature of the researcher. (See Appendix 11:
Certification of Originality Format.)

Abstract
The abstract is a brief descriptive summary of the thesis
or dissertation. It contains the main objective of the study, a
brief description of the research method, major findings,
conclusions or main arguments, and recommendations.
Type triple-spaced from the top margin, centered,
boldfaced, and in ALL CAPS the heading ABSTRACT. Type
triple-spaced from the heading the following data that are
double-spaced, flush with the left margin: thesis/dissertation
title, researcher’s name, degree, name of granting institution,
year of completion, and adviser’s name.
The abstract, which shall not exceed five (5) pages, is
double-spaced, except the title of the thesis or dissertation
which is typed single-spaced. (See Appendix 12: Abstract
Format.)

Table of Contents
The Table of Contents comes after the abstract. It shall
list all elements of the preliminaries, the chapter titles, the
main headings, the references, and the appendices. The
beginning page number of each section is indicated along
the right margin. The numbering of the chapters and the
wording, capitalization, and punctuation of titles and
headings, shall be exactly the same as they are in the text.
(See Appendix 14: Table of Contents Format).

Type triple-spaced from the top margin, centered,

boldfaced, and in ALL CAPS the heading TABLE OF
CONTENTS.

List of Tables

57
 Type triple-spaced from the top margin, centered,
boldfaced, and in ALL CAPS the heading LIST OF TABLES. Type
double-spaced from the heading the following data in three
columns: number, title, and page. Single-space the title of
each table, but double-space between titles.

List of Figures

Follow the format of the List of Tables for the List of

Figures.

The Text or Body of the Paper

1. Chapter and Subheadings

1.1 Begin each chapter on a new page.
1.2 Capitalize only the first letter of the heading
Chapter but type in ALL CAPS and centered the
title of the chapter. Both the chapter heading and
the title are typed in bold font.
1.3 Use Arabic numerals for the chapter numbers.
1.4 Type triple-spaced from the top margin and
centered the chapter number heading. Type
double-spaced below the chapter heading the
title of the chapter.

Example:
Chapter 1

THE PROBLEM AND ITS BACKGROUND

1.5 Type the subheading or the first line of the

introductory paragraph of the chapter (if there is
no subheading) three spaces below the chapter
title.

58
 1.6 Type subheadings of a chapter in a bold face, flush
with the left margin, capitalizing only the first word,
all major words, and prepositions of five (5) or more
letters e.g., Background of the Study. Subheadings
do not have any end punctuation.
1.7 Start another subheading with one triple space (or
three spaces) below the last line of the immediately
preceding paragraph.
1.8 Have at least two full lines below a subheading at
the bottom of a page, or else carry over the
subheading to the next page.

2. Definition of Terms
2.1 Only the variables or key terms included in the
study shall be defined.
2.2 Acronyms need not be defined if their meanings
have been mentioned in the text.

Example:

The Housing and Land Use Regulatory Board

(HULRB), the agency that provides for the

preparation and implementation of the

comprehensive land use plans for local

government units…

2.3 A term shall be defined operationally (i.e., how it is

used or measured in the study), or
theoretically/technically (i.e., how it is defined in
published resources).
2.4 Definitions taken from a published source shall bear
an appropriate in-text citation and have a

59
 corresponding entry in the reference list/works
cited.
2.5 The terms shall be sequenced in alphabetical
order.
2.6 If the terms consist of more than one word,
capitalize every major word.
2.7 The term to be defined shall be indented, typed in
a bold face, and punctuated with a period. Two
spaces shall separate the period from the
definition.
2.8 Each definition shall be expressed in a complete
sentence, NOT a phrase.
2.9 Symbols and abbreviations shall be enclosed in
parentheses following their spelled-out
nomenclatures.

Example:

Pi( π ). This term refers to a mathematical

constant and a transcendental real

number,

approximately equal to 3.14159, which is the ratio of

a circle's circumference to its diameter in Euclidean

geometry, and has many uses in mathematics,

physics, and engineering. In this study, the π was

used to calculate…

60
3. Review of Related Literature and Studies
3.1 The Review of Related Literature and Studies shall
only have five subheadings, namely, Foreign
Literature, Local Literature, Foreign Studies, Local
Studies, and Synthesis of the Reviewed Literature
and Studies.
3.2 Except for classical theories and any other
relevant literature, all the literature and studies
included in the review shall have been published
or written (if unpublished, such as theses and
dissertations) at least ten (10) years before the
conduct of the current study.
3.3 Except for exploratory studies, the minimum
number of literature and studies to be reviewed
shall be pegged at ten (10) each for foreign
literature, local literature, foreign studies, and
local studies. These include at least 10 journal
articles for a thesis and 15 for a dissertation.

4. Tables and Figures

4.1 Type above every table its number and title.
4.2 Type the table number in a bold font, centered,
and triple-spaced from the last line of the
immediately preceding paragraph.
4.3 Type the title of the table in a bold font,
centered, double-spaced from the table
number, and triple-spaced to the table.
4.4 Type triple-spaced below the figure its number
and title.
4.5 Type the title of the figure in a bold font,
centered, and double-spaced from the figure
number.
4.6 If the title is long, type single-spaced, aligned to
the first word of the title, the rest of the title.
4.7 Capitalize only the first word, all major words, and
prepositions with five or more letter in all titles.

61
 4.8 Use the landscape page layout for large tables.
4.9 If the table is still too large to fit a single page,
reduce the Arial font to a minimum of 8 points.
4.10 If the table does not fit the page even after the
font-size reduction, continue the rest of its part to
the next page, but the cut part shall bear the
subtitle ‘Continuation’ flush with the left margin
(e.g., Continuation of Table 14). The cut part/s
shouldshall also show the column headings and
the legend (if any) like those in the first part of the
table.
4.11 Put two or moresmall tables or figures on a single
page if the page is big enough to
accommodate them all.
4.12 Have small tables and figures appear on the
same page along with the text; however, leave
three spaces between every table/figure and
the texts above and below it.

5. Drawings
5.1 Use permanent blank ink and heavy lines to draw
any materials (e.g., graphs, maps, etc.) that
cannot be typed or computer-generated.
Photographs of drawings are also acceptable.
5.2 Use photocopied reproductions of drawings if
they are reproduced on the right paper and if
they are of high contrast. No pressed-on letters
may, however, be used.

6. Photographs
6.1 Label as figures photographs to be included as
part of the text.
6.2 Properly scan or print photographs; do NOT just
mount them using any dry mounting material
(e.g., paste or glue).

62
 6.3 Use full-page photographs on 8.5” x 11” heavy
photography stock, as well as high clarity
photocopies of photographs.
6.4 In all copies, use originally scanned photographs
(or reproduced on a full sheet of photographic
paper), not mere clear photocopies.
6.5 Reduce large-sized figures and photographs to
the appropriate size before their insertion into the
text.
6.6 Put more than one photograph on a single page
if the page has sufficient space for them all.

7. Italics
Italicize Filipino and any other foreign terms used
in the text. When analyzing items from tables or figures,
use quotation marks, NOT italics, to cite the statement
or phrase.

Example:

The item “Establish a risk-based plan to determine

the priorities of the internal audit activities” obtained a

weighted mean of 3.51, which indicates a very

satisfactory rating by the respondents.

8. In-Text Citation
Use the American Psychological Association
(APA) style in the in-text citation of theses/ dissertations
of all programs except the MAELT/MATESL and MAF
programs that shall use the Modern Language
Association (MLA) style.

63
 The APA style follows the author-date method of
in-text citation. This means that the author's last name
and the year of publication for the source shall appear
in the text, e.g., (Jones, 2006) or “According to Jones
(2006)…”. A page number of the source, however, shall
be included in the in-text citation in case of direct
quotations, e.g., (Jones, 2006, p. 199).

The MLA style follows the author-page method of

in-text citation. This means that the author's last name
and the page number(s) from which the quotation or
paraphrase is taken shall appear in the text. The
author's name may appear either in the sentence itself
or in parentheses following the quotation or
paraphrase, but the page number(s) shall always
appear in parentheses, not in the sentence, e.g.,
(Wordsworth 263).

Refer to the PUPGS Guidelines for Writing Source

Materials for complete details.

9. Long Quotation
Place direct long quotations (four or more lines)
in a free standing block of typewritten lines and omit
quotation marks. Start the quotation on a new line,
with the entire quotation indented five spaces from the
left margin. Indent the first line of any subsequent
paragraph within the quotation. Maintain single-
spacing throughout the quotation and put the
parenthetical citation after the closing punctuation
mark.

64
10. Chapter 4 Subheadings
Subheadings in Chapter 4 are the findings of
every problem stated in Chapter 1. These subtitles are
numbered following the sequence of the statement
of the subproblems in the Statement of the Problem
and typed in block style and single-spaced and in a
bold face.

Do not put any enumeration under each subtitle,

whose first word, all major words, and prepositions of
five or more letters shall be capitalized.

Example:

1. Profile of the Respondents According to Sex, Age,

Civil Status, Religion, Educational Attainment,
Employment Status, Position, Eligibility, and Number
of Years in the Service

11. Chapter 5 Subheadings

Chapter 5 shall only have the following three
subheadings: Summary, Conclusions, and
Recommendations.

Follow the rules of spacing for subheadings.

12. Bibliography/References/Works Cited

The American Psychological Association (APA) or
the Modern Language Association (MLA) style of
documentation shall be used.

The references (for APA style) and works cited (for

the MLA style) shall include all references cited in the
text, and the entries shall be arranged alphabetically
and double-spaced from one another.

65
 With the exception of exploratory researches,
there shall be a citation of at least twenty (20) research
articles from refereed professional journals and
unpublished theses/dissertations for a thesis, and at
least thirty (30) research articles for a dissertation. These
shall be added to the references taken from books,
papers presented, manuals and any other publications,
and electronic sources.

Each reference shall be single-spaced with the

first line typed flush with the left margin and the
succeeding lines indented five spaces from the left
margin.

The references/works cited shall contain the

following parts that shall be arranged accordingly:
books, journals and monographs, unpublished theses/
dissertations, periodicals, electronic sources, other print
sources, non-print sources.

13. Appendices
Appendices shall be provided to give
supplementary materials that are not appropriate for
inclusion in the text.

13.1 Number the appendices using Arabic numerals,

e.g., Appendix 1, Appendix 2, etc.

13.2 Type the appendix title centered and triple-

spaced from the top margin.

13.3 If lengthy, supplementary illustrative materials such

as tables, figures, and references appear in
appendices, refer to them at appropriate places
in the text.

66
 13.4 List the appended work/document in the Table of
Contents as it is numbered and entitled in the
appendices, e.g., Appendix 1: Research
Instrument.

14. Curriculum Vitae

Limit the curriculum vitae to five (5) pages. Include
only relevant personal and professional data such as
educational attainment, employment background,
professional development (seminars,
conferences/trainings attended), extension services
(lectures, speakerships, projects organized, etc.),
publications (books, researches, etc.) if any, and
professional affiliations.

THE INTRODUCTION AND BACKGROUND OF THE STUDY

Although the format and content of dissertations differ

dramatically across different fields of study, several core
components appear in all disciplines. A typical dissertation
includes an introduction, a background or literature review,
discussions of the theories and methods used in the research,
data analysis and results, a discussion or interpretation of the
results, and a conclusion. Each of these components may be
in a separate section or chapter, or you may address multiple
components within sections.

Goals of the Introduction

If someone reads only one section of your dissertation, it’ll be

the introduction, so the introduction’s primary goal is to
demonstrate the importance, interest and originality of your
research project. Above all, it should include a statement of
the research question that your project investigates. This
statement should give readers a broad sense of the current

67
research on your topic, what’s at stake in learning more
about the topic and how your specific project changes what
people know about the topic. The introduction’s tone should
be confident without being arrogant or dismissive. Finally, the
introduction should define key terms you’ll use throughout the
study, as well as map out the rest of the dissertation.

Goals of the Background

The background section is often called the literature review.

“The literature” refers to other research on your topic. The
background synthesizes current knowledge on your research
question in far greater detail than your introductory section
does. Its goal is to articulate patterns within the literature and
to describe unresolved issues or questions, then to show how
your study fits into the larger body of work in your field. You
can organize your overview of other research in several ways,
including in chronological order, by significant issues, or from
broad information about your field to specific studies on your
topic.
Separate Introduction and Background
Some disciplines organize the introduction and background
as separate sections of the dissertation. Dissertations in the
social sciences, for instance, frequently have an introduction
followed by a literature review. The hard sciences also tend to
follow this format, with each subsequent chapter representing
a published article related to the broad research question.

Background Within the Introduction

In many humanities disciplines, the introduction will include a

section called “Background” or “Literature Review,” which
provides a history of criticism on your topic. Individual
dissertation chapters then contain further discussion of the
criticism related to specific texts that the chapter investigates.

68
It’s crucial to consult your adviser to find out how you’re
expected to organize the introduction and background.

THE RESEARCH PROBLEM

Sources of the Problem

A research problem, in general, refers to some difficulty
which a researcher experiences in the context of either a
theoretical or practical situation and wants to obtain a
solution for the same.

- CHARACTERISTICS OF RESEARCH PROBLEM

 The variables in the problem must be clear

 It should be limited in scope and should be specific,
 It must have a goal
 It should be free from ethical constraints
 Good research problem must be researchable.

- SOURCES OF RESEARCH PROBLEMS

 Social problems like unemployment, crimes, female

genital mutilation, etc.
 Theory deduction
 Funding agencies
 Past researches and literature review
 Casual observation
 Related literature
 Current social and economic issues
 Personal interest and experience
 Replication of previous studies
 Clarification of contradictory research results

Criteria for Selection of the Problem

69
Any research is a difficult task to achieve and research needs
to do a great effort. Selection of research topic is the first step
to success.

1. Research topic must be very clear and easy to

understand. It should not distract people.
2. A topic well defined is the only way to successful
research. The topic should not create doubt and
double impression.
3. Easy language is a key to success. Use technical
words if necessary otherwise focus of simplicity.
4. Research title should be according to the rules of
titling. There are different rules of titling, a researcher
must aware before writing a research title.
5. While selecting a research topic current importance
of a researcher should also be considered. Topic
should not be obsolete and it should have great
importance in the current day.

Statement of the Problem

The statement of problem deals with the gap in knowledge

you have noticed and intend to address through your
research work. Every research study should be based on two
variables mostly; the subject and the object. When you do a
statement of problem it means you have a problem you want
to address research-wise. SO, you need to build on the
problem and tell the audience what the problem you have
discovered is as well as how you intend mitigating the
problem.

Thesis problem statement

A problem statement deliberates on the issue. An accurately

defined problem statement translates to a feasible or good
solution to that problem; on the contrary, an inept definition
may lead to an inaccurate and inappropriate solution.

70
Hence, the thesis statement or research question is your
guide.

Therefore, the hypothesis comes after you state the problem

or state the question in your report

A statement of the problem is used in research work as a

claim that outlines the problem addressed by a study. The
statement of the problem briefly addresses the question:
What is the problem that the research will address?

What are the goals of a statement of the problem?

The ultimate goal of a statement of the problem is to

transform a generalized problem (something that bothers
you; a perceived lack) into a targeted, well-defined problem;
one that can be resolved through focused research and
careful decision-making.

Writing a statement of the problem should help you clearly

identify the purpose of the research project you will propose.
Often, the statement of the problem will also serve as the
basis for the introductory section of your final proposal,
directing your reader’s attention quickly to the issues that your
proposed project will address and providing the reader with a
concise statement of the proposed project itself.

A statement of problem need not be long and elaborate:

one page is more than enough for a good statement of
problem.

What are the key characteristics of a statement of the

problem?

71
A good research problem should have the following
characteristics:

1. It should address a gap in knowledge.

2. It should be significant enough to contribute to the
existing body of research
3. It should lead to further research
4. The problem should render itself to investigation
through collection of data
5. It should be of interest to the researcher and suit his/her
skills, time, and resources
6. The approach towards solving the problem should be
ethical

What is the format for writing a statement of the problem?

A persuasive statement of problem is usually written in three

parts:

Part A (The ideal): Describes a desired goal or ideal situation;

explains how things should be.

Part B (The reality): Describes a condition that prevents the

goal, state, or value in Part A from being achieved or realized
at this time; explains how the current situation falls short of the
goal or ideal.

Part C (The consequences): Identifies the way you propose to

improve the current situation and move it closer to the goal
or ideal.

Here is an example:

Example 1

72
 Part A: According to the XY university mission statement,
the university seeks to provide students with a safe, healthy
learning environment. Dormitories are one important
aspect of that learning environment, since 55% of XY
students live in campus dorms and most of these students
spend a significant amount of time working in their dorm
rooms.

However,

Part B: Students living in dorms A B C, and D currently do

not have air conditioning units, and during the hot
seasons, it is common for room temperatures to exceed 80
degrees F. Many students report that they are unable to
do homework in their dorm rooms. Others report having
problems sleeping because of the humidity and
temperature. The rooms are not only unhealthy, but they
inhibit student productivity and academic achievement.

Part C: In response to this problem, our study proposes to

investigate several options for making the dorms more
hospitable. We plan to carry out an all-inclusive
participatory investigation into options for purchasing air
conditioners (university-funded; student-subsidized) and
different types of air conditioning systems. We will also
consider less expensive ways to mitigate some or all of the
problems noted above (such as creating climate-
controlled dorm lounges and equipping them with better
study areas and computing space).

HYPOTHESIS

A hypothesis is a specific statement of prediction. It describes

in concrete (rather than theoretical) terms what you expect
will happen in your study. Not all studies have hypotheses.

73
Sometimes a study is designed to be exploratory
(see inductive research). There is no formal hypothesis, and
perhaps the purpose of the study is to explore some area
more thoroughly in order to develop some specific hypothesis
or prediction that can be tested in future research. A single
study may have one or many hypotheses.

Hypotheses in Quantitative Studies

Research hypotheses in quantitative studies take a familiar

form: one independent variable, one dependent variable,
and a statement about the expected relationship between
them. Generally the independent variable is mentioned first
followed by language implying causality (terms such as
explains, results in) and then the dependent variable; the
ordering of the variables should be consistent across all
hypotheses in a study so that the reader is not confused
about the proposed causal ordering. When both variables
are continuous in nature, language describing a positive or
negative association between the variables can be used (for
example, as education increases, so does income). For
hypotheses with categorical variables, a statement about
which category of the independent variable is associated
with a certain category of the dependent variable can be
made (for example, men are more likely to support
Republican candidates than women). Continuous variables
can also be spoken about it categorical terms (those with
higher education are more likely to have high incomes).

Most researchers prefer to present research hypotheses in a

directional format, meaning that some statement is made
about the expected relationship based on examination of
existing theory, past research, general observation, or even
an educated guess. It is also appropriate to use the null
hypothesis instead, which states simply that no relationship
exists between the variables; recall that the null hypothesis

74
forms the basis of all statistical tests of significance. A
compromise position is to present a research hypothesis
which states a possible direction for the relationship but
softens the causal argument by using language such as
“tends to” or “in general.”

Hypotheses in Qualitative Studies

Hypotheses in qualitative studies serve a very different

purpose than in quantitative studies. Due to the inductive
nature of qualitative studies, the generation of hypotheses
does not take place at the outset of the study. Instead,
hypotheses are only tentatively proposed during an iterative
process of data collection and interpretation, and help guide
the researcher in asking additional questions and searching
for disconfirming evidence.

Qualitative research is guided by central questions and

subquestions posed by the researcher at the outset of a
qualitative study. These questions usually employ the
language of how and what in an effort to allow
understanding to emerge from the research, rather than why,
which tends to imply that the researcher has already
developed a belief about the causal mechanism. In general
a qualitative study will have one or two central questions and
a series of five to ten subquestions that further develop the
central questions. These questions are often asked directly of
the study participants (through in-depth interviews, focus
groups, etc.) in recognition of the fact that developing an
understanding of a particular phenomenon is a collaborative
experience between researchers and participants.

SIGNIFICANCE OF THE STUDY

In writing the introduction of a thesis, a section is devoted to

the significance of the study. This article discusses how to write

75
this section and provides an example to illustrate the
technique.

Essentially, the section on significance of the study provides

information to the reader on how the study will contribute. It
must be specifically stated, however, what the study will
contribute and who will benefit from it.

You can figure out several important contributions of your

research paper if you let your mind flow. But I find the
following tips helpful in writing the significance of the study.

2 Tips in Writing the Significance of the Study

1. Refer to the statement of the problem

Your problem statement can guide you in identifying the

specific contribution of your study. You can do this by
observing a one-to-one correspondence between the
statement of the problem and the significance of the study.

For example, if you ask the question “Is there a significant

relationship between the teacher’s teaching style and the
students’ long quiz scores in Mathematics?” then the
contribution of your research would probably be a teaching
style or styles (among say, three teaching styles you
evaluated) that can help students perform better in
Mathematics. Your research will demonstrate that that
teaching style really works. That could be a groundbreaking
approach that will change the way teachers teach
Mathematics which many students abhor.

2. Write from general to specific contribution

Write the significance of the study by looking into the general

contribution of your study, such as its importance to society as
a whole, then proceed downwards—towards its contribution
to individuals and that may include yourself as a researcher.
76
You start off broadly then taper off gradually to a specific
group or person.

Coupled with reference to the problem statement, this

effectively stimulates the mind to think in a deductive mode,
i.e., from general to
o specific. This writing approach is similar
to the Inverted Pyramid Approach discussed in How to Write
a Good Thesis Introduction.

For example, in the study on teaching style given in #1, you

may write:

These two techniques will prevent your mind from wandering

w
wildly or aimlessly as you explore the significance of your
study. Applying them will save time thus allow you to focus on
the next section of your thesis. Who knows, this section may
also help justify why your study deserves a grant.

SCOPE AND LIMITATIONS

Guidelines in writing the scope and delimitation. The scope

and delimitation should include the following:

1. A brief statement of the general purpose of the study.

2. The subject matter and topics studied and discussed.
3. The locale of the study, where the data were gathered or

77
the entity to which the data belong.
4. The population or universe from which the respondents
were selected. This must be large enough to make
generalizations significant.
5. The period of the study. This is the time, either months or
years, during which the data were gathered.

Example: This investigation was conducted to determine the

status of the teaching of science in the high schools of
Province A as perceived by the teachers and students in
science classes during the school year 1989-1990. the aspects
looked into were the qualifications of teachers, their methods
and strategies, facilities forms of supervisory assistance,
problems and proposed solutions to problems.

General purpose: To determine the status of the teaching of

science.
Subject matter: The teaching of science.
Topics (aspects) studied: Qualifications of teachers. Their
methods and strategies, facilities, form of supervisory
assistance, problems and proposed solutions to the problems.
Population or universe: teachers and students
Locale of the study: High schools of province A.
Period of the study: School year 1989-1990.

78
THEORETICAL FRAMEWORK

Every dissertation needs to have a theoretical foundation or a

conceptual framework (or both) that is discussed in the
literature review section. The theoretical foundation is
important because it will be the lens through which you
evaluate your research problem and research questions. The
theoretical framework section is typically required for
quantitative studies, while a conceptual framework is used in
qualitative studies. Once you have identified the theory or
theories that you will be utilizing, it is important to know what
information to include and how to include it in your
discussion.

First and foremost, you should identify and describe your

selected theory or theories. This information should consist of
the author(s) of the theory and details regarding the original
study, such as the population, purpose, and results. Providing
this information will help you to highlight why the theory you
have selected is valid for use in your study, and will make it
easier for you to demonstrate how the theory relates to your
own study.

Once you’ve introduced and discussed your selected theory

in the Theoretical Foundation section of your literature review,
you need to outline why the theory is relevant to your study
and how you will use the information from it to conduct and
evaluate your research and findings. For example, if you
conduct a study on employee engagement and opt to use
Maslow’s (1943) Hierarchy of Needs as your theoretical
foundation, discussing the fulfillment of needs as outlined in
Maslow’s theory will help you to establish what his theory is
and why it is relevant to your research. Once that is done,
you can discuss how the fulfillment of needs (based on

79
Maslow) relates to motivation in employees and how your
study will use Maslow’s theory to evaluate and address your
identified problem.

Theories are formulated to explain, predict, and understand

phenomena and, in many cases, to challenge and extend
existing knowledge within the limits of critical bounding
assumptions. The theoretical framework is the structure that
can hold or support a theory of a research study. The
theoretical framework introduces and describes the theory
that explains why the research problem under study exists.

A theoretical framework consists of concepts and, together

with their definitions and reference to relevant scholarly
literature, existing theory that is used for your particular
study. The theoretical framework must demonstrate an
understanding of theories and concepts that are relevant to
the topic of your research paper and that relate to the
broader areas of knowledge being considered.
The theoretical framework is most often not something readily
found within the literature. You must review course readings
and pertinent research studies for theories and analytic
models that are relevant to the research problem you are
investigating. The selection of a theory should depend on its
appropriateness, ease of application, and explanatory
power.
The theoretical framework strengthens the study in the
following ways:

1. An explicit statement of theoretical assumptions permits

the reader to evaluate them critically.
2. The theoretical framework connects the researcher to
existing knowledge. Guided by a relevant theory, you

80
 are given a basis for your hypotheses and choice of
research methods.
3. Articulating the theoretical assumptions of a research
study forces you to address questions of why and how.
It permits you to intellectually transition from simply
describing a phenomenon you have observed to
generalizing about various aspects of that
phenomenon.
4. Having a theory helps you identify the limits to those
generalizations. A theoretical framework specifies
which key variables influence a phenomenon of
interest and highlights the need to examine how those
key variables might differ and under what
circumstances.

By virtue of its applicative nature, good theory in the social

sciences is of value precisely because it fulfills one primary
purpose: to explain the meaning, nature, and challenges
associated with a phenomenon, often experienced but
unexplained in the world in which we live, so that we may use
that knowledge and understanding to act in more informed
and effective ways.

Strategies for Developing the Theoretical Framework

I. Developing the Framework

Here are some strategies to develop of an effective
theoretical framework:

1. Examine your thesis title and research problem. The

research problem anchors your entire study and forms
the basis from which you construct your theoretical
framework.

81
 2. Brainstorm about what you consider to be the key
variables in your research. Answer the question, "What
factors contribute to the presumed effect?"
3. Review related literature to find how scholars have
addressed your research problem. Identify the
assumptions from which the author(s) addressed the
problem.
4. List the constructs and variables that might be relevant
to your study. Group these variables into independent
and dependent categories.
5. Review key social science theories that are introduced
to you in your course readings and choose the theory
that can best explain the relationships between the key
variables in your study [note the Writing Tip on this
page].
6. Discuss the assumptions or propositions of this theory
and point out their relevance to your research.

A theoretical framework is used to limit the scope of the

relevant data by focusing on specific variables and defining
the specific viewpoint [framework] that the researcher will
take in analyzing and interpreting the data to be gathered. It
also facilitates the understanding of concepts and variables
according to given definitions and builds new knowledge by
validating or challenging theoretical assumptions.
II. Purpose
Think of theories as the conceptual basis for understanding,
analyzing, and designing ways to investigate relationships
within social systems. To that end, the following roles served
by a theory can help guide the development of your
framework.

 Means by which new research data can be interpreted

and coded for future use,

82
  Response to new problems that have no previously
identified solutions strategy,
 Means for identifying and defining research problems,
 Means for prescribing or evaluating solutions to
research problems,
 Ways of discerning certain facts among the
accumulated knowledge that are important and
which facts are not,
 Means of giving old data new interpretations and new
meaning,
 Means by which to identify important new issues and
prescribe the most critical research questions that need
to be answered to maximize understanding of the issue,
 Means of providing members of a professional
discipline with a common language and a frame of
reference for defining the boundaries of their
profession, and
 Means to guide and inform research so that it can, in
turn, guide research efforts and improve professional
practice.

Structure and Writing Style

The theoretical framework may be rooted in a specific theory,

in which case, your work is expected to test the validity of
that existing theory in relation to specific events, issues, or
phenomena. Many social science research papers fit into this
rubric. For example, Peripheral Realism Theory, which
categorizes perceived differences among nation-states as
those that give orders, those that obey, and those that rebel,
could be used as a means for understanding conflicted
relationships among countries in Africa. A test of this theory
could be the following: Does Peripheral Realism Theory help
explain intra-state actions, such as, the disputed split

83
between southern and northern Sudan that led to the
creation of two nations?
However, you may not always be asked by your professor to
test a specific theory in your paper, but to develop your own
framework from which your analysis of the research problem
is derived. Based upon the above example, it is perhaps
easiest to understand the nature and function of a
theoretical framework if it is viewed as an answer to two basic
questions:

1. What is the research problem/question? [e.g., "How

should the individual and the state relate during
periods of conflict?"]
2. Why is your approach a feasible solution? [i.e., justify
the application of your choice of a particular theory
and explain why alternative constructs were rejected. I
could choose instead to test Instrumentalist or
Circumstantialists models developed among ethnic
conflict theorists that rely upon socio-economic-
political factors to explain individual-state relations and
to apply this theoretical model to periods of war
between nations].

The answers to these questions come from a thorough review

of the literature and your course readings [summarized and
analyzed in the next section of your paper] and the gaps in
the research that emerge from the review process. With this in
mind, a complete theoretical framework will likely not
emerge until after you have completed a thorough review of
the literature.
Just as a research problem in your paper requires
contextualization and background information, a theory
requires a framework for understanding its application to the
topic being investigated. When writing and revising this part
of your research paper, keep in mind the following:

84
 Clearly describe the framework, concepts, models, or
specific theories that underpin your study. This includes
noting who the key theorists are in the field who have
conducted research on the problem you are
investigating and, when necessary, the historical
context that supports the formulation of that theory. This
latter element is particularly important if the theory is
relatively unknown or it is borrowed from another
discipline.
 Position your theoretical framework within a broader
context of related frameworks, concepts, models,
or theories. As noted in the example above, there will
likely be several concepts, theories, or models that can
be used to help develop a framework for
understanding the research problem. Therefore, note
why the theory you've chosen is the appropriate one.
 The present tense is used when writing about
theory. Although the past tense can be used to
describe the history of a theory or the role of key
theorists, the construction of your theoretical framework
is happening now.
 You should make your theoretical assumptions as
explicit as possible. Later, your discussion of
methodology should be linked back to this theoretical
framework.
 Don’t just take what the theory says as a given! Reality
is never accurately represented in such a simplistic
way; if you imply that it can be, you fundamentally
distort a reader's ability to understand the findings that
emerge. Given this, always note the limitations of the
theoretical framework you've chosen [i.e., what parts
of the research problem require further investigation
because the theory inadequately explains a
certainphenomena].

85
CONCEPTUAL FRAMEWORK

Definition of Conceptual Framework

A conceptual framework represents the researcher’s synthesis

of literature on how to explain a phenomenon. It maps out
the actions required in the course of the study given his
previous knowledge of other researchers’ point of view and
his observations on the subject of research.
In other words, the conceptual framework is the researcher’s
understanding of how the particular variables in his study
connect with each other. Thus, it identifies the variables
required in the research investigation. It is the researcher’s
“map” in pursuing the investigation.
As McGaghie et al. (2001) put it: The conceptual framework
“sets the stage” for the presentation of the particular research
question that drives the investigation being reported based
on the problem statement. The problem statement of a thesis
presents the context and the issues that caused the
researcher to conduct the study.
The conceptual framework lies within a much broader
framework called theoretical framework. The latter draws
support from time-tested theories that embody the findings of
many researchers on why and how a particular phenomenon
occurs.

Step by Step Guide on How to Make the Conceptual

Framework
Before you prepare your conceptual framework, you need to
do the following things:

1. Choose your topic. Decide on what will be your research

topic. The topic should be within your field of specialization.
2. Do a literature review. Review relevant and updated
research on the theme that you decide to work on after

86
 scrutiny of the issue at hand. Preferably use peer-reviewed
and well-known scientific journals as these are reliable
sources of information.
3. Isolate the important variables. Identify the specific
variables described in the literature and figure out how
these are related. Some abstracts contain the variables
and the salient findings thus may serve the purpose. If
these are not available, find the research paper’s
summary. If the variables are not explicit in the summary,
get back to the methodology or the results and discussion
section and quickly identify the variables of the study and
the significant findings.
4. Generate the conceptual framework. Build your
conceptual framework using your mix of the variables from
the scientific articles you have read. Your problem
statement serves as a reference in constructing the
conceptual framework. In effect, your study will attempt to
answer a question that other researchers have not
explained yet. Your research should address a knowledge
gap.

87
DEFINITIONS

Much misunderstanding in human communication results

from people bringing different meanings to the words they
use in speaking and writing. Effective researchers seek to
avoid this difficulty by clearly explaining the meanings they
assign to key terms in their investigations. If, early in the
research process, you define precisely what you intend by
words and phrases crucial to your project, (a) you help
identify appropriate methods of gathering and interpreting
data and (b) your advisors can judge at the outset how well
they agree with your definitions, thereby saving you possible
trouble during subsequent stages of your project. The terms
key words and key phrases refer to concepts at the core of
your study, concepts that must be unambiguous if you are to
conduct your research with proper care and if the
procedures and outcomes are to be properly understood by
your reading audience. Among the most basic terms are
those found in a project's title or topic question. To illustrate
key words, in the following examples we have italicized each
term that calls for a definition:

Guidelines in defining terms:

1. Only terms, words, or phrases which have special or unique

meanings in the study are defined. For instance, the term
non-teaching facilities may be used in the study of the
teaching of science. Non-teaching facilities may be defined
as facilities needed by the students and teachers but are not
used to explain the lesson or to make instructions clearer.
Examples are toilets or comfort rooms, electric fans, rest rooms
or lounges, and the like. They may also be called non-
instructional facilities.
2. Terms should be defined operationally, that is how they are
used in the study. For instance, a study is made about early
marriage. What is meant by early marriage? To make the

88
meaning clear, early marriage may be defined as one in
which the contracting parties are both below eighteen years
of age.
3. The researcher may develop his own definition from the
characteristics of the term defined. Thus, a house of light
materials may be defined as one with bamboo or small
wooden posts, nipa, buri, or nipa walls; split bamboo floor
and cogon or nipa roof. This is also an operational definition.
4. Definitions may be taken from encyclopedias, books,
magazines and newspaper articles, dictionaries, and other
publications but the researcher must acknowledge his
sources. Definitions taken from published materials are called
conceptual or theoretical definitions.
5. Definitions should a\be brief, clear, and unequivocal as
possible.
6. Acronyms should always be spelled out fully especially if it is
not commonly known or if it is used for the first time.

Exercise:

This is a highly effective technique for checking student

progress, both in understanding the material and in reacting
to course material. Ask students to take out a blank sheet of
paper, pose a question (either specific or open-ended), and
give them one (or perhaps two - but not many more)
minute(s) to respond. Some sample questions include: "How
does John Hospers define "free will"?", "What is "scientific
realism"?", "What is the activation energy for a chemical
reaction?", "What is the difference between replication and
transcription?", and so on. Another good use of the minute
paper is to ask questions like "What was the main point of
today’s class material?" This tells you whether or not the
students are viewing the material in the way you envisioned.

Chapter 4:
Reviewing Studies and Literature

89
A literature review surveys books, scholarly articles, and any
other sources relevant to a particular issue, area of research,
or theory, and by so doing, provides a description, summary,
and critical evaluation of these works in relation to the
research problem being investigated. Literature reviews are
designed to provide an overview of sources you have
explored while researching a particular topic and to
demonstrate to your readers how your research fits within a
larger field of study.

SOURCES AND PURPOSE OF REVIEW

A literature review may consist of simply a summary of key

sources, but in the social sciences, a literature review usually
has an organizational pattern and combines both summary
and synthesis, often within specific conceptual categories. A
summary is a recap of the important information of the
source, but a synthesis is a re-organization, or a reshuffling, of
that information in a way that informs how you are planning
to investigate a research problem. The analytical features of
a literature review might:

 Give a new interpretation of old material or combine

new with old interpretations,
 Trace the intellectual progression of the field, including
major debates,
 Depending on the situation, evaluate the sources and
advise the reader on the most pertinent or relevant
research, or
 Usually in the conclusion of a literature review, identify
where gaps exist in how a problem has been
researched to date.

The purpose of a literature review is to:

90
  Place each work in the context of its contribution to
understanding the research problem being studied.
 Describe the relationship of each work to the others
under consideration.
 Identify new ways to interpret prior research.
 Reveal any gaps that exist in the literature.
 Resolve conflicts amongst seemingly contradictory
previous studies.
 Identify areas of prior scholarship to prevent duplication
of effort.
 Point the way in fulfilling a need for additional research.
 Locate your own research within the context of existing
literature [very important]

SCOPE AND METHOD OF REVIEW

I. Thinking About Your Literature Review

The structure of a literature review should include the
following:

 An overview of the subject, issue, or theory under

consideration, along with the objectives of the
literature review,
 Division of works under review into themes or categories
[e.g. works that support a particular position, those
against, and those offering alternative approaches
entirely],
 An explanation of how each work is similar to and how
it varies from the others,
 Conclusions as to which pieces are best considered in
their argument, are most convincing of their opinions,
and make the greatest contribution to the
understanding and development of their area of
research.

91
The critical evaluation of each work should consider:

 Provenance -- what are the author's credentials? Are

the author's arguments supported by evidence [e.g.
primary historical material, case studies, narratives,
statistics, recent scientific findings]?
 Methodology -- were the techniques used to identify,
gather, and analyze the data appropriate to
addressing the research problem? Was the sample size
appropriate? Were the results effectively interpreted
and reported?
 Objectivity -- is the author's perspective even-handed
or prejudicial? Is contrary data considered or is certain
pertinent information ignored to prove the author's
point?
 Persuasiveness -- which of the author's theses are most
convincing or least convincing?
 Value -- are the author's arguments and conclusions
convincing? Does the work ultimately contribute in any
significant way to an understanding of the subject?

II. Development of the Literature Review

FourStages

1. Problem formulation -- which topic or field is being

examined and what are its component issues?
2. Literature search -- finding materials relevant to the subject
being explored.
3. Data evaluation -- determining which literature makes a
significant contribution to the understanding of the topic.
4. Analysis and interpretation -- discussing the findings and
conclusions of pertinent literature.
Consider the following issues before writing the literature
review:

92
Clarify

If your assignment is not very specific about what form your

literature review should take, seek clarification from your
professor by asking these questions:

1. Roughly how many sources should I include?

2. What types of sources should I review (books, journal
articles, websites; scholarly versus popular sources)?
3. Should I summarize, synthesize, or critique sources by
discussing a common theme or issue?
4. Should I evaluate the sources?
5. Should I provide subheadings and other background
information, such as definitions and/or a history?

Find Models

Use the exercise of reviewing the literature to examine how

authors in your discipline or area of interest have composed
their literature review sections. Read them to get a sense of
the types of themes you might want to look for in your own
research or to identify ways to organize your final review. The
bibliography or reference section of sources you've already
read are also excellent entry points into your own research.

Narrow the Topic

The narrower your topic, the easier it will be to limit the

number of sources you need to read in order to obtain a
good survey of relevant resources. Your professor will
probably not expect you to read everything that's available
about the topic, but you'll make your job easier if you first limit
scope of the research problem. A good strategy is to begin
by searching the HOMER catalog for books about the topic
and review the table of contents for chapters that focuses on
specific issues. You can also review the indexes of books to

93
find references to specific issues that can serve as the focus
of your research. For example, a book surveying the history of
the Israeli-Palestinian conflict may include a chapter on the
role Egypt has played in mediating the conflict, or look in the
index for the pages where Egypt is mentioned in the text.

Consider Whether Your Sources are Current

Some disciplines require that you use information that is as

current as possible. This is particularly true in disciplines in
medicine and the sciences where research conducted
becomes obsolete very quickly as new discoveries are made.
However, when writing a review in the social sciences, a
survey of the history of the literature may be required. In other
words, a complete understanding the research problem
requires you to deliberately examine how knowledge and
perspectives have changed over time. Sort through other
current bibliographies or literature reviews in the field to get a
sense of what your discipline expects. You can also use this
method to explore what is considered by scholars to be a
"hot topic" and what is not.

94
III. Ways to Organize Your Literature Review
Chronology of Events
If your review follows the chronological method, you could
write about the materials according to when they were
published. This approach should only be followed if a clear
path of research building on previous research can be
identified and that these trends follow a clear chronological
order of development. For example, a literature review that
focuses on continuing research about the emergence of
German economic power after the fall of the Soviet Union.

By Publication
Order your sources by publication chronology, then, only if
the order demonstrates a more important trend. For instance,
you could order a review of literature on environmental
studies of brown fields if the progression revealed, for
example, a change in the soil collection practices of the
researchers who wrote and/or conducted the studies.

Thematic [“conceptual categories”]

Thematic reviews of literature are organized around a topic
or issue, rather than the progression of time. However,
progression of time may still be an important factor in a
thematic review. For example, a review of the Internet’s
impact on American presidential politics could focus on the
development of online political satire. While the study focuses
on one topic, the Internet’s impact on American presidential
politics, it will still be organized chronologically reflecting
technological developments in media. The only difference
here between a "chronological" and a "thematic" approach
is what is emphasized the most: the role of the Internet in
presidential politics. Note however that more authentic
thematic reviews tend to break away from chronological
order. A review organized in this manner would shift between

95
time periods within each section according to the point
made.

Methodological
A methodological approach focuses on the methods utilized
by the researcher. For the Internet in American presidential
politics project, one methodological approach would be to
look at cultural differences between the portrayal of
American presidents on American, British, and French
websites. Or the review might focus on the fundraising impact
of the Internet on a particular political party. A
methodological scope will influence either the types of
documents in the review or the way in which these
documents are discussed.
Other Sections of Your Literature Review
Once you've decided on the organizational method for your
literature review, the sections you need to include in the
paper should be easy to figure out because they arise from
your organizational strategy. In other words, a chronological
review would have subsections for each vital time period; a
thematic review would have subtopics based upon factors
that relate to the theme or issue. However, sometimes you
may need to add additional sections that are necessary for
your study, but do not fit in the organizational strategy of the
body. What other sections you include in the body is up to
you but include only what is necessary for the reader to
locate your study within the larger scholarship framework.
Here are examples of other sections you may need to include
depending on the type of review you write:

 Current Situation: information necessary to understand

the topic or focus of the literature review.
 History: the chronological progression of the field, the
literature, or an idea that is necessary to understand

96
 the literature review, if the body of the literature review
is not already a chronology.
 Selection Methods: the criteria you used to select (and
perhaps exclude) sources in your literature review. For
instance, you might explain that your review includes
only peer-reviewed articles and journals.
 Standards: the way in which you present your
information.
 Questions for Further Research: What questions about
the field has the review sparked? How will you further
your research as a result of the review?

IV. Writing Your Literature Review

Once you've settled on how to organize your literature
review, you're ready to write each section. When writing your
review, keep in mind these issues.
Use Evidence
A literature review section is, in this sense, just like any other
academic research paper. Your interpretation of the
available sources must be backed up with evidence
[citations] that demonstrates that what you are saying is valid.

Be Selective
Select only the most important points in each source to
highlight in the review. The type of information you choose to
mention should relate directly to the research problem,
whether it is thematic, methodological, or chronological.
Related items that provide additional information but that are
not key to understanding the research problem can be
included in a list of further readings.

Use Quotes Sparingly

Some short quotes are okay if you want to emphasize a point,
or if what an author stated cannot be easily paraphrased.
Sometimes you may need to quote certain terminology that

97
was coined by the author, not common knowledge, or taken
directly from the study. Do not use extensive quotes as a
substitute for your own summary and interpretation of the
literature.

Summarize and Synthesize

Remember to summarize and synthesize your sources within
each thematic paragraph as well as throughout the review.
Recapitulate important features of a research study, but then
synthesize it by rephrasing the study's significance and
relating it to your own work.

Keep Your Own Voice

While the literature review presents others' ideas, your voice
[the writer's] should remain front and center. For example,
weave references to other sources into what you are writing
but maintain your own voice by starting and ending the
paragraph with your own ideas and wording.

Use Caution When Paraphrasing

When paraphrasing a source that is not your own, be sure to
represent the author's information or opinions accurately and
in your own words. Even when paraphrasing an author’s work,
you still must provide a citation to that work.

98
V. Common Mistakes to Avoid
These are the most common mistakes made in reviewing
social science research literature.

 Sources in your literature review do not clearly relate to

the research problem;
 You do not take sufficient time to define and identify
the most relevent sources to use in the literature review
related to the research problem;
 Relies exclusively on secondary analytical sources
rather than including relevant primary research studies
or data;
 Uncritically accepts another researcher's findings and
interpretations as valid, rather than examining critically
all aspects of the research design and analysis;
 Does not describe the search procedures that were
used in identifying the literature to review;
 Reports isolated statistical results rather than
synthesizing them in chi-squared or meta-analytic
methods; and,
 Only includes research that validates assumptions and
does not consider contrary findings and alternative
interpretations found in the literature.

Exercises:

Compare Popular and Scholarly Literature

Find the original article for a research study mentioned in the

popular press. Compare the announcement to the study.
Analyze the accuracy of, and/or the discrepancies present
in, the announcement’s summary of the research findings.
Alternatively, find a research article and write a summary
announcement suitable for the popular press (Burkhardt et
al., 2010, p. 65).

99
SLOs: 1, 2, 3 Level: Bachelor’s, Master’s

Examine Coverage of a Controversial Issue

Examine the treatment of a controversial issue in several

information sources (newspapers, scholarly journals, trade
journals, blogs, Twitter). Discuss the influences behind and
implications of the process of information creation and
dissemination. SLOs: 1, 2, 3, 4 Level: Bachelor’s Examine a
Classical Work Assemble and assess the criteria used to deem
a work a "classic" in a discipline. Analyze the effect a classical
work has on a discipline.

SLOs: 1, 2, 3, 4, 5 Level: Master’s, Doctoral

100
 Chapter 5
Understanding Methodologies

The scientific method—i.e., the method used in

dissertations—is based, in part, on the ability of others to
replicate your study. Specifically, in order to faithfully
replicate your study, other researchers must know the Who,
What, Where, When, and How of your study. While there are
nuanced differences in the details between qualitative and
quantitative studies, the methodologies to both are similar. In
short, a methodology provides a blueprint for other
researchers to follow, allowing them to conduct their own
study while using your method to arrive at similar findings. The
following excerpts provide a brief overview of how the two
approaches to methodology formation differ.

Quantitative Research Methodology

While the design and rationale of a quantitative study

requires a written section, the nuts and bolts of the
quantitative method consists of describing the participants in
the study, the instruments used, the procedure used to
administer the instruments to the participants, and the
treatment of the data, or the data analysis plan. In the
sampling procedure, the researcher must describe the
process used to select participants from the population. In
the instrument section, the researcher must cite the reliability
and validity of the instrument used, typically from previous
research studies that have used the instrument. Of course,
the researcher must also base their sample size—typically
much larger than in qualitative studies—on the statistical tests
selected in the data analysis plan. Furthermore, the
researcher details in the data analysis plan any pre-analysis
data screening, reliability of the scales, and the assumptions
that will be tested for based upon the specific statistical

101
analyses chosen. Lastly, ethical procedures include informed
consent, data storage, and other safeguards.

Qualitative Research Methodology

Research design and rationale is the how and why of

data collection and analysis in qualitative approaches (Yin,
2009). Typically students use a phenomenological, grounded
theory, or case study, for their research. As such, the
researcher plays an integral part when it comes to qualitative
research methodology design. Specifically, how the samples
are selected, including final selection of sample sizes used,
and how the data are collected are all parts of the
methodology. On a related note, the data plan needs to
explain everything in a step-by-step process— from defining
units of meaning to the extracting of themes and summarizing
of interviews. Issues of trustworthiness (i.e., credibility,
dependability, and transferability) are also described and
can be facilitated by looking at Creswell (2014) or Merriam
(2009). Ethical procedures, ramifications for choices that are
made, the well-being of participants, and any risks to
participants are described here, too.

DESCRIPTIVE METHOD

Descriptive research is defined as a research

method that describes the characteristics of the population
or phenomenon that is being studied. This methodology
focuses more on the “what” of the research subject rather
than the “why” of the research subject.

In other words, descriptive research primarily focuses on

describing the nature of a demographic segment, without
focusing on “why” a certain phenomenon occurs. In other
words, it “describes” the subject of the research, without
covering “why” it happens.
102
 For example, an apparel brand that wants to
understand the fashion purchasing trends among New York
buyers will conduct a demographic survey of this region,
gather population data and then conduct descriptive
research on this demographic segment. The research will
then uncover details on “what is the purchasing pattern of
New York buyers”, but not cover any investigative details on
“why” the patterns exits. Because for the apparel brand trying
to break into this market, understanding the nature of their
market is the objective of the study.

Characteristics of Descriptive Research

The term descriptive research then, refers to research

questions, design of the research and data analysis that
would be conducted on that topic. It is called an
observational research method because none of the
variables that are part of the research study are influenced in
any capacity.

Some distinctive characteristics of descriptive research are:

1. Quantitative research: Descriptive research is a quantitative

research method that attempts to collect quantifiable
information to be used for statistical analysis of the
population sample. It is an popular market research tool
that allows to collect and describe the nature of the
demographic segment.
2. Uncontrolled variables: In descriptive research, none of the
variables are influenced in any way. This uses observational
methods to conduct the research. Hence, the nature of the
variables or their behavior is not in the hands of the
researcher.
3. Cross-sectional studies: Descriptive research is generally
a cross-sectional study where different sections belonging
to the same group are studied.
103
4. Basis for further research: The data collected and analyzed
from descriptive research can then be further researched
using different research techniques. The data also can help
point towards the types of research methods are to be
used for the subsequent research.

Applications of Descriptive Research with Examples

Descriptive research can be used in multiple ways and

for multiple reasons. Before getting into any kind of survey
though, the survey goals and survey design is very important.
Despite following these steps though, there is no way to know
if the research outcome will be met. To understand the end
objective of research goals, below are some ways
organizations currently use descriptive research today:

 Define respondent characteristics: The aim of using close-

ended questions is to draw concrete conclusions about the
respondents. This could be the need to derive patterns,
traits and behaviors of the respondents. It could also be to
understand from a respondent, their attitude or opinion
about the phenomenon in question. For example,
understanding from millennials the hours per week they
spend on browsing the internet. All this information helps the
organization conducting the research make informed
business decisions.
 Measure data trends: Data trends can be measured over
time with statistical capabilities provided by descriptive
research. Consider if an apparel company conducts
research between different demographics like age groups
from 24-35 and 36-45 on a new range launch of autumn
wear. If one of those groups doesn’t take too well to the
new launch, this provides an insight into what clothes are
like and what are not and the ones that are not, are
dropped.

104
 Conduct comparisons: Organizations also use descriptive
research to understand how different groups respond to a
certain product or service. For example, an apparel brand
creates a survey asking general questions that measure the
brands image. The same survey also asks demographic
questions like age, income, gender, geographical
location etc. This consumer research helps the organization
understand what aspects of the brand appeal to the
population and what aspects do not. It also helps in making
product or marketing fixes or in some cases even create a
new product line just to cater to a high growth potential,
group.

 Validate existing conditions: Descriptive research is widely

used to help ascertain the prevailing conditions and
underlying patterns of the research object. Due to the
noninvasive method of research and the use of
quantitative observation and some aspects of qualitative
observation, each variable is observed and an in-depth
analysis can be concluded. It is also used to validate any
existing conditions that maybe prevalent in a population.

 Conduct research at different times: To ascertain if there

are any similarities or differences, the research can be
conducted at different periods of times. This also allows any
number of variables to be evaluated. For the purpose of
verification, studies on prevailing conditions can also be
repeated to draw trends.

Descriptive Research Methods

There are 3 distinctive methods to conduct descriptive

research. They are:

 Observational Method

105
 The observational method is the most effective
method to conduct descriptive research and both
quantitative observation and qualitative observation are
used in this research method.

Quantitative observation is the objective collection of

data which is primarily focused on numbers and values – it
suggests “associated to, of or depicted in terms of a
quantity”. Results of quantitative observation are derived
using statistical and numerical analysis methods. It implies
observation of any entity that can be associated with a
numeric value such as age, shape, weight, volume, scale etc.
For example, the researcher can track if current customers will
refer the brand by using a simple Net Promoter
Score question.

Qualitative observation doesn’t involve measurements

or numbers but instead just monitoring characteristics. In this
case the researcher observes the respondents from a
distance. Since the respondents are in a comfortable
environment, the characteristics observed are natural and
effective. In descriptive research, the researcher can chose
to be either a complete observer, an observer as a
participant, a participant as an observer or a complete
participant. For example, in a supermarket, a researcher can
from afar monitor and track the selection and purchasing
trends of the customers. This offers a deeper insight into the
purchasing experience of the customer.

 Case Study Method

Case studies involve in-depth research and study of
individuals or groups. Case studies lead to a hypothesis and
widen a further scope of studying a phenomenon. However,
case studies should not be used to determine cause and
effect as they don’t have the capacity to make accurate
predictions because there could be a bias on the part of the
106
 researcher. The other reason why case studies are not an
accurate way of conducting descriptive research is because
there could be an atypical respondent in the research and
describing them leads to poor generalizations and move
away from external validity.

 Survey Research
In survey research, respondents answer
through surveys or questionnaires, or polls. They are a popular
market research tool to collect feedback from respondents.
In order for a survey to gather good quality data, it should
have good survey questions, which should be a balanced
mix of open-ended questions and close ended-questions. The
survey method can be conducting online or offline which is
makes it the go-to option for descriptive research where
the sample size is very large.

EXPERIMENTAL METHOD

The word experimental research has a range of

definitions. In the strict sense, experimental research is what
we call a true experiment.
This is an experiment where the
researcher manipulates one variable,
and control/randomizes the rest of the variables. It has
a control group, the subjects have been randomly assigned
between the groups, and the researcher only tests one effect
at a time. It is also important to know what variable(s) you
want to test and measure.

A very wide definition of experimental research, or

a quasi experiment, is research where the scientist actively
influences something to observe the consequences. Most
experiments tend to fall in between the strict and the wide
definition.

107
 A rule of thumb is that physical sciences, such as
physics, chemistry and geology tend to define experiments
more narrowly than social sciences, such as sociology and
psychology, which conduct experiments closer to the wider
definition.

Aims of Experimental Research

Experiments are conducted to be able to predict

phenomena. Typically, an experiment is constructed to be
able to explain some kind of causation. Experimental
research is important to society - it helps us to improve our
everyday lives.

Typical Designs and Features in Experimental Design

 Pretest-Posttest Design
Check whether the groups are different before the
manipulation starts and the effect of the manipulation.
Pretests sometimes influence the effect.
 Control Group
Control groups are designed to measure research bias and
measurement effects, such as the Hawthorne Effect or
the Placebo Effect. A control group is a group not receiving
the same manipulation as the experimental group.
Experiments frequently have 2 conditions, but rarely more
than 3 conditions at the same time.
 Randomized Controlled Trials
Randomized Sampling, comparison between an
Experimental Group and a Control Group and strict
control/randomization of all other variables
 Solomon Four-Group Design
With two control groups and two experimental groups. Half
the groups have a pretest and half do not have a pretest.

108
 This to test both the effect itself and the effect of the
pretest.
 Between Subjects Design
Grouping Participants to Different Conditions
 Within Subject Design
Participants Take Part in the Different Conditions - See
also: Repeated Measures Design
 Counterbalanced Measures Design
Testing the effect of the order of treatments when no
control group is available/ethical
 Matched Subjects Design
Matching Participants to Create Similar Experimental- and
Control-Groups
 Double-Blind Experiment
Neither the researcher, nor the participants, know which is
the control group. The results can be affected if the
researcher or participants know this.
 Bayesian Probability
Using bayesian probability to "interact" with participants is a
more "advanced" experimental design. It can be used for
settings were there are many variables which are hard to
isolate. The researcher starts with a set of initial beliefs, and
tries to adjust them to how participants have responded

Pilot Study

It may be wise to first conduct a pilot-study or two

before you do the real experiment. This ensures that the
experiment measures what it should, and that everything is
set up right.

Minor errors, which could potentially destroy the

experiment, are often found during this process. With a pilot
study, you can get information about errors and problems,
and improve the design, before putting a lot of effort into the
real experiment.

109
 If the experiments involve humans, a common strategy
is to first have a pilot study with someone involved in the
research, but not too closely, and then arrange a pilot with a
person who resembles the subject(s). Those two different
pilots are likely to give the researcher good information about
any problems in the experiment.

Conducting the Experiment

An experiment is typically carried out by manipulating

a variable, called the independent variable, affecting the
experimental group. The effect that the researcher is
interested in, the dependent variable(s), is measured.

Identifying and controlling non-experimental factors

which the researcher does not want to influence the effects,
is crucial to drawing a valid conclusion. This is often done
by controlling variables, if possible, or randomizing variables
to minimize effects that can be traced back to third
variables. Researchers only want to measure the effect of the
independent variable(s) when conducting an experiment,
allowing them to conclude that this was the reason for the
effect.

Analysis and Conclusions

In quantitative research, the amount of data measured

can be enormous. Data not prepared to be analyzed is
called "raw data". The raw data is often summarized as
something called "output data", which typically consists of
one line per subject (or item). A cell of the output data is, for
example, an average of an effect in many trials for a subject.
The output data is used for statistical analysis, e.g.
significance tests, to see if there really is an effect.

110
 The aim of an analysis is to draw a conclusion, together
with other observations. The researcher might generalize the
results to a wider phenomenon, if there is no indication
of confounding variables "polluting" the results.

If the researcher suspects that the effect stems from a

different variable than the independent variable, further
investigation is needed to gauge the validity of the results. An
experiment is often conducted because the scientist wants to
know if the independent variable is having any effect upon
the dependent variable. Variables correlating are not proof
that there is causation.

Experiments are more often of quantitative nature

than qualitative nature, although it happens.

111
ETHNOGRAPHY

Ethnography can help investigate very complicated

or critical design challenges. A good researcher is essential
when observing and/or interacting with target audiences in
their real-life environment.
What is ethnographic research?

Ethnographic research is a qualitative method

where researchers observe and/or interact with a study’s
participants in their real-life environment. Ethnography was
popularized by anthropology, but is used across a wide
range of social sciences.

Within the field of usability, user-centered design

and service design, ethnography is used to support
a designer’s deeper understanding of the design problem –
including the relevant domain, audience(s), processes,
goals and context(s) of use.

The aim of an ethnographic study within a usability

project is to get ‘under the skin’ of a design problem (and
all its associated issues). It is hoped that by achieving this, a
designer will be able to truly understand the problem and
therefore design a far better solution.

Methods associated with ethnography

Anthropological ethnographers often live amongst a

group/society for a year or more, in order to learn about
them. This fully immersive, long-term ‘live and work’
approach to ethnography has not proven popular within
the field of usability.

112
 Part of the reason may involve cost, but it is also the
case that anthropologists and usability practitioners are
interested in different things. Anthropologists use
ethnography in an attempt to fully understand as much as
possible about an entire society. Usability practitioners are
usually only interested in learning information that will
support their reasoning on a specific design problem.

We would argue that deep, immersive ‘live and work’

ethnography is rarely required within the field of user-
centered design. However, short ethnographic studies can
be very useful for user-centered projects.

For example: in order to understand the way in which

a Merchant Bank trades and operates, a usability
consultant might conduct an ethnographic study by
working and socializing with its employees for a month.

Individual methods which are available within an

ethnographic study include: participant observation,
interviews and surveys. All of these ethnographic methods
can be very valuable in gaining a deeper understanding of
a design problem. Usability practitioners often make use of
these in order to develop their understanding of the
relevant domain, audience(s), processes, goals and
context(s) of use.

When to use ethnography

Ethnography is most useful in the early stages of a

user-centred design project. This is because ethnography
focuses on developing an understanding of the design
problem. Therefore, it makes more sense to conduct
ethnographic studies at the beginning of a project in order

113
to support future design decisions (which will happen later in
the user-centered design process).

Ethnographic methods (such as participant

observation) could also be used to evaluate an existing
design – but their true value comes from developing an
early understanding of the relevant domain, audience(s),
processes, goals and context(s) of use.

We would normally recommend that ethnographic

methods are used for very complex and/or critical design
problems. More complex design problems (in terms of their
domain, audience(s), processes, goals and/or context(s) of
use) are likely to need the deeper understanding which
ethnographic studies can bring. Equally, highly critical
systems (where failure or error can lead to disaster) could
also justify significant ethnographic research.

For example: An insurance company wanted to re-

design their system dealing with the processing of insurance
claims. This system had evolved over many years and
actually represented a patchwork of previous systems. The
‘claim processing’ supported by this ‘system of systems’ is
itself a highly complex process. In this example,
ethnographic research should probably be considered.

Advantages of ethnography

One of the main advantages associated with

ethnographic research is that ethnography can help identify
and analyzes unexpected issues. When conducting other
types of studies, which are not based on in-situ observation
or interaction, it can very easy to miss unexpected issues.
This can happen either because questions are not asked, or
respondents neglect to mention something. An
114
ethnographic researcher’s in-situ presence helps mitigate
this risk because the issues will (hopefully) become directly
apparent to the researcher.

Ethnography’s other main benefit is generally

considered to be its ability to deliver a detailed and faithful
representation of users’ behaviors and attitudes. Because of
its subjective nature, an ethnographic study (with a skilled
researcher) can be very useful in uncovering and analyzing
relevant user attitudes and emotions.

Disadvantages of ethnography

One of the main criticisms levelled at ethnographic studies is

the amount of time they take to conduct. As discussed
above, ethnographic studies do not always require a long
period of time, but this consideration is nonetheless valid.
Because of its richer output, an ethnographic study will tend
to take longer to generate and analyze its data than many
other methods.

During previous ethnographic studies, we have found

that it is possible that subjects may not act naturally during
a short study. Longer studies normally counter-act this
because the subjects grow to trust the researcher and/or
get tired of any pretense.

For example: During the first week of an ethnographic

study into an insurance claim processing system, all the
subjects were observed to be following the strictest
interpretation of the correct procedures. As time
progressed, however, it became increasingly apparent that
almost all employees had ‘work-arounds’ and ‘short cuts’
which were liberally used in order to speed things up. These
behaviors were very instructive in helping to re-design the
115
process flow. Had the researcher not stayed in-situ long
enough to observe these, they may have gone unrecorded.

Risks associated with ethnography

As stated above, ethnographic studies consist of the

researcher observing and/or interacting with subjects within
the environment which the (future) design is intended to
support. The two main potential weaknesses with
ethnographic studies are:

Researcher

Ethnographic researchers need to be very highly-

skilled to avoid all the potential pitfalls of an ethnographic
study. Some of these include the detail & completeness of
observations, as well as potential bias (and mistakes) in data
collection or analysis.

Subjects

It is essential that any studies’ subjects are as true a

representation of the larger user audience as possible
(assuming that the study has been designed this way). It is
also vital that the subjects are open and honest with the
researcher. Of course, both of these issues are related to the
quality of the researcher themselves and their role in the
study’s design.
As we can see from the above, most of the risks associated
with ethnographic studies relate to the researcher, either
directly or indirectly. This, of course, means that the choice
of ethnographic researcher is critical to a study’s success.
We recommend choosing a researcher with a proven
background of past involvement in successful projects
across varying domains.
116
CASE STUDY

Basically, a case study is an in depth study of a

particular situation rather than a sweeping statistical survey. It
is a method used to narrow down a very broad field of
research into one easily researchable topic.

Whilst it will not answer a question completely, it will

give some indications and allow further elaboration
and hypothesis creation on a subject.

The case study research design is also useful for testing

whether scientific theories and models actually work in the
real world. You may come out with a great computer model
for describing how the ecosystem of a rock pool works but it is
only by trying it out on a real life pool that you can see if it is a
realistic simulation.

For psychologists, anthropologists and social scientists

they have been regarded as a valid method of research for
many years. Scientists are sometimes guilty of becoming
bogged down in the general picture and it is sometimes
important to understand specific cases and ensure a more
holistic approach to research.

The Argument for and Against the Case Study Research

Design

Some argue that because a case study is such a

narrow field that its results cannot be extrapolated to fit an
entire question and that they show only one narrow example.
On the other hand, it is argued that a case study provides
more realistic responses than a purely statistical survey.
The truth probably lies between the two and it is probably
best to try and synergize the two approaches. It is valid to

117
conduct case studies but they should be tied in with more
general statistical processes.

For example, a statistical survey might show how much

time people spend talking on mobile phones, but it is case
studies of a narrow group that will determine why this is so.

The other main thing to remember during case studies is

their flexibility. Whilst a pure scientist is trying to prove or
disprove a hypothesis, a case study might introduce new and
unexpected results during its course, and lead to research
taking new directions.

The argument between case study and statistical

method also appears to be one of scale. Whilst many
'physical' scientists avoid case studies, for psychology,
anthropology and ecology they are an essential tool. It is
important to ensure that you realize that a case study cannot
be generalized to fit a whole population or ecosystem.

Finally, one peripheral point is that, when informing

others of your results, case studies make more interesting
topics than purely statistical surveys, something that has been
realized by teachers and magazine editors for many years.
The general public has little interest in pages of statistical
calculations but some well-placed case studies can have a
strong impact.

How to Design and Conduct a Case Study

The advantage of the case study research design is

that you can focus on specific and interesting cases. This may
be an attempt to test a theory with a typical case or it can
be a specific topic that is of interest. Research should be
thorough and note taking should be meticulous and
systematic.

118
The first foundation of the case study is the subject and
relevance. In a case study, you are deliberately trying to
isolate a small study group, one individual case or one
particular population.

For example, statistical analysis may have shown that

birthrates in African countries are increasing. A case study on
one or two specific countries becomes a powerful and
focused tool for determining the social and economic
pressures driving this.

In the design of a case study, it is important to plan and

design how you are going to address the study and make
sure that all collected data is relevant. Unlike a scientific
report, there is no strict set of rules so the most important part
is making sure that the study is focused and concise;
otherwise you will end up having to wade through a lot of
irrelevant information.

It is best if you make yourself a short list of 4 or 5 bullet

points that you are going to try and address during the study.
If you make sure that all research refers back to these then
you will not be far wrong.

With a case study, even more than a questionnaire

or survey, it is important to be passive in your research. You
are much more of an observer than an experimenter and you
must remember that, even in a multi-subject case, each case
must be treated individually and then cross case conclusions
can be drawn.

How to Analyze the Results

Analyzing results for a case study tends to be more

opinion based than statistical methods. The usual idea is to try

119
and collate your data into a manageable form and construct
a narrative around it.

Use examples in your narrative whilst keeping things

concise and interesting. It is useful to show some numerical
data but remember that you are only trying to judge trends
and not analyze every last piece of data. Constantly refer
back to your bullet points so that you do not lose focus.

It is always a good idea to assume that a person

reading your research may not possess a lot of knowledge of
the subject so try to write accordingly.
In addition, unlike a scientific study which deals with facts, a
case study is based on opinion and is very much designed to
provoke reasoned debate. There really is no right or wrong
answer in a case study.

HISTORICAL METHOD

Historical research involves studying, understanding

and interpreting past events. The purpose of historical
research is to reach insights or conclusions about past persons
or occurrences. Historical research entails more than simply
compiling and presenting factual information; it also requires
interpretation of the information.

Typically, histories focus on particular individuals, social

issues and links between the old and the new. Some
historical research is aimed at reinterpreting prior historical
works by revising existing understandings and replacing them
with new, often politically charged ones.

The main emphasis in historical research is on

interpretation of documents, diaries and the like. Historical
data are categorized into primary or secondary
sources. Primary sources include firsthand information, such
120
as eyewitness reposts and original documents. Secondary
sources include secondhand information, such as a
description of an event by someone other than an
eyewitness, or a textbook author’s explanation of an event or
theory. Primary sources may be harder to find but are
generally more accurate and preferred by historical
researchers. A major problem with much historical research is
excessive reliance on secondary sources.

Researches cannot accept historical data at face

value, since many diaries memoirs, reposts and testimonies
are written to enhance the writer’s position, stature, or
importance. Because of this possibility, historical data has to
be examined for its authenticity and truthfulness. Such
examination is done through criticism; by asking and
researching to help determine truthfulness, bias, omissions
and consistency in data.

121
KNOW YOUR LIBRARY

Get to know your library (or libraries) before you begin

researching. It is very important to familiarize yourself with the
many resources available at the library. Time spent initially will
save more in the long run. You should find out what
references are available and where they are located. In
starting research you will have to examine a range of sources
that are pertinent to your topic.

GETTING STARTED
o Narrow initial search to encyclopedias, journals or
articles
o Make a list of keywords to guide your literature search
o Use titles and authors in searches
o Always record or photocopy bibliographic references
o Conduct keyword searches
o Search for books in library
o Consult computer databases to locate journal articles,
reports, and other publications
o Search internet for up to date information
o A useful way to narrow or broaden a keyword search is
to use AND, OR and NOT connectors

PROGRAM EVALUATION

Some Myths About Program Evaluation

1. Many people believe evaluation is a useless activity that

generates lots of boring data with useless conclusions. This
was a problem with evaluations in the past when program
evaluation methods were chosen largely on the basis of
achieving complete scientific accuracy, reliability and
validity. This approach often generated extensive data from
which very carefully chosen conclusions were drawn.
122
Generalizations and recommendations were avoided. As a
result, evaluation reports tended to reiterate the obvious and
left program administrators disappointed and skeptical about
the value of evaluation in general. More recently (especially
as a result of Michael Patton's development of utilization-
focused evaluation), evaluation has focused on utility,
relevance and practicality at least as much as scientific
validity.

2. Many people believe that evaluation is about proving the

success or failure of a program. This myth assumes that
success is implementing the perfect program and never
having to hear from employees, customers or clients again --
the program will now run itself perfectly. This doesn't happen
in real life. Success is remaining open to continuing feedback
and adjusting the program accordingly. Evaluation gives you
this continuing feedback.

3. Many believe that evaluation is a highly unique and

complex process that occurs at a certain time in a certain
way, and almost always includes the use of outside experts.
Many people believe they must completely understand terms
such as validity and reliability. They don't have to. They do
have to consider what information they need in order to
make current decisions about program issues or needs. And
they have to be willing to commit to understanding what is
really going on. Note that many people regularly undertake
some nature of program evaluation -- they just don't do it in a
formal fashion so they don't get the most out of their efforts or
they make conclusions that are inaccurate (some evaluators
would disagree that this is program evaluation if not done
methodically). Consequently, they miss precious opportunities
123
to make more of difference for their customer and clients, or
to get a bigger bang for their buck.

So What is Program Evaluation?

First, we'll consider "what is a program?" Typically,

organizations work from their mission to identify several overall
goals which must be reached to accomplish their mission. In
nonprofits, each of these goals often becomes a program.
Nonprofit programs are organized methods to provide certain
related services to constituents, e.g., clients, customers,
patients, etc. Programs must be evaluated to decide if the
programs are indeed useful to constituents. In a for-profit, a
program is often a one-time effort to produce a new product
or line of products.

So, still, what is program evaluation? Program

evaluation is carefully collecting information about a
program or some aspect of a program in order to make
necessary decisions about the program. Program evaluation
can include any or a variety of at least 35 different types of
evaluation, such as for needs assessments, accreditation,
cost/benefit analysis, effectiveness, efficiency, formative,
summative, goal-based, process, outcomes, etc. The type of
evaluation you undertake to improve your programs depends
on what you want to learn about the program. Don't worry
about what type of evaluation you need or are doing -- worry
about what you need to know to make the program
decisions you need to make, and worry about how you can
accurately collect and understand that information.

Where Program Evaluation is Helpful

124
Frequent Reasons

Program evaluation can:

1. Understand, verify or increase the impact of products or
services on customers or clients - These "outcomes"
evaluations are increasingly required by nonprofit funders as
verification that the nonprofits are indeed helping their
constituents. Too often, service providers (for-profit or
nonprofit) rely on their own instincts and passions to conclude
what their customers or clients really need and whether the
products or services are providing what is needed. Over time,
these organizations find themselves in a lot of guessing about
what would be a good product or service, and trial and error
about how new products or services could be delivered.

2. Improve delivery mechanisms to be more efficient and less

costly - Over time, product or service delivery ends up to be
an inefficient collection of activities that are less efficient and
more costly than need be. Evaluations can identify program
strengths and weaknesses to improve the program.

3. Verify that you're doing what you think you're doing -

Typically, plans about how to deliver services, end up
changing substantially as those plans are put into place.
Evaluations can verify if the program is really running as
originally planned.

Other Reasons
Program evaluation can:

125
4. Facilitate management's really thinking about what their
program is all about, including its goals, how it meets it goals
and how it will know if it has met its goals or not.

5. Produce data or verify results that can be used for public

relations and promoting services in the community.

6. Produce valid comparisons between programs to decide

which should be retained, e.g., in the face of pending
budget cuts.

7. Fully examine and describe effective programs for

duplication elsewhere.

Basic Ingredients: Organization and Program(s)

You Need an Organization:

This may seem too obvious to discuss, but before an
organization embarks on evaluating a program, it should
have well established means to conduct itself as an
organization, e.g., (in the case of a nonprofit) the board
should be in good working order, the organization should be
staffed and organized to conduct activities to work toward
the mission of the organization, and there should be no
current crisis that is clearly more important to address than
evaluating programs.

You Need Program(s):

To effectively conduct program evaluation, you should first
have programs. That is, you need a strong impression of what
your customers or clients actually need. (You may have used
a needs assessment to determine these needs -- itself a form
126
of evaluation, but usually the first step in a good marketing
plan). Next, you need some effective methods to meet each
of those goals. These methods are usually in the form of
programs.

It often helps to think of your programs in terms of inputs,

process, outputs and outcomes. Inputs are the various
resources needed to run the program, e.g., money, facilities,
customers, clients, program staff, etc. The process is how the
program is carried out, e.g., customers are served, clients are
counseled, children are cared for, art is created, association
members are supported, etc. The outputs are the units of
service, e.g., number of customers serviced, number of clients
counseled, children cared for, artistic pieces produced, or
members in the association. Outcomes are the impacts on
the customers or on clients receiving services, e.g., increased
mental health, safe and secure development, richer artistic
appreciation and perspectives in life, increased effectiveness
among members, etc.

127
 Planning your program evaluation depends on what
information you need to make your decisions and on your
resources.

Often, management wants to know everything about

their products, services or programs. However, limited
resources usually force managers to prioritize what they need
to know to make current decisions.

Your program evaluation plans depend on what

information you need to collect in order to make major
decisions. Usually, management is faced with having to make
major decisions due to decreased funding, ongoing
complaints, unmet needs among customers and clients, the
need to polish service delivery, etc. For example, do you
want to know more about what is actually going on in your
programs, whether your programs are meeting their goals,
the impact of your programs on customers, etc? You may
want other information or a combination of these. Ultimately,
it's up to you.

But the more focused you are about what you want to
examine by the evaluation, the more efficient you can be in
your evaluation, the shorter the time it will take you and
ultimately the less it will cost you (whether in your own time,
the time of your employees and/or the time of a consultant).

There are tradeoffs, too, in the breadth and depth of

information you get. The more breadth you want, usually the
less depth you get (unless you have a great deal of resources
to carry out the evaluation). On the other hand, if you want
128
to examine a certain aspect of a program in great detail, you
will likely not get as much information about other aspects of
the program.

For those starting out in program evaluation or who

have very limited resources, they can use various methods to
get a good mix of breadth and depth of information. They
can both understand more about certain areas of their
programs and not go bankrupt doing so.

Key Considerations:
Consider the following key questions when designing a
program evaluation.

1. For what purposes is the evaluation being done, i.e., what

do you want to be able to decide as a result of the
evaluation?
2. Who are the audiences for the information from the
evaluation, e.g., customers, bankers, funders, board,
management, staff, customers, clients, etc.
3. What kinds of information are needed to mfake the
decision you need to make and/or enlighten your intended
audiences, e.g., information to really understand the process
of the product or program (its inputs, activities and outputs),
the customers or clients who experience the product or
program, strengths and weaknesses of the product or
program, benefits tof customers or clients (outcomes), how
the product or program failed and why, etc.
4. From what sources should the information be collected,
e.g., employees, customers, clients, groups of customers or
clients and employees together, program documentation,
etc.
129
5. How can that information be collected in a reasonable
fashion, e.g., questionnaires, interviews, examining
documentation, observing customers or employees,
conducting focus groups among customers or employees,
etc.
6. When is the information needed (so, by when must it be
collected)?
7. What resources are available to collect the information?

Exercise
Compare Databases
Students should select two databases (SuperSearch, Google
Scholar, or a subject-specific option) and explore the
coverage, currency, journal titles, search interfaces, and
features of the two. Using think/pair/share or a reflective
paper, student can discuss how these databases might
directly or indirectly influence their research (Burkhardt et al.,
2010, p. 77).
SLOs: 2, 3/ Level: Bachelor’s, Master’s, Doctoral

130
 Chapter 6
Gathering of Data Techniques

Under the main three basic groups of research

methods (quantitative, qualitative and mixed), there are
different tools that can be used to collect data. Interviews
can be done either face-to-face or over the phone.
Surveys/questionnaires can be paper or web based.
Observations and experiments can be conducted to collect
either quantitative, qualitative or a mixture of the two
methods. Records can also be used to study previous
information by other researchers.

Tips:

 Organize collected data as soon as it is available

 Begin with the end in mind - know what message you
want to get across and then collect data that is
relevant to the message
 Collect more data
 Create more data
 Regularly run experiments or collect data
 Challenge your assumptions
 Set reasonable expectations
 Take note of interesting or significant data
 Quantity is good but quality is even better

SAMPLING PROCEDURE

What are sampling methods?

In a statistical study, sampling methods refer to how we

select members from the population to be in the study.

131
 If a sample isn't randomly selected, it will probably be
biased in some way and the data may not be representative
of the population.

There are many ways to select a sample—some good

and some bad.

Bad ways to sample

Convenience sample: The researcher chooses a

sample that is readily available in some non-random way.
Example—A researcher polls people as they walk by on the
street.
Why it's probably biased: The location and time of day and
other factors may produce a biased sample of people.

Voluntary response sample: The researcher puts out a

request for members of a population to join the sample, and
people decide whether or not to be in the sample.
Example—A TV show host asks his viewers to visit his website
and respond to an online poll.
Why it's probably biased: People who take the time to
respond tend to have similarly strong opinions compared to
the rest of the population

Good ways to sample

Simple random sample: Every member and set of

members has an equal chance of being included in the
sample. Technology, random number generators, or some
other sort of chance process is needed to get a simple
random sample.

Example—A teachers puts students' names in a hat and

chooses without looking to get a sample of students.
Why it's good: Random samples are usually fairly
representative since they don't favor certain members.
132
 Stratified random sample: The population is first split into
groups. The overall sample consists of some members from
every group. The members from each group are chosen
randomly.

Example—A student council surveys 100100 students by

getting random samples
of 2525 freshmen, 2525 sophomores, 2525 juniors,and 2525 sen
iors.

Why it's good: A stratified sample guarantees that

members from each group will be represented in the sample,
so this sampling method is good when we want some
members from every group.

Cluster random sample: The population is first split into

groups. The overall sample consists of every member from
some of the groups. The groups are selected at random.

Example—An airline company wants to survey its

customers one day, so they randomly select 55 flights that
day and survey every passenger on those flights.

Why it's good: A cluster sample gets every member

from some of the groups, so it's good when each group
reflects the population as a whole.

Systematic random sample: Members of the

population are put in some order. A starting point is selected
at random, and every n^{\text{th}}nth member is selected to
be in the sample.
Example—A principal takes an alphabetized list of
student names and picks a random starting point.
Every 20^{\text{th}}20th student is selected to take a survey.

133
RESEARCH TOOLS

Experiment

The most systematic technique of data collection is

laboratory experiment, where it is hoped all variables will be
able to be controlled by the researcher. Since physics and
chemistry (including biochemistry) have gone the farthest in
developing the experimental "method", they are (sometimes
inappropriately) taken as the gold standard of what science
has to be like. Although it should be obvious that great strides
have been made in many of the other physical sciences
where laboratory experiment is not possible such as
astronomy, geology and many areas of biology, the social
sciences are sometimes criticized as not being real sciences
because they are not based enough on experiment,
particularly lab experiment. While small issues of human social
behavior can be studied in the artificial setting of a
laboratory, attempts to study larger issues have frequently
resulted in results that are very controversial because it is not
completely clear that all of the variables were actually either
being controlled or even measured.

While variables sometimes can be manipulated in a

"field" setting, field experiments can be contaminated by
factors the experimenter is unaware of.

Strengths:
 Provides numerical results in most cases.
 Focus on trivial behaviors can yield results that are of
low importance to the subject, resulting in more natural
behavior.
 Careful matching of experimental and control group
members can give increased confidence in the results.
 Generally replicable.

134
Weaknesses:
 Ethical guidelines prevent many experiments. Informed
consent of the subjects required by scientific societies,
universities and funding agencies alerts the subjects
and puts them on their guard.
 Humans are very self-conscious when they know they
are being observed and may intentionally or
unintentially alter their behavior.
 The higher the "risk" in the experiment the higher the
self-consciousness.
 In trying to control the situation, the behaviors that are
being measured may become so limited that there is
little point to the experiment.
 Failure to clearly isolate and manipulate variables
between a control and an experimental group can
give an illusion of an effect.
 The experimental subjects may not properly reflect the
population and therefore prevent generalization of the
experimental results to the population.

Survey

Surveys are any form of questioning of a sample of

people with hopes of getting an insight into a larger
population of people. Since people are either being asked to
fill out a questionnaire, or are being asked by a door-to-door
interviewer or by someone on the telephone, or responding
to questions from a computer, surveys usually depend on self-
reporting of the data by the subject. In the past door-to-door
or phone interviewers were often instructed to fill out some
information by observation, such as age, gender or "race", or
impressions of the subject while responding: slow? hesitant?
answer with a rising tone? facial responses such as looking
down or into the eyes of the interviewer? Trained interviewers
also can easily follow a branching survey. With today's
phone-robot and online surveys there is very little door-to-

135
door surveying done and the phone robots cannot usually
discern gender or speaking accent. However, computers and
phone robot interviewers can time responses and note
changes to answers, and phone robots could (I do not know
whether they do) register rising tone answers. And
computerized surveys can easily be programmed to branch
depending on the previous answers.

Since surveys almost always deal with a sample, one of

the most critical issues is to understand the sample. Many
people mistakenly believe the larger the sample the more
likely it is to be random, but the Literary Digest Presidential Poll
of 1936 is a good demonstration that that is not true. In that
election George Gallup stunned everyone when he correctly
predicted the outcome of the election with a much smaller,
but well constructed and understood, sample. Since the
1960's national elections have been successfully predicted
with samples of ~1200. In addition, other national surveys of
opinion or self-reported behavior have been successfully
studied with similar sample sizes. Ironically, samples of local
and state-level opinions or behavior usually need samples
nearly as large, 500-1000.

Small samples frequently are stratified, which means

that the less numerous categories of people in the population
are oversampled to insure they are included, while larger
categories such as white men are undersampled because
they are much more likely to be included even if
undersampled. To compute numerical values for the different
answers the answers of the overcounted categories and
undercounted categories are multiplied by a weighting
factor to come closer to the population projection.

In addition, small samples are usually done on a forced

basis, which means the subjects have been selected to
represent certain characteristics and if the first person

136
meeting those characteristics declines to participate, s/he will
be replaced by someone else with the same characteristics.
For example, since place of residence is frequently strongly
correlated with other social factors such as ethnicity, income,
education, age, etc., addresses are sometimes chosen as
means of getting respondents fitting a certain set of
characteristics. The instructions to the door-to-door (d2d)
interviewer might be to interview the residents of the house
on the northwest corner of each block in a particular
neighborhood. If no one is home or they refuse to participate,
go 4 doors to the left until success is achieved. For computer
or telephone interviews, phone numbers are often chosen
because they are in particular neighborhoods, so the same
algorithm can be used.

Strengths:
 Anonymity can encourage people to be more
forthcoming than in an experiment or other kind of
direct observation
 Easy to analyze with modern data management and
analysis software. Some analysis tools are usually
installed on most PCs.
 Relatively cheap to mail out questionnaires or one-time
keys to take the survey online.
 Even forced sampling is more easily accomplished with
computers using auto dialing routines.
 Interviews by humans or computers can be better than
questionnaires in capturing information about how the
respondents reacted to the individual questions.
 As we lose privacy with the increase in commercial and
governmental data mining it will be increasingly easy to
collect behavior without the knowledge or consent of
the people being studied.
 If anonymity is assured, there are relatively few ethical
issues in scientific polling. Political push polling is another
matter entirely.

137
Weaknesses:
 Regardless of perceived anonymity people do, both
willingly and unwillingly, misreport their characteristics or
behavior to be more acceptable than they actually
are.
 People do try to figure out what you are trying to get at
and may be influenced by that in their answers.
 Particularly with questionnaires that are distributed
through the mail, even though the initial sample to
whom things were mailed may be either properly
random or properly stratified, the sampling may be
disrupted by differential response rates by different sorts
of people. For example, on politically charged topics
people who feel strongly one way or the other
participate more heavily than do the people in the
middle who are usually more numerous. On lifestyle
issues people whose behavior may seem extreme or
radical compared to the norm are sometimes more
likely to fill out the questionnaire.
 Interviews, which are more likely to used forced
sampling to maintain randomness or structure of the
sample even in the face of people who decline to
participate, are more expensive since they involve
more members of the team who have to trained and
frequently paid to conduct the interviews. D2d
interviews are even more time consuming and thus
more expensive.
 Ultimately, even when reporting behavior, surveys are
more about thoughts and ideas than the actual
behavior.
 As we lose privacy with the increase in commercial
data mining our behavior will increasingly be reported
without our knowledge or consent.

Archival Research

138
 In the past this was thought to be primarily the domain
of historians, but increasingly after the mid-1900's, some
historians themselves began to think of their discipline as a
kind of social science, and historical data began to be used
by sociologists, anthropologists, economists, political scientists
and cultural geographers. People often think of the past as
something that is easily reconstructed from written records,
but the cross fertilization of all these fields raised such basic
questions as "What is a document of the past?" Folktales?
Myths? Legends? Oral histories? Architecture? Art? "Why are
some things from the past preserved and others are not?"
Might behavior in the past be revealed by looking in new
ways for remains of that past behavior?

139
 As a basic example, historians and social scientists from
sociology and anthropology had for a long time assumed
that the modern nuclear family, increasingly today becoming
a broken nuclear family, was a result of the breakdown of
traditional society caused by the industrial revolution.

Historians' views of families of the past were distorted by

their past concentration on the history of the elites, the ones
who wrote letters, kept diaries and were frequently
documented by others. Even the artifacts produced for and
used by the elites are more likely to survive over the years
because of their quality, value and historical significance.
Such families tended to be multigenerational and even to be
extended with several married children sharing country
estates and elaborate urban residences, depending on the
season.

Sociologists' views were distorted by their tendency to

focus on contemporary social problems of whatever period
the individual sociologist was living in. Comparing the
sometimes broken families of the mid to late 1800s which had
been studied at that time, to the increasingly broken families
of the 20th century without adequate detailed attention to
the histories of those poorer people before industrialization
led sociologists to infer a regression line back to an imagined
state of the peasants living in the past in families like those of
the wealthy.

Anthropologists' views were distorted by their tendency

to treat the aboriginal people in all of the conquered parts of
the world that the Europeans had colonized in the Age of
Exploration and the Industrial Revolution as though those
natives had been living from time immemorial in the manner
in which they appeared to the Europeans in the 18th and
19th centuries.

140
 All three of these disciplines began to wake up at
about the same time that the history of these "people without
history" was perhaps very different from what had been
assumed based on the history of "people with history."

The problem was how to recover the history where

there seemed to be none? Social scientists began to look for
different documents and for previously overlooked
information in documents that had previously been used to
study the behavior of the elites. For example, travel passes
issued by slave owners to slaves to visit neighboring
plantations, followed by transfers of slaves between
plantation owners which had not been looked at seriously
before, revealed a pattern of courting and family formation
among some African American slaves that hitherto had not
been noticed by historians.

Content analysis of paintings could be used to reveal

past attitudes and behaviors, such as the changing
depictions of children in paintings from medieval to modern
times, suggesting that in the past children went straight from
being babies to being workers on farms and in mills and
factories.

Another of the tools is archeology (I prefer the simpler

spelling) which treats physical remnants of past behavior as
documents. With the advent of radiological and chemical
dating techniques following World War II it was possible to dig
up an ancient Native American, African or English peasant
house from several hundred years ago and have some
confidence in how old it is. Increasing sophistication in
recording the materials coming out of archeological sites
allowed fuller reconstruction of the activities at the time the
sites were "alive", including how many people lived there.

141
 By the 1950's archeologists were using punch card
sorters to analyze the distributions of artifacts dug out of the
ground, and by the 1960's some were fortunate enough to
have access to computers to catalog, store and analyze their
data. Sites that were being lived in were excavated using
archeological techniques to check what can be learned
using just archeology versus archeology and other data-
gathering techniques together. These kinds of comparisons
helped archeologists make better sense of the artifacts they
were able to recover and the way in which the artifacts had
originally been distributed in the sites. As computers become
more powerful reconstructions and simulations become
easier to do and more complete.

DNA mapping is being correlated with other evidence

of past behavior giving better insights into migrations and
other demographic dynamics of the past.

Satellite mapping is revealing past civilizations in such

unlikely places as the depths of the Amazon jungle and the
Western Sahara.

Ground penetrating radar can be used to locate and

map buildings buried under the surface of the earth without
having to dig and disrupt the present-day use of the land.

Better understanding of accomplishments of non-

Western peoples before contact with Europeans and the
frequent serious disruptions of native societies even before
they came into direct contact with Europeans is increasing
our awareness of unintened consequences of our actions
and the fragility of even the most-developed civilizations in
the face of environmental and economic change.

Strengths:

142
  The people being studied cannot alter their behavior
because of the current study or the presence of the
current researcher. However the researcher needs to
be aware of the context in which the "document" of
the past was created, but that has always been the
essence of the historical method, historical criticism.
What are the likely sources of bias, sampling error or
other possible sources of distortion?
 Some sources of data are readily available either in
microfilm or microfiche or in electronically readable
format, some even readily available online for free.
Examples: Mormon genealogical database, GSS,
various government surveys. The entire body of ancient
Greek documents has been available in electronically
readable form since at least the 1970's. Even for sources
that are not online or in portable format, indexes for
more and more sources of data are going online
making it possible for more researchers to locate
archival materials all around the world.
 With imagination it is possible to study more things
through archives than once thought possible.
 It is frequently one of the least expensive ways for the
lone scholar with limited expenses to study significant
problems.

Weaknesses:
 Even though research subjects are not intruded on in
any way there are still ethical issues inherent in possibly
making people or authorities aware of information that
has been considered private. The U.S. census enforces
a strict 72-year-confidentiality rule on its raw data, but
the Mormon Church exposes data about families and
even living individuals in its database that are still
considered private or confidential by the people in
question. This has led some organizations to issue
takedown requests with the LDS Church, for example

143
 Jewish groups requesting the takedown of Holocaust
data and other Jewish material.
 As an extension of the previous point, some material
may be so closely guarded that only people who
agree to follow guidelines set by those who control the
material are given access. While on the surface this
seems fair and reasonable, it has interfered with some
important potential studies. For example, Sigmund
Freud's descendents still guard his papers so jealously
that scholars who raise legitimate questions about
Freud's methods and interpretations cannot gain
access to his papers where those things might be
made more clear. Those who do have access
sometimes are not as critical as they should be of
Freud's work.
 You can only study what has left traces, although
imagination can sometimes find traces where none
were thought to exist.
 For archives that are not portable, travel and living
expenses can be considerable. If the archives have not
been catalogued the researcher will have to spend a
lot of time cataloguing and calendaring the materials.
At least the archeologist frequently gets to take his/her
materials back to the home lab and spend years
cataloguing and analyzing them.
 Archeology as a means of accessing "archives" of
remnants of past behavior can be horribly expensive.
Add to the travel and living expenses for the lone
researcher listed above those of all the members of the
team who are working away from home. Then add the
costs of local labor, consumable tools such as shovels,
picks and trowels and equipment rental for earth-
moving equipment. Also add the costs of satellite
imaging (going down in recent years), GP radar,
radiological and chemical tests. I am sure I am missing
some major categories.

144
  Archeological sites and non-portable archives
frequently become unavailable for long periods of time
because of military and political conflicts, or because
of budget cutbacks for the institutions housing the
documents or artifacts.

Participant Observation

The basic premise of participant observation (P.O.) is

that it is possible for the researcher to observe behavior in real
time in its full context without having to depend on self-
reporting, the accidents of behavior or attitudes being
documented or the accidents of artifacts of past behavior
being preserved to be studied in the future.

Strengths:
 It is possible to gain an understanding of the complete
context of the behavior that is being studied.
 Every aspect of the society/culture of the small
community can be studied if the period of study is long
enough. This is called the holistic approach.
 Can be carried out by a lone researcher.
 Lengthy presence in the community makes it less likely
that the research subjects will continue to be able to try
to mislead the researcher about their real behavior.
What is recognized as unacceptable behavior by the
outside world or by members of the local community is
more likely to be exposed. On the flipside, posing
behavior designed to impress outsiders in a negative or
positive fashion is frequently discontinued by the
research subjects as the researcher becomes part of
the daily situation.
 Lots of studies of similar communities in lots of different
parts of the world to compare findings with.

Weaknesses:

145
  The ethics of P.O. deal mostly with questions of
disrupting the lives of the research subjects during the
research or as a consequence of the publication of the
research. Additional ethical dilemmas have to do with
how much to reveal. Lengthy observation with no
consent by the subjects is rarely attempted, but would
be considered unethical today by most social scientists.
Over involvement in the lives of the subjects raise
questions of both ethics and bias.
 Gaining rapport may be difficult, taking as long as
several months to several years. Some studies have
been abandoned because of this difficulty.
 Time consuming. The best studies last a year or longer,
during which the researcher frequently has reduced or
no income. This requires researchers to compete for
grants to support the research.
 Can be dangerous. Exotic diseases, poisonous animals,
ferocious beasts, violent people, food that is quite at
odds with personal preferences or past experience all
can be problems.
 Small sample size, particularly if only one researcher is
carrying out the study. Is my neighborhood or village
comparable to other similar ones elsewhere?
 Researcher bias can influence findings. One researcher
of a village in Mexico saw a cooperative community in
which people worked for the common good,
sometimes sacrificing their own opportunities. A later
researcher of the same village found people constantly
engaging in gossip and criticism of others, particularly
those who seemed to be doing better. Which one was
right? Both. Subsequent studies of peasant villages in
similar circumstances even in other societies show that
members of the peasant community frequently coerce
cooperation from their neighbors through criticism and
other negative informal social sanctions. We are
fortunate to have the two studies, but if either study

146
 had to be evaluated on its own, we would be
suspicious of it and probably not be misled despite the
fame and reputation of both scholars.

Properties of Instruments: Validity and Reliability

Instrument is the general term that researchers use for a

measurement device (survey, test, questionnaire, etc.). To
help distinguish between instrument and instrumentation,
consider that the instrument is the device and instrumentation
is the course of action (the process of developing, testing,
and using the device).
Instruments fall into two broad categories, researcher-
completed and subject-completed, distinguished by those
instruments that researchers administer versus those that are
completed by participants. Researchers chose which type of
instrument, or instruments, to use based on the research
question. Examples are listed below:
Researcher-completed Subject-completed Instruments
Instruments
Rating scales Questionnaires
Interview schedules/guides Self-checklists
Tally sheets Attitude scales
Flowcharts Personality inventories
Performance checklists Achievement/aptitude tests
Time-and-motion logs Projective devices
Observation forms Sociometric devices

Usability
Usability refers to the ease with which an instrument
can be administered, interpreted by the participant, and
scored/interpreted by the researcher. Example usability
problems include:

147
1. Students are asked to rate a lesson immediately after
class, but there are only a few minutes before the next
class begins (problem with administration).
2. Students are asked to keep self-checklists of their after
school activities, but the directions are complicated and
the item descriptions confusing (problem with
interpretation).
3. Teachers are asked about their attitudes regarding school
policy, but some questions are worded poorly which
results in low completion rates (problem with
scoring/interpretation).

Validity and reliability concern (discussed below) will help

alleviate usability issues. For now, we can identify five usability
considerations:
1. How long will it take to administer?
2. Are the directions clear?
3. How easy is it to score?
4. Do equivalent forms exist?
5. Have any problems been reported by others who used it?

Validity is the extent to which an instrument measures

what it is supposed to measure and performs as it is designed
to perform. It is rare, if nearly impossible, that an instrument be
100% valid, so validity is generally measured in degrees. As a
process, validation involves collecting and analyzing data to
assess the accuracy of an instrument. There are numerous
statistical tests and measures to assess the validity of
quantitative instruments, which generally involves pilot testing.
The remainder of this discussion focuses on external validity
and content validity.

148
 External validity is the extent to which the results of a
study can be generalized from a sample to a population.
Establishing eternal validity for an instrument, then, follows
directly from sampling. Recall that a sample should be an
accurate representation of a population, because the total
population may not be available. An instrument that is
externally valid helps obtain population generalizability, or the
degree to which a sample represents the population.
Content validity refers to the appropriateness of the
content of an instrument. In other words, do the measures
(questions, observation logs, etc.) accurately assess what you
want to know? This is particularly important with achievement
tests. Consider that a test developer wants to maximize the
validity of a unit test for 7th grade mathematics. This would
involve taking representative questions from each of the
sections of the unit and evaluating them against the desired
outcomes.
Reliability can be thought of as consistency. Does the
instrument consistently measure what it is intended to
measure? It is not possible to calculate reliability; however,
there are four general estimators that you may encounter in
reading research:
1. Inter-Rater/Observer Reliability: The degree to which
different raters/observers give consistent answers or
estimates.
2. Test-Retest Reliability: The consistency of a measure
evaluated over time.
3. Parallel-Forms Reliability: The reliability of two tests
constructed the same way, from the same content.
4. Internal Consistency Reliability: The consistency of results
across items, often measured with Cronbach’s Alpha.

Relating Reliability and Validity

149
 Reliability is directly related to the validity of the
measure. There are several important principles. First, a test
can be considered reliable, but not valid. Consider the SAT,
used as a predictor of success in college. It is a reliable test
(high scores relate to high GPA), though only a moderately
valid indicator of success (due to the lack of structured
environment – class attendance, parent-regulated study, and
sleeping habits – each holistically related to success).
Second, validity is more important than reliability. Using
the above example, college admissions may consider the SAT
a reliable test, but not necessarily a valid measure of other
quantities colleges seek, such as leadership capability,
altruism, and civic involvement. The combination of these
aspects, alongside the SAT, is a more valid measure of the
applicant’s potential for graduation, later social involvement,
and generosity (alumni giving) toward the alma mater.
Finally, the most useful instrument is both valid and
reliable. Proponents of the SAT argue that it is both. It is a
moderately reliable predictor of future success and a
moderately valid measure of a student’s knowledge in
Mathematics, Critical Reading, and Writing.
Thus far, we have discussed Instrumentation as related
to mostly quantitative measurement. Establishing validity and
reliability in qualitative research can be less precise, though
participant/member checks, peer evaluation (another
researcher checks the researcher’s inferences based on the
instrument (Denzin & Lincoln, 2005), and multiple methods
(keyword: triangulation), are convincingly used. Some
qualitative researchers reject the concept of validity due to
the constructivist viewpoint that reality is unique to the
individual, and cannot be generalized. These researchers
argue for a different standard for judging research quality.

Standard Tests vs. Non- Standard Tests

150
 Both standardized and non-standardized assessments
have their own benefits and drawbacks. Standardized
assessments seek to measure the measurable, while non-
standardized tests measure student skills that are noticeable
and maybe significant, but can't be quantified. Luckily, both
forms of assessment can operate alongside each other in a
class curriculum.

State Mandates for Standardized Testing

Standardized test results play a major role in how states

fund schools and in the decisions that policymakers make.
However, states are less likely to make funding decisions
based on non-standardized tests. Standardized tests also help
make schools accountable for student achievement. By
measuring how much students improve in a particular
classroom, standardized tests cue administrators to check on
instructional methods in place and whether they need to be
changed.

Measurability

The No Child Left Behind Act requires that standardized

tests be the main metric by which schools assess student
achievement. Standardized tests are fairer, while non-
standardized tests do not allow schools to compare teachers
and students, since schools can feign improvement by
making non-standardized assessment easier for students.
Standardized tests are quantifiable. However, standardized
tests cannot accurately assess subjects, such as creative
writing, which are skills that are not easily measurable. This
leaves non-standardized tests as the better option for these
skills.

151
Types of Questions

With standardized tests, students answer the same or

very similar questions. Non-standardized tests can give
students different questions and sometimes have them
perform different activities. For example, a class might have
students create projects where they use visual aids to explain
addictive substances to the rest of the class. One student
may choose to show a video, while another student might
use a poster board.

Grading Criteria

Non-standardized tests may have varying conditions

that influence how the tests are scored. For example, students
may receive higher or lower grades on tests based on their
progress compared to how they scored on earlier tests. Non-
standardized tests are often geared more toward developing
the skills of the students.

Formality

Standardized assessments rely on easily measurable

data, such as multiple-choice questions. The tests give
students questions and they must fill in bubbles that
correspond with their chosen answers. There are several false
answers along with the true answer. Filling in a false answer
will cause the student to not receive credit for the answer.
Non-standardized assessments are less formal and rely on
situations, such as teacher observations, portfolios, informal
questioning, student participation in the class and checklists.

Levels of Academic Achievement

Non-standardized assessment methods consider that

students have different levels of academic achievement.
Standardized tests assess students at the same level of

152
difficulty, so the tests will either be “dumbed down” to help
struggling students pass, or be so hard that struggling students
never pass.

Consistency

Standardized assessments are controlled assessments

that test all students in a given geographic area evenly. This
allows schools and policymakers to accurately compare the
success of groups, such as teachers, schools and students.
Since the tests are standardized, the poor-achieving groups
cannot argue that the tests are unevenly matched.
Standardized tests have specific skills or content areas that
they measure. States often align tests with state standards on
what they want schools to teach students.

153
Observation

Observation is a systematic data collection approach.

Researchers use all of their senses to examine people in
natural settings or naturally occurring situations.

Observation of a field setting involves:

 prolonged engagement in a setting or social situation

 clearly expressed, self-conscious notations of how
observing is done
 methodical and tactical improvisation in order to
develop a full understanding of the setting of interest
 imparting attention in ways that is in some sense
'standardized'
 recording one's observations

Participant Observation

Some researchers draw a distinction between

participant observation and observation. This distinction is
murky.

Participant observation "combines participation in the

lives of the people being studied with maintenance of a
professional distance that allows adequate observation and
recording of data" (Fetterman, 1998, pp. 34-35).

Participant observation underscores the person's role as

participant in the social setting he or she observes. The range
of roles one may play as a participant observer have been
describe by Gold (1958), Adler and Adler (1984) and others.

Bernard (1998) suggests that participant observation

must be learned in the field. However, he identifies serveral

154
skills associated with participant observation. Click here for
more.

For a more developed discussion of the distinction

between observation and participant observation see
Savage (2000) and for a discussion of participant observation
as a methodology see Jorgensen (1989).

When might observation be used?

There are a variety of reasons for collecting

observational data. Some of these reasons include:

 When the nature of the research question to be

answered is focused on answering a how- or what-type
question
 When the topic is relatively unexplored and little is
known to explain the behavior of people in a particular
setting
 When understanding the meaning of a setting in a
detailed way is valuable
 When it is important to study a phenomenon in its
natural setting
 When self-report data (asking people what they do) is
likely to be different from actual behavior (what people
actually do). One example of this seen in the
difference between self-reported versus observed
preventive service delivery in health care settings.
 When implementing an intervention in a natural setting,
observation may be used in conjunction with other
quantitative data collection techniques. Observational
data can help researchers evaluate the fidelity of an
intervention across settings and identify when 'stasis' has
been achieved.

155
Non-Participant Observation

Non-participant observation is observation with limited

interaction with the people one observes. For example, some
observational data can be collected unobtrusively (e.g. worn
out carpet as indicators of high use areas in a physical
setting).

Researchers who study how people communicate

often want to examine the details of how people talk and
behave together. Non-participant observation involving the
use of recording devices might be a good choice.

This data collection approach results in a detailed

recording of the communication and provides the researcher
with access to the contours of talk (e.g. intonation) as well as
body behavior (e.g. facial expression, eye gaze). Even a
great observer cannot record these aspects in detail.

Non-participant observation may provide limited insight

into the meaning of the social context studied. If this
contextual understanding is important, participant
observation might be needed. These two data collection
techniques can complement each other and be used
together.

Observing by video or audio recording

If people are to be observed in a closed setting, the

researcher is not a participant observer, and tape- or video-
recording is permissable then this data recording approach
may be appropriate (e.g. physician-patient encounters).

Choosing to tape-record of video-record a setting will

depend in large part on what is permissible in that setting.
There are, however, a few things to keep in mind:

156
 Decisions regarding how to record observational data
depend largely on the focus of the research question and the
analytical approach proposed.

If the researcher is trying to understand how people

behave together and the people in question can see each
other, then the use of video may be recommended. This is
because of the important role that bodily-based behavior
plays in our social processes. Without this visual information,
the researcher may not fully understand what transpires (e.g.
physician-patient encounters). Additionally, capturing the
details of this behavior in fieldnotes will be difficult.

Audio-recording a telephone counseling session makes

sense because the two interactants (and the researcher) only
have access to verbal communication.

Audio and video recordings afford the researcher the

opportunity to transcribe what occurs in a setting and play it
over and over. This can be very useful in the analysis process.

Fieldnotes

Participant observers may use multiple methods to

gather data. One primary approach involves writing
fieldnotes. There are several guides for learning how to
prepare fieldnotes.

Researchers may be interested in creating or using a

template to guide a researchers' observations.

 Templates or observational coding sheets can be useful

when data is collected by inexperienced observers

157
  Templates or observational coding sheets should only
be developed after observation in the field that is not
inhibited by such a template
 Theories and concepts can be driven by templates
and result in focused data collection
 Templates can deflect attention from unnamed
categories, unimagined and unanticipated activities
that can be very important to understanding a
phenomenon and a setting

Benefits

 Immersion and prolonged involvement in a setting can

lead to the development of rapport and foster free
and open speaking with members.
 Observation fosters an in depth and rich understanding
of a phenomenon, situation and/or setting and the
behavior of the participants in that setting.
 Observation is an essential part of gaining an
understanding of naturalistic settings and its members'
ways of seeing.
 Observation can provide the foundation for theory and
hypothesis development.

Interview

If you are conducting qualitative research, you may be

wondering what is the best interview type for your study?
Well, the answer is that it depends on your research design
and what you want to accomplish. Let us start with the
different types of interviews. There are three types of
interviews: unstructured, semistructured, and structured.

Unstructured interviews: These are interviews that take

place with few, if any, interview questions. They often progress
in the manner a normal conversation would, however it

158
concerns the research topic under review. It is a relatively
formless interview style that researchers use to establish
rapport and comfort with the participant, and is extremely
helpful when researchers are discussing sensitive topics. The
researcher is expected to probe participants in order to
obtain the most rich and in-depth information possible. If you
select this interview style, just keep in mind that you may have
to conduct several rounds of interviews with your participants
in order to gather all the information you need. Since you do
not use a standard interview protocol, sometimes
participant’s narratives maneuver the conversation away
from other aspects of the research topic you want to explore;
it is a part of the conversational style this interview method
requires.

Semistructured interviews: These are interviews that use

an interview protocol to help guide the researcher through
the interview process. While this can incorporate
conversational aspects, it is mostly a guided conversation
between the researcher and participant. It does maintain
some structure (hence the name semistructured), but it also
provides the researcher with the ability to probe the
participant for additional details. If you decide to choose this
interview method, understand that it offers a great deal of
flexibility for you as a researcher. You do not have to worry
about needing to conduct several rounds of interviews
because your interview protocol will keep you focused on
gathering all the information that you need to answer your
research question. Even though that is the goal with an
interview protocol, there may be a need for additional
probing so that you can get more details about participants’
thoughts, feelings, and opinions.

Structured interviews: These are interviews that strictly

adhere to the use of an interview protocol to guide the
researcher. It is a more rigid interview style, in that only the

159
questions on the interview protocol are asked. As a result,
there are not a lot of opportunities to probe and further
explore topics that participants bring up when answering the
interview questions. This method can be advantageous when
researchers have a comprehensive list of interview questions,
since it helps target the specific phenomenon or experience
that the researcher is investigating. It makes for expedient
interviewing and will gather the correct information that you
need, so there should not be much need for you to do follow-
up interviews for missed or forgotten questions.

Each interview type has its advantages and its

disadvantages, as briefly outlined above. Selecting one
depends on your research design and what you want to get
out of your interviews. If you take some time to review your
research design, this will help you determine which interview
type works best for your study.

Questionnaire

A questionnaire is a research instrument consisting of a

series of questions for the purpose of gathering information
from respondents. Questionnaires can be thought of as a kind
of written interview. They can be carried out face to face, by
telephone, computer or post.

Questionnaires provide a relatively cheap, quick and

efficient way of obtaining large amounts of information from
a large sample of people. Data can be collected relatively
quickly because the researcher would not need to be
present when the questionnaires were completed. This is
useful for large populations when interviews would be
impractical.

However, a problem with questionnaire is that

respondents may lie due to social desirability. Most people

160
want to present a positive image of themselves and so may
lie or bend the truth to look good, e.g., pupils would
exaggerate revision duration.

Questionnaires can be an effective means of

measuring the behavior, attitudes, preferences, opinions and,
intentions of relatively large numbers of subjects more
cheaply and quickly than other methods. An important
distinction is between open-ended and closed questions.

Often a questionnaire uses both open and closed

questions to collect data. This is beneficial as it means both
quantitative and qualitative data can be obtained.

Closed Questions

Closed questions structure the answer by only allowing

responses which fit into pre-decided categories.

Data that can be placed into a category is called

nominal data. The category can be restricted to as few as
two options, i.e., dichotomous (e.g., 'yes' or 'no,' 'male' or
'female'), or include quite complex lists of alternatives from
which the respondent can choose (e.g., polytomous).

Closed questions can also provide ordinal data (which

can be ranked). This often involves using a continuous rating
scale to measure the strength of attitudes or emotions. For
example, strongly agree / agree / neutral / disagree /
strongly disagree / unable to answer.

Closed questions have been used to research type A

personality (e.g., Friedman & Rosenman, 1974), and also to
assess life events which may cause stress (Holmes & Rahe,
1967), and attachment (Fraley, Waller, & Brennan, 2000).

161
Strengths

They can be economical. This means they can provide

large amounts of research data for relatively low costs.
Therefore, a large sample size can be obtained which should
be representative of the population, which a researcher can
then generalize from.

The respondent provides information which can be

easily converted into quantitative data (e.g., count the
number of 'yes' or 'no' answers), allowing statistical analysis of
the responses.

The questions are standardized. All respondents are

asked exactly the same questions in the same order. This
means a questionnaire can be replicated easily to check for
reliability. Therefore, a second researcher can use the
questionnaire to check that the results are consistent.

162
Limitations

They lack detail. Because the responses are fixed, there

is less scope for respondents to supply answers which reflect
their true feelings on a topic.

Open Questions

Open questions allow people to express what they

think in their own words. Open-ended questions enable the
respondent to answer in as much detail as they like in their
own words. For example: “can you tell me how happy you
feel right now?”

If you want to gather more in-depth answers from your

respondents, then open questions will work better. These give
no pre-set answer options and instead allow the respondents
to put down exactly what they like in their own words.

Open questions are often used for complex questions

that cannot be answered in a few simple categories but
require more detail and discussion.

Lawrence Kohlberg presented his participants with

moral dilemmas. One of the most famous concerns a
character called Heinz who is faced with the choice
between watching his wife die of cancer or stealing the only
drug that could help her. Participants were asked whether
Heinz should steal the drug or not and, more importantly, for
their reasons why upholding or breaking the law is right.

163
Strengths

Rich qualitative data is obtained as open questions

allow the respondent to elaborate on their answer. This
means the research can find out why a person holds a
certain attitude.

Limitations

Time-consuming to collect the data. It takes longer for

the respondent to complete open questions. This is a problem
as a smaller sample size may be obtained.

Time-consuming to analyze the data. It takes longer for

the researcher to analyze qualitative data as they have to
read the answers and try to put them into categories by
coding, which is often subjective and difficult. However, Smith
(1992) has devoted an entire book to the issues of thematic
content analysis the includes 14 different scoring systems for
open-ended questions.

Not suitable for less educated respondents as open

questions require superior writing skills and a better ability to
express one's feelings verbally.

Designing a Questionnaire

With some questionnaires suffering from a response rate

as low as 5%, it is essential that a questionnaire is well
designed.

There are a number of important factors in

questionnaire design.

Aims

164
 Make sure that all questions asked address the aims of
the research. However, use only one feature of the construct
you are investigating in per item.

Length

The longer the questionnaire, the less likely people will

complete it. Questions should be short, clear, and be to the
point; any unnecessary questions/items should be omitted.

Pilot Study

Run a small scale practice study to ensure people

understand the questions. People will also be able to give
detailed honest feedback on the questionnaire design.

Question Order

Questions should progress logically from the least

sensitive to the most sensitive, from the factual and
behavioral to the cognitive, and from the more general to
the more specific.

The researcher should ensure that the answer to a

question is not influenced by previous questions.

Terminology

There should be a minimum of technical jargon.

Questions should be simple, to the point and easy to
understand.
The language of a questionnaire should be appropriate
to the vocabulary of the group of people being studied. Use
statements which are interpreted in the same way by

165
members of different subpopulations of the population of
interest.

For example, the researcher must change the

language of questions to match the social background of
respondents' age / educational level / social class / ethnicity
etc.

Presentation

Make sure it looks professional, include clear and

concise instructions. If sent through the post make sure the
envelope does not signify ‘junk mail.’
Ethical Issues
The researcher must ensure that the information provided by
the respondent is kept confidential, e.g., name, address, etc.

This means questionnaires are good for researching

sensitive topics as respondents will be more honest when they
cannot be identified. Keeping the questionnaire confidential
should also reduce the likelihood of any psychological harm,
such as embarrassment.

Participants must provide informed consent prior to

completing the questionnaire, and must be aware that they
have the right to withdraw their information at any time
during the survey/ study.

Problems with Postal Questionnaires

The data might not be valid (i.e., truthful) as we can

never be sure that the right person actually completed the
postal questionnaire.

Also, postal questionnaires may not be representative

of the population they are studying?

166
 This is because some questionnaires may be lost in the
post reducing the sample size.

The questionnaire may be completed by someone who

is not a member of the research population.
Those with strong views on the questionnaire’s subject are
more likely to complete it than those with no interest in it.

Benefits of a Pilot Study

A pilot study is a practice / small-scale study

conducted before the main study. It allows the researcher to
try out the study with a few participants so that adjustments
can be made before the main study, so saving time and
money.

 It is important to conduct a questionnaire pilot study for

the following reasons:

 Check that respondents understand the terminology

used in the questionnaire.

 Check that emotive questions have not been used as

they make people defensive and could invalidate their
answers.

 Check that leading questions have not been used as

they could bias the respondent's answer.

 Ensure the questionnaire can be completed in an

appropriate time frame (i.e., it's not too long).

Scales of Measurement: Checklist and Rating Scales

167
 Assessment Strategies and Tools: Checklists, Rating
Scales and Rubrics

Checklists, rating scales and rubrics are tools that state

specific criteria and allow teachers and students to gather
information and to make judgements about what students
know and can do in relation to the outcomes. They offer
systematic ways of collecting data about specific behaviours,
knowledge and skills.

The quality of information acquired through the use of

checklists, rating scales and rubrics is highly dependent on
the quality of the descriptors chosen for assessment. Their
benefit is also dependent on students’ direct involvement in
the assessment and understanding of the feedback
provided.

The purpose of checklists, rating scales and rubrics is to:

 provide tools for systematic recording of observations

 provide tools for self-assessment
 provide samples of criteria for students prior to
collecting and evaluating data on their work
 record the development of specific skills, strategies,
attitudes and behaviors necessary for demonstrating
learning
 clarify students' instructional needs by presenting a
record of current accomplishments.

Tips for Developing Checklists, Rating Scales and Rubrics

 Use checklists, rating scales and rubrics in relation to

outcomes and standards.

168
  Use simple formats that can be understood by students
and that will communicate information about student
learning to parents.
 Ensure that the characteristics and descriptors listed are
clear, specific and observable.
 Encourage students to assist with constructing
appropriate criteria. For example, what are the
descriptors that demonstrate levels of performance in
problem solving?
 Ensure that checklists, rating scales and rubrics are
dated to track progress over time.
 Leave space to record anecdotal notes or comments.
 Use generic templates that become familiar to students
and to which various descriptors can be added
quickly, depending on the outcome(s) being assessed.
 Provide guidance to students to use and create their
own checklists, rating scales and rubrics for self-
assessment purposes and as guidelines for goal setting.

Checklists usually offer a yes/no format in relation to

student demonstration of specific criteria. This is similar to a
light switch; the light is either on or off. They may be used to
record observations of an individual, a group or a whole
class.

Rating Scales allow teachers to indicate the degree or

frequency of the behaviours, skills and strategies displayed by
the learner. To continue the light switch analogy, a rating
scale is like a dimmer switch that provides for a range of
performance levels. Rating scales state the criteria and
provide three or four response selections to describe the
quality or frequency of student work.

Teachers can use rating scales to record observations

and students can use them as self-assessment tools. Teaching
students to use descriptive words, such as always, usually,

169
sometimes and never helps them pinpoint specific strengths
and needs. Rating scales also give students information for
setting goals and improving performance. In a rating scale,
the descriptive word is more important than the related
number. The more precise and descriptive the words for each
scale point, the more reliable the tool.

Effective rating scales use descriptors with clearly

understood measures, such as frequency. Scales that rely on
subjective descriptors of quality, such as fair, good or
excellent, are less effective because the single adjective
does not contain enough information on what criteria are
indicated at each of these points on the scale.

Rubrics use a set of criteria to evaluate a student's

performance. They consist of a fixed measurement scale and
detailed description of the characteristics for each level of
performance. These descriptions focus on the quality of the
product or performance and not the quantity; e.g., not
number of paragraphs, examples to support an idea, spelling
errors. Rubrics are commonly used to evaluate student
performance with the intention of including the result in a
grade for reporting purposes. Rubrics can increase the
consistency and reliability of scoring.

Rubrics use a set of specific criteria to evaluate student

performance. They may be used to assess individuals or
groups and, as with rating scales, may be compared over
time.

Developing Rubrics and Scoring Criteria

Rubrics are increasingly recognized as a way to both

effectively assess student learning and communicate
expectations directly, clearly and concisely to students. The

170
inclusion of rubrics in a teaching resource provides
opportunities to consider what demonstrations of learning
look like, and to describe stages in the development and
growth of knowledge, understandings and skills. To be most
effective, rubrics should allow students to see the progression
of mastery in the development of understandings and skills.

Rubrics should be constructed with input from students

whenever possible. A good start is to define what quality work
looks like based on the learning outcomes. Exemplars of
achievement need to be used to demonstrate to students
what an excellent or acceptable performance is. This
provides a collection of quality work for students to use as
reference points. Once the standard is established, it is easy
to define what exemplary levels and less-than-satisfactory
levels of performance look like. The best rubrics have three to
five descriptive levels to allow for discrimination in the
evaluation of the product or task. Rubrics may be used for
summative purposes to gauge marks by assigning a score to
each of the various levels.

When developing a rubric, consider the following:

 What are the specific outcomes in the task?

 Do the students have some experience with this or a
similar task?
 What does an excellent performance look like? What
are the qualities that distinguish an excellent response
from other levels?
 What do other responses along the performance
quality continuum look like?
 Is each description qualitatively different from the
others? Are there an equal number of descriptors at
each level of quality? Are the differences clear and
understandable to students and others?

171
 Begin by developing criteria to describe the
Acceptable level. Then use Bloom's taxonomy to identify
differentiating criteria as you move up the scale. The criteria
should not go beyond the original performance task, but
reflect higher order thinking skills that students could
demonstrate within the parameters of the initial task.

When developing the scoring criteria and quality levels

of a rubric, consider the following guidelines.

 Level 4 is the Standard of excellence level. Descriptions

should indicate that all aspects of work exceed grade
level expectations and show exemplary performance
or understanding. This is a "Wow!"
 Level 3 is the Approaching standard of excellence
level. Descriptions should indicate some aspects of
work that exceed grade level expectations and
demonstrate solid performance or understanding. This is
a "Yes!"
 Level 2 is the Meets acceptable standard. This level
should indicate minimal competencies acceptable to
meet grade level expectations. Performance and
understanding are emerging or developing but there
are some errors and mastery is not thorough. This is a
"On the right track, but …".
 Level 1 Does not yet meet acceptable standard. This
level indicates what is not adequate for grade level
expectations and indicates that the student has serious
errors, omissions or misconceptions. This is a "No, but …".
The teacher needs to make decisions about
appropriate intervention to help the student improve.

Creating Rubrics with Students

Learning increases when students are actively involved

in the assessment process. Students do better when they

172
know the goal, see models and know how their performance
compares to learning outcomes.

Learning outcomes are clarified when students assist in

describing the criteria used to evaluate performance. Use
brainstorming and discussion to help students analyze what
each level looks like. Use student-friendly language and
encourage students to identify descriptors that are
meaningful to them. For example, a Grade 3 class might
describe levels of quality with phrases such as the following.

 Super!
 Going beyond
 Meets the mark
 Needs more work.
 Use work samples to help students practice and
analyze specific criteria for developing a critical
elements list. They can also use samples to practice
assigning performance levels and compare criteria
from level to level.

Although rubrics are often used as assessment of

learning tools, they can also be used as assessment for
learning tools. Students can benefit from using rubrics as they
become more competent at judging the quality of their work
and examining their own progress.

Example:
Involve students in the assessment process by having
them participate in the creation of a rubric. This process
facilitates a deeper understanding of the intended outcomes
and the associated assessment criteria.
After a rubric has been created, students can use it to guide
their learning. Criteria described in a rubric serve to focus
student reflection on their work and facilitate the setting of
learning goals for a particular performance assessment.

173
Through self-assessment or peer-assessment, students can use
a rubric to assess work completed to date and use it to guide
their planning for the "next steps" in learning.

Exercises:

Identify Primary Research

Identify research articles that document an original

study. Differentiate between primary and secondary sources
in adiscipline. Discuss when and why it is appropriate to use
each in academic research (Burkhardt et al., 2010, p. 14).
SLOs: 2, 4

Level: Bachelor’s, Master’s

Locate and Evaluate the “Best” and “Worst” Website on a

Topic

Using a research topic provided by the instructor,

students will develop criteria for judging the credibility,
relevancy,accuracy, and purpose of the information they
find. Have students locate websites relevant to the topic,
highlightfeatures of the “best” site, and recommend
improvements for the “worst” site (Burkhardt et al., 2010, p. 88,
90).
SLOs: 2, 3, 4

Level: Bachelor’s

Locate Primary Sources from the Date of Your Birth

Students use a minimum of three different sources and three
different resource types (e.g., a newspaper article, a top40
hit, a painting). Write a short annotation of each source, and
include the complete bibliographic citation.
SLOs: 2, 3, 6

174
Level: Bachelor’s

Locate Two Scholarly Articles; Compare and Evaluate

Their Bibliographies

Using a predetermined research topic, access relevant

information, compare similar and unique references, and
consider the quality of sources and the authority of each
article.
SLOs: 1, 2, 3, 4

Level: Bachelor’s, Master’s

Locate and Evaluate an Alternative Press Article

On a topic previously determined, or assigned by

instructor, students locate and evaluate resources considered
to be
alternative press. Assess implications of the
information/publishing cycle on alternative information
creation,dissemination, and intellectual freedom.[Source:
http://library.minneapolis.edu/courses/infs1000/assignments/T
eam_Assignment_6.pdf]
SLOs: 2, 3, 4, 6

Level: Bachelor’s, Master’s

Research Careers

Students research a career of their choice, finding

information in at least one reference book, one general-
audiencebook, one journal article, and one website (use a
set number of each source type). Identify required
careerqualifications such as education, experience, and
licenses (if any). Research salary expectations,

175
employmentopportunities, and employment forecasts.
Reflect upon your interests, abilities and experience.[Source:
http://library.missouristate.edu/services/fyp/assignments.htm]
SLOs: 1, 2, 5

Level: Bachelor

176
 References

Abend, Gabriel. "The Meaning of Theory." Sociological

Theory 26 (June 2008): 173–199; Swanson, Richard
A. Theory Building in Applied Disciplines. San Francisco,
CA: Berrett-Koehler Publishers 2013.

Adapted from: Torraco, R. J. “Theory-Building Research

Methods.” In Swanson R. A. and E. F. Holton III ,
editors. Human Resource Development Handbook:
Linking Research and Practice. (San Francisco, CA:
Berrett-Koehler, 1997): pp. 114-137; Jacard, James and
Jacob Jacoby. Theory Construction and Model-
Building Skills: A Practical Guide for Social Scientists.
New York: Guilford, 2010; Ravitch, Sharon M. and
Matthew Riggan. Reason and Rigor: How Conceptual
Frameworks Guide Research. Second edition. Los
Angeles, CA: SAGE, 2017; Sutton, Robert I. and Barry M.
Staw. “What Theory is Not.” Administrative Science
Quarterly 40 (September 1995): 371-384.

Cook, Kathleen E. and Elise Murowchick. “Do Literature

Review Skills Transfer from One Course to
Another?” Psychology Learning and Teaching 13
(March 2014): 3-11; Fink, Arlene. Conducting Research
Literature Reviews: From the Internet to Paper. 2nd ed.
Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a
Literature Review: Releasing the Social Science
Research Imagination. Thousand Oaks, CA: Sage
Publications, 1998; Jesson, Jill. Doing Your Literature
Review: Traditional and Systematic Techniques.
London: SAGE, 2011; Literature Review Handout. Online
Writing Center. Liberty University; Literature Reviews. The
Writing Center. University of North Carolina;
Onwuegbuzie, Anthony J. and Rebecca Frels. Seven
Steps to a Comprehensive Literature Review: A
Multimodal and Cultural Approach. Los Angeles, CA:
177
 SAGE, 2016; Ridley, Diana. The Literature Review: A
Step-by-Step Guide for Students. 2nd ed. Los Angeles,
CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing
the Dissertation Literature Review."Practical Assessment,
Research, and Evaluation. vol. 14, June 2009; Sutton,
Anthea. Systematic Approaches to a Successful
Literature Review. Los Angeles, CA: Sage Publications,
2016; Taylor, Dena. The Literature Review: A Few Tips On
Conducting It. University College Writing Centre.
University of Toronto; Writing a Literature Review.
Academic Skills Centre. University of Canberra.

Essays, UK. (November 2013). Designing a Research Study:

Types of Research and Methods. Retrieved from
https://www.ukessays.com/dissertation/examples/man
agement/basic-concepts-of-research.php?vref=1

Fink, Arlene. Conducting Research Literature Reviews: From

the Internet to Paper. Fourth edition. Thousand Oaks,
CA: SAGE, 2014.

Fink, Arlene. Conducting Research Literature Reviews: From

the Internet to Paper. 2nd ed. Thousand Oaks, CA:
Sage, 2005; Hart, Chris. Doing a Literature Review:
Releasing the Social Science Research Imagination.
Thousand Oaks, CA: Sage Publications, 1998; Jesson,
Jill. Doing Your Literature Review: Traditional and
Systematic Techniques. Los Angeles, CA: SAGE, 2011;
Knopf, Jeffrey W. "Doing a Literature Review." PS:
Political Science and Politics 39 (January 2006): 127-132;
Ridley, Diana. The Literature Review: A Step-by-Step
Guide for Students. 2nd ed. Los Angeles, CA: SAGE,
2012.

Glass R. L., Ramesh V. &Vessey I. (2004). An analysis of

research in computing disciplines. Communications of

178
 the ACM, Volume 47, Issue 6. ACM Press. [available at
ACM]

Holz, H. J., Applin, A., Haberman, B., Joyce, D., Purchase, H.,
Reed, C. (2006). Research methods in computing: What
are they and how should we teach them,ACM SIGCSE
Bulletin,Volume 38,Issue 4,ACM Press. [available at
ACM]

http://schools.yrdsb.ca/markville.ss/history/honours/research
methods.html

http://thesisadviser.blogspot.com/2013/02/thesis-writing-
definition-of-terms.html

http://thesisadviser.blogspot.com/2013/02/thesis-writing-
scope-and-limitation-of.html

http://www.c.u-
tokyo.ac.jp/eng_site/info/academics/grad/purpose/

https://classroom.synonym.com/differences-between-
standardized-nonstandardized-assessments-4442.html

https://explorable.com/case-study-research-design

https://explorable.com/experimental-research

https://geographymaterials.blogspot.com/2015/02/problem-
research-problem-in-general.html

https://managementhelp.org/evaluation/program-
evaluation-guide.htm

https://researchrundowns.com/quantitative-
methods/instrument-validity-
reliability/https://paws.kettering.edu/~ellis/mycourse/ss
ci201/r-d-g-t-good-bad.htm

179
https://simplyeducate.me/2015/01/05/conceptual-
framework-guide/

https://www.editage.com/insights/the-basics-of-writing-a-
statement-of-the-problem-for-your-research-proposal

https://www.khanacademy.org/math/statistics-
probability/designing-studies/sampling-methods-
stats/a/sampling-methods-
review?utm_account=Grant&utm_campaignname=Gr
ant_Math_Dynamic&gclid=Cj0KCQiArenfBRCoARIsAFc
1FqeZKSdj7ey4zlPuBz-
4VRJnFhHQ4ZGKNzBjtw0x_h8IfAeC20RMRg4aAq-
MEALw_wcB

https://www.questionpro.com/blog/descriptive-research/

https://www.spotless.co.uk/insights/ethnography-when-
and-how/

https://www.statisticssolutions.com/how-to-begin-writing-
your-theoretical-framework/

https://www.statisticssolutions.com/research-hypotheses/

Huenneke, L.F., Stearns, D.M., Martinez, J.D. et al. Innov High

Educ (2017) 42: 421. https://doi.org/10.1007/s10755-
017-9394-y

The Conceptual Framework. College of Education. Alabama

State University; Corvellec, Hervé, ed. What is Theory?:
Answers from the Social and Cultural Sciences.
Stockholm: Copenhagen Business School Press, 2013;
Asher, Herbert B. Theory-Building and Data Analysis in
the Social Sciences. Knoxville, TN: University of
Tennessee Press, 1984; Drafting an Argument.
Writing@CSU. Colorado State University; Ravitch, Sharon
M. and Matthew Riggan. Reason and Rigor: How
180
 Conceptual Frameworks Guide Research. Second
edition. Los Angeles, CA: SAGE, 2017; Trochim, William
M.K. Philosophy of Research. Research Methods
Knowledge Base. 2006; Jarvis, Peter. The Practitioner-
Researcher. Developing Theory from Practice. San
Francisco, CA: Jossey-Bass, 1999.

The Conceptual Framework. College of Education. Alabama

State University; Conceptual Framework: What Do You
Think is Going On? College of Engineering. University of
Michigan;Drafting an Argument. Writing@CSU.
Colorado State University; Lynham, Susan A. “The
General Method of Theory-Building Research in Applied
Disciplines.” Advances in Developing Human
Resources 4 (August 2002): 221-241; Tavallaei, Mehdi
and Mansor Abu Talib. "A General Perspective on the
Role of Theory in Qualitative Research." Journal of
International Social Research3 (Spring 2010); Ravitch,
Sharon M. and Matthew Riggan. Reason and Rigor:
How Conceptual Frameworks Guide Research. Second
edition. Los Angeles, CA: SAGE, 2017; Reyes,
Victoria.Demystifying the Journal Article. Inside Higher
Education; Trochim, William M.K. Philosophy of
Research. Research Methods Knowledge Base. 2006;
Weick, Karl E. “The Work of Theorizing.” InTheorizing in
Social Science: The Context of Discovery. Richard
Swedberg, editor. (Stanford, CA: Stanford University
Press, 2014), pp. 177-194.

Two Tips on How to Write the Significance of the Study.

In SimplyEducate.Me. Retrieved
from https://simplyeducate.me/2015/02/09/two-tips-on-
how-to-write-the-significance-of-the-study/

181
182