 Attitude Measurement

Attitude is defined as the predisposition to respond to an idea or object

and in marketing it relates to the consumer’s predisposition to respond
to a particular product or service”

Attitudes are composed of

1) Beliefs about the subject

2) Emotional feeling (like-dislike)
3) Readiness to respond behaviorally- i.e. buy

More about Attitudes

 Attitudes are complex and not fully understood.

 Attitudes can be changed but they tend to persist.
 Attitude Measurement tends to focus on beliefs and emotional
feelings.

Questionnaire methods (i.e self-reports verbal reactions to

situations)
Observation methods (actions or physiological reaction like
sweating, pupils dilation etc).

 Specific Methods of Attitude Measurement

 A method is structured when there is a formal structure or
procedure for the questioning.
 It is disguised when the respondent does not know the purpose of
the interview.

1) Non- Structured methods- Non-Disguised Methods(dis·guised,

dis·guis·ing. to change the appearance or guise of so as to
conceal identity or mislead):
 Depth Interviews: conducted by psychologists in depth
discussion around the subject and each aspect introduced.
Respondent is encouraged to open up and depends on
interviewer’s skill and imagination. It is costly and also gives
errors due to small sample.

 Focus group Interviews: Here groups are interviewed and it

reduces costs and stimulates respondents by group
interaction. However, domination by one or more
individuals is possible.

2) Non Structured – disguised Methods:

These help to get responses in some cases where respondents
would not open up. Projective techniques are used to see
conscious/unconscious attitudes.

 Word associations: First reaction to words related to the

subject, responses are timed so as to take in to account
answers which are ‘reasoned out’. It’s like a game but
requires skill to interpret results.
  Sentence Completion: Respondent asked to complete
sentences with first thought and time. Here some
respondents may be able to see thru the disguises: “A
person who reads India today is……”
 Story Completion: give ending to a story.
 Pictorial Techniques: here pictures are used as stimuli:
 Thematic Apperception Test: shows series of ambiguous
pictures and respondents are asked to tell a story. It is
held that in describing the characters/ actions the
respondent indirectly tells about himself.
3) Structured Non- Disguised Techniques: (Scaling) since disguise
makes measurement indirect and comparisons become difficult
hence development of standardized attitude measuring
instruments is necessary.
 Ordinal scales: rank respondent according to some
characteristics or rank items/ brands in order of
preference.
 Interval Scales: Give items/individuals rank order plus
measure distance between rank positions in equal units.

Self-Rating Scales: Respondents classify themselves in one or two

categories (or do not know) categories. Further refined to give
additional alternatives in degrees of like or dislike. 2 variations of self-
rating scale are:

1) Graphic Rating Scales:

Graphic rating scales can be used to measure employee behavior,

sentiments, and aptitudes. Because these qualities vary across roles or
areas of work, the prompts used in employee assessments may vary as
well.

Some of the commonly measured behaviors are:

 Ability or willingness to communicate

 Quality of work
 Leadership
 Self-motivation or initiative
 Timeliness or punctuality
Example

Statement: This employee is a team player and thrives in a team

environment.
Answer:

Select the most appropriate statement from below.

1. Extremely unlikely (does not work well in a team,

sabotages(damage/destroy) colleagues, is difficult to work with)

2. Quite unlikely (doesn’t particularly enjoy working in a group)

3. Hard to say (hasn’t worked in many team situations, is neither

good nor bad at working in a group)

4. Quite likely (team player, enjoys working with colleagues,

colleagues want to work with them)

5. Extremely likely (always thrives in any team environment, exceeds

expectations within a team).

Example 3

Question: Rate the employee on their quality of work.

Answer: Give the employee a score between 1 and 10.

1 = The employee’s quality of work is consistently poor.

10 = The employee’s quality of work is always excellent.

Advantages of the graphic rating scale method

Let’s have a look at some of the graphic rating scale advantages and

why it’s generally considered an effective performance appraisal

method.

 Easy to understand and use

The graphic rating scale method is straightforward and easy to

understand. Most people have likely filled out a similar questionnaire in

the past; therefore, it requires minimal extra training to implement and

can be grasped by both managers and employees. That’s why it’s a

popular choice for performance rating.

 Quantifying behaviors makes the appraisal system easier

This method quantifies behavioral traits, making it far easier to analyze

the feedback and spot trends, patterns, and developments. Areas for

improvement can be easily identified and actioned, and progress can be

tracked. This streamlines the entire appraisal system.

 Inexpensive to develop
A graphic rating scale can be developed quickly, while many questions

will overlap across roles in the organization, which means they can be

used again. This makes it an affordable employee performance method

for almost all organizations.

Limitations of the graphic rating scale method

As with all performance appraisal forms, there are some limitations to

this type of performance appraisal method.

 Subjectivity of different evaluators

Although everyone in the organization may be using the same

numerical scale, this doesn’t guarantee that the way manager’s rate

employees will be uniform across the board. Each person will have a

different perception of the scale and what each rating means. What
one manager sees as “excellent”, another one might take for granted

and rate it as “good”.

Similarly, there may be subjectivity between the scores a manager gives

and how an employee receives them.

 Different types of biases

One of the most common biases of this method is the “halo effect,”

which can skew the evaluation results. This happens when employees

have a handful of exceptional strengths which overshadow any glaring

weaknesses that need addressing. The reverse is also a possibility—one

significant weakness can bring down an overall score and detract from

an employee’s many strengths. There’s also the recency effect – the

tendency to remember the most recent events best.

  Difficult to understand employees’ strengths

The scores are added at the end of the questionnaire, and each

employee is given an average final score. However, as mentioned

above, this may not accurately represent an employee’s total

performance, which can be considered a disadvantage.

For example, employee A consistently scores as average across the

scale, while employee B scores at both extreme ends of the scale, but

they will end up with a similar total score. Therefore, it would be wrong

to suggest similar changes and improvements to both these employees,

as they are clearly in different situations.

Semantic Differential Scales

Semantic differential scales are a popular type of survey question used
in market research to measure attitudes toward a particular concept or
object.

This type of scale uses a series of bipolar adjectives or phrases to

evaluate the respondent's perception and evaluation of a concept or
object.

In this blog post, our market research company explains what semantic
differential scales are, how they work, and provide examples of
different types of questions to help you understand how to use them in
your research.

Semantic Differential Scale: Definition

A semantic differential scale is a type of survey question that asks

respondents to select a specific rating with the two endpoints of each
scale being opposites.

It works much like a 1 to 5 or 1 to 10 Likert scale but instead of "1"

being least and "10" being most the numbers are replaced with words
opposite one another.
An example of a semantic differential scale is a fast food restaurant
asking: "How were the french fries you recently purchased from our
restaurant?"

As you can see from the picture above, one side of the scale is cold, and
the other is hot.

In between each word are radio buttons to allow the participant to lean
in one direction or another or select a midpoint.

Semantic Differential Scales vs. Likert Scales

Although the two terms are often used synonymously, semantic

differential scales and Likert scale are two different types of rating
scales used in research and surveys.

In short, the key difference between the two scales is that semantic
differential scales measure attitudes towards an object using a set of
bipolar adjectives, while the Likert scales measure the degree of
agreement or disagreement with a statement or question.

The differences between semantic differential scales and Likert scales

are outlined by our online survey agency in greater detail below.

Semantic Differential Scale:

 Semantic Differential Scale is a rating scale used to measure the

attitudes and opinions of respondents toward an object, person,
event, or idea.
 It uses a set of bipolar adjectives, such as "good-bad," "happy-
sad," "strong-weak," etc., placed at opposite ends of a continuum.
 Respondents are asked to rate the object of interest on each
adjective using a 7-point or 10-point scale.
 The scale measures the perceived distance or gap between two
opposite concepts, and thus, provides information about the
attitude of the respondent towards the object.

Likert Scale:

 Likert Scale is a rating scale used to measure the degree of

agreement or disagreement of respondents with a statement or
question.
 It consists of a set of statements or questions, and respondents
are asked to indicate their level of agreement or disagreement
with each statement using a 5-point or 7-point scale.
 The scale measures the intensity of the respondent's attitude
toward a statement or question.

Types of Semantic Differential Scale Questions (with Examples)

As discussed, This type of question uses a series of bipolar adjectives or
phrases to evaluate the attitude of a respondent toward a particular
concept or object.

Here are some examples of different types of semantic differential

scale questions:

1. Evaluative Semantic Differential Scale: Measures the overall

attitude of a respondent towards a concept or object. Example:
Please rate your attitude towards the new product as pleasant or
unpleasant.

2. Potency Semantic Differential Scale: Measures the perceived

strength or power of a concept or object. Example: Please rate
the product as strong or weak.

3. Activity Semantic Differential Scale: Measures the level of activity

associated with a concept or object. Example: Please rate the new
exercise program as active or inactive.

4. Control Semantic Differential Scale: Measures the level of control

that a respondent feels towards a particular concept or
object. Example: Please rate the new software as controllable or
uncontrollable.

5. Arousal Semantic Differential Scale: Measures the level of

excitement or arousal associated with a concept or
object. Example: Please rate the new movie as exciting or boring.

6. Evaluation of Product Attributes Semantic Differential

Scale: Measures the perception of different attributes of a
product. Example: Please rate the new car on the basis of its
comfort level as comfortable or uncomfortable.
What is an Example of a Semantic Differential Scale?

Let's say you want to better understand your brand. You want to
understand associations, image, perceptions, and other affiliations with
your company brand and mission.

If you were to create a semantic differential scale around this, it might

look something like this, using these words.

The example above details a basic semantic differential scale for brand
perception. Sometimes it is helpful to mix up the positives and
negatives on each side of the scale.

Final Thoughts

Semantic differential scale survey questions are a powerful tool for

measuring attitudes toward various concepts or factors. They allow
market research companies, like Drive Research, to collect quantitative
or measurable data on the perceptions of respondents on a variety of
attributes.

We hope the examples provided in this blog post have helped you
understand how to successfully use semantic differential scales in your
surveys.

If you're looking for more help with survey design, programming,

fieldwork, and analysis - we can help with that too! Our contact
information is listed below.

What is a Constant Sum Scale in Market Research

A constant sum scale is a type of question used in a market research

survey in which respondents are required to divide a specific number of

points or percents as part of a total sum. The allocation of points are

divided to detail the variance and weight of each category.

Constant sum scales are a less frequently used question in surveys

when compared to basic likert scales, single radio responses, or

checklists (i.e. multiple response options). They are an excellent way to

create variance among a data set and truly understand which factors

are key and which are not for customers or respondents.

They are especially helpful if you need to ask a question to a customer

or respondent where you believe several factors are critical or of high

importance. You are more likely to create differentiation in the data

with a constant sum when compared to other question types.

Survey writing is a science in market research. With any given objective

there may be 10, 20, or 30+ ways to ask the question. Part of the

expertise in survey writing comes from using the right question format
to best address an objective. It doesn't always involve picking the

easiest yes or no option, or scale. There are more complex ways to

address survey topicsThis is the main reason why it is so important to

lean on someone or a firm in the industry who has a deep background

in survey writing and market research. The outcomes or results from a

question are often unknown. Unfortunately, when you are reviewing

the analysis it is too late to come to the determination you should have

asked the question differently.

This is not a spot you want to be in as a brand commissioning marketing

research in-house. The expertise of a market research company

includes foresight into how results will pan out for analysis including

how the results can be interpreted and action can be taken. It is an

invaluable service and although anyone thinks they can write a survey
question, there are much deeper levels of understanding that help

formulate the best possible question and outcome for your project.

Let's dive into this a little deeper.

What is a Constant Sum Scale in Market Research?

Our Market Research 101 series discusses the basics of surveys and

other common methods used in the industry. Try reading our Market

Research 101 category for more information.

Example of a Single Response Question

The best way to explain how this works is to walk you through an

example of a constant sum scale question in a survey. This is a

theoretical question. Let's say you want to understand what factor(s)

matter most to a consumer when purchasing a home. There are

several ways you can ask this.

One option is a single response question:

Q: What one factor is most important to you when buying a home?

Select one.

A: Price, Location, School District, Inside Features, etc.

You are likely to get a mix of different responses here and your results

may look like: price (40%), location (25%), school district (10%), inside

features, (5%), etc. However, you do lack some context as to exactly

how much more important price is compared to others. For some price

may be very important but for others, it is not. However, a single

response is likely to create more differentiation than a multiple

response or likert scale option.

But there is always a risk that each option comes back with an equal

share within the margin of error. The results may appear as price (20%),

location (20%), school district (20%), inside features (20%), etc. It is

near impossible to for anyone to understand which of these choices

may be more or less important than the other.

Example of a Multiple Response Option

Next, let's try to ask the question as a checklist or multiple response

options. This would be asked as follows.

Q: What factor(s) are most important to you when buying a home?

Select all that apply.

A: Price, Location, School District, Inside Features, etc.

Here, if all of the categories are deemed very important by the

respondent you may end up with response breakdowns like price

(96%), location (93%), school district (85%), inside features (83%), etc.

This creates even less variance among responses making it more

difficult to differentiate.
Similar to the single response question, this also runs the risk of diluting

the analysis by respondents seemingly selecting everything as

important.

Example of a Likert Scale

As a third option, you may want to ask this question using a 1 to 5 or 1

to 10 scale to ask respondents to rate the importance of each factor.

Q: Using a scale of 1 to 5 where "1" is not at all important and "5" is

very important, how important are each of the following when buying a

home? Select a rating for each.

A: Price, Location, School District, Inside Features, etc. with 1 to 5 scale.

For the analysis, you can run a top-2 box breakdown which tallies up

those who rated each a "4" or "5" to provide percentages for each

category.

The other option is creating a mean score for each which may look like:

price (4.9), location (4.8), school district (4.4), etc. Again, the variance is

not strong here so it is difficult to interpret. The key takeaway may be

all of these factors are important.

Example of a Constant Sum Scale Option

Finally, let's give an example of the constant sum scale question. This is

asked in the following manner in your survey. It forces the respondent

to slow down a bit and think about how important each factor is as

they allot points.

Q: Using 100 points, please apply a number of points to each factor

based on how important each is to you when buying a home. You must

total 100 points divided among the factors.

A: Price, Location, School District, Inside Features, etc.

The respondent is given 100 points. They may choose to apply 80 to

price, 15 to location, and spread out the remaining 5 points among

other factors. When you analyze this data set, the differentiation

between factors becomes evident. Most survey software will

automatically tally and sum the point values to ensure they add to a

constant sum of 100.

This constant sum scale adds another layer of analytical thinking for the

respondent rather than just selecting one, running through a checklist

of choices, or selecting from a grid or scaling question. It forces

respondents to slow down and understand the relative value of each

factor and compare the importance of one over another. It maximizes

the chances of creating differentiation between your choices.

Always consult with a market research or survey design expert when

undertaking new projects. These analysts and firms can point you in the

right direction or even design the project from end-to-end for you.
Rating Scale

Forced ranking is a system in which employees are ranked from

best to worst based on their performance. This system can be
used to identify top talent, to help managers identify employees
who need development, and to provide a framework for awarding
bonuses and promotions. Forced ranking is a system in which
employees are ranked from best to worst based on their
performance. This system can be used to identify top talent, to
 help managers identify employees who need development, and to
provide a framework for awarding bonuses and promotions.

Applications
However, one common use of forced ranking is to help identify
high and low performers within a company. This information can
then be used to determine who should be promoted, who should
be given additional training, and who should be let go.

What is sampling?
Sampling is a technique of selecting individual members or a subset of the
population to make statistical inferences from them and estimate the
characteristics of the whole population. Different sampling methods are widely
used by researchers in market research so that they do not need to research
the entire population to collect actionable insights.

It is also a time-convenient and cost-effective method and hence forms the

basis of any research design. Sampling techniques can be used in research
survey software for optimum derivation.

For example, suppose a drug manufacturer would like to research the

adverse side effects of a drug on the country’s population. In that case, it is
almost impossible to conduct a research study that involves everyone. In this
case, the researcher decides on a sample of people from
each demographic and then researches them, giving him/her indicative
feedback on the drug’s behavior.

Learn more about Audience by QuestionPro

Types of sampling: sampling methods

Sampling in market action research is of two types – probability sampling and
non-probability sampling. Let’s take a closer look at these two methods of
sampling.

1. Probability sampling: Probability sampling is a sampling technique

where a researcher selects a few criteria and chooses members of a
population randomly. All the members have an equal opportunity to
participate in the sample with this selection parameter.
2. Non-probability sampling: In non-probability sampling, the researcher
randomly chooses members for research. This sampling method is not
a fixed or predefined selection process. This makes it difficult for all
population elements to have equal opportunities to be included in a
sample.

This blog discusses the various probability and non-probability sampling

methods you can implement in any market research study.

LEARN ABOUT: Survey Sampling

Types of probability sampling with examples:

Probability sampling is a technique in which researchers choose samples from
a larger population based on the theory of probability. This sampling method
considers every member of the population and forms samples based on a
fixed process.

For example, in a population of 1000 members, every member will have a

1/1000 chance of being selected to be a part of a sample. Probability
sampling eliminates sampling bias in the population and allows all members to
be included in the sample.

There are four types of probability sampling techniques:

 Simple random sampling: One of the best probability sampling
techniques that helps in saving time and resources is the Simple
Random Sampling method. It is a reliable method of obtaining
information where every single member of a population is chosen
randomly, merely by chance. Each individual has the same probability
of being chosen to be a part of a sample.
For example, in an organization of 500 employees, if the HR team
decides on conducting team-building activities, they would likely prefer
picking chits out of a bowl. In this case, each of the 500 employees has
an equal opportunity of being selected.
 Cluster sampling: Cluster sampling is a method where the researchers
divide the entire population into sections or clusters representing a
population. Clusters are identified and included in a sample based on
demographic parameters like age, sex, location, etc. This makes it very
simple for a survey creator to derive effective inferences from the
feedback.
For example, suppose the United States government wishes to evaluate
the number of immigrants living in the Mainland US. In that case, they
can divide it into clusters based on states such as California, Texas,
Florida, Massachusetts, Colorado, Hawaii, etc. This way of conducting a
survey will be more effective as the results will be organized into states
and provide insightful immigration data.
 Systematic sampling: Researchers use the systematic sampling
method to choose the sample members of a population at regular
intervals. It requires selecting a starting point for the sample and sample
size determination that can be repeated at regular intervals. This type of
sampling method has a predefined range; hence, this sampling
technique is the least time-consuming.
For example, a researcher intends to collect a systematic sample of 500
people in a population of 5000. He/she numbers each element of the
population from 1-5000 and will choose every 10th individual to be a
part of the sample (Total population/ Sample Size = 5000/500 = 10).
 Stratified random sampling: Stratified random sampling is a method in
which the researcher divides the population into smaller groups that
don’t overlap but represent the entire population. While sampling, these
groups can be organized, and then draw a sample from each group
separately.
For example, a researcher looking to analyze the characteristics of
people belonging to different annual income divisions will create strata
(groups) according to the annual family income. Eg – less than $20,000,
$21,000 – $30,000, $31,000 to $40,000, $41,000 to $50,000, etc. By
 doing this, the researcher concludes the characteristics of people
belonging to different income groups. Marketers can analyze which
income groups to target and which ones to eliminate to create a
roadmap that would bear fruitful results.

LEARN ABOUT: Purposive Sampling

Uses of probability sampling

There are multiple uses of probability sampling:

 Reduce Sample Bias: Using the probability sampling method,

the research bias in the sample derived from a population is negligible
to non-existent. The sample selection mainly depicts the researcher’s
understanding and inference. Probability sampling leads to higher-
quality data collection as the sample appropriately represents the
population.
 Diverse Population: When the population is vast and diverse, it is
essential to have adequate representation so that the data is not
skewed toward one demographic. For example, suppose Square would
like to understand the people that could make their point-of-sale
devices. In that case, a survey conducted from a sample of people
across the US from different industries and socio-economic
backgrounds helps.
 Create an Accurate Sample: Probability sampling helps the
researchers plan and create an accurate sample. This helps to obtain
well-defined data.

Types of non-probability sampling with examples

The non-probability method is a sampling method that involves a collection of

feedback based on a researcher or statistician’s sample selection capabilities
and not on a fixed selection process. In most situations, the output of a survey
conducted with a non-probable sample leads to skewed results, which may
not represent the desired target population. But, there are situations, such as
the preliminary stages of research or cost constraints for conducting research,
where non-probability sampling will be much more useful than the other type.

Four types of non-probability sampling explain the purpose of this sampling

method in a better manner:
 Convenience sampling: This method depends on the ease of access
to subjects such as surveying customers at a mall or passers-by on a
busy street. It is usually termed as convenience sampling because of
the researcher’s ease of carrying it out and getting in touch with the
subjects. Researchers have nearly no authority to select the sample
elements, and it’s purely done based on proximity and not
representativeness. This non-probability sampling method is used when
there are time and cost limitations in collecting feedback. In situations
with resource limitations, such as the initial stages of research,
convenience sampling is used.
For example, startups and NGOs usually conduct convenience
sampling at a mall to distribute leaflets of upcoming events or promotion
of a cause – they do that by standing at the mall entrance and giving out
pamphlets randomly.
 Judgmental or purposive sampling: Judgmental or purposive
samples are formed at the researcher’s discretion. Researchers purely
consider the purpose of the study, along with the understanding of the
target audience. For instance, when researchers want to understand the
thought process of people interested in studying for their master’s
degree. The selection criteria will be: “Are you interested in doing your
masters in …?” and those who respond with a “No” are excluded from
the sample.
 Snowball sampling: Snowball sampling is a sampling method that
researchers apply when the subjects are difficult to trace. For example,
surveying shelterless people or illegal immigrants will be extremely
challenging. In such cases, using the snowball theory, researchers can
track a few categories to interview and derive results. Researchers also
implement this sampling method when the topic is highly sensitive and
not openly discussed—for example, surveys to gather information about
HIV Aids. Not many victims will readily respond to the questions. Still,
researchers can contact people they might know or volunteers
associated with the cause to get in touch with the victims and collect
information.
 Quota sampling: In Quota sampling, members in this sampling
technique selection happens based on a pre-set standard. In this case,
as a sample is formed based on specific attributes, the created sample
will have the same qualities found in the total population. It is a rapid
method of collecting samples.
Uses of non-probability sampling

Non-probability sampling is used for the following:

 Create a hypothesis: Researchers use the non-probability sampling

method to create an assumption when limited to no prior information is
available. This method helps with the immediate return of data and
builds a base for further research.
 Exploratory research: Researchers use this sampling technique widely
when conducting qualitative research, pilot studies, or exploratory
research.
 Budget and time constraints: The non-probability method when there
are budget and time constraints, and some preliminary data must be
collected. Since the survey design is not rigid, it is easier to pick
respondents randomly and have them take the survey or questionnaire.

How do you decide on the type of sampling to use?

For any research, it is essential to choose a sampling method accurately to

meet the goals of your study. The effectiveness of your sampling relies on
various factors. Here are some steps expert researchers follow to decide the
best sampling method.

 Jot down the research goals. Generally, it must be a combination of

cost, precision, or accuracy.
 Identify the effective sampling techniques that might potentially achieve
the research goals.
 Test each of these methods and examine whether they help achieve
your goal.
 Select the method that works best for the research.

Difference between probability sampling and non-

probability sampling methods
We have looked at the different types of sampling methods above and their
subtypes. To encapsulate the whole discussion, though, the significant
differences between probability sampling methods and non-probability
sampling methods are as below:

Probability Sampling Methods Non-Probability Sampling Methods

 Probability Sampling is a sampling
technique in which samples from a larger
Definition
population are chosen using a method
based on the theory of probability.
Non-probability sampling is a sampling
technique in which the researcher selects
samples based on the researcher’s subjective
judgment rather than random selection.

Alternatively
Random sampling method. Non-random sampling method
Known as

Population
The population is selected randomly. The population is selected arbitrarily.
selection

Nature The research is conclusive. The research is exploratory.

Since there is a method for deciding the Since the sampling method is arbitrary, the
Sample sample, the population demographics are population demographics representation is
conclusively represented. almost always skewed.

Takes longer to conduct since the

This type of sampling method is quick since
research design defines the selection
Time Taken neither the sample nor the selection criteria of
parameters before the market research
the sample are undefined.
study begins.

This type of sampling is entirely biased, and

This type of sampling is entirely unbiased;
Results hence the results are biased, too, rendering
hence, the results are also conclusive.
the research speculative.

In probability sampling, there is an

underlying hypothesis before the study In non-probability sampling, the hypothesis is
Hypothesis
begins, and this method aims to prove the derived after conducting the research study.
hypothesis.
Pilot studies are a fundamental stage of the research process. They can help
identify design issues and evaluate a study’s feasibility, practicality, resources,
time, and cost before the main research is conducted.
It involves selecting a few people and trying out the study on them. It is
possible to save time and, in some cases, money by identifying any flaws in the
procedures designed by the researcher.

A pilot study can help the researcher spot any ambiguities (i.e., unusual
things), confusion in the information given to participants, or problems with
the task devised.

Sometimes the task is too hard, and the researcher may get a floor effect
because none of the participants can score at all or can complete the task – all
performances are low.

The opposite effect is a ceiling effect, when the task is so easy that all achieve
virtually full marks or top performances and are “hitting the ceiling.”

This enables researchers to predict an appropriate sample size, budget

accordingly, and improve the study design before performing a full-scale
project.

Pilot studies also provide researchers with preliminary data to gain insight
into the potential results of their proposed experiment.

However, pilot studies should not be used to test hypotheses since the
appropriate power and sample size are not calculated. Rather, pilot studies
should be used to assess the feasibility of participant recruitment or study
design.

By conducting a pilot study, researchers will be better prepared to face the

challenges that might arise in the larger study. They will be more confident
with the instruments they will use for data collection.

Multiple pilot studies may be needed in some studies, and qualitative and/or
quantitative methods may be used.

To avoid bias, pilot studies are usually carried out on individuals who are as
similar as possible to the target population but not on those who will be a part
of the final sample.
 Feedback from participants in the pilot study can be used to improve the
experience for participants in the main study. This might include reducing the
burden on participants, improving instructions, or identifying potential ethical
issues.

Experiment Pilot Study

In a pilot study with an experimental design, you would want to ensure that
your measures of these variables are reliable and valid.

You would also want to check that you can effectively manipulate your
independent variables and that you can control for potential confounding
variables.

A pilot study allows the research team to gain experience and training, which
can be particularly beneficial if new experimental techniques or procedures
are used.

Questionnaire Pilot Study

It is important to conduct a questionnaire pilot study for the following reasons:
 Check that respondents understand the terminology used in the
questionnaire.

 Check that emotive questions are not used, as they make people defensive
and could invalidate their answers.

 Check that leading questions have not been used as they could bias the
respondent’s answer.

 Ensure that the questionnaire can be completed in a reasonable amount of

time. If it’s too long, respondents may lose interest or not have enough time
to complete it, which could affect the response rate and the data quality.
 Advantages
 By identifying and addressing issues in the pilot study, researchers can
reduce errors and risks in the main study. This increases
the reliability and validity of the main study’s results.
 Assessing the practicality and feasibility of the main study
 Testing the efficacy of research instruments

 Identifying and addressing any weaknesses or logistical problems

 Collecting preliminary data
 Estimating the time and costs required for the project
 Determining what resources are needed for the study
 Identifying the necessity to modify procedures that do not elicit useful data
 Adding credibility and dependability to the study
 Pretesting the interview format
 Enabling researchers to develop consistent practices and familiarize
themselves with the procedures in the protocol
 Addressing safety issues and management problems

Limitations
 Require extra costs, time, and resources.
 Do not guarantee the success of the main study.
 Contamination (ie: if data from the pilot study or pilot participants are
included in the main study results).
 Funding bodies may be reluctant to fund a further study if the pilot study
results are published.
 Do not have the power to assess treatment effects due to small sample size.

A t-test is an inferential statistic used to determine if there is a

significant difference between the means of two groups and how they
are related. T-tests are used when the data sets follow a normal
distribution and have unknown variances, like the data set recorded
from flipping a coin 100 times.

When to use a t test

A t test can only be used when comparing the means of two groups (a.k.a. pairwise
comparison). If you want to compare more than two groups, or if you want to do multiple
pairwise comparisons, use an ANOVA test or a post-hoc test.

The t test is a parametric test of difference, meaning that it makes the same
assumptions about your data as other parametric tests. The t test assumes your data:

1. are independent
2. are (approximately) normally distributed
3. have a similar amount of variance within each group being compared (a.k.a.
homogeneity of variance)

ne-sample, two-sample, or paired t test?

 If the groups come from a single population (e.g., measuring before and after an
experimental treatment), perform a paired t test. This is a within-subjects design.
 If the groups come from two different populations (e.g., two different species, or people
from two separate cities), perform a two-sample t test (a.k.a. independent t test). This
is a between-subjects design.
 If there is one group being compared against a standard value (e.g., comparing the
acidity of a liquid to a neutral pH of 7), perform a one-sample t test.

One-tailed or two-tailed t test?

 If you only care whether the two populations are different from one another, perform
a two-tailed t test.
 If you want to know whether one population mean is greater than or less than the other,
perform a one-tailed t test.

t test exampleIn your test of whether petal length differs by species:

 Your observations come from two separate populations (separate species), so you perform a
two-sample t test.
 You don’t care about the direction of the difference, only whether there is a difference, so you
choose to use a two-tailed t test.

Performing a t test
The t test estimates the true difference between two group means using the ratio of the
difference in group means over the pooled standard error of both groups. You can
calculate it manually using a formula, or use statistical analysis software.
T test formula
The formula for the two-sample t test (a.k.a. the Student’s t-test) is shown below.

In this formula, t is the t value, x1 and x2 are the means of the two groups being
compared, s2 is the pooled standard error of the two groups, and n1 and n2 are the
number of observations in each of the groups.

A larger t value shows that the difference between group means is greater than the
pooled standard error, indicating a more significant difference between the groups.

ANOVA Test is used to analyze the differences among the means of various
groups using certain estimation procedures. ANOVA means analysis of
variance. ANOVA test is a statistical significance test that is used to check
whether the null hypothesis can be rejected or not during hypothesis testing.

An ANOVA test can be either one-way or two-way depending upon the

number of independent variables. In this article, we will learn more about an
ANOVA test, the one-way ANOVA and two-way ANOVA, its formulas and see
certain associated examples.

What is ANOVA Test?

ANOVA test, in its simplest form, is used to check whether the means of three
or more populations are equal or not. The ANOVA test applies when there are
more than two independent groups. The goal of the ANOVA test is to check
for variability within the groups as well as the variability among the groups.
The ANOVA test statistic is given by the f test.
ANOVA Test Definition
ANOVA test can be defined as a type of test used in hypothesis testing to
compare whether the means of two or more groups are equal or not. This test
is used to check if the null hypothesis can be rejected or not depending upon
the statistical significance exhibited by the parameters. The decision is made
by comparing the ANOVA test statistic with the critical value.
ANOVA Test Example
Suppose it needs to be determined if consumption of a certain type of tea will
result in a mean weight loss. Let there be three groups using three types of
tea - green tea, earl grey tea, and jasmine tea. Thus, to compare if there was
any mean weight loss exhibited by a certain group, the ANOVA test (one way)
will be used.

Suppose a survey was conducted to check if there is an interaction between

income and gender with anxiety level at job interviews. To conduct such a test
a two-way ANOVA will be used.

ANOVA Formula

There are several components to the ANOVA formula. The best way to solve
a problem on an ANOVA test is by organizing the formulas into an ANOVA
table. The ANOVA formulas are given below.

Sum of squares between groups, SSB = ∑nj(¯¯¯¯¯Xj−¯¯¯¯¯X)2∑𝑛𝑗(𝑋¯𝑗−𝑋¯)2.

Here, ¯¯¯¯¯Xj𝑋¯𝑗 is the mean of the jth group, ¯¯¯¯¯X𝑋¯ is the overall mean
and nj𝑛𝑗 is the sample size of the jth group.
¯¯¯¯¯X𝑋¯ = ¯¯¯¯¯X1+¯¯¯¯¯X2+¯¯¯¯¯X3+...+¯¯¯¯¯Xjj𝑋¯1+𝑋¯2+𝑋¯3+...+𝑋¯𝑗𝑗
Sum of squares of errors, SSE = ∑∑(X−¯¯¯¯¯Xj)2∑∑(𝑋−𝑋¯𝑗)2. Here, X refers to
each data point in the jth group.

Total sum of squares, SST = SSB + SSE

Degrees of freedom between groups, df1 = k - 1. Here, k denotes the number

of groups.
Degrees of freedom of errors, df2 = N - k, where N denotes the total number of
observations across k groups.
Total degrees of freedom, df3 = N - 1.

Mean squares between groups, MSB = SSB / (k - 1)

Mean squares of errors, MSE = SSE / (N - k)

ANOVA test statistic, f = MSB / MSE

Critical Value at α𝛼 = F(α𝛼, k - 1, N - k)

ANOVA Table
The ANOVA formulas can be arranged systematically in the form of a table.
This ANOVA table can be summarized as follows:

Source Degree
Mean F
of s of
Sum of Squares Square Valu
Variatio Freedo
s e
n m

f=
SSB = MSB =
Between df1 = k - MSB
Σnj(¯¯¯¯¯Xj−¯¯¯¯¯X)2𝑋¯𝑗− SSB /
Groups 1 /
𝑋¯)2 (k - 1)
MSE

MSE =
SSE = df2 = N -
Error SSE /
ΣΣ(X−¯¯¯¯¯Xj)2𝑋−𝑋¯𝑗)2 k
(N - k)

df3 = N -
Total SST = SSB + SSE 1

One Way ANOVA

The one way ANOVA test is used to determine whether there is any difference
between the means of three or more groups. A one way ANOVA will have
only one independent variable. The hypothesis for a one way ANOVA test can
be set up as follows:

Null Hypothesis, H0𝐻0: μ1𝜇1 = μ2𝜇2 = μ3𝜇3 = ... = μk𝜇𝑘

Alternative Hypothesis, H1𝐻1: The means are not equal

Decision Rule: If test statistic > critical value then reject the null hypothesis
and conclude that the means of at least two groups are statistically significant.

The steps to perform the one way ANOVA test are given below:

 Step 1: Calculate the mean for each group.

 Step 2: Calculate the total mean. This is done by adding all the means and dividing it by the total
number of means.
 Step 3: Calculate the SSB.
 Step 4: Calculate the between groups degrees of freedom.
 Step 5: Calculate the SSE.
 Step 6: Calculate the degrees of freedom of errors.
 Step 7: Determine the MSB and the MSE.
 Step 8: Find the f test statistic.
 Step 9: Using the f table for the specified level of significance, α𝛼, find the critical value. This is
given by F(α𝛼, df1. df2).
 Step 10: If f > F then reject the null hypothesis.

Limitations of One Way ANOVA Test

The one way ANOVA is an omnibus test statistic. This implies that the test will
determine whether the means of the various groups are statistically significant
or not. However, it cannot distinguish the specific groups that have a
statistically significant mean. Thus, to find the specific group with a different
mean, a post hoc test needs to be conducted.

Two Way ANOVA

The two way ANOVA has two independent variables. Thus, it can be thought
of as an extension of a one way ANOVA where only one variable affects the
dependent variable. A two way ANOVA test is used to check the main effect
of each independent variable and to see if there is an interaction effect
between them. To examine the main effect, each factor is considered
separately as done in a one way ANOVA. Furthermore, to check the
interaction effect, all factors are considered at the same time. There are
certain assumptions made for a two way ANOVA test. These are given as
follows:

 The samples drawn from the population must be independent.

 The population should be approximately normally distributed.
 The groups should have the same sample size.
 The population variances are equal

Suppose in the two way ANOVA example, as mentioned above, the income
groups are low, middle, high. T

Important Notes on ANOVA Test

 ANOVA test is used to check whether the means of three or more groups are different or not
by using estimation parameters such as the variance.
 An ANOVA table is used to summarize the results of an ANOVA test.
 There are two types of ANOVA tests - one way ANOVA and two way ANOVA
 One way ANOVA has only one independent variable while a two way ANOVA has two
independent variables.

Example 2: A trial was run to check the effects of different diets. Positive numbers
indicate weight loss and negative numbers indicate weight gain. Check if there is an
average difference in the weight of people following different diets using an ANOVA
Table.

Low Fat Low Calorie Low Protein Low Carbohydrate

8 2 3 2

9 4 5 2

6 3 4 -1

7 5 2 0

3 1 3 3

Solution:

H0𝐻0: μ1𝜇1 = μ2𝜇2 = μ3𝜇3 = μ4𝜇4

H1𝐻1: The means are not equal

(X (X (X
Low (X Low Low
Low - - -
Calori - Protei Carbohy
Fat 6. 3. 1.
e 3)2 n drate
6)2 4)2 2)2

0. 0.
8 2 2 1 3 2
2 6

5. 2. 0.
9 4 1 5 2
8 6 6

6 0. 3 0 4 0. -1 4.
 (X (X (X
Low (X Low Low
Low - - -
Calori - Protei Carbohy
Fat 6. 3. 1.
e 3)2 n drate
6)2 4)2 2)2

4 4 8

0. 1.
7 5 4 2 2 0
2 4

0. 3.
3 13 1 4 3 3
2 2

To To To
To
¯¯¯¯¯X1 tal ¯¯¯¯¯X2 ¯¯¯¯¯X3 tal tal
tal ¯¯¯¯¯X4𝑋
𝑋¯1 = = 𝑋¯2 = = 𝑋¯3 = = =
21 5. ¯4 = 1.2 10
6.6 3 10 3.4
.4 4 .6

Total mean, ¯¯¯¯¯X𝑋¯ = 3.6

n1𝑛1 = n2𝑛2 = n3𝑛3 = n4𝑛4 = 5, k = 4

SSB
= n1(¯¯¯¯¯X1−¯¯¯¯¯X)2𝑛1(𝑋¯1−𝑋¯)2 + n2(¯¯¯¯¯X2−¯¯¯¯¯X)2𝑛2(𝑋¯2−𝑋¯)2 +& n3(¯¯¯¯¯X3−¯¯¯
¯¯X)2𝑛3(𝑋¯3−𝑋¯)2 +n4(¯¯¯¯¯X4−¯¯¯¯¯X)2𝑛4(𝑋¯4−𝑋¯)2

= 75.8

SSE = 21.4 + 10 + 5.4 + 10.6 = 47.4

Z Test T-Test

A z test is a statistical test that A t-test is used to check if the means of two
is used to check if the means data sets are different when the population
of two data sets are different variance is not known.
 Z Test T-Test

when the population variance

is known.

The sample size is greater

The sample size is lesser than 30.
than or equal to 30.

The data follows a normal

The data follows a student-t distribution.
distribution.

The t test statistic is given

The one-sample z test
statistic is given
by ¯¯¯x−μσ√n𝑥¯−𝜇𝜎𝑛
as ¯¯¯x−μs√n𝑥¯−𝜇𝑠𝑛 where
s is the sample standard
deviation

What is Z Test?

A z test is a test that is used to check if the means of two populations are
different or not provided the data follows a normal distribution. For this
purpose, the null hypothesis and the alternative hypothesis must be set up
and the value of the z test statistic must be calculated. The decision criterion
is based on the z critical value.
Z Test Definition
A z test is conducted on a population that follows a normal distribution with
independent data points and has a sample size that is greater than or equal to
30. It is used to check whether the means of two populations are equal to
each other when the population variance is known. The null hypothesis of a z
test can be rejected if the z test statistic is statistically significant when
compared with the critical value.
Z Test Formula

The z test formula compares the z statistic with the z critical value to test
whether there is a difference in the means of two populations. In hypothesis
testing, the z critical value divides the distribution graph into the acceptance
and the rejection regions. If the test statistic falls in the rejection region then
the null hypothesis can be rejected otherwise it cannot be rejected. The z test
formula to set up the required hypothesis tests for a one sample and a two-
sample z test are given below.
One-Sample Z Test
A one-sample z test is used to check if there is a difference between the
sample mean and the population mean when the population standard
deviation is known. The formula for the z test statistic is given as follows:
z = ¯¯¯x−μσ√n𝑥¯−𝜇𝜎𝑛. ¯¯¯x𝑥¯ is the sample mean, μ𝜇 is the population
mean, σ𝜎 is the population standard deviation and n is the sample size.

The algorithm to set a one sample z test based on the z test statistic is given
as follows:

Left Tailed Test:

Null Hypothesis: H0𝐻0 : μ=μ0𝜇=𝜇0

Alternate Hypothesis: H1𝐻1 : μ<μ0

Mann-Whitney U test is the non-parametric alternative test to the

independent sample t-test. It is a non-parametric test that is used to
compare two sample means that come from the same population, and used
to test whether two sample means are equal or not.

The Mann-Whitney U test is used to compare differences

between two independent groups when the dependent variable is
either ordinal or continuous, but not normally distributed.