Here we focus primarily on testing done during the concept development

phase. In a concept test, the development team solicits a response to a

description of the product concept from potential customers in the
target market. This type of testing may be used to select which of two
or more concepts should be pursued, to gather information from
potential customers on how to improve a concept, and to estimate the
sales potential of the product. Note that various other types of testing
with potential customers may be completed at times other than during
concept development. For example, some kind of customer test, usually
based on only a verbal description of a concept, may be used in
identifying the original product opportunity that forms the basis of the
mission statement for the project. A test may also be used to refine the
demand forecast after the development of a product is nearly complete,
but before a firm commits to full production and launch. Concept testing
is distinct in that it is based on data gathered directly from potential
customers and relies to a lesser degree on judgments made by the
development team. The reason that concept testing generally follows
concept selection is that a team cannot feasibly test more than a few
concepts directly with potential customers. As a result, the team must
first narrow the set of alternatives under consideration to very few
 A team may choose not to do any concept testing at all if the time
required to test the concept is large relative to the product life
cycles in the product category, or if the cost of testing is large
relative to the cost of actually launching the product. For example,
in the Internet software business, some observers and practitioners
argue that just launching a product and iteratively refining it with
subsequent product generations is a better strategy than carefully
testing a concept before developing it fully. While perhaps
appropriate for some products, this strategy would be foolish in the
development of, for example, a new commercial airplane, where
development costs and time are huge and failure can be disastrous.
Most product categories fall between these extremes, and in most
cases some form of concept testing is warranted.
Seven-step method for testing product concepts:
1. Define the purpose of the concept test.
2. Choose a survey population.
3. Choose a survey format.
4. Communicate the concept.
5. Measure customer response.
6. Interpret the results.
7. Reflect on the results and the process.
 As a first step in concept testing, it is
recommended that the team explicitly articulate in
writing the questions that the team wishes to
answer with the test. Concept testing is essentially
an experimental activity, and as with any
experiment, knowing the purpose of the
experiment is essential to designing an effective
experimental method. The primary questions
addressed in concept testing are typically:
 Which of several alternative concepts should be pursued?
 How can the concept be improved to better meet
customer needs?
 Approximately how many units are likely to be sold?
 Should development be continued?
 An assumption underlying the concept test is that the
population of potential customers surveyed reflects that of the
target market for the product. If the survey population is
either more or less enthusiastic about the product than will be
the eventual target audience for the product, then inferences
based on the concept test will be biased. As a result, the team
should choose a survey population that mirrors the target
market in as many ways as possible. In the actual survey, the
first few questions are called the screener questions and
generally are used to verify that the respondent fits the
definition of the target market for the product.
 Often a product addresses multiple market segments. In such
cases, an accurate concept test requires that potential
customers from each target segment be surveyed. Surveying
every possible segment may be prohibitively expensive in cost
or time, and in such cases, the team may choose to survey
potential customers from only the largest segment. However,
when only one segment is sampled, inferences about the
response of the entire market are likely to be biased.
The sample size of the survey should be large
enough that the team’s confidence in the
results is high enough to guide decision making.
Sample sizes for concept testing are sometimes
as small as 10 (e.g., when gathering qualitative
feedback on a new surgical device for a highly
specialized procedure) or as large as 1,000 (e.g.,
when trying to quantitatively assess the potential
demand for a new portable telephone that is
targeted at a market segment comprising 10
million households). Although there are no
simple formulas for determining sample size,
some of the factors driving sample size are
shown in next slide.
 The following formats are commonly used in concept testing:
Face-to-face interaction: In this format, an interviewer interacts directly
with the respondent. Face-to-face interactions can take the form of intercepts
(i.e., stopping people at a mall, in a park, or on a city street), interviews
prearranged by telephone, interviews with potential customers at a trade-show
booth, or focus groups (i.e., pre-arranged group discussions with 6–12 people).
 Telephone: Telephone interviews may be prearranged and targeted at very
specific individuals (e.g., pediatric dentists) or may be “cold calls” of
consumers from a target population.
 Postal mail: In mail surveys, concept-testing materials are sent and
respondents are asked to return a completed form. Postal surveys are
somewhat slower than other methods and suffer from relatively poor
response rates. Some kind of incentive—often cash or a gift—is sometimes
offered to increase response.
 Electronic mail: Electronic mail surveys are very similar to postal mail
surveys, except that (as of this writing) respondents seem slightly more likely
to reply than via postal mail. With the proliferation of unwanted e-mail, this
tendency may not persist. Many electronic mail users react extremely
negatively to unsolicited commercial correspondence. We therefore
recommend that electronic mail surveys be used only when respondents are
likely to perceive a benefit to their participation, or when the team has
already established some kind of positive relationship with the target
population.
 Internet: Using the Internet, a team may create a virtual concept-testing site
in which survey participants can observe concepts and provide responses. An
electronic mail message is usually used to recruit respondents to visit the
test site.
Each of these formats presents risks of sample bias. For example, the
use of electronic formats may bias the sample toward those who are
technologically sophisticated. For some products, this sophistication is
part of the profile of the target market (e.g., the target market for
Internet software products is likely to be comfortable with electronic
survey formats). Conversely, an Internet survey might be a particularly
bad format for testing a television-based computer concept targeted at
people without personal computers.
Exploratory testing, typical in the early phases of concept development,
benefits from open-ended interactive formats. We recommend that the
team use face-to-face formats when presenting multiple concept
alternatives or when soliciting ideas for improving a concept. In these
settings, the product developers themselves benefit from performing the
interviews because they can directly observe reactions to the product in
rich detail. As the purpose of the concept test becomes more focused,
more structured formats such as mail and telephone become more
appropriate. If the questions are very focused, the team can hire a
market research firm to implement the concept test. When gathering
data intended primarily for use in forecasting demand, third parties are
generally used to collect the data in face-to-face formats. This helps to
avoid a sympathy bias—respondents indicating that they like the concept
in order to please an anxious product developer.
 The choice of survey format is closely linked to the way in which the
concept will be communicated. Concepts can be communicated in any of
the following ways, listed in order of increasing richness of the
description.

Verbal description
Sketch
Photos and renderings
Storyboard
Video
Simulation
Interactive multimedia
Physical appearance models
Working prototypes

Matching the Survey Format with the Means of Communicating the Concept
The choice of survey format is tightly linked to the means of communicating
the product concept
 Most concept test surveys first communicate the
product concept and then measure customer response.
When a concept test is performed early in the concept
development phase, customer response is usually
measured by asking the respondent to choose from two or
more alternative concepts. Additional questions focus on
why respondents react the way they do and on how the
product concepts could be improved. Concept tests also
generally attempt to measure purchase intent. The most
commonly used purchase-intent scale has five response
categories:
Definitely would buy.
Probably would buy.
Might or might not buy.
Probably would not buy.
Definitely would not buy.
 If the team is simply interested in comparing two or more concepts,
interpretation of the results is straightforward. If one concept dominates
the others and the team is confident that the respondents understood
the key differences among the concepts, then the team can simply
choose the preferred concept. If the results are not conclusive, the team
may decide to choose a concept based on cost or other considerations,
or may decide to offer multiple versions of the product. Note that care
must be applied in making this judgment for cases in which
manufacturing costs are dramatically different among the concepts
under comparison and in which no price information is communicated to
the respondents. In such cases, respondents may be biased to select the
most costly alternative.
 In many cases the team is also interested in estimating the demand for a
product in the period following launch, usually one year. Here we
present a model for estimating the sales potential of durables. By
durables we mean products that last several years, and for which there
is, therefore, a negligible repeat-purchase rate. These products are in
contrast to consumer packaged goods, like razor blades, toothpaste, or
frozen food, for which forecasting models must consider rates of trial
and subsequent repeat purchase.
 We estimate Q, the quantity of the product expected to be sold
during a time period, as
Q=N xAxP
N is the number of potential customers expected to make
purchases during the time period. For an existing and stable
product category N is the expected number of purchases to be
made of existing products in the category over the time period.
A is the fraction of these potential customers or purchases for
which the product is available and the customer is aware of the
product. (In situations where awareness and availability
are assumed to be separate independent factors, they are
multiplied together to generate A.)
P is the probability that the product is purchased if available and if
the customer is aware of it. P is estimated in turn by
 Fdefinitely is the fraction of survey respondents
indicating in the concept test survey that they would
definitely purchase (often called the “top box” score).
 Fprobably is the fraction of survey respondents
indicating that they would probably purchase (often
called the “second box” score).
 Cdefinitely and Cprobably are calibration constants usually
established based on the experience of a company
with similar products in the past. Generally the values
of Cdefinitely and Cprobably fall in these intervals: 0.10 <
Cdefinitely < 0.50, 0 < Cprobably < 0.25. Absent prior
history, many teams use values of Cdefinitely = 0.4 and
Cprobably = 0.2.
Note that these values reflect the typical bias of respondents to
overestimate the probability that they would actually purchase the
product.
 The primary benefit of the concept test is in getting
feedback from real potential customers. The qualitative
insights gathered through open-ended discussions with
respondents about the proposed concepts may be the
most important result of concept testing, especially early
in the development process.
The team benefits from thinking about the impact of the
three key variables in the forecasting model:
 (1) the overall size of the market, (2) the availability and
awareness of the product, and (3) the fraction of
customers who are likely to purchase. In reflecting on the
results of the concept test, the team should ask two key
diagnostic questions:
 First, was the concept communicated in a way that is likely
to elicit customer response that reflects true intent?
 Second, is the resulting forecast consistent with observed
sales rates of similar products?