WORLD CLASS
DESIGN TO MANUFACTURE
The R&D Effectiveness Index:
A Metric for Product Development Performance
Michael E. McGrath and Michael N. Romeri
Pittiglio Robin Todd & McGrath
“How effective are my company’s product
development efforts? Is our product development
process improving, or possibly deteriorating? We
can measure specific projects but have no way of
measuring how we are doing overall.” This
CEO’s comments are typical of the frustration in
measuring overall product development success.
Many companies are now realizing that both
the number and the success of new products rely
on the performance of their product development
process. Until recently, improving this process
was such a clear opportunity that companies did
not need to measure performance. They just
started to improve their processes. However, now
that some companies have made major
improvements to their product development
process, they need an appropriate way to measure
its performance.
However, product development is much more
difficult to measure than other business processes
and because of this, there are no broadly accepted
performance metrics as there are for other
business processes. For example, in
manufacturing, inventory turnover and gross
margin percentage can be used as metrics of the
manufacturing process.
Some metrics are applied. The investment
community and the press, for example, have
resorted to using the percentage of R&D spending
as a comparative measure – with more spending
as better. This is a little like saying that more
spending on plant and equipment is always better,
even though the appropriate level of capital
investment considers expected return and a
company’s long-term strategy. In the same way,
business executives sometimes compare their
R&D expenditure (as a percent of sales) to that of
their competition. The comparison is only valid if
World Class Design to Manufacture, Vol. 1 No. 4, 1994, pp. 24-31
MCB University Press, 1352-3074
24
their strategies and opportunities are similar and
their product development processes are equally
effective.
Studies have generally shown little correlation
between the level of R&D spending and success.
Some[2,6] have measured limited correlation
between the percentage of R&D spending and
revenue growth. Although these studies show
some relationship, it is not enough to suggest that
spending more on R&D is always better.
Spending 50% of revenue on R&D, for example,
will likely increase revenue, but it is unlikely that
the return on investment would justify this for
very long.
Product development cycle times are a key
performance measure of the product development
process[3]. Short cycle times generally increase
product success and improve development
productivity. However, this does not measure
success across products or guide the amount of
spending on R&D.
Companies are now beginning to focus on the
effectiveness of their R&D investment instead of
how much they invest. They want to get more
from the same level of investment instead of
increasing their investment. Even Business Week’s
annual R&D Scoreboard[1], which ranks
companies by R&D spending, points out that
R&D efficiency is becoming a strategic necessity.
Pittiglio Rabin Todd & McGrath (PRTM)
recently conducted a comprehensive study of
development metrics for electronic systems
products. The study included extensive
benchmarking of time-to-market, sources and
Reprinted by permission of the publisher from “The
R&D Effectiveness Index: A Metric for Product
Development Performance” by Michael E. McGrath
and Michael N. Romeri, Journal of Product Innovation
Management, Vol. 11 No. 3, pp. 213-220.  1994 by
Elsevier Science Inc.
 VOLUME 1 NUMBER 4
1994
uses of R&D, performance to targets, and best
practices. Forty-five companies participated in the
study. Although they varied by size and industry,
all developed similar products – electronic
systems where there is both electronic and
software content. Table I shows a profile of the
participating companies.
Study participants included American,
European, and Asian companies. Each provided
detailed data on its product development
activities, enabling the analysis of the data by
product complexity, type of product, and level of
effort involved. They study’s benchmarks are
used by participants to evaluate their product
development process and provide cycle-time
guidelines for scheduling new development. The
study showed that the participants did not
consistently use any single overall metric to
measure their product development process, but
they indicated that one was badly needed.
As part of this study, PRTM introduced a new
metric for measuring overall product development
performance – the R&D effectiveness index.
R&D Effectiveness Index
The R&D effectiveness index is an aggregate
measure of the overall success of a company’s
product development efforts. It fills the void for
an overall metric at a level above individual
products and development projects, and provides
management with a tool to measure the long-term
effectiveness of its product strategy and product
development process.
Overall or aggregate performance involves
many new products and product development
Industries %
Communication systems 35.6
Industrial/Medical electronics 22.2
Consumer electronics 15.6
Computers 13.3
Peripherals 13.3
100
Size (revenue)
$250 million + 28.2
$100-$250 million 25.6
$50-$100 million 20.5
$25-$50 million 10.3
<$25 million 15.4
100
Table I.
Profile of Participating Companies
25
projects and is therefore much more difficult to
measure. Unsuccessful projects and wasted efforts
need to be offset against successful ones. Projects
with varying degrees of success need to be
weighted together. Finally, R&D investment that
spans several periods needs to be matched against
a payback that spans future periods.
Although these complications limit the
precision of any aggregate performance metric, a
metric – even an approximate one – is needed
to measure overall performance. Without such a
metric, management of the product development
process is purely subjective.
The R&D effectiveness index (EI) compares
the profit from new products to the investment in
new product development, using the following
formula (all% are stated as a percentage of
revenue):
EI =
% New Product Revenue × (Net Profit % + R&D %)
R&D %
As a simple interpretation, the index computes the
ratio of increased profits from new products
divided by the investment in product
development. When the index is above 1.0, the
return from new products is running at a rate
greater than the investment.
For example, assume that a company has a 9%
net profit. Also assume that it invests 6% of
revenue in R&D and that 40% of its revenue is
derived from new products. Its R&D effectiveness
index would be 1.0 [computation: 40% × (9% +
6%) / 6%]. On the other hand, if this same
company derived only 20% of its revenue from
new products then it would have an index of 0.5
(20% × 15% / 6%). It would not be getting as
much from its investment in R&D.
The R&D effectiveness index was computed
for companies participating in the study. The
results were then analyzed and correlated to other
performance factors. Figure 1 shows the
21%
<0.5
39% 0.5 - 1.0
1.0 - 1.25
>1.25
18%
22%
Figure 1.
R&D Effectiveness Index Distribution from the Study
 WORLD CLASS
DESIGN TO MANUFACTURE

distribution of R&D effectiveness index values begins to flatten at that point, consistent with the
derived from the study. 2.5-year new product definition. In other
Of the participants, 39% (21% + 18%) had an industries, the curve would be different.
R&D effectiveness index greater than 1.0, Arguments could be made for different
indicating that new products generated more definitions of new products. For example, the
profit than the investment made in R&D. If any entire product life-cycle could be used in the
time-value of money is considered, then only 21% definition, because the entire life-cycle is used to
(companies with an EI greater than 1.25) evaluate a new product opportunity. This
generated a return on their investment in R&D. definition would include all revenue and could
Interpreting the index properly requires an not distinguish between new and old products.
understanding of the numerator and denominator: The index would simply be the ratio of profit to
new product profit and R&D investment. These R&D investment, losing the importance of new
are subject to differences in interpretation. products.
The definition could be longer or shorter, but
the half-line concept is reasonable. Consistency is
New Product Profit the most important consideration in determining
Computing new product profit starts with the the definition of a new product. Comparisons are
definitions of new products. In the R&D possible when the definition is consistent, and the
effectiveness index, new products are defined as real objective of the metric is relative
those that are still in the first half of their product comparisons, not absolute performance.
life-cycle. For electronic systems, the average New product profit is computed by multiplying
product life-cycle is approximately five years, so the percentage of revenue from products
new product revenue covers products introduced introduced in the last three years by the rate of net
in the last three years. (This averages to be 2.5 profit combined with the percentage of R&D
years because products introduced in the last half spending. Net profit is used instead of gross
of a year are less than 0.5-years-old, whereas margin because selling and administrative costs
others are more than 0.5-years-old.) must be covered by new product revenue. The
In this study, product life-cycle was supported percentage of R&D investment is added back into
by analyzing the new product curve for profitability because the index computes how well
participating companies. The curve plots the profit covers this investment. An EI of 1.1
percentage of revenue from products introduced a represents a return of the original investment, plus
year ago, two years ago, etc. Figure 2 illustrates 10%.
the new product curve for the electronic systems Either actual profit from new products or a
companies in the study. representative average profit can be used. Actual
For 1992, approximately 12% of revenue came profit is more precise, but may not be accurate.
from products introduced in the last year, whereas The allocation of administrative and other indirect
approximately 50% came from products costs makes it difficult for many companies to
introduced in the last three years. The curve accurately identify the actual profit of individual
products. Both alternative methods were
compared in the study, and there was no
Average %
of 1992 revenue significant difference in results. The profit rate
70 –
-
- may also need to be adjusted for cases where
-
-
60 –
-
there are significant nonrecurring expenses.
-
-
-
50 –
-
-
-
-
40 –
-
-
R&D Investment
-
-
30 –
-
The investment in R&D is based on spending as a
-
-
- percentage of current revenue. Although this does
20 –
-
-
-
not match the time period of the profit derived
-
10 –
-
-
from this investment, it does provide a reasonable
-
0–
- approximation. Companies tend to invest a
92 91 90 89 88
Years in which products were introduced
consistent percentage of revenue in product
development from year to year. The EI is
Figure 2. therefore a running average.
New Product Revenue as a Percentage of Total The definition of R&D investment also varies
Revenue by company. Some define R&D as spending by

26
 VOLUME 1 NUMBER 4
their engineering and research departments,
regardless of whether it is spent on new product
development or used to support existing products.
Companies that divert a significant portion of
their R&D investment to product support would
have a lower index than if they invested more in
new products. Some companies include the cost
of other departments involved in product
development, whereas some do not. Sensitivity
analysis of these variations did not show any
significant difference in results; however, an
individual company with accounting practices at
the extreme of the variation could have
noncomparable results.
Drivers of the R&D Effectiveness Index
The R&D effectiveness index provides insights
into the dynamics of the product development
process and how it affects profitability. In fact, the
R&D effectiveness index itself is the result of key
drivers of the product development process. This
is illustrated in Figure 3.
The percentage of total revenue derived from
new products is a result of how long it takes to get
these new products to market and their early sales
success once they are released. The first factor is
From new products
Profit
released in year Y (start date + TTM )
Profit From new products
released in year Y-1(start date + TTM +1)
From new products
Profit released in year Y-2(start date + TTM +2)

Project 1 run rate X TTM  Portion in
Project 2 run rate X TTM year Y
. 
+
. 
.
 Spending on
Project n run rate X TTM  cancelled
projects
Figure 3.
Drivers of the R&D Effectiveness Index
1994
driven by time-to-market (TTM), whereas the
second is driven by the characteristics of the
product as well as time-to-market.
The profitability of new products is a result of
how successful the products are in meeting the
customer needs compared to competitive
products. New product profit is the product of the
sales volume and the rate of profitability. It may
be reduced by a high degree of problems, such as
engineering change orders (ECOs), when new
products are released.
R&D spending comprises the number of
development projects and product development
productivity. Time-to-market is the most
significant driver of productivity. If a product is
developed in less time with fewer delays, then the
run rate (spending per month) costs are incurred
for fewer months. The cost of wasted
development is another productivity factor. This
is the amount spent on the development of
products that never come to market.
Understanding each of these dynamics and how
they influence the R&D effectiveness index is
essential to applying it to measure the success of
the product development process. Decreasing
time-to-market, improving product profitability,
increasing product development productivity, and
reducing wasted development all increase the
R&D effectiveness index.


 Cost of product
 launch Inefficiencies

New product
profit in
year Y
R&D
= effectiveness
index year Y
R&D spending in
year Y




27
 WORLD CLASS
DESIGN TO MANUFACTURE
Application of the Index
The R&D effectiveness index is a useful
quantitative metric of product development
performance that companies can use to manage
their product development process. However, it
should be interpreted with caution, as should all
metrics. Proper application requires an
understanding of its basis and limitations. The
following are some suggestions for application:
(1) The index is best used as a metric of relative
performance. Consistent improvement in the
product development process should be the
goal of all companies, and the R&D
effectiveness index can be used to measure
this continuous improvement. It is a long-
term metric. Improvements to the product
development process will be seen in future
years, when the benefits are achieved.
(2) The absolute value of the index can be used to
measure ROI. An index of less than 1.0
implies that a company may not be getting a
sufficient return on its investment in R&D.
There are some limitations to this
interpretation, including the definition of new
product profit. It is not a conclusive metric; it
is an indicative metric. Companies with an
index well below 1.0 would be wise to
investigate the reasons behind this, including
their portfolio of R&D projects.
(3) The index can measure performance across
multiple divisions. Large companies can use
the index as a comparative measure of
performance across all divisions. Because
market opportunities for each division can be
very different, the absolute index value may
not compare the success of their product
development process. Differences in the index
across divisions may identify which divisions
have better opportunities. Relative changes
will measure their improvement.
(4) The index measures the differences in
opportunities across industries. There was an
indication in the study that the R&D
effectiveness index varies by industry. Some
industries have a much higher payback on
their R&D investments than others. They
have a higher potential, whereas others have
significantly less. In some cases, a low R&D
effectiveness index may be more an
indication of an industry’s potential than the
performance of a particular company in that
industry.
(5) The index can be used as a benchmark
comparison to other companies.
28
Benchmarking provides a powerful tool to
understand performance relative to other
companies, particularly “best-in-class”
companies. Benchmarking studies, like the
one referenced here, can provide participants
with this comparison. Any such study,
however, needs to be thorough enough to
enable comparability of the benchmarks.
Case Study: The R&D Effectiveness Index
as a Management Tool
The following case study provides an illustration
of how the R&D effectiveness index can be used
as a management tool guide and measure
improvement to the product development process.
The company has six divisions, each developing
different products. Its CEO was not satisfied with
the results of its new product development efforts.
He believed intuitively that while the company
was investing heavily in research and
development, it was not getting enough in return.
“Our growth has been less than 10% per year,”
he said, “while the industry has been growing at
25%. Our profit margins are shrinking every year,
and we don’t seem to have as many new products
as our competitors.”
He did not know what to do. “The board of
directors thinks that I should cut R&D spending.
They also believe that we shouldn’t be investing
the same percentage in R&D for each division,
but they don’t have any specific suggestions on
how to shift it. Marketing and engineering, of
course, say that we are not spending enough.
Lately, I’ve begun to wonder if we have
fundamental problems in our product
development processes.”
The CEO did not have a way to measure the
performance of his product development process,
and as a result he did not know how to fix it. The
R&D effectiveness index provided him with the
performance measure he needed.
Overall the company had an R&D effectiveness
index of 0.5. Approximately 20% of its revenue
came from new products, and the average profit
on these products was 12%. It invested 8% of
revenue in R&D. The index of 0.5, [20% × (12%
+ 8%)] / 8%, however, varied widely by division,
as can be seen in Table II.
Even though it invested heavily in new product
development, Division A had the highest index
due to a higher percentage of new products and a
higher rate of profitability. Divisions C and D also
invested heavily in R&D, but their index was
lower because new products were less profitable.
Division B, E and F has a low percentage of new
 VOLUME 1 NUMBER 4
1994

New 50 –
Effectiveness index
– 1.40

product Profit (%) Investment Effective- 45 –

– 1.20
revenue of new in R&D ness 40 –
(%) products (%) index

Percentage of revenue
35 – New product – 1.00

Effectiveness index
revenue
Division A 30 25 9 1.13 30 –
– 0.80
Division B 15 18 7 0.54 25 –
Division C 20 10 8 0.45 20 –
– 0.60
Profit %
Division D 30 10 9 0.63 15 – – 0.40
Division E 10 10 6 0.27 10 – R&D %
Division F 5 12 7 0.14 – 0.20
5–

0– – 0.00
Table II. 0 1 2 3 4 5
Division Performance Year

products, and as a result had a low R&D
effectiveness index.
No one, including the CEO, knew why each
division invested what it did in R&D. A budget of
6% to 9% seemed to be acceptable within the
company, and the R&D budget just continued
from year to year, increasing at about the same
rate that the division grew.
Differences in the R&D effectiveness index
across divisions were not easily explained.
Perhaps some had better market opportunities
than others. Division A, for example, was in a
“hot” market, whereas Division F was in a very
mature market. Perhaps some divisions had a
better product development process than others.
There was no consistency in the way each
developed products.
The CEO set an ambitious goal for
improvement. Using the R&D effectiveness index
as the overall metric, he set a target to improve
effectiveness from 0.5 to 1.4 over the next five
years. The underlying goals were to increase new
product content of revenue from 20% to 40%,
improve the new product profitability from 12%
to 15%, and reduce R&D investment from 8% to
6% of revenue. This would result in an index of
1.4 [40% × (15% + 6%)] / 6%. Each of these
underlying goals, as well as the R&D
effectiveness index, were set by year. These are
charted in Figure 4.
These individual goals were the expected result
of an aggressive improvement plan. The product
development process would be upgraded to
world-class status. The percentage of R&D
spending would be reduced starting in year 2.
Improvements in the number of new products and
product profitability were not expected until year
2, with most of the improvement not coming until
year 3. New product profit was expected to
increase starting in year 2. These were composite
Figure 4.
Performance Goals
goals for the company as a whole; each division
has its own individual goals.
The company’s plan included a number of
changes to its product development process and
the way it did business:
● A new product development process[4] would
be implemented throughout the company. This
was expected to reduce time-to-market and
increase R&D productivity. As a result, more
new products were expected starting in year 2.
● More rigorous financial screening of new
product proposals was implemented as part of
a new phase review process. This would stress
higher profitability as a requirement for
funding new projects.
● A strategic evaluation was also initiated,
focusing on the growth opportunities in
Divisions E and F with the expectation that
R&D spending would be reduced in these
divisions if they did not have acceptable new
product opportunities.
Although the work was just beginning, the CEO
of the company now felt like he was in control,
having set a clear direction. “We certainly have
our work cut out for us, but we know what we
need to do and how we are going to get there. The
R&D effectiveness index is our overall guide.
There is no magic to the goal of 1.4. Perhaps we
could do even better. I do know, however, that an
improvement of that magnitude is both achievable
and badly needed.”
This case study illustrates the R&D
effectiveness index in action as a management
tool. It provided the metric that the company
needed to guide its efforts to become more
competitive. The case also illustrates how a

29
 WORLD CLASS
DESIGN TO MANUFACTURE

company can sort through the complexity of growth also shows a strong relationship between
managing new product development without growth and the R&D effectiveness index.
oversimplifying this complex problem. Companies with a high index grew twice as fast
as the average from 1988 to 1992.
Other characteristics of companies with a high
Characteristics of Companies with a High R&D effectiveness index are also worth noting:
R&D Effectiveness Index ● They invested less on R&D and got more out
The benchmarking study also provided an of it. As a percentage of revenue, the
interesting analysis of the characteristics of companies with a high R&D effectiveness
companies with a high R&D effectiveness index. index invested approximately a fifth less than
These characteristics help not only to understand average in R&D. At the same time, they also
the index a little better, but also provide insights had approximately a third more revenue from
into what it takes to perform well. new products. This reflects the productivity
Companies with a high R&D effectiveness aspect measured by the index.
index (> 1.25) outperformed the average in
● They wasted less on products that did not
growth and profitability. These comparisons are
come to market. Companies with a high R&D
shown in Figure 5. Although higher profitability
effectiveness index cancelled marginal
would be expected (as it is part of the
computation), the magnitude of this difference is projects much earlier in the development
surprising. These companies were twenty-two cycle. As a result, they lost only 14% of their
times more profitable! The difference in revenue R&D budget on work that never came to
market, compared to 21% for the others.
Companies with the lowest R&D effectiveness
index (>0.5) lost 28%.
Profitability
25 – (% revenue) ● They invested proportionally more of their
-
-
-
22.3 R&D budget on totally new products and
-
20 – platforms. They invested 30% of their R&D
-
-
-
budget on totally new platforms compared to
-
15 –
-
less than 25% for the other companies. This
-
- supports the importance of managing the
-
10 –
-
amount invested in new product
-
- platforms[5,7].
-
5–
-
-
● They met their original project goals more
0.98
-
- often. Companies with a high R&D
0–
High R&D Average effectiveness index met their revenue goals
effectiveness
index 50% more often, their profitability goals 20%
more often, and their return on investment
goals twice as often. This suggests that their
Revenue growth product development process was more in
1988-1992
350 –
-
(% change in revenue) control.
- 318.6
-
-
300 –
-
-
-
-
250 –
-
-
Conclusion
-
-
200 –
- This article introduces the R&D effectiveness
-
-
-
150 –
154.9 index as an overall measure of product
-
-
-
-
development success. An aggregate metric like
100 –
-
- this is clearly needed by companies that want to
-
-
50 –
-
measure their improvement, as well as by
-
0–
-
- companies that are not satisfied by their results.
High R&D Average The R&D effectiveness index reflects the
effectiveness
index dynamics of the product development process.
Decreased time-to-market, improved product
Figure 5. profitability, increased development productivity,
R&D Effectiveness Index Correlated to Revenue Growth and reduced development waste all increase the
and Profitability index.

30
 VOLUME 1 NUMBER 4
1994
A study of product development in 45
electronic systems companies provided data to
compute and analyze a profile of index values.
This analysis showed a strong relationship
between the index and both growth and
profitability. The study also provided data to
analyze the characteristics of companies with a
high R&D effectiveness index. These companies
show some outstanding attributes in their product
development process.
The R&D effectiveness index has several
applications as a management tool. It is strongest
at measuring relative performance, but can also
measure absolute performance similar to ROI. It
can identify performance differences across
divisions and opportunity differences among
industries. It can also be used as a benchmark
comparison. Some of these are illustrated in a
case study.
n Product Strategy for High-Technology Companies
References
1. Coy, P, In the Labs, the Fight to Spend Less,
Get More, Business Week (New York:
McGraw-Hill), pp. 102-104 (July 5, 1993).
2. Gilman, J. Inventivity, New York: Van
Nostrand Reinhold, 1992, pp. 7-27.
3. Griffin, A. Metrics for measuring product
development cycle time, Journal of Product
Innovation Management 10:112-125 (March
1993).
4. McGrath, M., Anthony, M. and Shapiro, A.,
Product Development: Success Through
Product and Cycle-time Excellence,
Stoneham, MA: Butterworth-Heinemann,
1992.
5. McGrath, M. Product Strategy in High-
Technology Companies, Burr Ridge, IL: Irwin
Professional Publishing, 1994 (in press).
6. Morbey, G. R&D: Its relationship to company
performance, Journal of Product Innovation
Management 5: 191-200 (March 1988).
31
7. Wheelwright, S. and Clark, K. Revolutionizing
Product Development, New York: The Free
Press, 1992.
Michael E. McGrath is a founding Director of
Pittiglio Rabin Todd & McGrath (PRTM), an
internationally recognized management
consulting firm to technology-based companies.
He is based in the firm’s Weston, MA office.
Seven years ago, Michael initiated the firm’s
product-development consulting practice referred
to as PACE (Product and Cycle-time Excellence).
Michael has written numerous articles on product
development, operations management,
manufacturing strategy, and international
manufacturing and is co-author of the book
Product Development: Success Through Product
and Cycle-time Excellence and author of the book
to be published by Irwin Professional Publishing.
Prior to founding PRTM, Michael worked for
Texas Instruments, Computervision, Price
Waterhouse & Co., and McCormack & Dodge.
He graduated magna cum laude from Boston
College with a BSc in Computer Science and
Management Science and received an MBA from
Harvard Business School. He is also a CPA and
CMA.
Michael N. Romeri is a Director in PRTM’s
Weston, MA office. He works extensively in the
firm’s product development consulting practice,
directing large international projects. Mike has
worked extensively within the electronics
industry since 1979. Prior to joining PRTM, he
worked with AMD and Flextronics. Mike’s
academic experience includes a BA from Harvard
College and an MBA from Harvard Business
School.
For further information, contact Michael
McGrath, Managing Director, PRTM, 9 Riverside
Rd, Weston, MA 02193, USA. Tel: (617) 647
2800; Fax: (617) 647 2804.