See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/233719383

Measuring the Effectiveness of R&D

Article in Research Technology Management · September 2011

DOI: 10.5437/08956308X5405008

CITATIONS READS
26 8,874

4 authors, including:

Larry Schwartz
Industrial Research Institute
5 PUBLICATIONS 131 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Sustainably View project

All content following this page was uploaded by Larry Schwartz on 29 June 2016.

The user has requested enhancement of the downloaded file.

 MEASURING THE EFFECTIVENESS OF R&D
R&D metrics continue to be an important topic for measuring the effectiveness of R&D.
Practitioners share their issues and recommendations.
Lawrence Schwartz, Roger Miller, Daniel Plummer, and
Alan R. Fusfeld
OVERVIEW: Measuring the effectiveness of R&D has
been a perennial challenge. IRI’s Research-on-Research
working group Measuring the Effectiveness of R&D
sought to investigate how managers define R&D effec-
tiveness and what metrics they use to measure it. Via
surveys and questionnaires, attendees at IRI meetings
revealed that while the three top metrics are unchanged
over the past 15 years, there were significant differences
in metrics used depending on the industry type. The
study also revealed issues with metrics in general and
the need for new metrics to meet the changing R&D en-
vironment.
KEY CONCEPTS: Metrics, Technology value pyramid,
Innovation games, R&D effectiveness, Research-on-
research groups
Lawrence Schwartz is a vice president and principal
of Intellectual Assets, Inc., a California-based pro-
fessional services company linking business deci-
sions and intellectual property. His areas of technical
expertise are in materials and sustainability. Previ-
ously he was vice president of strategic development
for Aurigin Systems. At Raychem (Tyco), he worked
for 25 years in all phases of technology management.
He holds a PhD in chemistry from the University of
Arizona, an MBA from San Jose State University and
a BS in chemistry from San Diego State University.
larryschwartz333@aol.com
Roger Miller is a founding partner of Secor, a strategy
consulting firm with offices in Montreal, Toronto, New
York, and Paris. He is presently a Distinguished Research
Fellow at the Said Business School at Oxford University.
He was previously the Jarislowsky Professor of Innova-
tion and Project Management at Ecole Polytechnique in
Montreal, Canada. rmiller@groupesecor.com
DOI: 10.5437/08956308X5405008
September—October 2011
0895-6308/11/$5.00 © 2011 Industrial Research Institute, Inc.
The creation of a set of metrics to measure the effective-
ness of R&D has been a major need for research man-
agers for some time. In recent surveys of Industrial
Research Institute (IRI) participants, the need for met-
rics has ranked in the top three for the past three years
(Cosner 2010). The enhanced importance of reliable
metrics is being driven by several forces: the need to
justify the investment in R&D to senior management,
the desire to improve efficiency in the use of R&D re-
sources, and the need for a means to estimate the value
of the R&D investment for the future growth of the com-
pany.
Because R&D tends to be both longer term and more
subjective than a sales or manufacturing target, effective
metrics must encompass the broad influence R&D has
Dan Plummer is the manager of R&D for Sasol North
America in Lake Charles, Louisiana, a manufacturer
of surfactants, surfactant intermediates, and specialty
chemicals. He has 27 years of industrial experience with
Sasol North America and its predecessor companies. He
has filled roles in product management, sales and mar-
keting, quality development, and global R&D manage-
ment. Dan received a BA in chemistry from Kenyon
College and a PhD in inorganic chemistry from Iowa
State University. dan.plummer@us.sasol.com
Alan R. Fusfeld is president and CEO of The Fusfeld
Group, Inc., a consultancy practice specializing in strate-
gic development and technology management. Formerly,
he was the founder of the technology management group
of Pugh-Roberts Associates, Inc., where he was also se-
nior vice president and director. His current interests in-
clude R&D leadership, strategy for designing the future
organization, R&D metrics, and portfolios. He received
his B.E.S. degree in mechanics from the Johns Hopkins
University and was a member of the Massachusetts Insti-
tute of Technology’s PhD program in the management of
technology. www.fusfeldgroup.com; afusfeld@aol.com
1
over an organization, including such diverse elements as
new products, cost savings, customer support, competi-
tive evaluations, and IP protection. The difficulty in cre-
ating metrics is further complicated by the need for
changing measures of R&D effectiveness as business
conditions and technology strategies change. Over the
years, there have been many attempts to create metrics
to quantify the success or failure of R&D organizations.
The result has been a proliferation of metrics, ranging
from financial measures, such as R&D spending as a
percent of sales (Andrew et al. 2008, Hauser 1996), to
more complex measures, such as strategic alignment
(Roussel, Saad, and Erickson 1991). A 2004 study by the
European Industrial Research Management Association
(EIRMA) lists over 250 potential R&D metrics (EIRMA
2004).
While metrics clearly draw a high level of interest from
R&D leaders, it isn’t clear whether this continued focus
is because of the inadequacy of existing choices for met-
rics or a lack of knowledge of the topic. To address these
questions, the IRI formed the Measuring the Effective-
ness of R&D Research-on-Research (ROR) Working
Group in 2005 as a follow-up to previous ROR groups
on the topic. After a review of the literature on R&D
metrics, the ROR group polled members on the metrics
they currently used. The subcommittee further attempted
to evaluate whether metric choices were dependent on
the type or industry of the organization. Using the Tech-
nology Value Pyramid (TVP), a rediscovered and mod-
ernized model created by a 1994 ROR group, the group
classified and organized existing metrics, providing a
guide for choosing relevant, useful metrics, and identi-
fied gaps in available metrics.
The Technology Value Pyramid
One of the key challenges of implementing R&D met-
rics is matching metrics to the various levels and func-
tions of the R&D organization, so that metrics are
meaningful to the appropriate personnel. A bench scien-
tist’s metrics, for instance, should be related to his or her
accomplishments, while the CEO or general manager
should have more overarching metrics related to the per-
formance of the organization under his or her responsi-
bility. A metric for financial return makes sense for a
CEO or CTO but has little direct meaning for a bench
scientist. Conversely, the number of patents issued may
make sense as a measure of performance for the bench
scientist but is not directly meaningful for the CEO. Cat-
egorizing metrics by their relevance to the particular
components that make up the eventual value of the R&D
investment addresses this by allowing metrics to be tar-
geted to the most relevant levels of the organization. In
this way, bench scientists can access relevant measures
of their group’s performance, while managers get the
quantitative measures they need—not just to rate the
2 Research • Technology Management
return on the R&D investment but to answer the key
question, “Are we doing the right things?”
An earlier ROR group developed the TVP (Figure 1), a
model that takes this approach to categorizing metrics.
The TVP provides a hierarchy of metrics based on the
fundamental elements of R&D value and the relation-
ships of those elements to business results in the long
and short term (Tipping, Zeffren, and Fusfeld 1995).
Value creation metrics at the apex of the TVP investigate
the financial returns arising from the investment in
R&D. The returns are impacted by the pyramid’s next
segments, portfolio assessment and integration with the
business. Portfolio assessment metrics examine the dis-
tribution of the R&D investment in terms of risk, timing,
and potential return. Metrics addressing integration with
the business focus on the interaction of R&D and the
business groups in terms of process, teamwork, and or-
ganization. The foundations of the TVP are the asset
value of the technology and the practices of R&D. Asset
value of technology metrics address the development
of core competencies that are essential for growth and
competitiveness. Metrics assessing the practices of R&D
investigate the procedures, culture, and operations of the
R&D organization and their ability to contribute to en-
hancing technology development.
The TVP classified 33 R&D measures identified by the
original ROR group. For example, financial metrics fall in
the outcome and strategy tier of the pyramid, while a mea-
sure of the number of technical reports produced falls in
the foundations level.1 The group then surveyed 161 IRI
members about the R&D metrics they used, asking par-
ticipants to rank the 33 identified metrics in order of im-
portance. Over the years, some companies have expressed
a sense that the metrics available did not adequately quan-
tify their own corporate needs. This resulted in the
creation of newer metrics (Donnelly and Fink 2000;
Germeraad 2003). Some of these new metrics were even-
tually added to the TVP dictionary, so that the list now
incorporated into the TVP includes a total of 50 metrics.
Fifteen years after the publication of the original TVP
survey, a new ROR subcommittee convened to revitalize
the TVP, determine whether there has been a shift in the
metrics used to measure R&D, and identify any new
metrics that should be added to the TVP. The group also
sought to determine whether the importance of particu-
lar metrics varies by industry or innovation strategy.
1
The group also produced a comprehensive dictionary of the identi-
fied metrics, including for each entry its definition, a list of advan-
tages and disadvantages, instructions for use, a catalogue of options
and variations, and contacts and references for consultation. After the
article was published in Research-Technology Management in 1995,
the dictionary was stored on a floppy disk; over time, the dictionary
fell into disuse and was largely forgotten. The new version of the
TVP produced by this ROR group, and the accompanying metrics
dictionary, is now available to IRI members on the IRI website.
New Surveys on R&D Metrics
In an effort to answer these questions, the current group,
Measuring the Effectiveness of R&D, administered two
One of the key
surveys. The first, survey A, was largely a repeat of the
original 1994 survey; the intent of this instrument was to challenges of
provide data to facilitate comparison in metrics usage
between the original survey and this one and to measure
changes in practice with regard to R&D metrics. Survey
implementing R&D
A was administered at IRI’s 2009 Annual Meeting; at-
tendees were typically high-level R&D managers, vice
presidents of R&D, and chief technical officers (CTOs).
metrics is matching
The survey listed the 33 metrics included in the original
1994 survey and asked respondents to rank them by or- metrics to the
der of importance in their organizations. We did not ask
respondents to identify their organizations, but we did
ask if they represented for-profit or not-for-profit corpo-
various levels and
rations.
The second survey, Survey B, was a broader instrument
functions of the R&D
that asked participants to rate the expanded list of 50
metrics on their importance to the respondents’ organi-
zations, using a scale of 1 (no importance) to 5 (top im-
organization.
portance). Again, organizations were not identified, but
we did ask respondents to identify the industry in which
they worked. The group used this data to distinguish re-
lationships between preferred metrics and industry, Summit; again, this group comprised high-level R&D
company type, and innovation game. Additionally, the managers, vice presidents of R&D, and CTOs.
survey solicited open-ended comments on the value and
usage of metrics and methods used to collect data. Results
Respondents were also asked to describe metrics that
they felt were needed and list metrics that their corpora- Survey A. Survey A was completed by 56 respondents
tion was using but that were not on the list. Survey B from both corporate and not-for-profit organizations;
was administered to attendees at IRI’s 2008 Member the 1994 survey did not ask about for-profit versus

Figure 1.—The Technology Value Pyramid (TVP)

September—October 2011 3
not-for-profit status, but given that IRI’s membership did
not include not-for-profit organizations at that time, the
original sample most likely included only for-profit cor-
porations.
Data from Survey A were compared to data from the
1994 administration of the same survey (Table 1). The
2009 data are sorted by for-profit or not-for-profit to al-
low a better comparison of the data from 1994 to 2009.
Interestingly, the top three metrics from 1994—financial
return to the business, strategic alignment with the busi-
ness, and projected value of the R&D pipeline—main-
tained their importance for for-profit corporations in the
2009 survey, but only the strategic alignment metric was
judged of high value by the not-for-profit group.
A number of metrics ranked higher in 2009 rankings
than they did in 1994, including achievement of R&D
pipeline objectives, quality of R&D personnel, level of
business approval of projects, comparative manufac-
turing costs, and effectiveness of transfer to manufac-
turing; none of these made the top 12 list in 1994, and
all did in 2009. Other metrics lost ground, including
portfolio distribution of R&D projects, market share,
current spending level for technology, and customer
satisfaction surveys; all were in the top 10 in 1994, and
none were in 2009. Metrics ranked highly by for-profit
respondents to the 2009 survey revealed a focus on fi-
nancial returns, strategic business alignment, and qual-
ity, while not-for-profit respondents were focused on
strategic alignment, accomplishment of milestones,
quality of people, portfolio distribution, and clarity of
project goals. The common thread in both groups was
strategic alignment.
Survey B. Survey B asked about companies’ practices
with regard to R&D metrics; the survey also asked
participants to rate the metrics currently included in the
TVP structure. There were 52 respondents.
Questions about company practices with regard to col-
lecting and using metrics were revealing. While a mi-
nority of respondents indicated that their companies
did not collect any metrics, most companies apparently
feel compelled to measure the effectiveness of their
R&D efforts. The most-reported uses for R&D metrics
included:
• Providing data for project justification and decision-
making processes;
• Enabling portfolio analysis, balancing, and tracking;
• Calculating ROI for R&D;
• Enhancing efficiency in product development;
• Driving performance and defining goals;
Ensuring that R&D is aligned to the business strat-
•
egy; and
• Benchmarking against intracompany units as well
as outside companies.
About half of participants reported that their companies
collect data for metrics on a quarterly basis; a quarter
collect data either semiannually or annually. Most data
must be collected by hand; only a small percentage of
participants reported that their companies had automated
data collection and metric generation processes. ROI
calculations, according to respondents, require some
sort of project time tracking, in addition to other data
collection.
Participants also rated the importance to their compa-
nies of each entry in the expanded list of TVP metrics
(including those added after the original 1994 study),
sorted by TVP level (Table 2). As in survey A, financial
return is a dominant theme, especially in the top two
levels of the TVP. The top metrics for the value

Table 1.—Summary of Survey A results: Top ten metrics in 1994 and 2009
1994
Financial return to the business
Strategic alignment with the business
Projected value of R&D pipeline
Sales or gross profits from new products
Accomplishment of project milestones
Portfolio distribution of R&D projects
Market share
Customer satisfaction surveys
Development cycle time
Gross profit margin
Product quality and reliability (tie)
2009
For-Profit
Financial return to the business
Strategic alignment with the business
Projected value of R&D pipeline
Gross profit margin
Product quality and reliability
Sales or gross profits from new products
Accomplishment of project milestones
Achievement of R&D pipeline objectives
Quality of R&D personnel
Level of business approval of projects
Comparative manufacturing costs (tie)
Not-For Profit
Strategic alignment with the business
Accomplishment of project milestones
Quality of R&D personnel
Portfolio distribution of R&D projects
Clarity of project goals
Product quality and reliability
Rating of project benefits by customers
External peer evaluation of R&D
Customer rating of technical capabilities
Number of technical reports

4 Research • Technology Management

creation level are all financial, as are most of the top
metrics at the strategy level. Other important strategy
metrics include strategic alignment and probability of
Most companies
success. Foundation metrics are associated with quan-
titative measures, such as number of patents, as well as
more abstract metrics associated with people develop-
apparently feel
ment and creativity.
In an attempt to distinguish how innovation strategy in-
compelled to
fluenced the choice of metrics, we analyzed results in
the context of innovation strategy, using Roger Miller’s
concept of “innovation games” (Miller and Floricel
measure the
2004). Miller and his colleagues identified eight distinct
approaches to innovation strategy, each most likely to
effectiveness of their
be used by firms in particular industrial contexts; for
instance, pharmaceutical firms are likely to engage in
“science-based journeys” while biotechnology firms en-
R&D efforts.
gage in “technology races” and cellphone manufacturers
engage in “battles of architectures” (Miller and Floricel
2004; Tirpak et al. 2006; Miller and Olleros 2008). envelope” (integrated systems) industries, such as com-
Clearly, the structure of the innovation game should in- puters, machines, aircraft, or computer chips; consumer
fluence the shape and objective of the R&D effort and, products industries, such as food, soap, or cosmetics;
hence, the metrics an organization uses to measure the and services to innovator firms, such as financial, techni-
effectiveness of its R&D effort. cal, and governmental services. Representatives taking
Because of our small sample size, the original list of the survey from the services sector were typically from
eight innovation games yielded data sets too small to government labs, universities, and research institutes;
provide meaningful information. As a result, for the pur- these were largely not-for-profit organizations.
poses of analysis, the eight innovation games were con- Once overall data were analyzed for responses from
solidated into four: “new and improved” (standalone) each of the four defined innovation games, the results
industries, such as chemicals or petroleum; “pushing the revealed some interesting variations (Table 3). Financial
return, for instance, is much less important in services
than it is in the other groups, perhaps a reflection of the
preponderance of not-for-profit organizations in this cat-
egory. With reduced emphasis on financials, the founda-
Table 2.—Survey B results: Top five metrics by TVP level tion metrics focusing on intellectual property, R&D
Metric Ranking processes, and people development rank much higher.
Outcome Product quality is much more important in consumer
Financial Return 1
products than in the other groups, and the metrics for
integrated systems industries is different from the others
Gross Profit 2
in that pathways for exploiting technology, environmen-
Market Share 3 tal management, process efficiency, and management
Projected Value of Pipeline 4 systems are significantly more important than they are in
IP Management 5 other sectors.
Strategy
Financial Return 1
Respondents to Survey B also identified a number of
metrics that they felt should be added to the 50 currently
Projected Value of Pipeline 2
included in the TVP; these fell into four broad groups:
Gross Profit 3
R&D Investment/Sales 4 • Open innovation metrics, including external vs. in-
Strategic Alignment 5 ternal R&D, percent of projects outsourced, and exter-
Foundations nal innovation;
IP Management 1 • Metrics to measure the effectiveness of R&D pro-
Number and Quality of Patents 2 cesses beyond financial, including, for instance, mea-
People Development 3 sures of efficiency in project management;
Creativity 4
Cost versus Budget 5
• Value creation metrics for technical service and sup-
port for existing products; and

September—October 2011 5
 Table 3.—Survey B results: Top metrics by innovation game
TVP Level
Innovation Game Value Creation Strategy Foundation
New & Improved Financial Return Financial Return None identified at statistically
(Standalone) Projected Value of R&D Projected Value of R&D Pipeline significant level.
Pipeline Gross Profit
Gross Profit Gross Margin
Gross Margin
Pushing the Envelope Financial Return Financial Return People Development
(Integrated Systems) Projected Value of R&D Projected Value of R&D Pipeline Intellectual Property Management
Pipeline Gross Profit Number and Quality of Patents
Gross Profit Gross Margin
Gross Margin R&D Investment as % of Sales
Consumer Products Financial Return Financial Return People Development
Product Quality & Reliability Gross Margin Intellectual Property Management
Gross Margin Gross Profit Number and Quality of Patents
Gross Profit R&D Investment as % of Sales
Market Share Probability of Success
Services Intellectual Property Financial Return Idea Generation and Creativity
Management R&D Process
Financial Return People Development
Quality of Personnel
Number and Quality of Patents

• Not-for-profit metrics to measure the development
of new capabilities, awards, professional activities,
and bibliometrics.
Of particular interest were calls for additional metrics in
R&D effectiveness and value creation. While the litera-
ture does describe some metrics in the area of R&D ef-
fectiveness beyond financial measures (e.g., McGrath
1994), most companies found these existing measures
inadequate. Similarly, many participants reported a need
for metrics to measure the effectiveness of technical ser-
vice and support groups, which are often associated with
R&D budgets, in order to show their value to the com-
pany. The closest metrics currently available in this area
are related to cost savings, but these do not adequately
address the issue.
Respondents also identified a number of issues with ex-
isting metrics, including the difficulty of collecting data
and calculating and quantifying metrics, the lack of uni-
form standards for different business units or global
sites, and inconsistency in the application of business-
specific metrics across the company. Respondents were
sometimes skeptical of the value of metrics, attitudes re-
flected in comments that the easiest-to-measure metrics
generally have the lowest value, that metrics are trailing
measures that tend to focus on the short term, and that
metrics tend to focus on inputs rather than outcomes.
Respondents also worried that a reliance on metrics can
lead to gaming the numbers to make a group look good.
Respondents generally commented on the danger that
metrics can influence behavior, for good or bad, and that
poor choices of metrics can lead to bad business deci-
sions. Finally, they commented on the difficulty of creat-
ing metrics that are meaningful to shareholders. And
they lamented that, even with perfect metrics, doing all
the right things does not guarantee success.
Discussion
The comparative analysis enabled by Survey A reveals
some interesting trends, but also suggests areas where
R&D focus has remained the same. Financial return,
strategic alignment, and projected pipeline value were
the leading metrics in 1994, and they remained so in
2009, in spite of dramatic changes in the context and
understanding of R&D in the intervening decade and a
half. These three metrics remain important because they
reflect core sources of value. The quantitative financial
measures (return and value of pipeline) are indicators of
the absolute value of R&D in terms the world under-
stands as generally accepted accounting. Strategic align-
ment measures the likelihood that the R&D effort will
have a significant impact on the core business. Taken
together, these three measures also offer a wide perspec-
tive of measurement. Financial return is a retrospective
measure, while projected pipeline value and strategic
alignment are outcome measures. Not-for-profit organi-
zations, which do not have the direct financial criteria
for success of most corporations, tend to place more im-
portance on strategy and foundation than on value cre-
ation metrics.

6 Research • Technology Management

Shifts in ranking may reflect a number of changes in cor-
porate culture and the context of innovation: the effects
of the recession that began in 2008, which has driven a
As the R&D model
more inward and short-term focus and increased worries
about finances, cash flow, and excess spending; the shift
from total quality management (TQM) with its emphasis
is changing to an
on quality and the voice of the customer; the increased
attention to corporate governance; and the rise of global open innovation
competition. Recessionary anxieties are reflected in the
increased importance of financial measures, such as the
value of the R&D pipeline, manufacturing costs, and
perspective, metrics
transfers to manufacturing, and in the decreased signifi-
cance accorded to such metrics as portfolio distribution,
market share, spending on technology, and customer sat-
depicting these
isfaction. The increased importance of business approv-
als and transfer to manufacturing may be a reflection of
changes, and
a closer focus on corporate governance. The shift away
from TQM as a management paradigm may have driven
down the importance of customer satisfaction as an
measuring their
R&D metric, allowing other measures to take its place.
A third explanation for the emphasis on costs is global
outcomes, are
competition. In 1994, China was just beginning to re-
place some U.S. manufacturing. In today’s worldwide
marketplace, low price has become one of the most im-
required.
portant criteria for product sales. This is reflected in the
increased importance of manufacturing costs and trans- As the R&D model is changing from exclusively inter-
fers to manufacturing as R&D metrics. nal development to an open innovation perspective,
metrics depicting these changes, and measuring their
There were also some differences noted in the metrics outcomes, are required. Proposals for such metrics as
preferred depending on a company’s innovation game percent of new intellectual property from licensing in or
and between corporations and not-for-profit organiza- percent of new business created by licensed technology
tions. For instance, consumer product companies tend to have been proposed. Fine-tuning these and integrating
rely more heavily on quality metrics than other compa- them into the TVP are important next steps; one part of
nies do, possibly because these industries lack the well- that process will be determining in which levels of the
defined specifications and standards that generally guide hierarchy they belong. Since R&D will continue to
companies working in the standalone or integrated sys- evolve, identifying and integrating new metrics is a con-
tems areas. Companies in these areas are more likely to tinuous process. The TVP should continue to develop as
emphasize number of defects, efficiency, and system the repository of these metrics, along with their defini-
management and R&D metrics. Consumer products tions, recommended uses, and references.
companies, on the other hand, must deal in a consumer
market where a contaminated or poor quality product Conclusion
could generate major lawsuits, negative publicity, and
bad will. Future events could influence following sur- Our respondents, along with the data they provided, of-
veys; for instance, we might expect a higher emphasis fer some concrete advice for selecting R&D metrics.
on quality metrics in the petrochemical industry in the First, it is important to recognize that some metrics are
wake of the 2010 BP oil spill in the Gulf of Mexico. more important to specific industries and innovation
strategies than others. Choosing a metric, or bench-
There are also discernible differences in preferred met- marking results, is best done within similar business
rics between for-profit and not-for-profit organizations. types. In a similar vein, before choosing a metric, it is
Not-for-profit companies are typically less concerned important to understand what the company (and its
about financials and product sales; instead, they are fo- stakeholders) will do with the results. To be effective,
cused on meeting goals and managing people and port- metrics must align with the business and technical
folios. The common thread linking corporations and strategies as well as with the immediate goals of mea-
not-for-profits is strategic alignment; both are concerned surement. The number of metrics that are collected
to make sure they’re working on the right projects to should be limited to the minimum necessary to acquire
support the core business. a complete picture; at the same time, a few robust and

September—October 2011 7
clearly measurable metrics should be selected from
each portion of the TVP.
Beyond the surveys, the ROR subcommittee has success-
fully integrated the TVP process into the IRI website. An
updated list of the metrics with contemporary language and
examples, and their definition and usage has been captured
to enhance future knowledge sharing. As new metrics are
developed, these will need to be incorporated into the list.
Ultimately, a goal would be for a user of the TVP to com-
plete a survey and be able to compare the results with those
of others playing the same innovation game.
Even as measuring R&D performance becomes more im-
portant, metrics remain difficult and sometimes elusive.
IRI should continue with working groups that continue to
monitor, survey and make available to others these evolu-
tions. The TVP is an ideal system to support this effort.
References
Andrew, J.P., Hannæs, K., Michael, D.C., Sirkin, H.L., Taylor,
A. 2008. Measuring Innovation Survey 2008: Squandered
opportunities. Boston Consulting Group. http://www.bcg.com/
documents/file15302.pdf (accessed November 11, 2010).
8 Research • Technology Management
Cosner, R. R. 2010. The Industrial Research Institute’s R&D trends
forecast for 2010. Research-Technology Management 53(1):
14–22.
Donnelly, G., and Fink, R. 2000. A P&L for R&D. CFO Magazine
February: 44–50.
European Industrial Research Management Association (EIRMA).
2004. Assessing R&D Effectiveness: WG62 Report. EIRMA
Working Group Reports. Paris. www.eirma.org/pubs/reports/
wg62.pdf (accessed April 11, 2011).
Germeraad, P. 2003. Measuring R&D in 2003. Research-Technology
Management 46(6): 47–56.
Hauser, J. R. 1996. Metrics to value R&D: An annotated bibliography.
Sloan Working Papers. Cambridge, MA: Sloan School of
Management. Massachusetts Institute of Technology.
McGrath, M. E., and Romeri, M. N. 1994. The R&D Effectiveness
Index: A metric for product development performance. Journal of
Product Innovation Management 11(3): 213–220.
Miller, R., and Floricel, S. 2004. Value creation and games of
innovation. Research-Technology Management 47(6): 25–37.
Miller, R., and Olleros, X. 2008. To manage innovation, learn
the architecture. Research-Technology Management 51(3):
17–27.
Roussel, P. A., Saad, K. N., and Erickson, T. J. 1991. Third Generation
R&D: Managing the Link to Corporate Strategy. Boston, MA:
Harvard Business Press.
Tipping, J., Zeffren, E., and Fusfeld, A. 1995. Assessing the value
of your technology. Research-Technology Management 38(5):
22–39.
Tirpak, T., Miller, R., Schwartz, L., and Kashdan, D. 2006. R&D
structure in a changing world. Research-Technology Management
49(5): 19–26.
 NUMBER 1 OF 1

AUTHOR QUERIES
DATE 7/12/2011
JOB NAME RTM
ARTICLE 2780
QUERIES FOR AUTHORS Lawrence Schwartz et al.

PLEASE ANSWER THE AUTHOR QUERIES WHERE THEY APPEAR IN THE TEXT.

THERE ARE NO QUERIES

View publication stats