Journal of Business Research xxx (xxxx) xxx–xxx

Contents lists available at ScienceDirect

Journal of Business Research

journal homepage: www.elsevier.com/locate/jbusres

How to evaluate innovative ideas and concepts at the front-end?

A front-end perspective of the automotive innovation process
Marisa Dziallas
Innovation Economics, Technical University Berlin, Marchstraße 23, 10587 Berlin, Germany

A R T I C LE I N FO A B S T R A C T

Keywords: Despite growing research on front-end indicators, the understanding of the evaluation of innovative ideas in the
Front-end innovation evaluation early stages of the innovation process within a company remains limited. A reason for this might be a lack of data
Selecting innovative ideas on front-end evaluation. This paper investigates existing theoretical and new indicators in terms of their im-
Innovation indicators portance and practical applicability in the industry. This study follows a qualitative empirical research design,
Agile decision-making
using a new dataset based on 32 interviews with experts conducted in six international automotive original
Automotive industry
equipment manufacturers (OEMs). To improve generalization, the perspectives of volume, premium, and entry-
level manufacturers are included. The four most relevant indicators are high customer relevance, strategic fit,
high communication potential, and vision potential. The data suggest the need for more flexible and faster front-
end evaluation approaches. The findings enhance the quality of innovation decisions by presenting novel in-
sights into innovation evaluation at the front end.

1. Introduction
Companies face the challenge of selecting suitable ideas very early
in the innovation process—at the front end—because resource and
budget constraints limit the pursuit of every proposed idea (Achiche,
Appio, McAloone, & Di Minin, 2013; De Brentani & Reid, 2012;
Frishammar, Florén, & Wincent, 2011; Kock, Heising, & Gemünden,
2015; Montoya-Weiss & O'Driscoll, 2000). The term front end refers to
the early stages of the innovation process, including idea generation,
concept definition, opportunity recognition and evaluation1 (De
Brentani & Reid, 2012; Khurana & Rosenthal, 1998; Mohan, Voss, &
Jiménez, 2017). Many promising new ideas and concepts for innova-
tions do not become products. This may be because companies often
lack knowledge about appropriate evaluation approaches to identify
the most relevant topics out of many good ideas. When combined with
the uncertainty that characterizes the front end, this lack of knowledge
means that companies rarely dedicate enough resources for a structured
evaluation process (Dewangan & Godse, 2014; Edison, bin Ali, &
Torkar, 2013; Kock, Heising, & Gemünden, 2016). Front-end evaluation
approaches and indicators for new product development, however,
offer a solution to this problem. They help companies align idea se-
lection with their goals and resources. The use of indicators leads to
more transparent, traceable, and reliable innovation decisions, thus
reducing complexity at the front end of the innovation process. By
making the right decisions early on, a company can reduce its risk of
large-scale failure and changing costs in the later stages of the devel-
opment. Therefore, using the right indicators and an efficient evalua-
tion approach at the front end leads to an improved innovation per-
formance (Martinusuo & Poskela, 2011). This may result in higher
returns on investment (ROI) and research and development (R&D) cost
savings (Reid & de Brentani, 2004; Verworn, Herstatt, & Nagahira,
2008).
While the industrial importance of the subject grows, the research
on innovation evaluation at the front end is also increasing (De Brentani
& Reid, 2012; De Oliveira, Rozenfeld, Phaal, & Probert, 2015; Kock
et al., 2015; Martinusuo & Poskela, 2011; Mohan et al., 2017; Van den
Ende, Frederiksen, & Prencipe, 2015). Scholars have recognized the
front end as a critical stage in the innovation process, one that creates
value and sets the path for new product development success (see, for
example, Koen et al., 2002; Reid & de Brentani, 2004). Therefore, the
front end is perceived as an important driver for new product success
(De Oliveira et al., 2015; Kock et al., 2015; Reid & de Brentani, 2004;
Verworn et al., 2008; Zhang & Doll, 2001). Researchers have found that
a comprehensive evaluation positively influences front-end perfor-
mance (Mohan et al., 2017), which, in turn, influences new product
development performance overall (Markham, 2013). Thus, an impoved
evaluation approach at the front end is essential to maintain a com-
petitive advantage (Porter & Heppelmann, 2015).
Despite the large amount of existing research on innovation in-
dicators at the front end, there remains a pressing need to further

E-mail address: marisa.dziallas@campus.tu-berlin.de.

1
Evaluation refers to making a decision about whether to further develop an idea and invest more resources in it.

https://doi.org/10.1016/j.jbusres.2018.05.008
Received 1 May 2017; Received in revised form 3 May 2018; Accepted 5 May 2018
0148-2963/ © 2018 Elsevier Inc. All rights reserved.

Please cite this article as: Dziallas, M., Journal of Business Research (2018), https://doi.org/10.1016/j.jbusres.2018.05.008
M. Dziallas
investigate front-end decisions and indicators. Research on innovation
indicators typically emphasizes the later stages of the innovation pro-
cess while neglecting the early stages. Thus, the majority of existing
research excludes deeper insights into the precise indicators that are
applicable to the front end (Klenner, Hüsig, & Dowling, 2013), a subject
that consequently demands more clarification. In practice, efficient
evaluation performance systems (Dewangan & Godse, 2014) and eva-
luation strategies (Edison et al., 2013) are often defectively structured,
while the companies using them disagree about what should be mea-
sured and how. Some researchers argue that it is unclear whether
academic metrics and parameters are applicable to companies, sug-
gesting that the recommended innovation evaluation approach and
indicators may be too theoretical (see, for example, Adams, Bessant, &
Phelps, 2006; Cruz-Cázares, Bayona-Sáez, & García-Marco, 2013). Ac-
cordingly, there are no publications of expert industrial knowledge
about front-end indicators and evaluation approaches. Another area
that requires more research is companies' organizational agility cap-
abilities, which are needed to implement efficient evaluation indicators
(Roberts & Grover, 2012). The existing research lacks qualitative in-
sights into front-end evaluation based on agile methods.
To close the research gap, this study delivers a new dataset that
presents industrial experts' views on front-end indicators and evalua-
tions. The analysis augments the understanding of innovation evalua-
tion in practice, validating existing indicators and offering new indexes.
This leads to the following research questions:
1. Are front-end innovation indicators and evaluation approaches ap-
plied in industry, and is their application considered an efficient
evaluation performance approach?
2. Is the existing scientific literature on front-end innovation indicators
consistent within the industrial application?
3. Which specific evaluation indicators are suitable for industrial ap-
plication at the front end?
Specificially, this study presents innovation experts' views on 25
front-end innovation indicators and front-end evaluation. The dataset
includes 32 interviews with experts from six internationally operating
automotive original equipment manufacturers (OEMs). Thus, it in-
cludes the perspectives of experts from volume, premium, and entry-
level manufacturers. The indicators investigated in the study were de-
veloped from an extensive literature review and supplemented by the
information collected from the surveyed innovation experts. In addition
to the indicators, the data suggest more flexible and faster front-end
evaluation (see, for example, Cooper & Sommer, 2016b). The findings
emphasize the high relevance of front-end innovation indicators as
demonstrated by the 100% response rate to the interview requests.
The contributions of this study are threefold: First, since there is a
lack of information on front-end indicators in practice in the research,
this study delivers novel insights into front-end indicators and evalua-
tion approaches based on the experiences of internationally operating
companies. This study could benefit from the practitioner research
principles by working on innovation projects and having access to in-
ternal reports and confidential contact with experts (Jupp, 2006).
Therefore, valuable insights into the front-end of the innovation process
could be gained, which are usually not or only hardly open to external
partners. Second, the study has strategic value, as the examined in-
dicators are linked to the strategic and operational capabilities of
companies (Hart, Hultink, Tzokas, & Commandeur, 2003). A company's
strategic capability should allow for high innovation performance so
that the company can gain and maintain a competitive advantage. The
industrial application of the indicators identified in this study helps
companies improve their evaluation of ideas; in other words, the results
of this study enhance the research relating to indicators to improve
future innovation performance. From an operational viewpoint, the
indicators show which data, knowledge, tasks, or competencies should
be created at the front end to develop ideas into products or services
2
Journal of Business Research xxx (xxxx) xxx–xxx
(Hart et al., 2003; Martinusuo & Poskela, 2011). Based on this in-
formation, the operational capability of a company helps to align
available resources to the overall company strategy and customer va-
lues. To apply an efficient evaluation approach and indicators, com-
panies require organizational agility, including adaptive capabilities
such as vigilance and open marketing. Section 4.5 describes how this
required knowledge can be generated and how an agile method can be
implemented. Finally, this study is not merely descriptive (Hart et al.,
2003). It also provides specific indicators that can be used in practice by
combining theoretical indicators with practical insights. Moreover, by
synthesizing the data and results, this study provides information on
how companies can implement front-end evaluation to improve future
innovation performance.
This paper will briefly review the existing literature on front-end
indicators. After describing the methods and data and presenting the
results, the paper will discuss the findings of the study, focusing on
managerial and theoretical implications.
2. Prior research
The term innovation has been defined differently in theory and in
practice. This paper uses the acknowledged definition based on the
OECD that defines innovation as the implementation of a new or sig-
nificantly improved product, process or service (2005). This definition
also includes the commercialization of an innovation (see also
Dewangan & Godse, 2014). This study focuses on incremental innova-
tions such as the technical improvement of a product or service related
to, for example, user experience, autonomous driving, human-machine
interfaces, or driver assistance systems. Incremental innovations are the
focus of this study, because most innovation projects include im-
provements to existing products rather than entirely new inventions (or
radical innovations). This distinction is important because incremental
and radical innovations require different evaluation processes (Aarikka-
Stenroos & Lehtimäki, 2014; Bessant, Stamm, Moesleon, & Neyer, 2010;
Eling, Griffin, & Langerak, 2016; Holahan, Sullivan, & Markham, 2014;
Verworn et al., 2008). For radical innovations, for example, a manager's
support and belief in the potential for new markets might be more
important than evaluation indicators (Leifer, O'Connor, & Rice, 2001).
Another important term for this study is indicator. An indicator is a
measured value that provides information about the status quo and
which can be used to detect problems in an innovation system (Borrás &
Edquist, 2013).
2.1. Literature review
In recent years, considerable efforts have been made to study in-
novation indicators throughout the innovation process (Cooper &
Kleinschmidt, 1987; Hart et al., 2003) and innovation success factors
(Evanschitzky, Eisend, Calantone, & Jiang, 2012). For instance,
Krishnan and Ulrich (2001) conducted a review of product develop-
ment decisions. Research has also confirmed that the front-end is highly
relevant to a company's product performance (see, for example,
Markham, 2013). Front-end evaluation requires different indicators
than the evaluation of innovations that have already been introduced to
the market. It is more difficult to estimate an idea's potential at the front
end than in the later stages because the more the process progresses, the
broader the availability of relevant information becomes. This might be
the reason why the later stages of innovation have been more re-
searched than the early stages (Barczak, Griffin, & Kahn, 2009; Cooper,
2008; Hart et al., 2003).
Different methods for studying innovation indicators have been used
in the literature. Hart et al. (2003) used a questionnaire and examined all
phases of the innovation process in Dutch and British companies that
develop and manufacture industrial goods. By analyzing twenty in-
dicators, they identified four front-end indicators: technical feasibility,
intuition, market potential, and product uniqueness.
M. Dziallas
Carbonell-Foulquié, Munuera-Alemán, and Rodrıguez-Escudero
(2004) also identified four front-end indicators—strategic fit, technical
feasibility, customer acceptance, and market opportunity—by using a
questionnaire and conducting interviews with experts involved in the
development and launch of innovative products in Spanish manu-
facturing companies in medium- to high-technology industries (me-
chanical, electronic, or computer equipment; electrical machinery;
measurement instruments; motor vehicles; and other transportation). In
total, the authors analyzed sixteen indicators.
Martinusuo and Poskela (2011) used a cross-sectional questionnaire
and focused on ideas and concept evaluation in Finish industrial com-
panies. The researchers identified front-end indicators similar to those
highlighted in earlier research, such as strategic, market, and technical
criteria. Their results confirmed that the use of evaluation indicators is
significant for aligning the front end of innovations with business po-
tential. Furthermore, Martinusuo and Poskela emphasized the need for
a holistic and informal evaluation system that should be aligned with a
company's development goals.
In contrast, Evanschitzky et al. (2012) conducted a meta-analysis on
the success factors of product innovations based on Henard and Szy-
manski's meta-analysis (2001). The authors highlighted the importance
of considering national culture as a factor to better understand suc-
cessful new product development.
Magnusson, Netz, and Wästlund (2014) used quantitative (survey)
and qualitative (thinking out loud) studies to analyze idea screening in
the telecommunication sector in Europe. They identified six front-end
indicators: originality, user value, producibility, intuitive assessment,
strategic fit, and profitability. Frederiksen and Knudsen (2017) con-
ducted a case study and used an assessment by students and industrial
experts in Denmark to analyze idea selection, identifying novelty,
usefulness, and market potential as useful front-end indicators.
Overall, the literature on front-end evaluation discusses diverse,
important, and useful indicators and shows that different indicators can
be linked to the front end. However, the question of which specific
indicators can be applied to the evaluation of ideas at the front end of
the innovation process remains. Moreover, a systematic overview of
practical front-end indicators is missing.
Prior research criticized the lack of evaluation approaches based on
the perceived importance of innovation criteria (Hart et al., 2003), yet
it remains unclear which theoretical indicators are applicable at the
front end and which indicators are interesting from companies' per-
spectives. Information about how experts evaluate ideas in practice is
not available, while the understanding of practical front-end indicators
remains incomplete. Furthermore, quantitative research on indicators
has been conducted, but qualitative research is also needed to gain
deeper insights. As the literature review makes clear, analyses of front-
end indicators in the automotive industry have been neglected thus far.
Front-end indicators have likely been insufficiently addressed in the
literature due to a deficiency of data. Companies avoid publishing in-
sights into their front-end processes for nondisclosure and competitive
reasons. The combination of uncertainty and vagueness—due to the
lack of reliable and tangible data—impedes a clear assessment of an
innovation’s potential (Poskela & Martinsuo, 2009). Another challenge
concerning the evaluation of innovative ideas lies in the fact that the
idea needs to be evaluated at the point of market entry, which, in the
automotive industry, is likely five years in the future. In order to make
more successful innovation investment decisions, companies use me-
trics such as the internal rate of return (IRR) (Magni, 2010). However,
this metric is based on numbers that are often unavailable or vague in
the early stages of the innovation process. This means that the identi-
fication of the most promising ideas remains difficult. Moreover, based
on the fuzziness that characterizes the front end, researchers argue that
further investigation is needed to better understand the early stages of
the innovation process and to improve new product development and
performance (Eling & Herstatt, 2017).
3
Journal of Business Research xxx (xxxx) xxx–xxx
2.2. Innovation indicators used in this study
The innovation indicators used in this analysis are based on an ex-
tensive literature review of innovation indicators from 1980 to 2015.2
The indicators that are most significant for front-end evaluation were
selected for this study based on the knowledge of experts working in the
innovation management department of Company I. Additional in-
dicators were added based on the experience of the author of this paper.
Table 1 shows the identified indicators.
2.3. Agile development methods and capabilities
Recent research discusses the use of Agile–Stage-Gate hybrid models
to increase the success rate of a new product (Cooper & Sommer,
2016b). Agile development methods are usually applied in software
development, but new studies demonstrate that they can be integrated
into the traditional predevelopment processes of manufacturing com-
panies (Cooper, 2016; Cooper & Sommer, 2016b). The shift to digiti-
zation may lead to changes in user behavior and rebound effects that
require shorter and faster iterations (Cooper, 2016). For instance, cus-
tomers' shopping-related tasks may change based on digital technology.
This research shows the advantages of agile processes, including im-
proved communication and coordination, a better ability to cope with
uncertainty, quicker product releases, and faster responses to changing
market and customer requirements (Begel & Nagappan, 2007). Will
experts in the manufacturing industry agree on the need for more agile
and flexible evaluation approaches?
The application of appropriate evaluation indicators also requires
organizational agility capabilities. Researchers argue that agile com-
panies tend to be more successful than those that are not agile (Roberts
& Grover, 2012). Despite the importance of agility, little research has
been done on the topic, especially as it relates to automotive manu-
facturing companies. The term organizational agility refers to a compa-
ny's ability to respond quickly and flexibly to its environment and to
meet emerging challenges with innovations (Bessant, Francis, Meredith,
& Kalinsky, 2001; Day, 2000). Organizational agility requires strong
dynamic capabilities to understand the deep uncertainty associated
with innovations (Teece, Peteraf, & Leih, 2016).
Researchers have shown that a company's customer agility (sense
and response capabilities) is also important for its success (Roberts &
Grover, 2011). The term customer agility is defined as the “degree to
which a firm is able to sense and respond quickly to customer-based
opportunities for innovation and competitive action” (Roberts &
Grover, 2011, p. 580). Day (2011) argued that adaptive capabilities are
closely linked to agility and are important for innovations as well. Day
(2011) identified three adaptive capabilities: vigilant market learning
to improve market insights with a warning system to anticipate market
changes and unmet needs, adaptive market experimentation to con-
tinuously learn, and open marketing using new media and social net-
working technologies. In order to realize its adaptive capabilities, a
company must be resilient and have vigilant leadership with more
adaptive business models (Day, 2011).
3. Methodology
This section describes the data gathering process, the data used for
this analysis, and the innovation process. Following Cooper and
2
The extensive literature review is based on the working paper: “Dziallas, M. and
Blind, K. (2017). Innovation indicators throughout the innovation process: An extensive
literature analysis.” Within the literature review, indicators of the innovation manage-
ment literature from 1985 to 2015 have been consolidated. The review is based on the
framework of Becheikh et al. (2006 published in Technovation) indicating internal and
contextual factors and innovation related findings (e.g., type of innovation). The work of
Dziallas and Blind has been presented at the OECD Blue Sky III Forum in Ghent,
September 2016.
M. Dziallas Journal of Business Research xxx (xxxx) xxx–xxx

Fig. 1. Innovation in the context of the product development process. The illustration is the author's own, based on Cooper, 1990 and Hart et al., 2003.

Schindler (2013), this study uses an empirical qualitative research ap-
proach to explore expert opinions about front-end innovation indicators
in the manufacturing industry. The study's goal is to gain deep, high-
quality insights into the complex organizational processes behind in-
novation evaluation rather than generalize statistical results (Yin,
2003). The analysis is based on in-depth individual interviews with
experts who work in six internationally operating companies. The
analysis was first conducted for Company I, a volume manufacturer
with a large number of ideas for innovations that require a front-end
decision about wether to move forward. Then, to ensure general-
izability, the same analysis was conducted with data from additional
volume, premium, and entry-level manufacturers. The sample is cross-
sectional, with a multi-perspective scope of the research population
(Cooper & Schindler, 2013).
The automotive industry is suitable for this analysis, as it experi-
ences immense changes and promotes numerous innovations per year
(Sharmelly, 2016). It also faces the challenge of combining hardware
and software developments with different development times and life
cycles that might require agile processes (Cooper, 2016).
The international automotive companies featured in this study in-
clude four volume manufacturers, one premium manufacturer, and one
entry-level manufacturer. These companies were chosen because of
their wide variety of innovative solutions. The variety of products and
services they provide is combined with numerous innovative ideas at
the front end, increasing the complexity of integrating new develop-
ments into products and necessitating a clear evaluation process. The
companies all belong to a publicly traded group. This corporation is one
of the largest internationally operating OEMs in the automotive in-
dustry. The companies follow an innovation process that is based on the
Stage-Gate system (Cooper, 1990). Each of the experts surveyed con-
tributes decisively to the companies' integration of innovative vehicle
ideas. The experts' impact on the innovation process is shown in Fig. 1.

Table 1
Innovation indicators identified as relevant for the study based on an extensive literature review.
Indicator References

Communication potential cf. Yang et al. (2015), based on innovation management experts
Customer orientation e.g., Cooper (1999), Lenfle (2008), Suwannaporn and Speece (2010), Caird et al. (2013), Dewangan and Godse (2014)
Customer satisfaction Astebro and Michela (2005), Enkel et al. (2005), Chiesa et al. (2009), Sawang (2011), De Felice and Petrillo (2013), Dewangan and
Godse (2014), Fleuren et al. (2014)
Degree of innovation De Brentani (2001)
Degree of novelty for the company Cooper (1981)
Ease of use for customer Cooper (1981); Astebro and Michela (2005)
Economic efficiency (potential) Czarnitzki and Kraft (2004), Flor and Oltra (2004), Palmberg (2006), Chiesa et al. (2009), Sawang (2011), Idris and Tey (2011), Caird
et al. (2013), De Felice and Petrillo (2013), Ivanov and Avasilcăi (2014), Dewangan and Godse (2014), Kim (2014), Hittmar et al.
(2015)
Extension of product portfolio e.g., Adams et al. (2006), Kim (2014)
Future product lifetime Astebro and Michela (2005)
Innovation design Alcaide-Marzal and Tortajada-Esparza (2007), Yang et al. (2015)
Innovation quality Griffin and Page (1993), Sawang (2011), De Felice and Petrillo (2013), Edison et al. (2013), Mendes Luz et al. (2015)
Number of innovative ideas Chiesa et al. (2009), Hittmar et al. (2015)
Patents and standards Chiesa et al. (2009), Makkonen and van der Have (2013), Rocha et al. (2015), Roper and Hewitt-Dundas (2015)
Service potential De Brentani (2001), Astebro and Michaela (2005), Schmoch and Gauch (2009), De Felice and Petrillo (2013), Yang et al. (2015)
Sustainability Kobayashi (2006), van Hoof et al. (2014)
Synergy potential with other innovations Griffin and Page (1993), Astebro and Michela (2005)
Technical feasibility cf. Chiesa et al. (2009)
Time to market Tipping et al. (1995)
Trend-setting e.g., Littell (1994)
Unique selling proposition potential Cooper (1981, 1999), Astebro and Michela (2005)
Brand defining Included in the analysis based on innovation management experts
Degree of maturity Included in the analysis based on innovation management experts
Installation rate potential Included in the analysis based on innovation management experts
Package potential Included in the analysis based on innovation management experts
Resistance of the innovation Included in the analysis based on innovation management experts

4
 M. Dziallas

Table 2
Interviews with innovation experts from Company I.
Department of respondent Management role Interview no. Transcribed pages Interview length Working experience

Innovation management Innovation manager 1 22 01:10:54 > 10 years

Innovation management Innovation manager 2 11 00:59:10 > 10 years
Strategy and technology management (Top) executive 3 19 02:25:56 > 10 years
Strategy and technology management Market experts 4 8 01:09:49 > 10 years
Project controlling: controlling of predevelopment projects Financial expert 5 17 02:32:52 > 10 years
Project controlling: controlling of predevelopment projects Financial expert 6 13 01:36:48 < 5 years
Group IT: new IT solutions (Top) executive 7 10 00:54:11 5–10 years
Group IT: evaluation new ventures (Top) executive 8 6 00:29:43 > 10 years
Futurology and trend transfer: early detection and communication Executive 9 15 01:22:41 > 10 years
Procurement: collaboration on innovation topics Executive 10 8 00:51:17 > 10 years
Procurement: innovation management Procurement expert 11 11 01:09:00 5–10 years
Business Development: development new business solutions (Top) executive 12 11 01:01:36 > 10 years

5
Predevelopment user experience: development of ideas and concepts for innovations (Top) executive 13 10 00:45:34 > 10 years
Predevelopment construction: development of ideas and concepts for innovations Predevelopment expert 14 9 00:49:16 < 5 years
Sales and marketing: innovation management Marketing innovation expert 15 9 00:39:28 > 10 years
Sales and marketing: innovation marketing (Top) executive/product marketing expert 16 15 00:45:19 > 10 years
Sales and marketing: trend and consumer research for innovations Brand management and consumer insights expert 17 5 00:27:24 > 10 years
Sales and marketing: strategic communication of innovations (Top) executive/brand management expert 18 20 01:23:16 > 10 years
Sales and marketing: strategic price planning for innovations, innovative price approaches Innovation sales experts 19 25 01:16:21 5–10 years
Sales and marketing: innovation analyses (Top) executive/brand management and consumer insights 20 8 00:50:36 > 10 years
Digitalization: development of ideas and concepts for mobility solutions (Top) executive/smart mobility expert 21 9 00:38:23 > 10 years
Digitalization: development of ideas and concepts for mobility solutions Product and service experts 22 11 00:54:58 > 10 years
Sales and marketing: customer monitoring and analyses (Top) executive/brand management and consumer insights 23 12 00:57:41 > 10 years
Futurology and trend transfer: early detection and communication (Top) executive 24 5 00:28:13 > 10 years
Predevelopment chassis: development of ideas and concepts for innovations (Top) executive 25 10 00:46:37 > 10 years

Note: All names are pseudonymized.

The work experience of the expert within the company is categorized as < 5 years, 5–10 years, > 10 years.
Journal of Business Research xxx (xxxx) xxx–xxx
M. Dziallas
Table 3
Interviews with innovation experts from Companies II–V.
Company no. Department of respondent Management role
II Innovation management Innovation manager/(top) executive
III Innovation management Innovation manager
IV Product marketing and strategy Senior innovation specialist
V Digitalization Innovation consultant
V Innovation management (Top) executive
V Innovation management Executive
VI Project management Project innovation management expert
Note: All names are pseudonymized.
The work experience of the expert within the company is categorized as < 5 years, 5–10 years, > 10 years.
In this study, the industrial innovation process is structured into six
phases and corresponds to the Stage-Gate process of Cooper (1990) and
Hart et al. (2003). Although this process appears to be a linear one, it is
important to note that the linear portrayal is quite dynamic and itera-
tive, with diverse feedback loops to adapt to market requirements
(Frederiksen & Knudsen, 2017; Kline & Rosenberg, 1986). The futur-
ology and trend department uses a timeline of about 10–20 years and
transfers future trends and customer needs to the present to develop
new new solutions. The other departments work within the product
development process and use a timeline of about 5–8 years. In the first
phase, the strategy phase, the long-term positioning of innovations is
determined based on company goals and trends, as well as brand con-
cept and product strategy (strategy and technology management de-
partment). In the second phase, the product is defined and ideas are
generated by the predevelopment department and evaluated with the
help of the innovation management. At this time, the digitization and
business development departments work on new mobility and service
solutions. In the third phase, the product is conceptualized according to
the product definition. Potential costs and required resources are esti-
mated based on business case calculations by the finance department.
In the fourth phase, the validation phase, prototypes are built and
tested. Ideally, in the fifth phase, the innovative concepts are integrated
into the vehicle projects and produced in high volumes. In the sixth
phase, the manufactured products are introduced to the market by the
sales and marketing department using a communication and marketing
strategy in order to achieve the highest sales figures possible. From an
innovation perspective, the procurement department serves as the link
to suppliers who deliver innovative ideas. The information technology
(IT) department contributes to the overall process with IT solutions.
The data in this study was gathered through 32 formal semi-struc-
tured interviews, each with a duration of 60 to 90 min (see Tables 2 and
3). These interviews resulted in 33 h of recording.
As outlined in Tables 2 and 3, the position and work experience of
the interviewees qualified them to appropriately validate front-end
innovation indicators and provide well-informed answers to the inter-
view questions (cf. Hart et al., 2003; Markham, 2013).
The study employed a questionnaire with open- and closed-ended
questions to further explore specific opinions and responses (Bogner &
Menz, 2009). The interview guidelines were created based on the re-
searcher's in-depth analysis of the academic literature dealing with
innovation indicators and review of expert feedback in order to ensure
the consistency and reliability of the analysis. Test surveys were con-
ducted in three rounds to guarantee a clear understanding of the
question and answer categories (Hart et al., 2003). The interviews were
structured in three parts: evaluation of existing innovation within the
company, evaluation of existing indicators, and evaluation of innova-
tion process. The 25 innovation indicators used for this analysis were
selected based on the extensive literature review (see Table 1) and were
then validated in terms of importance and applicability. For the analysis
of the indicators, the interviewees completed a questionnaire.
After an approval procedure, the interviews were done between
September 2016 and September 2017 using the snowball strategy
6
Journal of Business Research xxx (xxxx) xxx–xxx
Interview no. Transcribed pages Interview length Working experience
26 15 01:10:54 > 10 years
27 12 01:00:56 5–10 years
28 9 00:52:11 > 10 years
29 8 01:02:24 5–10 years
30 7 00:45:36 > 10 years
31 6 00:35:41 > 10 years
32 10 00:54:12 > 10 years
(Voicu, 2011). This strategy was chosen because it allowed for the in-
clusion of a high number of experts working on different aspects of
innovation decisions. The surveyed experts were managers and em-
ployees with decision-making roles in the innovation process. In the
first round of interviews, experts from the innovation management
department were interviewed. In the second round, innovation profes-
sionals from the departments responsible for predevelopment, digita-
lization, technology, and strategy were consulted. For the third round,
innovation experts were recruited for the study based on the advice of
the experts interviewed in the second round in order to ensure that the
information included in the analysis was complete and relevant. The
experts chosen for the third round of interviews belonged to the sales
and marketing, procurement, and IT divisions, as well as the finance,
futurology, and trend transfer departments. Finally, innovation experts
from five other companies were also interviewed. The interviewees'
perception and interpretation of the indicators were of primary interest
in the interviews.
Therefore, their different backgrounds were beneficial for building
an understanding of innovation decisions (Yin, 2014). All interviews
were conducted face-to-face at a location in the expert's company. Ex-
perts working for companies outside of Germany were interviewed via
video conference.
To analyze the data gathered from the interviews, an inductive
approach was chosen—findings were derived from detailed readings of
the raw data (Thomas, 2006). This qualitative approach reveals es-
sential particularities of the data that might otherwise go unnoticed.
The author's conclusions were drawn and validated in repeated dis-
cussions with the experts (Thomas, 2006). Thus, the author was able to
distinguish more general phenomena from those specific to individual
companies.
The interviews were recorded with the interviewees' permission. At
the same time, they were transcribed using the program f4transkript.
This resulted in 366 transcribed pages. For the transcript coding and
analysis, the software f4analyse was used. The necessary structural
coding was based on the interview question structure (Saldana, 2009).
After the interviews were coded, the established categories were revised
in one feedback loop to increase the reliability of the coding and de-
termine the most relevant innovation indicators (Thomas, 2006).
Therefore, the data was checked several times to ensure that the es-
sential findings were identified. Quantitative aspects of indicator eva-
luations based on qualitative methods were included in the analysis.
The results were translated into English.
As the author of this paper works for one of the companies included
in the study, this analysis benefited from the practitioner's research
principles (McCall & Simmons, 1969), generating a new dataset to
enhance the understanding of evaluation processes with insights from
project work, expert knowledge, and internal reports. These conditions
facilitated the research, helping to gain crucial insights into the eva-
luation of innovative ideas at the front end and implement an efficient
data collection process.
M. Dziallas Journal of Business Research xxx (xxxx) xxx–xxx

Table 4
The importance of innovation indicators for the front end of the innovation process.

Note: Three of the experts have not sent back the evaluation sheets. Therefore, they have been excluded in this graphic.

4. Analyses and results in relation to Company I. The subsequent section, 4.3, presents the
indicator applicability levels of all the companies, while Section 4.4
The surveyed experts were asked to describe their understanding of presents the indicator results based on a cockpit of indicators. Finally,
the term innovation. They defined a product or service innovation as “a Section 4.5 discusses approaches for improving the front-end innova-
novel solution that solves a customer problem or need which is suc- tion decision-making processes for Companies I to VI.
cessfully introduced to the market and perceived at the time of market
entry as subjectively new” (strategy expert, Interviewee 28). Further,
4.1. Front-end innovation indicators in practice
they noted that an innovation “creates an added value and a willingness
to pay an additional amount of money” (innovation sales expert,
The interviews focused on idea selection, the phase during which
Interviewee 15). The surveyed strategy and marketing experts argued,
innovative ideas and concepts have already been established. The aim
“We need to think about how we can evaluate customer problems in the
of this phase is to decide which of the proposed projects should be
early stages, how relevant is the problem, and how does the solution
developed and finalized as a product that will be available to the
address the problem. Many practitioners fail to implement this cor-
market. It should be noted that the need for creativity does not end with
rectly” (Interviewee 28).
this stage and should be pursued further. With this in mind, the fol-
Section 4.1 presents a view of the innovation indicators and in-
lowing excerpts from the interviews show how the experts assess the
novation evaluation conducted in Companies I to VI. Section 4.2 fea-
value of front-end evaluation.
tures an evaluation of the indicator results for all the companies, which
When asked about front-end evaluation indicators, one expert stated
includes the presentation of novel indicators. Sections 4.2.1 and 4.2.2
that they are “absolutely relevant. The so-called knockout indicators are
present a more specific view of Company I, as well as Companies II to VI
the most important ones in the early stages for the evaluation. To see

7
M. Dziallas Journal of Business Research xxx (xxxx) xxx–xxx

Table 5
New front-end innovation indicators and evaluation-related questions. The in-
dicators are arranged from the easiest to apply in practice to hardest.
Indicators and Evaluation-Related
Questions
High customer relevance
Is this solving a customer problem?
Is this suitable for everyday use?
High comfort and user
experience
Will the product be more
ergonomic or attractive for the
customer?
Strategic fit (meeting the
company’s strategy, vision,
aims, and future scenarios)
Is the idea intelligent and
prestigious?
Does it have situational relevance?
Does it have the potential to
engender enthusiasm?
Surveyed Experts
Sales and marketing, strategy,
innovation management, innovation
procurement, and predevelopment
experts
14). It appears that “it is difficult to get the information, which is re-
quired to evaluate the innovative idea, from the respective depart-
ments” (brand management and consumer insights, Interviewee 18).
According to an innovation management expert, “We conduct innova-
tion evaluation within the process using such indicators as brand de-
fining, innovation degree, customer benefit, the degree of maturity, and
campaign potential, but this is also rather subjective instead of having
weekly or monthly evaluation rounds” (Interviewee 1). Other indicators
used to evaluate innovations are search fields, strategic fit for a com-
pany, product cycle plan, budget, package, characteristics, and tech-
nology.
To summarize the experts' view, innovative ideas are already eval-
uated at the front end, but there is still room for improvement. A
structured front-end indicator system to improve future innovation
performance is needed. Some interviewees even argued that innova-
tions are evaluated subjectively instead of using consistent indicators.
Small companies are more likely to apply an indicator scheme for all
ideas, perhaps due to the smaller number of innovations they evaluate.
4.2. Front-end innovation indicators from the literature

Relevance at the time of market
entry
Can the idea be subjectively
perceived as new and original?
Does it support digital products or
have the ability to be updated?
Can the innovation be sold through
a free activation rate?
How many potential customers are
there?
What is their future frequency of
use?
Is the idea multifunctional and
marketable?
Would the customer be willing to
pay an additional charge?
Will the idea have a pricing
potential?
the right idea just in time that is the most important thing” (pre-
development chassis, Interviewee 25). A digitalization expert agreed
that front-end evaluation and its indicators are “necessary”. He ex-
plained, “In my opinion, it is very important that you start these things
to operationalize front-end innovation evaluation and make it mea-
surable … and we don't do that today professionally … such indicators
sets combined with evaluation rounds are a common way to do this
(Interviewee 21).
Some of the experts differentiated between two innovation types:
incremental and radical innovation. Incremental ideas require clear
innovation fields, whereas radical innovations require a higher level of
self-determination and a broader perspective. However, it is remarkable
that every expert argued that the front-end evaluation of innovative
ideas is highly important.
When asked whether they actually conduct innovation evaluation,
12 of the 32 experts said that they did. One replied:
“Yes, we conduct innovation evaluation. It is a classic innovation-
funnel and we ask at each of the gates ‘do we have to go to the next
gate with this innovation project?’ because of the large number of
ideas that have to be reduced on the way to the customer product.
However, we are in the process of setting the process so that we do
not decide about the innovation's future based on intuition only”
(Interviewee 21).
At the moment, “we use Excel files or one-pagers to assess ideas that
deserve further adaptions” (Interviewee 26).
A predevelopment expert explained, “Innovative ideas have been
evaluated in terms of time, money, trends and intuition” (Interviewee
The innovation indicators identified in the literature with the
highest importance for front-end evaluation were selected for this study
based on the expert opinion of the innovation management department
of Company I. Additional indicators were added based on the author's
industrial experience (see Table 1).
Table 4 shows the importance of indicator evaluation according to
the surveyed experts. The values range from one to ten, with one in-
dicating the lowest importance and ten the highest. The indicators are
arranged according to the mean value of their importance, starting with
the indicator of the highest importance. The experts' backgrounds can
be found in the second row of Table 4.
Table 4 indicates that although different experts have different
understandings of innovation indicators, a consensus nonetheless ex-
ists—especially within the same functional areas. The data suggest that
some indicators are more important than others at the front end. Each
department emphasizes the indicators that are most important to it.
According to the expert interviews, unique selling proposition
(USP), ease of use for the customer, communication potential, economic
efficiency, service potential, and customer satisfaction are the most
important innovation indicators based on the mean values. The in-
dicators with the highest numbers of values = 10 are USP, technical
feasibility, communication potential, customer satisfaction, ease of use
for the customer, economic efficiency, and customer orientation.
The number of innovative ideas and package potential are not
considered as important as the aforementioned indicators. The degree
of novelty for the company, the idea's maturity, resistance of innova-
tion, future product lifetime, and extension of the product portfolio are
considered the least important indicators. Based on the mean values,
these indicators seem unsuitable for a front-end evaluation.
When the experts were asked for additional important front-end
indicators, they named the indicators listed in Table 5.
4.2.1. Evaluation of Company I
Fig. 2 highlights the most important innovation indicators cate-
gorized by the departments of Company I. The results demonstrate that
customer, market, and strategic indicators are regarded as most im-
portant, followed by technical, product-related, and economic in-
dicators.
The numbers in the knots display the indicators (see legend). Those
in the white knots symbolize the departments of Company I. The
proximity of the indicator knots (1–25) to the department knots (a–j)
demonstrates the importance of the respective indicator to the depart-
ment (the closer the knots, the more important the indicator to the
department). The size of the knots suggests the overall importance of
the indicators. The more important the indicator, the bigger the knot

8
M. Dziallas Journal of Business Research xxx (xxxx) xxx–xxx

Fig. 2. Important indicators from the department's view of Company I. Note: The applicability values are counts indicating how often experts evaluated the respective
indicator as easy, middle or difficult.

(based on the mean value shown in Table 4). The width of the lines also
indicates the importance of the indicators to the respective department
(the thicker the line, the more important the indicator). The color of the
knots shows the applicability of the indicator in practice (see the legend
for Fig. 2). The darker the color, the easier the applicability of the
indicator in practice (for example, the information to apply this in-
dicator is available) according to the interviewed experts. The applic-
ability counts are presented in Table 6 (Appendix A). The applicability
values in Fig. 2 were calculated by substracting the number of difficult
applicable values for each indicator from the number of easy applicable

9
M. Dziallas
Table 7
Company profiles of interviewed experts.
Company number Country Segment
I Germany Volume
II Czech Republic Volume
III Spain Volume
IV United States of America Volume
V Germany Premium
VI Brazil Entry-level
values.
The indicators on which all the experts of Company I agreed can be
classified into five categories:
- Customer-related indicators such as customer orientation, satisfac-
tion, and ease of use
- Company-related indicators such as brand defining, trend-setting,
and USP
- Product-related indicators such as technical feasibility, innovation
degree, patents and standards related to the product, synergy po-
tential, communication and service potential, innovation design,
and sustainability (although sustainability focuses on different as-
pects such as strategy)
- Process-related indicators such as the time to market
- Financial-related indicators such as installation rate potential and
economic efficiency potential.
The data demonstrate that those indicators with a low level of im-
portance are degree of maturity, extension of product portfolio, future
product lifetime, package and synergy potential, degree of novelty,
innovation quality, number of ideas, and resistance of innovation.
Interestingly, the only departments that rated time to market as very
important were the finance and procurement departments. This may be
because an innovative idea related to time to market could influence
the financial performance of the product and competitive advantage of
the company. Other experts argue, “This indicator is useful for the
strategic orientation, but rather in order to categorize the innovative
idea, not to decide whether one does or does not implement the idea.”
(Interviewee 23).
An innovation manager rated degree of maturity as important be-
cause it is a “precondition for realizing the idea” (Interviewee 2). A
consumer insight expert agreed, adding that technical feasibility is
mandatory for the implementation of an idea (Interviewee 23). In
contrast, other financial experts stated that technical feasibility “is a
very basic requirement, which must not appear at the front end”
(Interviewee 6). A sales team expert noted, “The predevelopment is
responsible for this necessary condition, which refers to technical fea-
sibility (Interviewee 19).
Future product lifetime and extension of product portfolio are im-
portant indicators from the futurologist and technical perspective, but
other experts argued that “the portfolio is not as important as an in-
dicator, because it relates to the company strategy” (Interviewees 19),
whereas “the customer view might be more important instead of what is
new for the company” (Interviewee 4).
The findings indicate that brand defining and trend-setting are
considered less important for the front end than some of the other in-
dicators: “Whether the innovation will be trend-setting or brand de-
fining is decided in the future” (Interviewee 29). However, trend-setting
should not be neglected (IT, Interviewee 8).
4.2.2. Evaluation of Companies II to VI, compared to Company I
In order to generalize the results of the analysis of Company I, the
perspectives of experts from other international automotive companies
were included in the dataset. Table 7 presents some of the character-
istics of these companies, including the perspective on front-end
10
Journal of Business Research xxx (xxxx) xxx–xxx
decisions.
Fig. 3 highlights the most important indicators, categorized by
company. This figure is structured in the same way as Fig. 2.
Fig. 3 shows the following results for the volume manufacturers (see
also Table 4): The experts at Company II value USP, communication
potential, customer satisfaction, time to market, synergy potential, and
brand defining as important. Thus, the experts of Company I and II
agreed on the importance of USP, communication potential, and cus-
tomer satisfaction. They also agreed on the not important indicators.
Likewise, Company III views customer-oriented indicators as im-
portant, while also emphasizing those indicators related to the brand
image. The experts at Company III rated the following indicators as
unimportant with regard to the front-end: synergy potential, extension
of product portfolio, degree of novelty for the company, patents and
standards, and resistance of innovation.
Synergy potential was deemed unimportant because “it is actually
very good if the innovative project is detached from other projects
because in the best case, old techniques can be replaced. Only then we
can make technical jumps” (Interviewee 26). One expert stated that the
“degree of novelty for the company is not as important as other in-
dicators because we should evaluate innovative projects more from the
perspective of customers” (Interviewee 26).
Company II and III rated trend-setting as unimportant for the early
stages because “this indicator might be more important for trend-setters
and not for a volume producer” (Interviewee 27).
In contrast, Company IV rated customer satisfaction as highly im-
portant for the front end. One expert explained, “We need to evaluate
innovative projects with the question: ‘Do we address a customer pro-
blem and how intense is it?’ to be able to launch successful innova-
tions”. The expert continued, “As customer satisfaction is reached
through the elimination of a problem, this indicator should rather be
called ‘addressing of a customer problem’, which is very important” ...
Service potential is highly important because this indicator is really
differentiating” (Interviewee 28). The expert argued:
“The installation rate potential is important for the economic con-
sideration but inappropriate for the evaluation in the early stages.
Especially in the USA, this indicator would not be a good indicator
because the customers go to distributors and buy a vehicle, as also
happens in China. The installation rates are predefined. So this rate
is not influenced by the customer.”
The premium Company V rejected the number of innovative ideas as
a front-end indicator and deemed future product lifetime and package
potential equally unimportant. Company V rated USP, customer or-
ientation and satisfaction, and communication potential very highly.
One expert argued, “Everything that cannot be explained to the cus-
tomer is not an innovation, because the customer cannot perceive it”.
He continued, “Some innovations do not require a campaign, others
do”. Furthermore, “the service potential is increasingly important in the
digital world” (Interviewee 30).
The entry-level Company VI called customer satisfaction, sustain-
ability, and patents and standards unimportant for the front end.
However, the design of the product plays an important role for
Company VI, which is located in Brazil, for instance (see Table 4). In
addition, Company VI also places a high value on synergy potential
because it aims to launch innovations that are affordable for numerous
customers. Lastly, the indicators communication potential and degree
of novelty are of high importance for Company VI. These indicators are
important to the experts of the other companies as well.
Comparing the evaluation processes for Companies I to VI illumi-
nates the need for different approaches. Smaller companies, which have
fewer ideas to evaluate, tend to show less of a need for decision im-
provements than companies with more ideas per year. The companies
that wanted to improve their innovation decisions—Company I and
Company V—have a full range of products into which innovations must
be integrated, as compared to the smaller companies, which have fewer
M. Dziallas
Fig. 3. Important indicators from the company's view and applicability of indicators.
products and ideas. More ideas, combined with a full range of products,
are more difficult to manage at the front end than a few ideas combined
with a limited range of products. Therefore, volume companies with a
high number of innovation projects and a full range of products (e.g.,
small vehicles to premium vehicles), in particular, need to invest in the
front end of the innovation process more than small companies.
Nevertheless, all of the companies included in this study identified a
need for front-end decision processes.
11
Journal of Business Research xxx (xxxx) xxx–xxx
4.3. Practical applicability of front-end indicators
Because the existing literature criticizes the practicability of theo-
retical indicators, this analysis also investigates the ease of application
of the indicators within the companies. Interviewees were asked to rate
each indicator according to three applicability levels: easy, middle,
difficult. These levels indicate the ease of applying the indicators when
evaluating innovation. Applicability is also related to the availability of
data. Figs. 2 and 3 show the applicability level of the indicators as
M. Dziallas
Fig. 4. Innovation evaluation cockpit for the front end. The indicators can be applied with a five-point Likert scale.
The illustration is the author's own, based on above-mentioned literature and expert statements.
evaluated by the experts (see also Table 6 in Appendix A). The experts
argued that some indicators are easy to apply, while others that are
based on financial or technical numbers are more difficult to use. Ac-
cording to the experts, important indicators that are easy to apply at the
front end are communication potential, ease of use, and USP.
Those that are difficult to apply are economic efficiency potential,
service potential, and customer satisfaction.
The experts also identified number of innovative ideas, product
portfolio extension, and technical feasibility as easy to apply. The data
show that trend-setting and service potential are rather difficult to
apply. Based on this analysis, it seems that estimating the potential
service-related characteristics of an idea is rather difficult for experts. It
is also apparent that customer-related indicators are difficult to eval-
uate in the early stages of innovation. This might indicate the need for
higher customer integration and more agile processes at the front end.
4.4. Synthesis of results: Indicator cockpit
This section of the paper combines the existing indicators with new
data aligned with the automotive industry. The resulting indicators
present an innovation evaluation system or cockpit (Fig. 4). These in-
dicators are the results of the previous analysis, confirmed in a second
validation round with innovation experts of Companies I and V. The
indicators are based on the literature review and the expert experience
provided in the interviews. The novel indicators include higher com-
fort, improved experience, high future frequency of use, support of
digital products, or higher willingness to pay.
Fig. 4 presents suitable indicators for industrial applications, as well
as potential questions related to those indicators. The findings are in
line with Cooper (2011), who grouped “go” and “no-go” indicators into
the categories of strategic fit, technical feasibility, customer acceptance,
market opportunity, and financial performance (Cooper, 2011). How-
ever, Fig. 4 presents some new indexes, as well as related questions and
a prioritized set of existing indicators based on a new dataset. It shows
that businesses must invest enough time in defining their vision and
strategy to be able to directly align these with their innovation goals.
4.5. Front-end evaluation approach
This section describes how evaluation approaches can be im-
plemented in the context of flexible front-end innovation decisions. The
literature argues that a perception-based assessment must be applied to
evaluate innovative ideas (Bertels, Kleinschmidt, & Koen, 2011;
Shenhar, Dvir, Levy, & Maltz, 2001). Expert evaluation (based on
subjective human reasoning) can be used for innovation decision-
making at the front end to reduce uncertainty (Chang, Wei, & Lin,
2008). At this point, the knowledge and experience provided by these
experts must be used for the evaluation because concrete numbers
about the innovative idea in these early stages do not exist yet. How-
ever, the intuitive evaluation of ideas within a structured process can be
beneficial for the decision-making process because it leads to a more
strategic view (Eling, Griffin, & Langerak, 2014; Eling, Langerak, &
12
Journal of Business Research xxx (xxxx) xxx–xxx
Griffin, 2015; Khurana & Rosenthal, 1998). The decision-making pro-
cess should be multilevel, meaning that it includes central innovation
managers from different departments. An interdisciplinary and self-
organized innovation team should filter ideas in terms of go/no-go in-
dicators. Intense discussions among colleagues testing the value of ideas
are also helpful. Moreover, a micro-targeting (project) perspective
should be followed to achieve a specific innovation goal (Cooper,
2016), in line with the agile approach to innovation processes. An
overall evaluation by all departments is required to integrate expert
knowledge. Improved coordination and cooperation, combined with
sprint processes (Cooper, 2016), helps experts to evaluate ideas quickly
and react rapidly to market changes.
The innovation experts at Companies I to VI argued that “agile
methods are required to achieve faster validations and burying of non-
promising ideas at the front end. ... However, these methods should be
aligned to the entire innovation process” (Interviewee 30). The need for
more flexible processes actually depends on the type of company.
Companies that deal with many ideas each year (such as Company I or
Company VI) require an improved evaluation process, while companies
that have fewer ideas per year (like Companies II or III) do not. This
may be an indication of the need to implement agile pilot projects at
large companies with traditional manufacturing processes because the
large number of ideas requires a high degree of synchronizing and co-
operation efforts. Innovation search fields are required to align in-
novation goals with company goals and customer requirements. Based
on the expert opinions gathered for this study, top innovations and
innovative ideas must fit the companies’ automotive products. Like the
ex-ante evaluation, i.e., an evaluation of the idea in the front-end of the
innovation process, an ex-post evaluation, i.e., the evaluation of the
innovation performance after the idea’s commercialization, is very
important, according to the experts: the number of innovations in-
tegrated successfully into a product should be tracked.
5. Discussion
The study was designed to investigate front-end innovation deci-
sions based on indicators in the manufacturing industry. The qualitative
analysis and novel dataset reveal a need for front-end indicators, as well
as structured and manageable decision-making processes related to
innovative ideas and concepts.
The findings of this study are in line with the existing research,
which outlines the evaluation of innovative ideas at the front end (De
Oliveira et al., 2015; Kock et al., 2015; Mohan et al., 2017; Van den
Ende et al., 2015). The data confirm that specific indicators are re-
quired to manage the early stages and to enhance innovation decisions,
echoing the previous studies that highlight the necessity of specific
indicators (Birchall, Chanaron, Tovstiga, & Hillenbrand, 2011). Experts
agree that the industrial front end of innovation processes faces the
challenges of complexity and uncertainty. This study shows that the
front end should be structured to reduce this complexity and increase
the success of new product development (De Oliveira et al., 2015; Kock
et al., 2015; Verworn et al., 2008; Zhang & Doll, 2001).
M. Dziallas
Markham (2013) argued that companies should actively manage the
flow of ideas from the front end into more formal development pro-
cesses, which demonstrates the need for indicators. Kock et al. (2015)
confirmed the necessity of a formalized evaluation process, supporting
the opinion of the experts interviewed for this study. However, this
decision should be flexible and fast. If the speed of the evaluation
process can be increased, efficiency can be improved, which may lead
to greater front-end success (Brown & Eisenhardt, 1995; Kock et al.,
2015; Reinertsen, 1999; Verworn et al., 2008).
This study presents front-end indicators that demonstrate an idea's
value for customers and the company. The experts surveyed for this
project confirmed the high relevance of a strategic perspective of front-
end innovation decisions, thereby confirming the research by Birchall
et al. (2011). The data suggest the need for a strong customer focus,
which is in line with the existing research (see, for example, Cooper,
2011). Customers should be integrated into the innovation process al-
ready in the prelaunch stages (Cui & Wu, 2017) to test their acceptance
for the innovation (early reality check) (Ommen, Blut, Backhaus, &
Woisetschläger, 2016). It can be assumed that customer-related in-
dicators were highly rated by the experts because they suggest potential
customer acceptance, which is critical for innovation success. These
results reaffirm the previous research outlined in Section 2 of this
paper. In this study, the customer and business perspectives are struc-
tured into indicators (as shown in Fig. 4). The order of importance for
the indicators is slightly different in the data than in the literature;
overall, however, the data and literature are similar.
Further indicators are required to make decisions more transparent
and traceable. Martinusuo and Poskela's (2011) results underline the
need for a holistic but informal evaluation process aligned with a
company's objectives. This research also identified the need for more
flexible and agile processes, which are integrated into the innovation
process. Indicators help to enhance the understanding of the complexity
of innovation and to derive strategic actions from it (Hart et al., 2003).
Perhaps it is not the evaluation itself that improves the innovation
performance, but the improved understanding that results from the
evaluation (Martinusuo & Poskela, 2011).
The indicators used to evaluate innovations are soft rather than
hard, due to the lack of fixed values (i.e., sales figures or installation
rates) at the front end,3 Hart et al. (2003) underlined the need for in-
formal evaluation indicators to better make decisions and prevent
managers from failing in decision-making regarding innovative ideas.
The decision transparency provided by the innovation indicators can
enhance agility and the quality of the decision-making (Cooper &
Sommer, 2016b). It also helps a company to be more responsive to
external changes (Kock & Gemünden, 2016).
Apart from evaluation indicators, the experts confirmed that intui-
tion plays a crucial factor in innovation decision-making. Research also
recommends intuition as a front-end indicator to evaluate innovative
ideas (Magnusson et al., 2014).
It is evident that customer value plays an important role in the
success of new products and should, therefore, be integrated into the
innovation process as early as possible (Cooper & Sommer, 2016a). The
experts interviewed for this study agreed on the crucial role of custo-
mers in the selection of ideas. To better integrate the customer per-
spective, they referred to the “toothbrush test”, which Google's CEO,
Larry Page, uses to make acquisition decisions. When thinking about an
innovation, Page asks, “Is this something you will use once or twice a
day, and does it make your life better?”.4
This logic allows for fast and agile decision-making processes. The
results of this study confirm recent observations that agile processes are
3
In addition, the estimate of future values regarding the respective innovation is in-
accurate.
4
J. D’Onfro, “Here’s the Toothbrush Test Google’s CEO Uses to Make Acquisition
Decisions,” Aug. 18, 2014, http://www.businessinsider.com/larry-page-toothbrush-test-
google-acquisitions-2014-8?IR=T (accessed Oct. 11, 2017).
13
Journal of Business Research xxx (xxxx) xxx–xxx
required to improve decision-making. It therefore seems reasonable to
suggest that front-end evaluation teams working with soft indicators in
an agile way achieve quicker results (see, for example, Cooper &
Sommer, 2016b). To avoid hindering the evaluation, the decision-
making process should be conducted efficiently. The ability to react
flexibly to customer changes and market variations also requires an
available budget. Although the budget approval is a procedure that
takes a long time in the automotive industry, agile developments re-
quire a flexible and faster budget procedures.
The results of this study apply to the premium, volume, and entry-
level manufacturers at which the data was collected. The experts mostly
agree on the results, although they differ in terms of experience and
functions. Their varying backgrounds were beneficial in expanding the
understanding of the indicators; however, the results were also de-
pendent on the character of each individual expert. For example, some
experts frequently rated indicators with the top possible score, while
others rarely used values indicating high importance.
Although the data from each of the six companies is congruous, the
contingency approach must still be taken into account (Kock & Gemünden,
2016). According to the contingency approach, there is no generally ap-
plicable principle to manage a company (or, in this case, to manage in-
novations) because each company faces different situations, structures, and
leadership. In other words, there is no one-size-fits-all method for managing
the innovation process (Kok & Biemans, 2009). Therefore, each company
must decide individually which indicators are most relevant to it.
When evaluating ideas, the internal structure of a business plays an
important role because the way in which the relevant departments
collaborate at the front end of the product development process influ-
ences the decision-making process. Furthermore, the success of in-
novations depends on organizational ambidexterity, or a company’s
ability to efficiently manage today’s business while setting the path for
tomorrow. For innovation management, a company must exploit (in-
cremental innovation) and simultaneously explore (radical innovation)
its products in order to be successful (e.g., Gupta, Smith, & Shalley,
2006; Tushman & O’Reilly, 1996). The surveyed experts confirmed the
need for different processes for the evaluation and implementation of
radical and incremental ideas.
This study provides useful implications for practitioners. First, the
results underline the need for managers to pay more attention to in-
novation evaluation at the front end. Especially in the current time of
disruptive change facing automotive companies, managers must be able
to make agile innovation decisions that are future- and customer-or-
iented. Managers can benefit from understanding and utilizing in-
novation indicators because an improved quality of decisions positively
influences innovation and company performance (Kock et al., 2015).
Second, the application of innovation indicators has various ad-
vantages for companies. It helps managers cancel projects in a timely
manner to avoid wasting resources. Decisions on project termination
can achieve a higher strategic fit in the organizational product portfolio
(Unger Kock, Gemünden, & Jonas, 2012). However, the cockpit should
not be the sole basis for such an evaluation, though it can help auto-
motive managers better survey front-end innovative ideas and identify
the most promising ones.
Finally, companies can trigger the use of those indicators that are
the most important but difficult to apply, such as service potential, by
supporting incentives, organizational prerequisites, or initiatives. This
support may motivate different parties within the company to learn
enough about a topic to apply an indicator. Artificial intelligence (AI)
may also be helpful for the decision-making process; however, it re-
quires sufficient data for analysis and experts who can use AI methods.
Managers can follow an innovation strategy to better align in-
novation decisions with company goals (Kock et al., 2015). This
strategy should include strong customer (market pull) and technolo-
gical (technology push) perspectives that are in line with search fields
displaying potentially successful innovation areas (Kock et al., 2015).
With a defined innovation strategy, high-quality decisions can be made
M. Dziallas
(Cooper, 2011), and managers can ensure formal clarity about in-
novation evaluation processes in order to increase the probability of
front-end success. Furthermore, companies can foster radical innova-
tion by differentiating between incremental and radical innovation in
the budget allocation process and providing a separate budget for ra-
dical innovations. This could be done to ensure that radical projects are
implemented rather than tabled because of their high risk.
Another important finding from the study is that improving commu-
nication at the front end strengthens the customer’s perspective and the
visibility of an innovation in the market (Lee, Lee, & Schumann, 2002). This
view is strongly confirmed by the experts interviewed for the study.
6. Conclusion
The aim of this research was to identify a broad range of expert
opinions on front-end innovation indicators and evaluation approaches
using agile methods. The resulting indicator cockpit and evaluation
approach have been shown to improve future innovation performance.
The indicators investigated in this study are based on a systematic lit-
erature review and expert knowledge. By analyzing existing and novel
indicators in terms of their importance and practical applicability at the
front end, the author pinpointed the most relevant indicators.
The data suggest that the most relevant indicators for the front end
are high customer relevance, strategic fit, high communication poten-
tial, and vision potential. Other indicators such as resistance to the
product, innovation quality, number of innovative ideas, future product
duration, and extension of product portfolio are of only minor im-
portance for front-end decisions for innovations.
The findings demonstrate that front-end indicators are extremely
necessary in practice and that experts indicate a need for more flexible
and agile evaluation within the structured front end of the innovation
process. This means that agile methods are required to improve the
front-end evaluation of ideas. This, in turn, underlines the research that
argues that companies should have organizational agility, including
adaptive and strong dynamic capabilities, to be successful (e.g., Day,
2011; Roberts & Grover, 2011; Teece et al., 2016). Furthermore, this
research confirms the need for a stronger front-end focus on the cus-
tomer in order to meet customer needs and market trends. This requires
customer agility, including sense and response capabilities (Roberts &
Grover, 2011). To apply the indicators, an innovation strategy that
defines the company's and the innovation's aims is required. Overall,
the research shows that companies require strategic as well as opera-
tional capabilities to apply the indicators (Hart et al., 2003).
This study provides the first qualitative and systematic demonstra-
tion of front-end indicators based on the experience of innovation ex-
perts from volume, premium, and entry-level manufacturers.
Differences between these companies appear in the need for evaluation
approaches. Specificially, companies with fewer ideas per year (smaller
companies) tend to have a smaller need for evaluation improvements
than companies with more ideas per year (large companies). This may
be because a large number of ideas, combined with a full range of
products and services, is more difficult to manage at the front end than
a small number of ideas, combined with a low range of products and
services. Therefore, volume companies with a high number of innova-
tion projects and a full range of products need to invest more effort into
the front end of the innovation process than other types of companies.
This research deepens the understanding and quality of front-end
innovation indicators and evaluation. It helps to reduce the lack of
knowledge about the practical innovation selection process, identified
by Hart et al. (2003).
14
Journal of Business Research xxx (xxxx) xxx–xxx
7. Limitations and future research
The study is not without limitations, as the investigation covers a
limited timeframe. Next to cross-sectional data, a longitudinal research
design (panel data) could have been used. This would have taken the
time lag between innovation input and innovation success into account.
An exploratory factor analysis might also have been useful for analyzing
several evaluation indicators. This analysis would require a higher
number of the observation unit. Future research is necessary to test the
practicability of the relevant indicators and to elaborate on the success
of the front-end innovation portfolio (Kester, Hultink, & Griffin, 2014).
Such research could emphasize focus groups and workshops to gather
deeper insights. Corporate foresight methods and additional surveys
will require further investigation to better integrate the customer view
(Rohrbeck & Gemünden, 2011).
This analysis revealed deep insights into the complex and difficult
field of innovation evaluation. Although the study encompasses mul-
tiple segments within the automotive industry and various countries,
the experts' views on innovation indicators may not be transferable to
other industries. Various industrial and cultural aspects may have dif-
ferent influences on the innovation evaluation (cf. Ali & Park, 2016;
Cleveland, Rojas-Méndez, Laroche, & Papadopoulos, 2016). It is likely
that other industries that launch several innovations per year face
comparable challenges with regard to front-end decisions.
The results of this study only explain the evaluation phenomena of
several volume manufacturers, one premium producer, and one entry-level
manufacturer. A subsequent research step would be to further differentiate
between these types of producers. It might be reasonable, for example, to
analyze the importance of indicators for a luxury producer that focuses on a
different innovation strategy than a volume manufacturer, for example.
Additional volume producers also need to be analyzed in relation to pre-
mium and entry-level manufacturers to validate the results of the present
study. Future research is advised to elaborate front-end decisions in com-
panies dealing with many ideas. Thus, the generalization of the results from
this study should be further explored.
The results do not shed light on the relationship between front-end
evaluation and innovation success because the interplay of early cus-
tomer orientation and innovation sales rates requires further in-
vestigation. Likewise, the agile view of front-end indicators deserves
further investigation to validate the results of this study. There is also a
need to explore the front-end decision-making process for radical in-
novations in comparison to incremental innovations.
Despite these limitations, the findings presented in this paper serve
as the basis for further investigation of the characteristics of successful
innovations. Which types of innovative ideas are most likely to be in-
tegrated into products? To answer this question, future studies should
quantitatively explore the method of innovation tracking to analyze the
types of innovations that have been successful and the ones that have
not. This knowledge can be used to select the right evaluation in-
dicators. A next step would be to evaluate the innovation portfolios of
different companies with the identified indicators and measure in-
novation and business performance over time (panel data), which could
validate quantitatively the results of this research.
Acknowledgements
The author is grateful to the experts who have participated in the
interviews for their willingness to contribute and discuss their knowl-
edge. The author also thanks two anonymous reviewers for their helpful
comments to improve the manuscript significantly. All remaining
shortcomings are in her sole responsibility.
M. Dziallas Journal of Business Research xxx (xxxx) xxx–xxx

Appendix A

Table 6
Practical applicability of indicators based on experts, median value used.

Indicator Easily applicable indicators Middle applicable indicators Difficult applicable indicators

Ease of use for customer 20 1 4

Communication potential 19 1 4
Number of innovative ideas 16 1 3
Degree of novelty for the company 16 – 7
Extension of product portfolio 15 2 8
Unique selling proposition 15 2 8
Technical feasibility 14 2 6
Innovation quality 13 2 7
Patents and standards 13 1 7
Degree of maturity 11 2 8
Synergy potential 11 3 9
Future product lifetime 11 – 10
Brand defining 10 4 9
Degree of innovation 10 3 10
Innovation design 10 1 12
Resistance of innovation 9 2 8
Sustainability 8 5 5
Installation rate potential 8 5 11
Time to market 8 2 11
Service potential 7 2 12
Economic efficiency 7 3 13
Trend-setting 7 1 14
Customer orientation 6 5 10
Customer satisfaction 6 2 12
Note: The applicability values are counts indicating how often experts evaluated the respective indicator as easily, middle or difficult applicable.

References
findings, and future directions. Academy of Management Review, 20(2), 343–378.
Caird, S., Hallett, S., & Potter, S. (2013). The Open2-Innova8ion Tool—A software tool for
Aarikka-Stenroos, L., & Lehtimäki, T. (2014). Commercializing a radical innovation:
rating organisational innovation performance. Technovation, 33(10-11), 381–385.
Probing the way to the market. Industrial Marketing Management, 43(8), 1372–1384.
Carbonell-Foulquié, P., Munuera-Alemán, J. L., & Rodrıguez-Escudero, A. I. (2004).
Achiche, S., Appio, F. P., McAloone, T. C., & Di Minin, A. (2013). Fuzzy decision support
Criteria employed for go/no-go decisions when developing successful highly in-
for tools selection in the core front end activities of new product development.
novative products. Industrial Marketing Management, 33(4), 307–316.
Research in Engineering Design, 24(1), 1–18. http://dx.doi.org/10.1007/s00163-012-
Chang, H., Wei, C., & Lin, R. (2008). A model for selecting product ideas in fuzzy front end.
0130-4.
Thousand Oaks: SAGE Publications.
Adams, R., Bessant, J., & Phelps, R. (2006). Innovation Management measurement: A
Chiesa, V., Frattini, F., Lazzarotti, V., & Manzini, R. (2009). Performance measurement in
review. International Journal of Management Reviews, 8(1), 21–47.
R&D: Exploring the interplay between measurement objectives, dimensions of per-
Alcaide-Marzal, J., & Tortajada-Esparza, E. (2007). Innovation assessment in traditional
formance and contextual factors. R&D Management, 39(5), 488–519.
industries. A proposal of aesthetic innovation indicators. Scientometrics, 72(1), 33–57.
Cleveland, M., Rojas-Méndez, J. I., Laroche, M., & Papadopoulos, N. (2016). Identity,
Ali, M., & Park, K. (2016). The mediating role of an innovative culture in the relationship
culture, dispositions and behavior. A cross-national examination of globalization and
between absorptive capacity and technical and non-technical innovation. Journal of
culture change. Journal of Business Research, 69(3), 1090–1102. http://dx.doi.org/10.
Business Research, 69(5), 1669–1675. http://dx.doi.org/10.1016/j.jbusres.2015.10.
1016/j.jbusres.2015.08.025.
036.
Cooper, R. G. (1981). An empirically derived new product project selection model. IEEE
Astebro, T., & Michaela, J. L. (2005). Predictors of the survival of innovations. Journal of
Transactions on Engineering Management, EM-28(3), 54–61.
Product Innovation Management, 22(3), 322–335.
Cooper, R. G. (1990). Stage-gate systems: A new tool for managing new products. Business
Barczak, G., Griffin, A., & Kahn, K. B. (2009). Perspective: trends and drivers of success in
Horizons, 33(3), 44–54.
NPD practices: results of the 2003 PDMA best practices study. Journal of Product
Cooper, R. G. (1999). From experience: The invisible success factors in product innova-
Innovation Management, 26(1), 3–23.
tion. Journal of Product Innovation Management, 16(2), 115–133.
Begel, A., & Nagappan, N. (2007, September). Usage and perceptions of agile software
Cooper, R. G. (2008). Perspective: The Stage-Gate® idea-to-launch process—Update,
development in an industrial context: An exploratory study. Empirical Software
what's new, and NexGen systems. Journal of Product Innovation Management, 25(3),
Engineering and Measurement, 2007. ESEM 2007. First International Symposium on (pp.
213–232.
255–264). IEEE.
Cooper, R. G. (2011). Perspective: The innovation dilemma - How to innovate when the
Bertels, H. M. J., Kleinschmidt, E. J., & Koen, P. A. (2011). Communities of practice versus
market is mature. Journal of Product Innovation Management, 28(S1), 2–27.
organizational climate: Which one matters more to dispersed collaboration in the
Cooper, R. G. (2016). Agile-Stage-Gate Hybrids: The next stage for product development.
front end of innovation? Journal of Product Innovation Management, 28, 757–772.
Research-Technology Management, 59(1), 21–29.
Bessant, J., Francis, D., Meredith, S., Kaplinsky, R., & Brown, S. (2001). Developing
Cooper, R. G., & Kleinschmidt, E. J. (1987). Success factors in product innovation.
manufacturing agility in SMEs. International Journal of Technology Management, 22(1-
Industrial Marketing Management, 16, 215–223.
3), 28–54.
Cooper, D. R., & Schindler, P. S. (2013). Business research methods (12th ed.). New York:
Bessant, J., Von Stamm, B., Moeslein, K. M., & Neyer, A. K. (2010). Backing outsiders:
McGraw-Hill Irwin.
selection strategies for discontinuous innovation. R&D Management, 40(4), 345–356.
Cooper, R. G., & Sommer, A. F. (2016a). Agile-Stage-Gate. New idea-to-launch method for
Birchall, D., Chanaron, J.-J., Tovstiga, G., & Hillenbrand, C. (2011). Innovation perfor-
manufactured new products is faster, more responsive. Industrial Marketing
mance measurement: Current practices, issues and management challenges.
Management, 59, 167–180. http://dx.doi.org/10.1016/j.indmarman.2016.10.006.
International Journal of Technology Management, 56(1), 1–20.
Cooper, R. G., & Sommer, A. F. (2016b). The Agile-Stage-Gate Hybrid Model. A promising
Bogner, A., & Menz, W. (2009). The theory-generating expert interview: epistemological
new approach and a new research opportunity. Journal of Product Innovation
interest, forms of knowledge, interaction. Interviewing experts (pp. 43–80). London:
Management, 33(5), 513–526. http://dx.doi.org/10.1111/jpim.12314.
Palgrave Macmillan.
Cruz-Cázares, C., Bayona-Sáez, C., & García-Marco, T. (2013). You can't manage right
Borrás, S., & Edquist, C. (2013). The choice of innovation policy instruments.
what you can't measure well. Technological innovation efficiency. Research Policy,
Technological Forecasting and Social Change, 80(8), 1513–1522.
42(6–7), 1239–1250. http://dx.doi.org/10.1016/j.respol.2013.03.012.
Brown, S. L., & Eisenhardt, K. M. (1995). Product development: Past research, present

15
M. Dziallas
Cui, A. S., & Wu, F. (2017). The impact of customer involvement on new product de-
velopment: Contingent and substitutive effects. Journal of Product Innovation
Management, 34(1), 60–80.
Czarnitzki, D., & Kraft, K. (2004). Innovation indicators and corporate credit ratings:
Evidence from German firms. Economics Letters, 82, 377–384.
Day, G. S. (2000). Managing market relationships. Journal of the Academy of Marketing
Science, 28(1), 24–30.
Day, G. S. (2011). Closing the marketing capabilities gap. Journal of Marketing, 75(4),
183–195.
De Brentani, U. (2001). Innovative versus incremental new business services: Different
keys for achieving success. Journal of Product Innovation Management, 18, 169–187.
De Brentani, U., & Reid, S. E. (2012). The fuzzy front-end of discontinuous innovation.
Insights for research and management. Journal of Product Innovation Management,
29(1), 70–87. http://dx.doi.org/10.1111/j.1540-5885.2011.00879.x.
De Felice, F., & Petrillo, A. (2013). Key success factors for organizational innovation in the
fashion industry. International Journal of Engineering Business Management, 5(27),
1–11.
De Oliveira, M. G., Rozenfeld, H., Phaal, R., & Probert, D. (2015). Decision making at the
front end of innovation: The hidden influence of knowledge and decision criteria. R&
D Management, 45(2), 161–180.
Dewangan, V., & Godse, M. (2014). Towards a holistic enterprise innovation performance
measurement system. Technovation, 34(9), 536–545. http://dx.doi.org/10.1016/j.
technovation.2014.04.002.
Edison, H., bin Ali, N., & Torkar, R. (2013). Towards innovation measurement in the
software industry. Journal of Systems and Software, 86(5), 1390–1407. http://dx.doi.
org/10.1016/j.jss.2013.01.013.
Eling, K., Griffin, A., & Langerak, F. (2014). Using intuition in fuzzy front-end decision-
making: A conceptual framework. Journal of Product Innovation Management, 31(5),
956–972.
Eling, K., Griffin, A., & Langerak, F. (2016). Consistency matters in formally selecting
incremental and radical new product ideas for advancement. Journal of Product
Innovation Management, 33(1), 20–33. http://dx.doi.org/10.1111/jpim.12320.
Eling, K., & Herstatt, C. (2017). Managing the front end of innovation—Less fuzzy, yet
still not fully understood. Journal of Product Innovation Management, 34(6), 864–874.
Eling, K., Langerak, F., & Griffin, A. (2015). The performance effects of combining ra-
tionality and intuition in making early new product idea evaluation decisions.
Creativity and Innovation Management, 24(3), 464–477.
Enkel, E., Perez-Freije, J., & Gassmann, O. (2005). Minimizing market risks through
customer integration in new product development: Learning from bad practice.
Creativity and Innovation Management, 14(4), 425–437.
Evanschitzky, H., Eisend, M., Calantone, R. J., & Jiang, Y. (2012). Success factors of
product innovation. An updated meta-analysis. Journal of Product Innovation
Management, 29(3), 21–37. http://dx.doi.org/10.1111/j.1540-5885.2012.00964.x.
Fleuren, M., Paulussen, T., Dommelen, P., & van Burren, S. (2014). Towards a mea-
surement instrument for determinants of innovations. International Journal for Quality
in Health Care, 26(5), 501–510.
Flor, M. L., & Oltra, M. J. (2004). Identification of innovating firms through technological
innovation indicators: an application to the Spanish ceramic tile industry. Research
Policy, 33, 323–336.
Frederiksen, M. H., & Knudsen, M. P. (2017). From creative ideas to innovation perfor-
mance: The role of assessment criteria. Creativity and Innovation Management, 26(1),
60–74.
Frishammar, J., Florén, H., & Wincent, J. (2011). Beyond managing uncertainty: Insights
from studying equivocality in the fuzzy front end of product and process innovation
projects. IEEE Transactions on Engineering Management, 58(3), 551–563.
Griffin, A., & Page, A. L. (1993). An interim report on measuring product development
success and failure. Journal of Product Innovation Management, 10, 291–308.
Gupta, A. K., Smith, K. G., & Shalley, C. E. (2006). The interplay between exploration and
exploitation. Academy of Management Journal, 49(4), 693–706. http://dx.doi.org/10.
5465/AMJ.2006.22083026.
Hart, S., Hultink, E. J., Tzokas, N., & Commandeur, H. R. (2003). Industrial companies'
evaluation criteria in new product development gates. Journal of Product Innovation
Management, 20(1), 22–36. http://dx.doi.org/10.1111/1540-5885.201003.
Hittmar, S., Varmusa, M., & Lendela, V. (2015). Proposal of evaluation system for suc-
cessful application of innovation strategy through a set of indicators. Procedia
Economics and Finance, 26, 17–22.
Holahan, P. J., Sullivan, Z. Z., & Markham, S. K. (2014). Product development as core
competence: How formal product development practices differ for radical, more in-
novative, and incremental product innovations. Journal of Product Innovation
Management, 31(2), 329–345.
Idris, A., & Trey, L. (2011). Exploring the motives and determinants of innovation per-
formance of Malaysian offshore international joint ventures. Management Decision,
49(10), 1623–1641.
Ivanov, C. I., & Avasilcăi, S. (2014). Measuring the performance of innovation processes:
A Balanced Scorecard perspective. Procedia - Social and Behavioral Sciences, 109,
1190–1193.
Jupp, V. (2006). The SAGE dictionary of social research methods. Thousand Oaks,
California: SAGE Publications.
Kester, L., Hultink, E. J., & Griffin, A. (2014). An empirical investigation of the ante-
cedents and outcomes of NPD portfolio success. Journal of Product Innovation
Management, 31(6), 1199–1213.
Khurana, A., & Rosenthal, S. R. (1998). Towards holistic “front-ends” in new product
development. Journal of New Product Development, 15(1), 57–74.
Kim, S.-K. (2014). Explicit design of innovation performance metrics by using analytic
hierarchy process expansion. International Journal of Mathematics and Mathematical
Sciences, 2014, 1–7.
16
Journal of Business Research xxx (xxxx) xxx–xxx
Klenner, P., Hüsig, S., & Dowling, M. (2013). Ex-ante evaluation of disruptive suscept-
ibility in established value networks—When are markets ready for disruptive in-
novations? Research Policy, 42(4), 914–927. http://dx.doi.org/10.1016/j.respol.
2012.12.006.
Kline, S. J., & Rosenberg, N. (1986). An overview of innovation. In R. Landau, & N.
Rosenberg (Eds.). The positive sum strategy: Harnessing technology for economic growth.
Washington: National Academy Press.
Kobayashi, H. (2006). A systematic approach to eco-innovative product design based on
life cycle planning. Advanced Engineering Informatics, 20, 113–125.
Kock, A., & Gemünden, H. G. (2016). Antecedents to decision-making quality and agility
in innovation portfolio management. Journal of Product Innovation Management,
33(6), 670–686. http://dx.doi.org/10.1111/jpim.12336.
Kock, A., Heising, W., & Gemünden, H. G. (2015). How ideation portfolio management
influences front-end success. Journal of Product Innovation Management, 32(4),
539–555. http://dx.doi.org/10.1111/jpim.12217.
Kock, A., Heising, W., & Gemünden, H. G. (2016). A contingency approach on the impact
of front-end success on project portfolio success. Project Management Journal, 47(2),
115–129.
Koen, P. A., Ajamian, G. M., Boyce, S., Clamen, A., Fisher, E., Fountoulakis, S., ... Seibert,
R. (2002). Fuzzy front end: Effective methods, tools, and techniques. In P. Bellviav, A.
Griffin, & S. Somemeyer (Eds.). The PDMA tool-book for new product development (pp.
5–36). New York: John Wiley & Sons.
Kok, R. A., & Biemans, W. G. (2009). Creating a market-oriented product innovation
process: A contingency approach. Technovation, 29(8), 517–526.
Krishnan, V., & Ulrich, K. T. (2001). Product development decisions: A review of the
literature. Management Science, 47(1), 1–21.
Lee, E. J., Lee, J., & Schumann, D. W. (2002). The influence of communication source and
mode on consumer adoption of technological innovations. Journal of Consumer
Affairs, 36(1), 1–27.
Leifer, R., O'Connor, G. C., & Rice, M. (2001). Implementing radical innovation in mature
firms. The role of hubs. Academy of Management Executive, 15(3), 102–113. http://dx.
doi.org/10.5465/AME.2001.5229646.
Lenfle, S. (2008). Exploration and project management. International Journal of Project
Management, 26, 469–478.
Littell, C. (1994). Innovation in medical technology: Reading the indicators. Health
Affairs, 13(3), 226–235.
Magni, C. A. (2010). Average internal rate of return and investment decisions: a new
perspective. The Engineering Economist, 55(2), 150–180.
Magnusson, P. R., Netz, J., & Wästlund, E. (2014). Exploring holistic intuitive idea
screening in the light of formal criteria. Technovation, 34(5–6), 315–326.
Makkonen, T., & van der Have, R. (2013). Benchmarking regional innovative perfor-
mance: Composite measures and direct innovation counts. Scientometrics, 94,
247–262.
Markham, S. K. (2013). The impact of front-end innovation activities on product per-
formance. Journal of Product Innovation Management, 30(4), 77–92. http://dx.doi.org/
10.1111/jpim.12065.
Martinusuo, M., & Poskela, J. (2011). Use of evaluation criteria and innovation perfor-
mance in the front end of innovation. Journal of Product Innovation Management, 28,
896–914.
McCall, G. J., & Simmons, J. L. (1969). Issues in Participant Observation: A Text and Reader.
New York: Random House.
Mendes Luz, L., Carlos de Francisco, A., & Piekarski, C. (2015). Proposed model for as-
sessing the contribution of the indicators obtained from the analysis of life-cycle
inventory to the generation of industry innovation. Journal of Cleaner Production, 96,
339–348.
Mohan, M., Voss, K. E., & Jiménez, F. R. (2017). Managerial disposition and front-end
innovation success. Journal of Business Research, 70, 193–201. http://dx.doi.org/10.
1016/j.jbusres.2016.08.019.
Montoya-Weiss, M. M., & O'Driscoll, T. M. (2000). From experience: Applying perfor-
mance support technology in the fuzzy front end. Journal of Product Innovation
Management, 17, 143–161.
Ommen, N. O., Blut, M., Backhaus, C., & Woisetschläger, D. M. (2016). Toward a better
understanding of stakeholder participation in the service innovation process: More
than one path to success. Journal of Business Research, 69(7), 2409–2416.
Palmberg, C. (2006). The sources and success of innovations – Determinants of com-
mercialisation and break-even times. Technovation, 26, 1253–1267.
Porter, M. E., & Heppelmann, J. E. (2015). How smart, connected products are trans-
forming companies. Harvard Business Review, 93(10), 96–114.
Poskela, J., & Martinsuo, M. (2009). Management control and strategic renewal in the
front end of innovation. Journal of Product Innovation Management, 26(6), 671–684.
http://dx.doi.org/10.1111/j.1540-5885.2009.00692.x.
Reid, S. E., & de Brentani, U. (2004). The fuzzy front end of new product development for
discontinuous innovations: A theoretical model. Journal of Product Innovation
Management, 21, 170–184.
Reinertsen, D. G. (1999). Taking the fuzziness out of the fuzzy front end. Research-
Technology Management, 42(6), 25–31.
Roberts, N., & Grover, V. (2012). Investigating firm's customer agility and firm perfor-
mance: The importance of aligning sense and respond capabilities. Journal of Business
Research, 65(5), 579–585.
Rocha, A., Quintella, C., & Torres, E. (2015). Biodiesel in Brazil: Science, technology and
innovation indicators. International Journal of Technology Management, 69(3/4),
246–260.
Rohrbeck, R., & Gemünden, H. G. (2011). Corporate foresight: Its three roles in enhancing
the innovation capacity of a firm. Technological Forecasting and Social Change, 78(2),
231–243.
Roper, S., & Hewitt-Dundas, N. (2015). Knowledge stocks, knowledge flows and
M. Dziallas
innovation: Evidence from matched patents and innovation panel data. Research
Policy, 44, 1327–1340.
Saldana, J. (2009). The coding manual for qualitative researchers. London, England: SAGE.
Sawang, S. (2011). Key performance indicators for innovation implementation:
Perception vs. actual usage. Asia Pacific Management Review, 16(1), 23–29.
Schmoch, U., & Gauch, S. (2009). Service marks as indicators for innovation in knowl-
edge-based services. Research Evaluation, 18(4), 323–335.
Sharmelly, R. (2016). Innovation for Emerging Markets Confronting Institutional
Environment Challenges: Perspectives from Visionary Leadership and Institutional
Entrepreneurship. International Journal of Business and Management, 11(6), 108.
Shenhar, D., Dvir, D., Levy, O., & Maltz, A. C. (2001). Project success: A multidimensional
strategic concept. Long Range Planning, 34(6), 699–725.
Suwannaporn, P., & Speece, M. W. (2010). Assessing new product development success
factors in the Thai food industry. British Food Journal, 112(4), 364–386.
Teece, D., Peteraf, M., & Leih, S. (2016). Dynamic capabilities and organizational agility.
California Management Review, 58(4), 13–35.
Thomas, D. R. (2006). A General Inductive Approach for Analyzing Qualitative
Evaluation Data. American Journal of Evaluation, 27, 237–246.
Tipping, J. W., Zeffren, E., & Fusfeld, A. R. (1995). Assessing the value of your tech-
nology. Research-Technology Management, 38(5), 22–39.
Tushman, M. L., & O'Reilly, C. A., III (1996). Ambidextrous organizations: Managing
evolutionary and revolutionary change. California Management Review, 38(4), 8–29.
Unger, B. N., Kock, A., Gemünden, H. G., & Jonas, D. (2012). Enforcing strategic fit of
17
Journal of Business Research xxx (xxxx) xxx–xxx
project portfolios by project termination: An empirical study on senior management
involvement. International Journal of Project Management, 30(6), 675–685.
Van den Ende, J., Frederiksen, L., & Prencipe, A. (2015). The front end of innovation.
Organizing search for ideas. Journal of Product Innovation Management, 32(4),
482–487. http://dx.doi.org/10.1111/jpim.12213.
Van Hoof, G., Weisbrod, A., & Kruse, B. (2014). Assessment of progressive product in-
novation on key environmental indicators: Pampers® Baby Wipes from 2007-2013.
Sustainability, 6, 5129–5142.
Verworn, B., Herstatt, C., & Nagahira, A. (2008). The fuzzy front end of Japanese new
product development projects: Impact on success and differences between incre-
mental and radical projects. R&D Management, 38(1), 1–19.
Voicu, M.-C. (2011). Using the snowball method in marketing research on research on
hidden populations. Challenges of the Knowledge Society, 1, 1341–1351.
Yang, C., Zhang, Q., & Ding, S. (2015). An evaluation method for innovation capability
based on uncertain linguistic variables. Applied Mathematics and Computation, 256,
160–174.
Yin, R. K. (2003). Case study research: Design and methods (third ed.). Beverly Hills, CA:
SAGE Publications.
Yin, R. K. (2014). Case study research: Design and methods (13th ed.). Thousand Oaks,
California: SAGE.
Zhang, Q., & Doll, W. J. (2001). The fuzzy front end and success of new product devel-
opment: A causal model. European Journal of Innovation Management, 4(2), 95–112.