Annual Reviews in Control 48 (2019) 165–177

Contents lists available at ScienceDirect

Annual Reviews in Control

journal homepage: www.elsevier.com/locate/arcontrol

Full Length Article

A maturity assessment approach for conceiving context-specific

roadmaps in the Industry 4.0 era ✩
M. Colli a,∗, U. Berger b, M. Bockholt a, O. Madsen a, C. Møller a, B. Vejrum Wæhrens a
a
Aalborg University, Aalborg Øst, 9220 Denmark
b
Brandenburg University of Technology, Cottbus, 03046 Germany

a r t i c l e i n f o a b s t r a c t

Article history: The digital transformation of the production sector is setting the scene for a major industrial change.
Received 15 February 2019 The need for supporting companies in this transformation is currently covered by several maturity mod-
Revised 4 April 2019
els, generally operationalized through standardized questionnaires, which provide, as an outcome, an as-
Accepted 5 June 2019
sessment of the current maturity stage and a set of general improvement recommendations according
Available online 19 June 2019
to it. However, to provide companies with a more tangible support, there is a need for more individ-
Keywords: ual approach. In order to deal with this need, this paper proposes, following a design science research
Digital transformation framework, a novel approach based on Problem-Based Learning for structuring the assessment procedure
Industry 4.0 as a dialectic process. This approach aims at facilitating the contextualization of the assessed company
Maturity model and, consequently, the identification of context-specific improvement recommendations. The proposed
Maturity assessment approach, supported by a maturity model used for framing information collected during the assessment
Problem based learning
process, is tested in three industrial cases. Although these have been assessed at the same maturity
stage, different improvement recommendations have been proposed according to contextual factors such
as strategic goals, core processes and key performance indicators.
© 2019 Elsevier Ltd. All rights reserved.

1. Introduction their business, offering significant business opportunities within

Globally, the production industry is experiencing a tremendous
change labeled as “The Fourth Industrial Revolution”. This change,
also known as Industry 4.0, is triggered by an exponential growth
in new digital technologies such as cloud computing and the in-
ternet of things. Through the enabling of data ubiquity and con-
nectivity capabilities, these provide an increasing number of new
possibilities for the development of new products, processes and
services (Iansiti & Lakhani, 2014; Porter & Heppelmann, 2014).
There are many speculations concerning the potential related to
the implementation of these new technologies (McKinsey & Com-
pany, 2015), ranging from the improvement in operational effec-
tiveness to the increase of value provided through products and
services (Schrauf Berttram, 2016) and entirely new business mod-
els. Not surprisingly, there is a great interest in Industry 4.0 driven
by these expectations. A study performed among Danish compa-
nies shows that 76% of them expect digitalization to transform
✩
This research has been supported by the Manufacturing Academy of Denmark
(MADE) Digital (Grant No. 6151-0 0 0 06B), within its Digital program.
∗
Corresponding author.
E-mail addresses: colli@mp.aau.dk (M. Colli), ulrich-berger@b-tu.de (U.
Berger), mack@business.aau.dk (M. Bockholt), om@mp.aau.dk (O. Madsen),
charles@business.aau.dk (C. Møller), bvw@business.aau.dk (B.V. Wæhrens).
the next three years, and 60% of them already have ongoing Indus-
try 4.0-related initiatives (Ericsson, 2015). Nevertheless, the digital
transformation involves multi-disciplinary activities and requires,
therefore, a number of experts on diverse domains, which may not
be present in all companies. This makes it difficult for many orga-
nizations to grasp the Industry 4.0 idea, still in its infancy, and to
set up comprehensive strategies to address the digital transforma-
tion (Andulkar, Tho Le, & Berger, 2018). Due to that, there is a need
for methods to support companies in the operationalization of this
transformation.
In order to answer this need, several maturity models describ-
ing the evolution path concerning the digital transformation of
the industrial sector have been published (Mittal, Khan, Romero, &
Wuest, 2018), as their application (i.e. digital maturity assessment)
is considered to be beneficial due to the generation of awareness
regarding the addressed domain and the provision of a frame for
systematically design improvement activities (Van de Vrande, De
Jong, Vanhaverbeke, & De Rochemont, 2009). The assessment of
digital maturity involves a maturity model, which is outlining the
different maturity stages and the considered areas in the company,
and of an assessment approach, providing a structured procedure
to operationally assess the maturity of a company. A descriptive
use of a digital maturity assessment is providing, as an outcome,

https://doi.org/10.1016/j.arcontrol.2019.06.001
1367-5788/© 2019 Elsevier Ltd. All rights reserved.
166 M. Colli, U. Berger and M. Bockholt et al. / Annual Reviews in Control 48 (2019) 165–177
an indication of the current maturity stage of a company in regards
to the evolution path outlined by the maturity model. However, in
order to support companies in their evolution, some of the exist-
ing models also provide, as an outcome of the assessment process,
a set of general improvement recommendations based on the ma-
turity stage of the assessed company but often detached from the
actual company context. However, this prescriptive use of maturity
models requires the ability to adapt to “organization-specific situ-
ational characteristics” (Röglinger, Pöppelbuß, & Becker, 2012) and,
therefore, to the context, as there could be different maturation
paths (Teo & King, 1997). This is still a critical aspect for the oper-
ationalization of maturity models (Röglinger et al., 2012), as there
is a need for conceiving tailored roadmaps consisting of context-
specific improvement recommendations for guiding the assessed
companies in their own digital transformation (Mittal et al., 2018).
This research deals with this literature gap proposing an as-
sessment approach, meant to be used for the operationalization
of a digital maturity assessment, which aims at supporting the
contextualization of its outcome. The objective is to make pos-
sible for the assessing party to adapt improvement recommen-
dations to company-specific needs. This artefact has been devel-
oped and tested following a design science research framework
(Hevner, March, Park, & Ram, 2004). It consists of a learning ap-
proach, based on the Problem Based Learning (PBL) model, with
the aim of structuring the digital maturity assessment around a
dialectic process with the assessed company. This is meant to fa-
cilitate the understanding of its contextual factors and the tailoring
of the provided improvement recommendations. There are many
learning approaches that can be used for supporting the design of
an artefact such as the one that the researchers intend to propose.
The PBL model has been chosen by the authors as a backbone for
the proposed approach due to its theoretical anchoring to the prox-
imal zone of development concept (Vygotsky, 1978) and, there-
fore, to the concept of maturity. This argues that maturity pro-
gression is facilitated by learning and collaborative problem solv-
ing (Harland, 2003). Furthermore, the PBL model fits particularly
well in contexts characterized by complex problems “that do not
have a single correct answer” (Hmelo-Silver;, 2004). Through the
enabling of collaboration, PBL facilitates the identification of the
key elements to be learned in order to develop an effective solu-
tion to a specific problem (Hmelo-Silver;, 2004) as well as the un-
derstanding of the related implications (Schmidt, Rotgans, & Yew,
2011).
The contribution of this paper consists, as an extension of
Colli et al. (2018), in bringing up the need for contextualization in
relation to digital maturity assessments and in proposing a novel
assessment approach to cope with this need. Furthermore, in addi-
tion to the previous research paper, it provides a reference model –
based on the Acatech maturity model (Schuh et al., 2017), chosen
due to its focus on data and connectivity – which relates matu-
rity stages and dimensions for mapping the information collected
during the assessment process as well as a more in-depth descrip-
tion of the proposed assessment procedure, testing it in three cases
and discussing its contextualization capabilities and the emerged
key contextual factors. The proposed approach is meant as a learn-
ing tool, based on PBL, for assisting practitioners in the provision
of company-specific guidelines as an outcome of a digital maturity
assessment, and scholars in their study of the transition across dig-
ital maturity stages in diverse contexts. This leads to the following
research question:
“How can Problem-Based Learning be used to operationalize
a digital maturity assessment, leading to context-specific im-
provement recommendations?”
This paper firstly presents the design science research frame-
work (Hevner et al., 2004) as the research approach used by the
authors to develop the novel approach based on PBL proposed
and tested in this research. Afterwards, it proposes a review of
literature concerning maturity models and their general charac-
teristics, their application in the digital transformation (or Indus-
try 4.0) domain – with a focus on the Acatech maturity model
(Schuh et al., 2017) - and the PBL model with its core principles
used for facilitating the understanding of contextual factors. The
proposed assessment approach, i.e. 360 Digital Maturity Assess-
ment (360DMA), is then presented. Initially, the maturity model
used to frame the collected information is described. The assess-
ment procedure, contribution of this paper, is presented after-
wards, and all the related activities described in detail. Eventu-
ally, the approach is tested in three industrial cases, and the pro-
posed improvement recommendations are discussed in relation to
the maturity stage and the contextual factors that justified them.
The contribution of the proposed model to the extant body of lit-
erature, its supporting function for practitioners, its limitations and
the need for further research are addressed in the conclusion.
2. Research approach
In order to propose a model which is applicable and in order to
be able to evaluate the related benefits, it is usually not enough to
define its content and a structure to frame it, but these have to be
tested (Wendler, 2012). Because of that, a design-oriented research
approach fits with the development of maturity models, which can
be designed and tested as artefacts according to the design sci-
ence research framework proposed by Hevner et al. (2004) and
Wendler (2012). The validation process is generally carried out
through case studies or action research, focusing on qualitative ob-
servations (Wendler, 2012).
According to that, this research is structured around a design
science research framework (Hevner et al., 2004) (Fig. 1) and starts
by the current business needs from the manufacturing industry,
target user of this research, concerning tangible support for oper-
ationally guiding their digital transformation journey. In order to
establish a solid foundation to build on top of for answering busi-
ness needs and to ensure the relevance of the research, a litera-
ture review is performed, identifying the current state of the art
regarding the addressed topic and the related literature gaps. Its
objective is, specifically, to identify the key attributes of the exist-
ing maturity models and, specifically, assessment approaches, their
use in regards to the digital transformation domain and the cur-
rent literature gaps. The proposal of an artefact, consisting of a new
learning-based assessment approach is following. This is built with
the aim of providing new ground to address the initial business
needs. Specifically, the proposed assessment approach, key contri-
bution of this research, is based on the PBL model and supported
by a maturity model used to structure the collected information
and based on an existing maturity model. The contextualization ca-
pabilities of the proposed artefact in terms of assessment outcome
(i.e. context-specific improvement recommendations) are eventu-
ally tested in three cases. A digital maturity assessment is per-
formed following the proposed approach and its outcome for the
three cases, consisting in improvement recommendations, is com-
pared and discussed in relation to the maturity stage of the cases
and to the specific contextual factors that have been identified dur-
ing the assessment process. According to the context-aware system
capability validation for a knowledge-based system performed by
Sánchez-Pi, Carbó, and Molina (2012), the contextualization capa-
bilities of the proposed assessment approach are considered val-
idated if, during the testing of the model in real cases, different
contextual factors (such as the investigation scope or the company
strategic goal) determines the assessment outcome (i.e. improve-
ment recommendations).
 M. Colli, U. Berger and M. Bockholt et al. / Annual Reviews in Control 48 (2019) 165–177
Fig. 1. Design science research framework (Hevner et al., 2004) applied for this research.
This will be provide a foundation for further research concern-
ing the refining of the proposed model, addressing its efficiency
(i.e. reducing the need for resources for performing the assess-
ment) and its effectiveness (i.e. making sure that the provided im-
provement recommendations are not only tailored to the assessed
company but also supports, in fact, its digital transformation).
3. Literature review
3.1. The concept of maturity and its use
The concept of maturity emerged in the quality management
domain in the 1930s (Shewhart, 1931), in order to describe the
development of an entity, from human beings to organizations. A
large variety of maturity models have been designed since then
(Klimko, 2001) and their key characteristic became the ability to
translate this development path in a structured progression of dif-
ferent maturity stages (Crosby & Free, 1979; Nolan, 1979) built on a
cumulative capabilities perspective (Miller & Roth, 1994) and to de-
fine a number of measurement categories to be considered in de-
scribing this evolution (Crosby & Free, 1979). The aim of a maturity
model is, according to that, to describe the anticipated, desired or
typical evolution path (Becker, Knackstedt, & Pöppelbuß, 2009) in
a simplified way (Klimko, 2001). In 1979, Crosby and Free (Crosby
& Free, 1979), with their quality management process maturity grid,
provided a tool to systematically measure the current level of ma-
turity of an entity along its evolution path. In the early 1990s, with
the advent of the Capability Maturity Model Integration (CMMI)
framework, this concept has been translated for the software in-
dustry and a set of best practices for supporting its evolution have
been provided (Wendler, 2012). Since then, multiple studies have
been performed aiming at widening the applicability spectrum of
this concept (Wendler, 2012), in order to support the development
of further industrial fields.
However, the use of maturity models for outlining an evo-
lution path has been criticized as an oversimplification of real-
ity often lacking empirical foundation (De Bruin, Freeze, Kaulka-
rni, & Rosemann, 2005). One of the key critical elements regard-
ing this aspect is that these models generally do not consider the
possibility of multiple maturation paths (Teo & King, 1997), but
provide a generic development receipt. In 2011, Pöppelbuß and
Röglinger (2011) identified a number of general principles for busi-
ness process management maturity models in order to use them
for designing the maturity model (i.e. basic design principles), for
167
supporting companies in identifying their current maturity stage
(i.e. descriptive purpose) and, at a more advanced level, for opera-
tionalizing the progression along their evolution path (i.e. prescrip-
tive purpose). By analyzing extant literature in relation to these de-
sign principles, Röglinger et al. (2012) identified the need for hav-
ing assessment approaches (i.e. assessment or adoption method-
ologies) that make possible to adapt the maturity assessment (i.e.
descriptive level) as well as the related improvement measures (i.e.
prescriptive level) to “organization-specific situational characteris-
tics”.
3.2. Assessing maturity in the digital transformation era
As the current digital transformation towards Industry 4.0 is
considered to be an evolutionary journey (Kagermann, Wahlster, &
Helbig, 2013) across a number of sequential stages characterized by
the cumulative achievement of capabilities, the concept of maturity
is being also translated into this domain. The aim remains to out-
line the related evolution path, identifying the key characteristics
related to its progression in order to provide companies with a tool
to plan their development. Both researchers, research institutions
and companies have published several maturity models within
this domain, part of them with a well-defined assessment ap-
proach. These have been reviewed by Mittal et al. (2018) and dis-
cussed in regards to their applicability in small and medium enter-
prises. Starting from this review, consisting of 15 maturity models,
the authors selected the nine maturity models presenting either
a well-defined assessment approach, either an indication of their
prescriptive outcome (i.e. improvement recommendations). These
have been mapped using, as comparison criteria, their assessment
approaches and prescriptive assessment outcomes (Table 1).
The general aim of these digital maturity models is represented
by their intention to assess the digital maturity stage of a com-
pany (i.e. descriptive purpose) and to recommend, accordingly, im-
provement activities in order to guide the company towards the
next maturity stage (i.e. prescriptive purpose). According to what
has been identified for generic business process management ma-
turity models by Röglinger et al. (2012), there is a lack of ability in
adapting the provided improvement recommendations to the spe-
cific contexts. All the reviewed digital maturity models are oper-
ationalized through a questionnaire and are, therefore, meant to
process structured data. Only two models put a focus on providing
an understanding of the specific company context and operational-
ize it by proposing also, in support of data collection, either com-
168 M. Colli, U. Berger and M. Bockholt et al. / Annual Reviews in Control 48 (2019) 165–177

Table 1
Digital maturity assessment approaches review.

Digital maturity model name Assessment approach Prescriptive assessment outcome

and reference

Schumacher, Erol, and Questionnaire Indication of standard elements to be implemented to be mature

Sihn (2016)
Acatech - Schuh et al. (2017)
Impuls - Lichtblau et al. (2015)
Dreamy - De Carolis, Macchi,
Negri, and Terzi (2017)
Qin, Liu, and Grosvenor (2016)
Geissbauer, Vedso, and
Schrauf (2016)
Rockwell Automation (2014)
Akdil, Ustundag, and
Cevikcan (2018)
Amm - Scremin et al. (2018)
Questionnaire combined with company visits
On-line self-assessment based on a questionnaire
Questionnaire with answers based on the maturity
stages, focused on management aspects
No explicit assessment approach
On-line self-assessment based on a questionnaire,
initially classifying companies based on industry,
region, country and annual revenue
No explicit assessment approach
Questionnaire and index to translate answers in a
specific maturity level
Questionnaire linked to maturity archetypes,
addressing also the company context through an
interview
Proposal of generic archetypes related to the different digital maturity
stages but no explicit indications of activities for enabling maturity
stage transition
Indication of general activities enabling maturity stage transition
No explicit indications for maturity improvement
Indication of general activities enabling maturity stage transition,
focused of technology implementation and automation capabilities
Indication of general activities enabling maturity stage transition,
focused on strategy mapping, pilot approach, needed capabilities
definition, data analytics, digitalization of the enterprise and
ecosystem thinking
Indication of general activities enabling maturity stage transition,
focused on securing and upgrading networks and controls, defining
and organizing data and focusing on analytics and collaboration
Indication of a set of standard principles (e.g. integration, automation,
data sharing, interoperability) and technologies that have to be
fulfilled to be mature
Indication of maturity items to be addressed for enabling maturity
stage transition, but no indication of activities concerning their
integration

pany visits (Schuh et al., 2017) or interviews (Scremin, Armellini,
Brun, Solar-Pelletier, & Beaudry, 2018) and, therefore, address the
processing of unstructured data. However, one of the two models
is not accessible for further investigations (Scremin et al., 2018)
and the other one (Schuh et al., 2017) is not describing in de-
tail its assessment approach. This is limiting the support, for com-
panies and assessment parties, concerning the understanding of
the “organization-specific situational characteristics”, mentioned by
Röglinger et al. (2012), needed for connecting the assessment out-
come to the nature of the assessed company and of its core busi-
ness processes. This confirms, also within the digital transforma-
tion domain, the gap identified by Röglinger et al. (2012) for
generic business process management maturity models and sup-
ports the argumentation of Mittal et al. (2018) which, in their
critical review concerning maturity models for Industry 4.0, iden-
tify the need for being able to conceive digital transformation
roadmaps that are tailored to company-specific needs.
3.3. The Acatech maturity model
The Acatech maturity model (Schuh et al., 2017) has been iden-
tified, among the reviewed ones, as the only one (i.e. the model
proposed by Scremin et al., 2018, is not accessible for further in-
vestigations) built for mapping company information consisting
also of unstructured data (i.e. collected from company visits and
not only through pre-defined questionnaires), and therefore being
able to adapt to different context-specific information. The Acat-
ech model, in accordance with the maturity model needs iden-
tified in extant academic literature, proposes a number of well-
defined sequential maturity stages built on cumulative capabilities
(Table 2) (e.g. Crosby & Free, 1979; Miller & Roth, 1994; Nolan,
1979) and a spectrum of dimensions to be considered within the
company during the assessment process (Table 3) (e.g. Crosby &
Free, 1979). The maturity model satisfies all the basic design prin-
ciples for designing a maturity model identified by Pöppelbuß and
Röglinger (2011) by clarifying the application domain, focused on
data and connectivity aspects, by defining the constructs related to
the maturation process, the maturity levels and the addressed di-
mensions and by clarifying how the maturity model is related to
these constructs.
3.4. Problem-based learning model
The PBL model has been introduced within the medical do-
main (Schmidt, 1983) to facilitate the learning process, answering
the need for contrasting the pre-defined diagnosis based on the
first detected symptoms. According to PBL, every case is unique
and has to be addressed based on its specific contextual aspects,
through active interaction with the case environment (Savery &
Duffy, 1996). Furthermore, the concept of proximal zone of devel-
opment (Vygotsky, 1978), theoretical foundation of the PBL model
(Harland, 2003), argues that the development process is enabled
by the help of more capable peers and operationalized through
learning and collaborative problem solving (Harland, 2003). In fact,

Table 2
Acatech maturity stages (Schuh et al., 2017).

Maturity stages Computerization Connectivity Visibility Transparency Predictive Adaptability

progression capability

Capabilities Data generation Data transmission Data availability Data correlation and Data analysis to Autonomous data
from assets and across different regarding events interpretation to simulate future driven decision
isolated IT assets connected to support understand causes of scenarios and making and
deployment through IT decision making events and support evaluate their continuous
deployment complex decision occurrence adaptation to the
making probability business
environment changes
 M. Colli, U. Berger and M. Bockholt et al. / Annual Reviews in Control 48 (2019) 165–177 169

Table 3
Acatech maturity dimensions (Schuh et al., 2017).

Maturity dimension Organizational structure Resources Information system Culture

Related aspects Organizational structure,
strategy, collaborations
within the company and
with external partners, etc.
Available assets, tangible and
physical resources (e.g.
humans, equipment,
materials)
Available IS infrastructure
where information is
provided by people or assets
for supporting decision
making
Corporate culture, behavioral
aspects, employees mindset
and skills

the presence of some kind of tutoring which guides the learn-
ing process is critical to the effectiveness of the learning process
(Savery, 2006).
The PBL core principles (Barge, 2010), necessary for enabling
this dialectic process and the consequent contextualization capa-
bilities, are:
• Problem orientation: problems related to the addressed environ-
ment are the basis for the learning process
• Project organization: the performed activities represent both the
mean through which the problems are addressed and the foun-
dation for active learning
• Integration of theory and practice: theories and practical knowl-
edge are interrelated in the performed activities
• Participant direction: the investigated problem is clearly defined
as well as the success criteria
• Team-based approach: the problem investigation, as well as the
consequent activities, are conducted in groups
• Collaboration and feedback: collaboration, critique, feedback and
reflection are fundamental elements to improve the outcome of
the performed activities
2. Basic: digital data are generated (e.g. machines on the produc-
tion floor generate digital data related to their process) col-
lected and handled locally
3. Transparent: data is available across the organization according
to value streams needs (e.g. alert data from the equipment are
collected and transmitted to the service department)
4. Aware: data, structured and transmitted according to recog-
nized standards, is analyzed to capture valuable information in
order to understand the business insights (e.g. proactive activi-
ties identification by crossing error data, product number, ma-
chine downtime, etc.)
5. Autonomous: decision making is performed autonomously
based on automatically synchronized data, structured and
transmitted following a universal standard, from the organi-
zation and its direct customers and suppliers (e.g. logistics
scheduling is automatically performed based on production
state, customer orders and location, traffic condition etc.)
6. Integrated: decision making is performed autonomously based
on automatically synchronized data from the whole organiza-
tion’s network (e.g. suppliers’ suppliers and customers’ cus-
tomers)

Dimension wise, a “value creation” dimension has been added

4. Model development
4.1. The maturity model
In order to frame the information collection performed during
the digital maturity assessment, a maturity model has to be de-
fined and, according to Section 3.3, the Acatech maturity model
(Schuh et al., 2017) has been considered. However, the authors per-
ceived the need for translating the Acatech maturity model in a
more tangible one for the companies that have to be addressed
during the testing of the proposed artefact (i.e. the assessment ap-
proach). The maturity stages and dimension of the Acatech model
have been translated, therefore, into a novel model.
In regards to maturity stages, an additional stage, “none”, has
been added to the model due to the need for a “level zero” stage
(identified by Mittal et al., 2018), in order to make possible to po-
sition on the evolution path companies that did not start the dig-
ital transformation process yet. The “computerization” stage has
been renamed as “basic” and the “connectivity”, “visibility” and
“transparency” stages have been grouped together in a “transpar-
ent” stage, as it has been found challenging to intuitively perceive
the difference between them. An “aware” is substituting the “pre-
dictive capability” one in order to provide a more generic label
to the related capabilities, i.e. analytics and interpretation. Because
of the same reason, the “adaptability” stage has been renamed as
“autonomous”. An “integrated” stage has been added in order to
include, as a final goal, the Industry 4.0 vision consisting in an
integrated supply network (Jeschke, Brecher, Meisen, Özdemir, &
Eschert, 2017). The obtained maturity stages progression consists,
therefore, of six sequential stages:
1. None: no digital awareness, idea or plan nor presence of digital
data within the organization (e.g. everything is registered on
paper or not registered)
to the initial four due to the need for clarifying how the assessed
company is generating value out of data, in order to have a more
clear understanding of the context; the other dimensions have
been renamed. They consist of:
• Governance: organizational level (e.g. strategy and plan, organi-
zational structure, management of projects, resource allocation,
digital awareness, engagement on different hierarchical levels).
• Technology: assets that make possible to generate, process and
use digital data (e.g. business intelligence tool, cloud computing
platform, MES, ERP, augmented and virtual reality tools)
• Connectivity: infrastructural elements needed for data transmis-
sion inside and outside the organization (e.g. data sharing ca-
pabilities, IT security, standard data structuring or data trans-
mission architectures)
• Value creation: models used to generate and capture value from
data (e.g. pay-per-use business model, take-back program, data
usage for orders or maintenance forecasting)
• Competences: mind-set and skills (internally or based on exter-
nal partnerships) needed for addressing the digital transforma-
tion process (e.g. digital competences, training culture, learning
culture)
In order to provide the assessment team with a well-defined
and tangible reference for evaluating the digital maturity of the as-
sessed companies in regards to their different dimensions, these
have been related to the proposed maturity stages and a clear
reference frame for the maturity assessment has been defined
(Table 4).
4.2. The assessment approach
The assessment approach, artefact and contribution of this re-
search, is based on the PBL model and on its core principles and
170 M. Colli, U. Berger and M. Bockholt et al. / Annual Reviews in Control 48 (2019) 165–177

Table 4
Reference frame for the maturity assessment.

Governance Technology Connectivity Value creation Competences

None No awareness regarding No presence of assets that There is no IT No value is created nor There are no competences
the digital generate digital data infrastructure captured out of data related to data handling
transformation within the organization
Basic There is a willingness Digital processes are in The IT infrastructure is Data is collected and There are competences
towards the digital place and operative as developed in separate available for eventual related to data handling
transformation from assets generate digital modules that address needs and these are involved
the management side data different tasks and when needed
cannot communicate
with each other
Transparent There is a digitalization There are interfaces that The IT infrastructure is not Data is collected and Training and learning
plan from the make possible for who standardised but all the shared according to value activities are proposed to
management side needs it to access data different modules can streams needs facilitate knowledge
and visualize it communicate with each dissemination
other
Aware There is a clear There are tools that make The IT infrastructure is not Data is analyzed to capture Solid partnerships are
digitalization agenda possible to process data fully integrated but is valuable information in established in order to
shared at all correlating and analysing based on a number of order to understand facilitate access to
hierarchical levels it and communicate recognised standards and business insights further knowledge.
with specific results to the user when new modules have Training and learning
objectives to be developed, this is activities are part of the
done accordingly company culture
Autonomous Digital development is There are assets or tools The IT infrastructure is Decisions are performed Both top down and bottom
a well-established that can act based on a single autonomously based on up co-creation culture is
company practice at autonomously according standard and new automatically supported and training
all hierarchical levels to information received modules are developed synchronized data and learning as well as IT
after an analytic process accordingly enabling security are a solid part
interoperability of it
Integrated Digital development is The assets deployed across The IT infrastructure in the Decisions are performed Both top down and bottom
a well-established the supply chain can whole supply chain is autonomously based on up co-creation culture is
practice at all interact together and based on standards that automatically a common practice and
hierarchical levels reconfigure themselves to allows plug and play synchronized data from the IT security culture is
within the whole optimize performance inter-organization the company’s network absorbed by all the
company’s network communication, enabling players along the supply
interoperability chain

it is meant to act as a dialog tool, enabling a dialectic process be-

tween a company and the assessment party. This is done with the
objective to gradually focus the scope of the assessment and to fa-
cilitate the identification of current digital transformation-related
capabilities as well as of contextual factors conditioning the need
for the development of the addressed company, as strategic goals
or specific limitations related to the business domain. The inten-
tion is to be able to facilitate the formulation of case-specific im-
provement recommendations as an assessment outcome. Due to
focus on enabling a dialog, the assessment paradigm, therefore,
shifts from the currently used expert model, where a diagnosis is
completely outsourced to external experts (Schein, 2009) to the ex- Fig. 2. 360DMA assessment approach.
ternal helper model, which bases the diagnostic process, facilitated
by external experts, on the interaction with the assessed party
At first, through the creation of awareness, the assessing party is
(Schein, 1995, 2008). The involvement of the company in its own
setting the scene for the upcoming investigation, integrating the-
assessment process not only enables better learning of the case but
ory and practice according to the PBL model (Barge, 2010). An
also the acquisition of more valid data related to it (Lewin, 1997).
overview of digital transformation-related concepts and of what is
In order to enable a team-based approach and a continuous collabo-
included in the industry 4.0 agenda in terms of technologies (e.g.
ration and feedback, core principles of the PBL model (Barge, 2010),
cloud computing), implications (e.g. IT security) and use cases (e.g.
the operationalization of the maturity assessment involves, from
predictive maintenance or autonomous guided vehicles) is pro-
the assessing side, the presence of a team composed by field ex-
vided. This activity can be performed through study visits, work-
perts for the areas that have to be investigated, a mediator that
shops with external experts and demonstrations in imaginary In-
directs the activities according to the defined approach and a rap-
dustry 4.0 factories (e.g. Madsen & Møller, 2017). The aim is to pro-
porteur that collects the information along the way.
vide inspiration regarding the digital transformation possibilities
The proposed assessment approach consists of five sequential
and implications, identifying the levers that can initiate the dialog.
steps (Fig. 2) which, according to the PBL’s project organization
The assessed company, on his side, provides the assessment team
principle, aim to address the problem, measuring the level of digi-
with an overview of the business the company is operating in, the
tal maturity of the assessed company, and to enable active learning
company itself along with its organizational structure, its strate-
concerning company-specific situational characteristics at the same
gic goals, core processes, key performance indicators and current
time.
performance levels. These elements provide the assessment team
 M. Colli, U. Berger and M. Bockholt et al. / Annual Reviews in Control 48 (2019) 165–177
with a foundation for contextualizing the digital maturity assess-
ment that is going to be performed.
A definition of scope is then performed, with the aim of clarify-
ing the investigated problem and the related success criteria (e.g.
10% cost reduction through efficiency improvement), identifying a
unit of analysis for the learning process (e.g. a production line, a
department, a whole factory or a part of the supply chain). This
step is addressing the participant direction and problem orientation
PBL model core principles (Barge, 2010). A collaborative problem-
scoping is operationalized through a presentation, from the com-
pany side, of their current and future strategic focus and develop-
ment perspectives and an open discussion, involving all the stake-
holders (i.e. cross-organizational) and facilitated by the assessment
team, aiming at highlighting all the emerged key challenges and
limitations concerning them. These are clustered according to the
area of the company they are related to and clusters are eventu-
ally discussed in order to identify the most critical one (or ones),
leading to the arbitrary definition of a scope for the assessment
process.
Data collection is then performed, aiming at gathering infor-
mation related to the company within the defined assessment
scope. This step is strongly anchored to the PBL model team-
based approach (Barge, 2010) as it involves the whole assessing
team. Through the analysis of information material provided by the
company and according to the defined assessment scope, a “self-
assessment” questionnaire consisting of 25 multiple-answer ques-
tions addressing the five maturity dimensions is submitted to rel-
evant stakeholders (e.g. managers of functional areas involved in
the digital transformation within the assessment scope). The goal
of the questionnaire is to identify critical areas in order to further
focus the assessment process and provide some ground to start a
dialog with the company representatives, and not to provide a col-
lection of data for performing the maturity assessment. Hence, the
proposed answers are not necessarily directly related to the def-
inition of the different maturity levels, but they are still follow-
ing their progression. One example of a question, related to the
competences dimension, and of its possible answers is: “Are for-
mal training sessions (presentations and/or workshops) related to the
introduction of new technologies existing?”
1. No, there is no training regarding new technologies
2. Yes, there is little training at the point of demand (e.g. the sup-
plier of new production equipment introduces it to the com-
pany internal operators)
3. Yes, there is peer-to-peer training between experts and opera-
tors throughout the value chain
4. Yes, we are buying service packages in connection to new tech-
nologies to receive continuous training in order to stay up-to-
date
5. Yes, we have formulated a comprehensive strategy for training
and competence development in regards to new technologies
6. Unknown
By analyzing the registered answers and isolating the low-
grade, unknown and mismatching ones (i.e. different stakeholders
answer differently), the assessment team identifies the critical as-
pects to be discussed vis-à-vis with field experts from the com-
pany. The utility of the self-assessment questionnaire is to provide
the assessment team with a support for further focusing the in-
terview process, maximizing its effectiveness by enabling a more
in-depth investigation concerning the critical aspects. An expert in-
terview workshop is, therefore, prepared and executed accordingly.
The expert interview workshop is performed at the assessed com-
pany to facilitate the availability of the needed company represen-
tatives and it is divided in a number of separate sessions that cover
the main critical areas (three or four, e.g. IT, logistics, etc.) that
emerged from the “self-assessment” questionnaire. The interview
171
is coordinated by a mediator that defines the topic to address and
facilitates the discussion in the available time. The experts from
the assessment team question company representatives in 30 min
sessions with the aim of obtaining an in-depth understanding of
these emerging critical areas and of the reasons why they are crit-
ical for the company. A rapporteur is taking notes in the meantime,
in order to collect the information that emerges from the discus-
sion. Furthermore, the sessions are recorded in to support the rap-
porteur in processing the notes afterwards.
The following evaluation and solution selection performed by the
whole assessment party, sticking to the team-based approach pro-
posed by the PBL model (Barge, 2010), represents the core of the
assessment process. Operationally, it firstly consists of mapping the
information collected during the previous step within the defined
digital maturity model. Collected data – consisting of statements
from the interviewed company representatives - are clustered ac-
cording to their relationship with the different maturity dimen-
sions (e.g. information concerning the IT infrastructure are grouped
under the “connectivity” label, according to the definitions pre-
sented in the maturity model). Afterwards, this data is related, for
each dimension cluster, to the maturity stages definition accord-
ing to Table 4. A maturity stage is then identified for each dimen-
sion by the assessing team by identifying where the majority of the
statements concerning a dimension are located in terms of digital
maturity. Secondly, the statements that are related to lower ma-
turity stages compared to the majority of the others are consid-
ered as “key gaps” for the improvement of that specific dimension.
Thirdly, these “key gaps” are discussed by the assessment team in
order to identify eventual causality relations between them. These
are then related to the contextual aspects emerged during the cre-
ation of awareness and the definition of scope steps, specific initia-
tives for addressing the key limitations for the company transfor-
mation (i.e. improvement recommendations) are formulated. The
assessment team knowledge and experience plays a key role in the
latter part of this evaluation and solution selection step.
Eventually, during a debriefing session and according to the col-
laboration and feedback PBL model principle (Barge, 2010), the as-
sessment outcome is presented plotted on a spider graph (Figs. 3–
5) and discussed with the company representatives along with the
identified “key gaps” for the company transformation. Afterwards,
the recommended short, medium and long-term improvement ac-
tivities are recommended through the presentation of project pro-
posals from the assessment team side, specifying the reason for
the project, a description of the activities and an objective.
The assessment process is intended to be performed iteratively
in order to be able to adjust the transformation direction, after
the execution of recommended improvement projects, according to
company goals changes and newly available digital technologies.
5. Model testing
The testing of the proposed 360DMA consisted in the applica-
tion of the artefact in three diverse industrial cases, in order to
assess their digital maturity and to suggest improvement activities
accordingly. The involved companies are all large Danish manufac-
turing companies interested in gaining a deeper understanding of
the Industry 4.0 agenda and in being guided towards the definition
of a roadmap for approaching the digital transformation. However,
these companies all differ in terms of context. From a research
point of view, the testing process aims at validating the contex-
tualization capabilities of the proposed approach according to the
validation criteria stated in Section 2. This is done by studying how
the proposed approach is facilitating the identification of company
contextual factors - such as strategic goals, key performance in-
dicators or existing criticalities concerning the core business pro-
172 M. Colli, U. Berger and M. Bockholt et al. / Annual Reviews in Control 48 (2019) 165–177
cesses - and leading towards specific improvement recommenda-
tions accordingly.
5.1. Case one
The first case consists of a large Danish manufacturing company
that operates worldwide. The company produces and supplies me-
chanical parts in multiple production facilities all over the world.
The assessment process has been performed by a team of four
interdisciplinary experts in the robotics and automation, IT and
manufacturing technology domains. It started with the creation of
awareness related to the industry 4.0 agenda. During a first work-
shop, through a presentation, the experts provided an overview of
new technologies, use cases and research projects to company rep-
resentatives and stakeholders directly involved in the digital trans-
formation of the organization. From their side, company represen-
tatives presented the company business, its key performance indi-
cators and strategic goals. The company target that emerged con-
cerns the increase of operations efficiency in the Danish production
facility to increase the annual turnover of 10% by reducing the im-
pact of labor on product costs. The aim is to reduce the revenue
gap between this plant and other company plants located in the
Far East. The company intends to achieve it through the enabling
of production data visualization and analytics and to the intro-
duction of autonomous internal logistics for production efficiency
improvement. Hence, the scope of the investigation, in order to
move towards this vision, has been framed around the production
floor.
A multiple answer questionnaire has been provided to the rele-
vant company stakeholders (including the COO/executive vice pres-
ident – project sponsor - the project manager for the digital-
ization agenda and four project team members: a corporate se-
nior manager, an operations controller and the corporate senior
director from the IT department and the supply chain director).
Through the analysis of questionnaire answers, critical points have
been identified as an undefined digitalization strategy (in regards
to both resources and plan), unclear data accessibility and usage,
a non-standardized IT infrastructure, low training practices, un-
clear digital capabilities, no benchmarking in relation to competi-
tors, low digital devices in production and remote work possibil-
ities. The data collection workshop has been planned, involving
the same company stakeholders and with a focus on these crit-
ical points. These have been addressed in four sessions: customer
ordering and production setup and execution, procurement and in-
ternal logistics, product development, IT. By discussing them, the
non-standardized IT architecture and the unclear data accessibility
and usage have been identified as particularly critical points in re-
lation to the company intentions for achieving its strategic goals.
The assessment process was finalized by mapping the collected in-
formation within the maturity model according to the definitions
of the different maturity stages.
The company has been assessed as aiming for the transpar-
ent maturity stage. The key limitations for further transformation
that need to be addressed concern the connectivity area (Fig. 3).
They consist of the company tendency of developing tailor-made
branches of the IT infrastructure anytime they are needed and in
the consequent lack of data structure when data is generated and
collected. These aspects limit the capability of collecting and shar-
ing data according to value stream needs, i.e. transparent stage, a
necessary backbone for enabling the “production data visualization
and analytics” and “autonomous internal logistics” the company is
addressing. According to what has been learned regarding the cur-
rent limitations for the digital transformation of the case company,
one recommendation was to consider the introduction of a stan-
dard in regards to the IT infrastructure in order to facilitate the
integration of new elements. As a follow-up, it has been suggested
Fig. 3. Digital stage in relation to the five digital dimensions (case 1).
to standardize the way data are structured in order to enable data
analytics and, eventually, sharing and visualization.
5.2. Case two
The second case is a large Danish company operating world-
wide and providing commodities. The company takes care of prod-
uct design, sales and supply chain coordination, and controls part
of the manufacturing processes.
The assessment process has been performed by two experts
in the organizational and manufacturing technology domains and
by a group of five research assistants. It started with the creation
of awareness following the same approach used in case one. The
company representatives who are directly involved in the digital
transformation process (i.e. innovation and logistics managers) pre-
sented the company business, its key performance indicators and
strategic goals. The paramount company goal was the improve-
ment of the supply chain responsiveness - through the rethink-
ing of internal business processes or the improvement of their ef-
ficiency - as the key selling point of the organization is delivery
time. A secondary goal was to reduce overproduction, a conse-
quence of the current need for stock in order to cope with deliv-
ery time requirements. The investigation, according to company in-
terests and know-how regarding potential efficiency improvement
margins, has been scoped around supply chain management activi-
ties and looking, specifically, at order management and production
planning.
A multiple answer questionnaire has been provided to the rel-
evant company stakeholders from the different functional areas of
the organization (i.e. innovation and logistics managers, sales sup-
porter, purchaser and IT project manager). Through the analysis
of questionnaire answers, critical points have been identified in
the lack of strategy and resource allocation concerning the digi-
tal transformation, the lack collection and usage of data and the
lack of information sharing with customers and with suppliers. A
data collection workshop took place afterwards, involving the same
company stakeholders and with a focus on these critical points.
These have been addressed in a single session to enable an open
discussion where each manager could provide its point of view
on the emerged problems. It has been identified that, despite an
agenda related to the digital transformation was missing, internal
resources to start-up the transition process have been allocated.
The use of data as an asset was currently limited to the forecast-
ing process, based on the analysis of historical sales data, in order
to identify trends and on the monitoring of orders shipping. How-
 M. Colli, U. Berger and M. Bockholt et al. / Annual Reviews in Control 48 (2019) 165–177
Fig. 4. Digital maturity stage in relation to the five digital dimensions (case 2).
ever, available data from the production site is not analyzed to ob-
tain further business insights for supporting the planning processes
(e.g. to optimize production scheduling and material requirement
planning). In regards to information sharing, on the customer side,
an online portal is in place for digitalizing the ordering process.
However, most of the customers are still using e-mails or faxes,
causing the need for time and resources on the company side for
manually process customer orders. On the supplier side, an en-
terprise resource planning (ERP) system is used to interconnect
the company with its partially controlled manufacturing supplier.
However, the scheduling process to provide the supplier with a set
of orders to manufacture is manually performed, delivered to the
production planning office on the supplier side which, after man-
ually defining an internal production schedule, shares it with the
production floor. The assessment process was finalized by mapping
collected information within the maturity model.
The company has been assessed across the transparent maturity
level (Fig. 4), with key limitations for further transformation con-
cerning the connectivity and the value creation dimensions. These
have been identified in the lack of infrastructure for the informa-
tion sharing on the supplier side and in the need for understand-
ing how to obtain and capitalize value related to available data
respectively. According to that and to the need for competitive-
ness increase regarding supply chain responsiveness, an investiga-
tion regarding the development of an integrated scheduling system
that has been proposed. This consisted of automating the schedul-
ing process, interconnecting the orders database and the company
ERP system with the related scheduling engine and the supplier’s
production floor.
5.3. Case three
The third case consists of a large manufacturing company based
in Denmark and operating worldwide. The company is produc-
ing in multiple plants, located all over the world, consumer-ready
products that are then provided to a number of distributors. The
assessment has been performed in collaboration with a business
unit in one of the plants located in Denmark.
The assessment process has been performed by seven interdis-
ciplinary experts in the robotics and automation, organizational, IT
and manufacturing technology domains. A mediator to manage the
assessment process and a rapporteur that could focus the atten-
tion on data collection has been appointed. It started with a work-
shop aiming for the creation of awareness related to the digital
transformation, provided by the assessment team, and to the or-
173
Fig. 5. Digital maturity stage in relation to the five digital dimensions (case 3).
ganization, provided by company representatives. At first, the as-
sessment team provided an overview of the Industry 4.0 agenda
discussing its enabling technologies and implications. Afterwards,
through the use of the Smart Production Laboratory at Aalborg
University, a showcase of new technologies applied to specific use
cases has been proposed to 16 company representatives (at a man-
agement level). Eventually, the corporate vice president and senior
director of the involved business unit presented the company busi-
ness and the strategic goal. This consisted in reducing the ramp-up
time related to the introduction of new products or equipment, as
the addressed factory has been designed as innovation factory for
the whole company and, nevertheless, has to cope with the same
performance targets of the other company plants. Through a round
table session, critical points concerning it have been highlighted by
each one of the 16 managers and discussed during a plenary ses-
sion afterwards. According to them, the pre-assembly department
has been defined as a scope for the assessment process.
A multiple answer questionnaire has been provided to company
stakeholders involved in the digital transformation agenda of the
defined business unit (including the corporate vice president, the
project manager for the digitalization agenda, the head of quality,
two directors and two department managers). Through the anal-
ysis of questionnaire answers, critical points have been identified
in the absence of a defined digital transformation agenda, in the
lack of data usage and sharing and in the need for a training cul-
ture. The data collection workshop has been divided into four ses-
sions in order to further discuss these topics in the different busi-
ness areas of the pre-assembly department. Each session consisted
in a 20 min presentation from the responsible of the addressed
business area followed by a 40 min discussion addressing material
and/or information flow, the process of introducing new projects,
key performance indicators, criticalities and future plans. The main
critical aspect consists in the lack of innovation agility, a key ele-
ment of the analyzed business unit, due to the rigid and complex
protocol for introducing and running new projects, due to inter-
nal safety reasons. Furthermore, the need for establishing internal
competences for supporting the identification of technology and
connectivity needs emergeing from the discussion. The assessment
process was finalized by mapping collected information within the
maturity model according to the definitions of the different matu-
rity stages.
The organization has been assessed as fully transparent (Fig. 5)
and governance has been identified as the main driver of the trans-
formation, due to the willingness of the management for support-
174 M. Colli, U. Berger and M. Bockholt et al. / Annual Reviews in Control 48 (2019) 165–177

ing digital transformation activities and to the available resources.

However, the key limitation for further transforming, being related
to the need for a more agile project structure, is part of the gov-
ernance dimension too. In order to increase the digital maturity,

technologies and processes in

Improvement recommendations
it was recommended, therefore, to establish an innovation fac-

culture (e.g. adopt SCRUM)

scheduling linking it to the
order database in the ERP

a protected environment
tory based on agile principles for enabling shorter innovation cy-

Adopt a standard for the IT

and establish a learning

Introduce an agile project

factory for testing new

cles, encouraging an innovation culture through the use of inno-

Automating production
secondarily, for data
infrastructure (and,
vation methodologies (e.g. design thinking) and providing an envi-
ronment for building the needed internal competences. Following

structuring)
these guidelines, it has been suggested to run pilots regarding, in
particular, data analysis, visualization and sharing.

system
5.4. Model validation

Rigid and inflexible protocol for

The proposed assessment approach is analyzed in regards to

caused by manual processing

structuring, its accessibility

and a production schedule

infrastructure, limiting the

innovation projects due to

its contextualization capabilities by comparing its application and

monitoring or automation
and consequent usage for

Delay between order intake

and unavailability of data

standardization of data
the assessment outcome for the three cases where it has been
tested (summarized in Table 5). To cope with the validation cri-

Non-standardized IT
teria stated in Section 2, the three cases are mapped in regards

safety reasons
supporting it
to the main contextual factors that emerged during the assess-
ment processes and conditioned the direction of the assessment

Key gap
and to the outcome of the assessment process, consisting of the

tasks
critical points defining the digital maturity stage of the company
(here not mapped as the same for all the three cases), the key gap
acting as the root limitation for it and of the improvement recom-

Lack of innovation agility and

support a currently manual

of competences concerning
mendations to deal with it in the company context. The mapped

Low use of available data to

structuring, unclear data

new digital technologies

contextual factors concern the company key performance indica-

accessibility and usage

non-standardized data

and slow scheduling

tors (KPIs), strategic goals and investigation scope.

Non-standardized IT
The three cases have all been assessed at a transparent maturity

infrastructure,
stage. However, due to the emerged contextual factors, different
Critical points

for the three cases and which conditioned the focus of the assess-

processes
ment, different improvement recommendations related to differ-
ent dimensions (i.e. connectivity for case 1 and 2 and governance
for case 3) have been provided as an outcome to all the three
cases. Furthermore, these improvement recommendations are de-

area, order management and

Factory level, production floor,

Factory level, production floor,

rived from these factors. In case 1 the absence of the recom-

pre-assembly processes in
Supply chain level, planning
mended standardized IT infrastructure represents the root cause
manufacturing processes
(automation of internal

production scheduling

the innovation factory

logistics, performance

of the current limitation of data accessibility and usage. These are

needed for automating internal logistics and monitor production
Investigation scope

performance, company activities for addressing manual labor re-

monitoring)

duction and manufacturing efficiency improvement (the company

processes

strategic goals). In case 2 the manual production scheduling pro-

cess has a relevant weight on the total lead time, a key perfor-
mance indicator of the company. Due to the availability of an ERP
system which is including the order database, this could be auto-
mated. In case 3 the absence of an agile project culture and of a
manufacturing processes’

Supply chain management

efficiency improvement

efficiency improvement
Manual labor reduction,

protected environment for safely testing new products or equip-

processes’ efficiency

Innovation processes’

ment delays their introduction on the production floor, where ev-

ery test has to follow strict procedures. In accordance to the val-
improvement
Strategic goals

idation criteria stated in Section 2, the presence of a different

outcome (i.e. improvement recommendations) determined by the
identified contextual factors, in cases at the same digital matu-
rity stage, is validating the contextualization capabilities of the
360DMA. This is translated into the ability of the proposed ap-
Summary of the 360DMA testing.

proach in supporting who is performing the assessment in identi-

new products or
Ramp-up time for
responsiveness

fying improvement recommendations that are derived, other than

from the maturity stage, from the context (e.g. company strategic
Supply chain

equipment
Product cost

goals, KPIs and investigation scope).

KPIs

6. Discussion
Table 5

Case

The proposed digital maturity assessment approach is, by

1

facilitating the identification of contextual factors and of their

implications in the digital transformation of an organization,
supporting the identification of context-specific improvement
recommendations which can drive the digital transformation of
 M. Colli, U. Berger and M. Bockholt et al. / Annual Reviews in Control 48 (2019) 165–177
an organization. By doing so, it addresses the need identified
by Röglinger et al. (2012) first and by Mittal et al. (2018) af-
terwards (specifically in regards to the current industrial
digital transformation) for novel maturity assessment ap-
proaches which can adapt their outcome to organization-specific
characteristics.
The enabling of a dialog with the assessed companies has been
the key learning element for identifying the contextual factors that
played a role in defining a tailored digital transformation roadmap
consisting of specific improvement recommendations. The use of
the PBL model core principles (Barge, 2010) as guidelines for de-
veloping an assessment approach helped in structuring a compre-
hensive learning process, which led to the identification of con-
textual factors that directly conditioned the provided improvement
recommendations. These are the key performance indicators, the
strategic goals and the scope of the development (and, therefore,
defined for the assessment process). The identification of the ma-
turity stage for each digital dimension – supported by the related
reference frame (Table 4) - acted as a tool, for the assessment
team, to identify and communicate to the assessed company the
main weak points. It is, however, the qualitative data collected dur-
ing the interviews – including the information about the identified
contextual factors - that made possible for the assessment team
to formulate the improvement recommendations to address the
weak points. The developed approach, through its dialectic pro-
cess, also caused the emerging of a number of unexpected in-
sights that, eventually, shaped – and filtered - the formulation of
these improvement recommendations. Even though all cases have
been assessed across a transparent maturity stage, the insights con-
cerning the identified contextual factors and learned through the
enabled dialectic process led to the formulation of different im-
provement recommendations. In particular, these have been for-
mulated by linking these contextual factors, the assessed digital
maturity stage and the related critical points and key gap led to
its definition (Table 5) validating, according to the criteria stated
in Section 2, the contextualization capabilities related to the pro-
posed approach.
In particular, four key elements emerged as facilitators for this
dialectic process. Firstly, the creation of awareness through a tan-
gible presentation of solutions from the Industry 4.0 agenda ap-
plied in an industrial environment (i.e. use of the Smart Produc-
tion Lab at Aalborg University, showcase of a number of industrial
demonstrators) enabled company representatives to openly reflect
on possible application of the presented solutions in their specific
use case. Secondly, the organization of a data collection workshop
divided in a number of sessions according to the domains that
have to be discussed, but clearly defining a key responsible for
each session according to the domain of expertise, helped in pre-
venting biases related to the company hierarchy. Thirdly, the avail-
ability of an assessing team that covered a wide spectrum of do-
mains of expertise enabled the provision of comprehensively well-
defined improvement recommendations. Lastly, the institution of
the mediator and the rapporteur roles helped in managing the as-
sessment process by directing questions in order to cover all the
critical areas and in collecting data continuously and consistently
along it. In general, the assessment activity led the three involved
companies to start one or more pilot projects to cover the pro-
posed improvement recommendations. The performance of the as-
sessment by an external team and following an approach based
on the PBL core principles helped to generate a “situational inter-
est” (Schmidt et al., 2011) and a sense of urgency that led to the
legitimization of management actions towards the digital transfor-
mation facilitating, therefore, the change. Furthermore, the effect
of enabling active learning emerged in the form of an aroused in-
terest regarding a more in-depth understanding of the topic (i.e.
175
planning a digital twin workshop in case 2 and getting in touch
with specialized consultants in case 3).
However, some challenges related to the developed 360DMA
emerged during the testing process. It has been pointed out by
the involved companies the need for providing operational indi-
cations concerning how to implement the proposed improvement
recommendations (case 2) and for a follow-up meeting at the be-
ginning of the project execution, when these are operationalized
by the company (case 3). Furthermore, there is a need for a large
team of experts to perform the assessment process, especially for
translating the collected and mapped data into improvement rec-
ommendations. For instance, in case 3 governance was the most
mature dimension. However, the main limitation of maturity in-
crease and operational improvement was related to governance as
well. This shows how the dialectic process helped to identify un-
expected elements but, at the same time, how this model relies
upon a qualitative analysis of the collected information and to the
expertise of the assessment team. This is limiting the replicability
of the outcome, especially compared to most of the existing digi-
tal maturity assessment approaches which are based on standard-
ized questionnaires and have a structured measurement approach
and corresponding outcomes. In addition to that, the need for a
qualitative data analysis implies a loss in terms of efficiency of the
assessment process and, together with the need for a large team
of experts, limits the usability of the 360DMA to large companies
only, as the implied resources are, most likely, not be available in
small and medium enterprises.
7. Conclusion
This research is addressing the need for contextualization re-
lated to maturity models in order to guide companies in their evo-
lution process (Röglinger et al., 2012). Specifically, due to the cur-
rent industrial scenario, it focuses on the digital transformation do-
main and on the related Industry 4.0 agenda, whose integration is
supported by generic improvement recommendations built on top
of information collected through questionnaires (see Section 3.2)
and which are not, therefore, reflecting the uniqueness of individ-
ual companies.
The contribution of this paper consists in answering this need
for assisting diverse companies in their digital transformation ac-
cording to their specific contextual factors. This aspect was already
considered by Schuh et al. (2017) in the Acatech model; how-
ever, it is not clearly stated how to operationally address it dur-
ing the assessment process, other than by performing company
visits. This research is proposing a novel digital maturity assess-
ment approach, built on top of the PBL model core principles (see
Section 3.4), which enables the contextualization of the outcome
of the digital maturity assessment (i.e. the improvement recom-
mendations) by addressing it as a learning process. The approach,
developed and tested according to a design science research frame-
work (Hevner et al., 2004), consists in a structured learning pro-
cess based on establishing a dialectic between the assessing team
and the assessed company. This facilitates the understanding of
the contextual factors that condition the evolution of the com-
pany, providing a foundation for deriving context-specific improve-
ment recommendations. Hence, the proposed assessment approach
based on PBL, being validated in regards to its contextualization
capabilities according to the defined criteria, answers the research
question as it leads a digital maturity assessment towards the for-
mulation of context-specific improvement recommendations. This
procedural steps makes it possible to assist practitioners from dif-
ferent organizations in operationalizing the digital transformation
and researchers for studying this transition in diverse environ-
ments.
176 M. Colli, U. Berger and M. Bockholt et al. / Annual Reviews in Control 48 (2019) 165–177
As the proposed assessment approach has been designed fol-
lowing a design science research framework, further research
would primarily concern its “refine” phase, aiming at improving
the robustness of the proposed artifact. In order to do so, clear
indicators for testing the functionalities of the artifact have to
be defined. On one hand, for supporting the improvement of the
360DMA, the effectiveness of the approach in supporting the digi-
tal transformation of the assessed company has to be verified. This
has to be done by observing an actual digital maturity stage in-
crease when performing multiple iterations of the assessment in
the same company. On the other hand, further work has to be di-
rected towards an efficiency improvement of the proposed assess-
ment approach. While this has been tested in large companies, its
extensive need for resources (e.g. large assessment team and time
to be dedicated to the assessment process) is limiting its appli-
cability in SMEs. In addition to that, different development limi-
tations identified during multiple assessments have to be studied
in relation to the contextual factors that led to the formulation of
the proposed improvement recommendations. This would enable
researchers to identify the key contextual factors to be considered
during an assessment and to build an understanding of their link
with development limitations and improvement recommendations.
Eventually, this would support the realization of a context-specific
prescriptive framework for digital maturity growth.
Declaration of Competing Interest
None.
References
Akdil, K. Y., Ustundag, A., & Cevikcan, E. (2018). Maturity and readiness model
for industry 4.0 strategy. In Industry 4.0: Managing the digital transformation
(pp. 61–94). Cham.: Springer.
Andulkar, M., Tho Le, D., & Berger, U. (2018). A multi-case study on Industry 4.0 for
SME’s in Brandenburg, Germany. In HICSS-51 proceedings: 51st Hawaii interna-
tional conference on system sciences, 2-6 January 2018: 5 (pp. 3575–3584). ISBN:
978-1-5108-5655-4.
Barge, S. (2010). Principles of problem and project based learning. The Aalborg PBL
model. Aalborg: Aalborg University.
Becker, J., Knackstedt, R., & Pöppelbuß, J. (2009). Developing maturity models for IT
management. Business & Information Systems Engineering, 1(3), 213–222.
Colli, M., Madsen, O., Berger, U., Møller, C., Wæhrens, B. V., & Bockholt, M. (2018).
Contextualizing the outcome of a maturity assessment for Industry 4.0. Ifac-pa-
personline, 51(11), 1347–1352.
Crosby, P. B., & Free, Q. I. (1979). The art of making quality certain: 17. New York:
New American Library.
De Bruin, T., Freeze, R., Kaulkarni, U., & Rosemann, M. (2005). Understanding the
main phases of developing a maturity assessment model. ACIS 2005 proceedings.
De Carolis, A., Macchi, M., Negri, E., & Terzi, S. (2017). A maturity model for as-
sessing the digital readiness of manufacturing companies. In IFIP international
federation for information processing 2017. Advances in production management
systems. The path to intelligent, collaborative and sustainable manufacturing. APMS
2017, Part I, IFIP AICT 513 (pp. 13–20). 2017.
Ericsson. (2015). A study of the adoption of “Internet of Things” among
Danish companies digital.di.dk/SiteCollectionDocuments/Analyser/IoT_
Report_onlineversion.pdf Accessed 12 July 2018.
Geissbauer, R., Vedso, J., & Schrauf, S. (2016). Industry 4.0: Building the digital
enterprise. Retrieved from PwC Website: https://www.pwc.com/gx/en/industries/
industries- 4.0/landing- page/industry- 4.0- building- your- digital- enterprise-
april-2016.pdf
Harland, T. (2003). Vygotsky’s zone of proximal development and problem-based
learning: Linking a theoretical concept with practice through action research.
Teaching in Higher Education, 8(2), 263–272 2003.
Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science in information
systems research. Management Information Systems, 28(1), 75–105.
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students
learn? Educational Psychology Review, 16(3), 235–266.
Iansiti, M., & Lakhani, K. R. (2014). Digital ubiquity: How connections, sensors, and
data are revolutionizing business.
Jeschke, S., Brecher, C., Meisen, T., Özdemir, D., & Eschert, T. (2017). Industrial inter-
net of things and cyber manufacturing systems. In Industrial internet of things
(pp. 3–19). Cham.: Springer.
Kagermann, H., Wahlster, W., & Helbig, J. (2013). Recommendations for implementing
the strategic initiative industrie 4.0. Acatech April 2013.
Klimko, G. (2001). Knowledge management and maturity models: Building common
understanding. In Proceedings of the 2nd European conference on knowledge man-
agement (pp. 269–278).
Lewin, K. (1997). Action research and minority problems. In Resolving social con-
flicts (pp. 143–154). Washington D.C.: American Psychological Association. Orig-
inal publication, 1946.
Lichtblau, K., Stich, V., Bertenrath, R., Blum, M., Bleider, M., Millack, A., et al. (2015).
IMPULS - Industrie 4.0- Readiness. Impuls-Stiftung des VDMA.
Madsen, O., & Møller, C. (2017). The AAU smart production laboratory for teaching
and research in emerging digital manufacturing technologies. Procedia Manufac-
turing, 9, 106–112.
McKinsey & Company. (2015). Industry 4.0: How to navigate digitalization of the man-
ufacturing sector. McKinsey and Company www.mckinsey.de/files/mck_industry_
40_report.pdf Accessed 31 March 2016.
Miller, J. G., & Roth, A. V. (1994). A taxonomy of manufacturing strategies. Manage-
ment Science, 40(3), 285–304.
Mittal, S., Khan, M. A., Romero, D., & Wuest, T. (2018). A critical review of
smart manufacturing & Industry 4.0 maturity models: Implications for small
and medium-sized enterprises (SMEs). Journal of Manufacturing Systems, 49,
194–214.
Nolan, R. L. (1979). Managing crises of data processing. Harvard Business Review,
3(4).
Pöppelbuß, J., & Röglinger, M. (2011). What makes a useful maturity model? A
framework of general design principles for maturity models and its demonstra-
tion in business process management. 19th European conference on information
systems (ECIS 2011).
Porter, M. E., & Heppelmann, J. E. (2014). How smart, connected products are trans-
forming competition. Harvard Business Review, 92(11), 64–88.
Qin, J., Liu, Y., & Grosvenor, R. (2016). A categorical framework of manufacturing for
industry 4.0 and beyond. Procedia Cirp, 52, 173–178.
Röglinger, M., Pöppelbuß, J., & Becker, J. (2012). Maturity models in business process
management. Business Process Management Journal, 18(2), 328–346.
Rockwell Automation. (2014). The connected enterprise maturity model Re-
trieved in April 2018 from Rockwell Automation; 2014 http://literature.
rockwellautomation.com/idc/groups/literature/documents/wp/cie- wp002_- en- p.
pdf .
Sánchez-Pi, N., Carbó, J., & Molina, J. M. (2012). A knowledge-based system approach
for a context-aware system. Knowledge-Based Systems, 27, 1–17.
Savery, J. R. (2006). Overview of Problem-based Learning: Definitions and Distinc-
tions. Interdisciplinary Journal of Problem-Based Learning, 1(1). https://doi.org/10.
7771/1541-5015.1002.
Savery, J. R., & Duffy, T. M. (1996). Problem based learning: An instructional model
and its constructivist framework. In B. G. Wilson (Ed.), Constructivist learning
environments: Case studies in instructional design (pp. 135–148).
Schein, E. H. (1995). Process consultation, action research and clinical inquiry, are
they the same? Journal of Managerial Psychology, 10(6), 14–19.
Schein, E. H. (2008). Clinical inquiry/research. In P. Reason, & H. Bradbury (Eds.), The
SAGE handbook of action research (pp. 266–279). London: SAGE Publications.
Schein, E. H. (2009). Helping. San Francisco, CA: Berrett-Kohler.
Schmidt, H. G. (1983). Problem-based learning: Rationale and description. Medical
Education, 17, 11–16.
Schmidt, H. G., Rotgans, J. I., & Yew, E. H. (2011). The process of problem-based
learning: What works and why. Medical Education, 45(8), 792–806.
Schrauf, S., & Berttram, P. (2016). How digitization makes the supply chain more effi-
cient, agile and customer-focused. PWC Published September 7, 2016.
Schuh, G., Anderl, R., Gausemeier, J., ten Hompel, M., & Wahlster, W. (2017). Indus-
trie 4.0 maturity index managing the digital transformation of companies. Acatech
Study April 2017.
Schumacher, A., Erol, S., & Sihn, W. (2016). A maturity model for assessing Industry
4.0 readiness and maturity of manufacturing enterprises. In Proceedings of sixth
international conference on changeable, agile, reconfigurable and virtual production
(CARV 2016) 4-6 Sep. 2016.
Scremin, L., Armellini, F., Brun, A., Solar-Pelletier, L., & Beaudry, C. (2018). Towards a
framework for assessing the maturity of manufacturing companies in industry
4.0 adoption. In Analyzing the impacts of industry 4.0 in modern business environ-
ments (pp. 224–254). IGI Global.
Shewhart, W. A. (1931). Economic control of quality of manufactured product. ASQ
Quality Press.
Teo, T. S., & King, W. R. (1997). Integration between business planning and infor-
mation systems planning: An evolutionary-contingency perspective. Journal of
Management Information Systems, 14(1), 185–214.
Van de Vrande, V., De Jong, J. P., Vanhaverbeke, W., & De Rochemont, M. (2009).
Open innovation in SMEs: Trends, motives and management challenges. Techno-
vation, 29(6–7), 423–437.
Vygotsky, L. S. (1978). Mind in society: the development of higher mental processes.
Cambridge: Harvard University Press.
Wendler, R. (2012). The maturity of maturity model research: A systematic mapping
study. Information and Software Technology, 54(12), 1317–1339.
 M. Colli, U. Berger and M. Bockholt et al. / Annual Reviews in Control 48 (2019) 165–177
M. Colli Ph.D. candidate at Aalborg University, Cen-
ter for Industrial Production, working with the digital
transformation of the manufacturing industry in collab-
oration with the Manufacturing Academy of Denmark
(MADE Digital). With a background in Mechanical En-
gineering (Dott.-Ing., Universita’ degli Studi di Bergamo,
Italy) and Manufacturing Technology (M.Sc., Aalborg Uni-
versity, Denmark), Michele Colli approached the Industry
4.0 agenda in 2016, developing and testing an end-to-end
cloud-based platform for remote monitoring of automa-
tion solutions through the use of the internet of things.
He is currently investigating, in close collaboration with
multiple Danish companies, drivers and barriers for ad-
dressing the digital transformation process.
U. Berger Mechanical Engineer (Dipl.-Ing., University of
Stuttgart and Dr.-Ing., University of Bremen). Career steps
as Project Manager at the Robert Bosch Group, Stuttgart,
Technical Director at the BIBA Institute and Chief Engi-
neer at the Institute of Material Science, Bremen. Since
2001 Full Professor (Chair of Automation Technology) at
Brandenburg University of Technology. Founder of the In-
novation Center Modern Industry Brandenburg and the
German SME Competence Center Industry 4.0 Cottbus. In
2016 appointed as Adjunct Professor at Aalborg University
(Laurits Andersen Professor for Smart Production). Techni-
cal and scientific portfolio in Robotics, Automation Tech-
nology and Industry 4.0.
M. Bockholt Environmental Engineer (BSc., University of
Applied Science RheinMain) and Master of Science in
Global Innovation Management (University of Strathclyde
in Glasgow, UK). Currently working as a project manager
in the field of Circular Economy at Grundfos A/S, Den-
mark and as an industrial PhD Candidate at the University
of Aalborg. Academic research areas are Circular Economy
and the role of Digitalization as potential key enabler.
177
O. Madsen hold a Laurids Andersen professorship at the
Department of Materials and Production Engineering at
Aalborg University, Denmark, where he is head of the re-
search group on Robotics and Automation. His research
interest includes flexible robotics, reconfigurable manu-
facturing systems and smart production. Ole Madsen has
more than 100 international publications and has partici-
pated in a number of national projects (e.g. IPS, BinPicker,
ROBOCUT, CARMEN, MADE) as well as international EU-
funded research projects (e.g. RobVision, Flexpaint, Dock-
laser, Eco2painter, TAPAS, GISA, ACAT, CARLOS).
C. Møller (born 1962) is full professor at the Center for
Industrial Production (CIP) at the Department of Mate-
rials and Production, Aalborg University, Denmark. His
research interests include topics such as Supply Chain
Management, Enterprise Information Systems and Busi-
ness Process Management. Recently he has been engaged
in industrial research into Industry 4.0 related topics and
digitalization in close collaboration with Danish indus-
tries and is currently managing a number of research pro-
grams including a joint AAU initiative on Smart Produc-
tion.
B. Vejrum Wæhrens Professor Brian Vejrum Wæhrens
currently works at the Department of Materials and
Production, Aalborg University. His research domain is
related to sustainable manufacturing (competitive and
green) and with a particular focus to the interaction be-
tween technical and organizational aspects of distributed
operations networks. Current research interests are re-
lated to the ongoing digital transformation of produc-
tion and supply chains, and specifically the link between
emerging technologies, organizational processes & perfor-
mance.