Behavioral Competencies of Digital Professionals

Behavioral Competencies
of Digital Professionals
Understanding the Role of
Emotional Intelligence
Sara Bonesso
Elena Bruni
Fabrizio Gerli
Behavioral Competencies of Digital Professionals
“I encounter many data scientists and analysts whose sole focus is solving analytical
problems and developing accurate models. They are not very effective in their
roles because they can’t build trust and interact effectively with people. They all
need to read this excellent book and adopt its recommendations!”.
—Thomas H. Davenport, Distinguished Professor, Babson College, Research
Fellow, MIT Initiative on the Digital Economy, Author of Competing on
Analytics and The AI Advantage
“Big data, Digital disruption, new jobs and competencies: we are familiar with the
big picture but we are not equipped to have a practical and helpful framework to
guide us. Sure, technical skills will remain necessary but are not sufficient. This
book provides a compelling, credible and sound narrative to de-code complexity
by developing a set of competencies (action, social, awareness, cognitive, explora-
tion and organizational) supported by emotional intelligence. A must read for
Leaders and HR practitioners, for the intellectual curious eager to understand that
Human Beings will have to remain central to Human Development”.
—Paolo Gallo, Author, Executive Coach, former CHRO at World Economic
Forum, World Bank and European Bank, www.paologallo.net
Sara Bonesso • Elena Bruni
Fabrizio Gerli
Behavioral
Competencies of
Digital Professionals
Understanding the Role of Emotional Intelligence
Sara Bonesso Elena Bruni
Department of Management Department of Management
Ca’ Foscari University of Venice Ca’ Foscari University of Venice
Venice, Italy Venice, Italy
Department of Business and
Fabrizio Gerli Management
Department of Management LUISS Guido Carli University
Ca’ Foscari University of Venice Rome, Italy
Venice, Italy
ISBN 978-3-030-33577-9 ISBN 978-3-030-33578-6 (eBook)

https://doi.org/10.1007/978-3-030-33578-6
© The Editor(s) (if applicable) and The Author(s), under exclusive licence to Springer
Nature Switzerland AG 2020
This work is subject to copyright. All rights are solely and exclusively licensed by the
Publisher, whether the whole or part of the material is concerned, specifically the rights of
translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on
microfilms or in any other physical way, and transmission or information storage and retrieval,
electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are
exempt from the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information
in this book are believed to be true and accurate at the date of publication. Neither the pub-
lisher nor the authors or the editors give a warranty, expressed or implied, with respect to the
material contained herein or for any errors or omissions that may have been made. The
publisher remains neutral with regard to jurisdictional claims in published maps and institu-
tional affiliations.
This Palgrave Pivot imprint is published by the registered company Springer Nature
Switzerland AG.
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Foreword
Titles of books are a major challenge. I am seriously bad at it. For example,
I titled my first book of research on competencies predicting outstanding
and superior performance in the private and public sector, The Competent
Manager. If I had known that same year, 1982, In Search of Excellence
would sell 4 million copies in its first couple of years, I could have used
that title and invited readers to a more exciting experience. After all, that
was exactly what I was describing with a voluminous set of data. The
German publishers all but insured that our 2002 international best seller,
Primal Leadership, would not even make up the advance in Germany by
calling it The Emotional Leader. Titles are a tough choice.
Don’t be misled by the somewhat boring title of Sara Bonesso’s, Elena
Bruni’s, and Fabrizio Gerli’s latest book, Behavioral Competencies of
Digital Professionals. They have written the Rosetta Stone of the digital
mind and lifestyle! It is an exciting book about how professionals in our
digital age can navigate the interpersonal and conceptual domain of their
subordinates and colleagues, competitors and hard technology to adapt,
innovate and perform better than others. Working with technology and
digital transformations is not an individual effort, it is a team sport.
Without others, no one will buy your goods or services, no one will remain
working with you, and your great ideas will be relegated to the trash heap
of things that “could have been.”
Let me explain two underlying discoveries that illustrate how important
their work about behavioral and emotional intelligence competencies are
to digital work. The first discovery comes to us from neuroscience.
Professor Anthony Jack’s opposing domains theory and work on opposing
v
vi FOREWORD
poles of reasoning has shown that among the many neural networks, two
are particularly important to our work: the Analytic Network (formally
called the Task Positive Network); and the Empathic Network (formally
called the Default Mode Network).
The Analytic Network enables us to solve problems, make decisions
and focus our perceptual and mental work. Any time we engage in analytic
work with abstractions, like building an information system or securing a
system from cybercrime, or numerical work, like analyzing financial data,
we use the Analytic Network. When companies place major emphasis on
the financial performance, metrics or goals, they activate the AN repeat-
edly. People who go into financial, analytic, software and digital work,
often have a disposition to engage in such activities. People who have IQs
above the normal range are also disposed to be analytic and think in
abstractions.
The Empathic Network enables us to be open to new ideas, scan the
environment for observations of trends and patterns, be open to people
and emotions. We need this network when interacting with others, under-
standing them or learning and innovating. When organizations emphasize
staying in touch with customers, patient experience, understanding your
staff, they emphasize the EN.
Sadly, these two neural networks suppress each other! Yes, activating
one suppresses the other. Activating one repeatedly, suppresses the other
repeatedly. In fact, activating one repeatedly on top of a possible pre-
disposition to engage that network over the other is a recipe for narrow
minded approaches to anything! Often, the appeal of digital work is
greater to people more comfortable with the AN than the EN – and the
nature of the work feeds that predisposition and over-emphasizes the AN
over the EN.
In the past, many scholars and consultants have discussed the differ-
ences in management or leadership styles and approach of those that are
task versus people oriented. This is further exaggerated by people claiming
rational versus emotional difference sin approaches to thinking. The
underlying causes of these distinctions are these two networks. Both the
AN and EN are cognitive processes. Both involve reasoning. But they base
the reasoning on different stimuli. Digital work invites a lopsided activa-
tion at work which can easily contagiously spread to how people live
their lives.
This new book on behavioral competencies and in particular EI helps
the reader orient themselves toward a more effective balance of the AN
FOREWORD vii
and EN. It helps to establish the empirical basis for working with both
neural networks. It explains and shines a light on how the intricate combi-
nation and integration of a person’s cognitive and emotional competen-
cies results in outstanding performance.
But the person is not static. The second major discovery is about
change. The last 25 years of medical and behavioral research has shown us
that humans are malleable from how we act with others to our DNA. Yes,
we do affect our genetics in two major ways. First, we have experiences,
consume certain foods and manage our moods to turn our genes on and
off. Geneticists call it gene expression. Second, our life experiences (and
even before we are born our birth Mother’s experiences) actually can
change out genetic make-up in profound ways. All of that brings us back
to the point that not only does our body renew itself (or die) all of the
time at the cellular levels but our spirit and what excites us also changes
(life and career cycle changes).
We now know that adult humans can create new neurons from stem
cells in parts of our brain. It is called neurogenesis. We also know that
“annoying” stress episodes can cause hormones to enter our bloodstream
that inhibit or stop this neurogenesis. We know that the deluge of annoy-
ing stress, not to mention acute stress that bombard us daily cause a dete-
rioration in our cognitive, emotional and perceptual capabilities – the
effect of activating our Sympathetic Nervous System. Meanwhile, our
bodies have the amazing capability to reverse that through another part of
our autonomic nervous system, the Parasympathetic Nervous System.
At the behavioral level, and in terms of the specific competencies that
predict effectiveness in a wide range of leadership and professional roles in
most countries of the world, we know that adults can dramatically develop
these competencies. Whether you are focusing on those we call emotional,
social or cognitive intelligence, they can be developed and the changes
sustained over years. The published research studies of my colleagues,
including Bonesso, Bruni and Gerli and myself have shown that in the past
25 years. Others have been showing this effect during the same
period of time.
So why do we persist in thinking that we cannot change? First, change
is difficult. Most training programs in government and industry, as well as
graduate education programs produce little sustained, desired change in
these competencies. So we often conclude that people do not change
because we have typically been so ineffective at inspiring and engaging
durable or sustained changes. Second, people often focus on traits. These
viii FOREWORD
are the deeper, relatively stable characteristics we have, like openness or

agreeableness, conscientiousness or introversion. But here as well, recent
research is showing that people can change on even these deep traits.
Our recent neuroimaging and hormonal studies have affirmed an
approach to helping people change called coaching with compassion. That
is, helping people change toward their dreams, values and calling (i.e.,
sense of purpose). The studies show, quite clearly, that the more typical
approach of giving people feedback and trying to help of fix them does the
reverse. It slows change and makes it less sustaining. It is like New Year’s
Eve resolutions – the effort lasts no more than 3 weeks, if that long.
This brings us back to the hope which you will experience in reading
Sara, Elena and Fabrizio’s new book. You CAN change! You CAN become
more effective! You CAN move closer to your dreams. This is not some
naïve hope. It is based on our experiences and decades of our and col-
leagues published academic research studies.
Richard E. Boyatzis is Distinguished University Professor, Case Western
Reserve University, Co-author of the international best seller, Primal
Leadership and the new book, Helping People Change.
Cleveland, OH, USA Richard E. Boyatzis

Preface
Why do some people get exceptional results? How can we improve the
performance of individuals and teams? These are some questions that
probably every CEO, HR director, and manager, asks himself every day.
And they are not the only ones asking these questions. Business schools,
universities, teachers, and trainers are all asking the same questions. In
addition, of course, to all those who work within organizations of every
kind and want to improve themselves.
To provide an answer to this kind of questions the Ca’ Foscari
Competency Centre was founded in 2012 within the Ca’ Foscari University
of Venice, Italy. A team of researchers works with the aim to increase the
performance of people, through the development of their behavioral com-
petencies. People think that having more technical skills allows them to
obtain better results. But for more than thirty years, scientific research has
taught us that although technical skills are required to perform a job, they
alone do not allow to obtain an excellent performance. On the contrary,
behavioral competencies, like emotional, social and cognitive competen-
cies, are the actual determinants of an outstanding performance. Within
the Ca’ Foscari Competency Centre, and in collaboration with the best
scholars and research centers in the world on these topics, we develop
training courses to improve these skills, tools to evaluate them and – above
all – we do a lot of research to identify the most relevant ones for specific
roles and for carrying out specific processes.
This book seeks to answer the above questions, exploring some big data
roles, which are emerging jobs extremely requested and critical for the
competitiveness of organizations, and contains the results of our research
ix
x PREFACE
on their behavioral competencies. This book is a journey into the still

unexplored world of data scientists and data analysts, into the world of the
problems they have to solve every day, of the situations that require their
intervention and of the behaviors they adopt to address them. For those
who already do these jobs, it is also a guide for evaluating themselves and
comparing themselves to others, to better understand one’s own strengths
and weaknesses and to learn the ways to acquire some competencies that
today are not adequately possessed. For those who are approaching these
jobs, it is a way to immerse themselves into the reality and understand
what their real contents are. For those who are in charge of training pro-
grams, and for educational institutions, it is a tool to guide the design of
new courses and curricula, which include the development of behavioral
competencies and use consistent tools and methods. For all the others, it
aims to be a map to move towards a better awareness of what is needed for
a better performance.
Venice, Italy Sara Bonesso

Venice, Italy Elena Bruni
Venice, Italy Fabrizio Gerli
Acknowledgments
There are so many people who have contributed to our research over the
last few years, that it is certainly impossible to name them all. Here we
want to express gratitude to some of them, who have been particularly
outstanding for their ideas, points of view, discussions, intellectual stimuli,
encouragements and criticisms. To Richard Boyatzis we owe the inspira-
tion and the countless opportunities to share our ideas and findings. His
suggestions are of immeasurable value and his tireless support have ori-
ented us within the line of research on behavioral competencies and have
helped us to give life to the Ca’ Foscari Competency Centre, which cur-
rently helps thousands of people to perform better in their job and in their
life. We express all our gratitude also to the colleagues of the GLEAD –
Leadership Development Research Center of ESADE – Ramon Llull
University, and in particular to Joan Manuel Batista-Foguet and Ricard
Serlavós Serra, for their continuous willingness to share methods, tools,
experiences and perspectives, and to Robert Emmerling and Ferran
Velasco Moreno for their support in the methodology and in the data
analysis of this research.
Another big thank you goes to all the colleagues at the Department of
Management of the Ca’ Foscari University of Venice with whom we had
the opportunity to share our research, and most especially to Anna
Comacchio and Andrea Pontiggia, who have shown to believe in this
research and have been a great source of encouragement and of precious
discussions, and to Laura Cortellazzo, for her enthusiastic and valuable
presence in all the activities of our research centre.
xi
xii ACKNOWLEDGMENTS
We are also grateful to the Scientific Committee of the Ca’ Foscari

Competency Centre, and particularly to Giovanni Giuriato, Luca
Giustiniano, Paolo Legrenzi and Pia Masiero, for having oriented our lines
of research and for the richness of the experiences they bring during each
and every meeting.
Of course, this book would not have been possible without the won-
derful people who play the roles of data scientists and data analysts, who
generously shared with us some moments of their job and allowed us to
understand the deep meaning of their behaviors and choices, to get into
their problems and to follow the decision-making processes they adopted.
The regret of not being able to thank them one by one, for reasons of
confidentiality, is as great as the gratitude towards them.
Finally, we wish to express our utmost thankfulness to Maria Raffaella
Caprioglio and to Giuseppe Venier, and to all the other marvelous people
we have met in UMANA S.p.A, for cooperating in both our research and
training activities and being a reliable and outstanding partner. In particu-
lar, Giulio De Biasio and Nicolò Capuzzo deserve our acknowledgment
for their support in this research.
And, of course, we cannot forget our families and friends, who kindly
and patiently endured our physical and mental absence while we con-
ducted this research and wrote this book. To them all we offer a hug of love.
To give to Caesar what is Caesar’s, errors, inaccuracies and omissions,
on the other hand, are to be charged to the authors.
Contents
1 The Organizational Challenges of Big Data 1
2 How Big Data Creates New Job Opportunities: Skill

Profiles of Emerging Professional Roles 21
3 Emotional and Social Intelligence Competencies in the

Digital Era 41
4 When Hard Skills Are Not Enough: Behavioral

Competencies of Data Scientists and Data Analysts 63
5 Managing Big Data Professionals through a Competency-

Based Approach 89
Index107
xiii
About the authors
Sara Bonesso is Associate Professor of Business Organization and Human

Resources Management at the Ca’ Foscari University of Venice, where she
received her Ph.D. in Management. She is one of the founders and the
vice-director of the Ca’ Foscari Competency Centre, a research centre aimed
to improve individuals’ performance and employability through the devel-
opment of behavioral competencies (www.unive.it/cfcc). She has been
visiting scholar at the Industrial Performance Center – Massachusetts
Institute of Technology (Boston, USA) and at the Fraunhofer Institute for
Systems and Innovation Research ISI (Karlsruhe, Germany). She is
Associate Editor of Frontiers in Psychology (section Organizational
Psychology) and member of the Consortium for Research on Emotional
Intelligence in Organizations. Her research interests lie in the fields of
Organizational Design, Organizational Behaviour and Human Resources
Management. Her recent research investigates the development and the
assessment of emotional and social competencies, the competency profile
of big data professionals and of entrepreneurs, the impact of behavioral
competencies on entrepreneurial intent, innovation, employability and
career development. Her research has appeared in various journals, includ-
ing Journal of Vocational Behavior, Industrial and Corporate Change,
Creativity and Innovation Management, the Journal of Small Business
Management, Frontiers in Organizational Psychology, Technovation, The
Journal of Technology Transfer and the European Management Journal.
Her recent works has been presented in several international conferences,
such as AOM (Academy of Management), EURAM (European Academy
of Management), BAM (British academy of management), IWHRM
xv
xvi ABOUT THE AUTHORS
(International Workshop on Human Resources Management), ICEI

(International Congress of Emotional Intelligence), and ISBE (Institute
for Small Business Entrepreneurship).
Elena Bruni is a post-doc research fellow at the Ca’ Foscari University of
Venice. She is also an Adjunct Professor in Human Resource Management
and Organizational Design for International Companies at Ca’ Foscari
University of Venice and Adjunct Professor in Organizational Design at
LUISS Business School, Rome (Italy). Her research interests are related to
innovation, organizational behavior and organization theory. Her doctoral
thesis was focused on the analysis of the mechanisms underlying novelty
generation, drawing on cognitive studies. She draws great attention on
how business model emerges as a concept, by linking recent contributions
from business model innovation literature and cognitive insights. Recently,
she has been devoting attention on how Big Data is impacting organiza-
tions, leadership, and job profiles. Her recent works has been presented in
several international conferences, such as AOM (Academy of Management),
EURAM (European Academy of Management), BAM (British academy of
management), and ISBE (Institute for Small Business Entrepreneurship).
Fabrizio Gerli is Associate Professor of Business Organization and
Human Resources Management at the Ca’ Foscari University of Venice,
where he received his Ph.D. in Management. He is also one of the found-
ers and the Director of the Ca’ Foscari Competency Centre (a research cen-
tre aimed to improve individuals’ performance and employability through
the development of behavioral competencies) and the Scientific
Coordinator of post-graduate courses in Human Resources Management
at Ca’ Foscari. He has been in charge of several research projects funded
by the Italian Ministry of Labor, the Italian Ministry of Research, the
European Institute of Public Administration and the European Social
Fund. He has been visiting professor at HIBA (Higher Institute for
Business Administration) in Damascus. He was also the founder and
Scientific Director of the Master Program in Innovation Management and
the Scientific Director of the full-time MBA at CUOA Business School,
where he was also the Scientific Director of the Competency Development
Department. He has also been member of the Board of Directors of
Fondazione Università Ca’ Foscari. He is also Associate Editor of Frontiers
in Psychology (section Organizational Psychology), member of the Editorial
Board of the Engaged Management Review and peer reviewer for some
management journals. In 2011 he was appointed as member of the
ABOUT THE AUTHORS xvii
Consortium for Research on Emotional Intelligence in Organizations. His

research embraces the fields of Human Resources Management,
Organizational Design and Organizational Behaviour, with a specific inter-
est on the development and evaluation of emotional, social and cognitive
competencies, and their impact on individual and organizational perfor-
mance. His research has been presented in several international confer-
ences and workshops, such as AOM (Academy of Management), EURAM
(European Academy of Management), BAM (British Academy of
Management), EGOS (European Group of Organizational Studies),
IWHRM (International Workshop on Human Resources Management),
ICEI (International Congress of Emotional Intelligence), EUROMA
(European Operations Management Association) and ISBE (Institute for
Small Business Entrepreneurship) and has been published in various jour-
nals, including: Journal of Vocational Behavior, Industrial and Corporate
Change, Industrial Relations, Frontiers in Organizational Psychology,
International Journal of Operations and Production Management,
Creativity and Innovation Management, Journal of Small Business
Management, European Management Journal, Journal of Management
Development, Cross Cultural Management, International Journal of
Training and Development.
List of Figures
Fig. 1.1 Google trends for the keyword “data science” (July 2019) 7
Fig. 3.1 The emotional and social intelligence competency framework 45
Fig. 3.2 The competency Hexagon 50
Fig. 4.1 Behavioral competencies framework: the competency hexagon 70
Fig. 4.2 Competencies of data scientists and data analysts according to
the frequency of manifestation 71
Fig. 4.3 Competencies of data scientists according to their frequency of
manifestation72
Fig. 4.4 Competencies of data analysts according to their frequency of
manifestation73
xix
List of Tables
Table 1.1 Classification of types of big data analytical methods 9

Table 1.2 A representation of the Gartner’s maturity model for data and
analytics10
Table 2.1 Summary of the main responsibilities of data architect, database
architect, database administrator, and data engineer 27
Table 2.2 Technical skills of data scientists identified in scholarly articles 31
Table 2.3 Soft skills of data scientists identified in scholarly articles 32
Table 3.1 Examples of professional roles analyzed in terms of performance
outcomes and related behavioral competencies 48
Table 3.2 Competency Hexagon: The thirty-three competencies and
related definition 51
Table 5.1 A representation of the five discoveries in Intentional Change
Theory94
Table 5.2 Experiential activities for developing self-awareness of the inner
identity and future work self, and the necessary behavioral
competency profile 96
xxi
CHAPTER 1
The Organizational Challenges of Big Data
Abstract How can big data be leveraged to create value and what are the
main barriers that prevent companies from benefiting from the full poten-
tial of data and analytics? This chapter describes the phenomenon of big
data and how its use through data science is dramatically changing the
basis of competition. The chapter also delves into the main organizational
challenges faced by companies in extracting value from data, namely the
promotion of a data-driven culture, the design of the internal and external
structures, and the acquisition of the technical and behavioral skills
required by big data professional roles. The aim and the structure of the
book are illustrated. Shedding light on the human side of big data through
the lense of emotional intelligence, the book aims to provide an in-depth
understanding of the behavioral competencies that big data profiles require
in order to achieve a higher performance.
Keywords Big data • Data science • Data analytics • Organizational

challenges • Emotional intelligence
The point is not to be dazzled by the volume of data, but rather to

analyze it – to convert it into insights, innovations, and business value.
(Davenport 2014: 2)
© The Author(s) 2020 1

S. Bonesso et al., Behavioral Competencies of Digital Professionals,
https://doi.org/10.1007/978-3-030-33578-6_1
2 S. BONESSO ET AL.
1.1 Competing in a Data-Driven Age

Every time we send an email or a message, visit a website, tap an icon on
a smartphone, or post and share comments, photos, and video in social
media, we generate digital data. But even when we perform actions in the
analog world we generate digital data: buying products at the supermar-
ket, driving our connected car, taking the train or the bus, watching an
on-demand movie, or simply taking a walk with our geo-localized smart-
phone in our pocket or using our credit card creates a huge amount of
data. Data is considered as the new raw material of the twenty-first century
(Berners-Lee and Shadbolt 2011), and its use through analytics is dra-
matically changing the basis of competition. The volume of available data
has grown exponentially in recent years due to the increasing number of
individuals, devices, and sensors that are connected by digital networks,
along with the development of more sophisticated algorithms, and the
improvements of computational power and data storage (McKinsey Global
Institute 2016).
Big data analysis is generating significant value across sectors, enhanc-
ing the competitiveness of companies. The more data-driven a firm is, the
more value it generates in terms of knowledge, higher productivity, profit,
and market value (BARC 2015; Brynjolfsson et al. 2011). Several classes
of value have been associated with the use of big data (Davenport 2014;
Lee 2017):
• Improvements in decision-making. Companies can use sophisticated

analytics and develop algorithms to optimize their decision pro-
cesses, such as the automatic fine-tuning of inventories and pricing in
response to real-time in-store and online sales, as well as to minimize
risks. For instance, through the use of optimization techniques it is
possible to identify the price of a product that is more likely to gener-
ate high profitability or the level of inventory that is more likely to
avoid stock-outs (Davenport and Kim 2013).
• Increase in process efficiency. The use of sensors and data analytics
favors cost savings in operations and improves companies’ reaction
time to issues in the supply chain, such as better demand forecasts,
optimized distribution network management, transportation, and
routing (Sanders 2016). For instance, in the fashion industry, the
Prada Group is using Oracle technology to analyze historical data
and market demands across its global retail network of 634 stores in
1 THE ORGANIZATIONAL CHALLENGES OF BIG DATA 3
order to optimize the merchandising process and detect trends as

well as for performance analysis, inventory management, and
allocation.
• Enhancement of customer experience. Granular data, namely
detailed data for each single customer, allows organizations to imple-
ment specific market segmentations and to tailor products and ser-
vices to meet specific customers’ needs. For instance, major retailers
analyzing preferences and sentiments data can deliver personalized
product/service recommendations and promotional offers, whereas
financial companies exploiting social media data are able to assess the
credit risk and financial needs of potential clients and provide new
types of financial products.
• Innovation of business models, products, and services. Data on cus-
tomers’ purchase decisions and social feedback mechanisms can be
complemented with digital payments and transaction data to delve
deeper into innovation and product adoption. The use of big data
can also promote the introduction of new business models in tradi-
tional industries, as in the case of Nike, which from a shoes manufac-
turer became a digital platform owner for data-driven fitness services,
or Under Armour, which from solely a sports apparel company part-
nered with IBM Watson to apply artificial intelligence to create UA
Record, an app that provides evidence-based coaching around sleep,
fitness, and nutrition.
• Improvements in customer service. Data on the same customer is
integrated from multiple channels, allowing service personnel to bet-
ter understand problems and address them quickly. Moreover, big
data analytics can be used to monitor transactions in real time and
detect fraudulent activities.
Besides the economic value mentioned above, big data analysis may
also generate social value, enhancing transparency, preventing frauds and
crimes, responding to natural disasters. Improving national security,
increasing transportation safety, and supporting the well-being of people
through better education and health care (Günther et al. 2017).
Organizations are still struggling to capture the full potential of big
data. As underlined in the Future of Jobs Report released by the World
Economic Forum (2018), by 2022 85 percent of the surveyed companies
are likely to invest in user and entity big data analytics and 75 percent are
likely to increase the use of Internet of Things and app- and web-enabled
4 S. BONESSO ET AL.
markets. Likewise, machine learning and cloud computing are receiving

considerable attention: respectively 73 percent and 72 percent of the sur-
veyed companies indicated their intention to adopt these technologies.
In addition to spurring the rate of technological advancement and its
related adoption in organizational contexts, big data is profoundly chang-
ing the job-relevant skills requested in the labor market. Indeed, while the
investment in big data technologies is becoming paramount, at least as
important is to attract those professionals with the skills profile relevant to
use these technologies effectively (Davenport and Patil 2012). Among the
most in-demand digital professional roles to emerge are those of data ana-
lysts and scientists, artificial intelligence and machine learning specialists,
and big data specialists (World Economic Forum 2018).
But what is big data, and how can it be leveraged to create value?
The term “big data” was coined in the mid-1990s but became wide-
spread after 2011 (Gandomi and Haider 2015; Mishra et al. 2017). In
providing definitions of big data, academics and practitioners have tried to
highlight the properties that characterize information in the digital era.
Specifically, big data has been conceptualized as information assets charac-
terized by a combination of volume, variety, and velocity (the so-called
Three Vs) that creates an opportunity for organizations to gain competi-
tive advantage in today’s digitized marketplace (Chen et al. 2012; Kwon
et al. 2014; De Mauro et al. 2016).
The size or the magnitude of data (“Volume”) is the first dimension
that comes to mind when defining data as “big.” Currently, exabytes (1
million terabytes) or zettabytes (1000 exabytes) qualify high-volume data,
even if bigger units of measure are occasionally developed, since – as the
data storage capacities continue to increase – the property “volume” is
relative and varies by time. One of the most important fuels of the increased
volume of data in recent years is the phenomenon of the Internet of
Things (IoT), namely the pervasive presence of a variety of objects –
phones, sensors, Radio-Frequency Identification (RFID) tags, and actua-
tors, among others – which can communicate and interact with each other,
over the Internet, and can be remotely monitored and controlled. Data
volume is expected to exponentially grow in the next years due to the
increasing number of Internet users and the billions of connected devices
and embedded systems that create, collect, and share data every day.
The second dimension, velocity, refers to the frequency of generation
of data and the speed at which it is analyzed. The diffusion of digital
devices, like smartphones and sensors, has increased the rate of data
c reation and the need for real-time analytics. For instance, the high speed
of data generated by mobile devices about geospatial location, demo-
graphics, and past buying patterns can be used to generate real-time, per-
sonalized offers to customers. In the retailing industry, the real-time
modeling, manipulation, and visualization of transactional data can explain
why sales declined in a particular product or category. The case of Walmart,
with over 20,000 stores in 28 countries, represents an example of how
companies can react to data quickly. The largest retailer in the world pro-
cesses 2.5 petabytes of data every hour through its Data Café, a state-of-
the-art analytics hub, combining more than 200 internal and external data
sources (transactional, economic, social media, and gas prices, among oth-
ers) to come up with solutions and make fast decisions (Marr 2017).
The last “V,” variety, is related to the fact that data is available in differ-
ent forms. The traditional structured format, in rows and columns stored
in Structured Query Language (SQL), represents a small percentage of all
data. Big data is predominantly in semi-structured format, like XML files,
or unstructured formats such as text, social media data, audio, and video.
Over time, other dimensions have been added to better describe the
concept of big data, such as veracity, which was introduced by IBM and
refers to the unreliability and uncertainty that characterize some sources of
data, due to their incompleteness, inaccuracy, or subjectivity. This is the
case of customers’ sentiments collected in the social media that derive
from individual judgments (Gandomi and Haider 2015). SAS added two
further properties of big data: variability and complexity. The former
underlines that the meaning of data is constantly changing, whereas the
latter refers to the fact that big data can be collected from different sources,
such as the following, according to George et al. (2014):
• public data, held by government institutions and local communities,

such as that concerning health care, education, transportation, and
financial markets;
• private data, held by individuals and private firms, which refers for
instance to mobile phone usage, consumer transactions, and move-
ment of company goods and resources;
• data exhaust, generated as trails or information by-products resulting
from all digital or online activities, that refers to storable choices,
actions, and preferences, such as log files, cookies, and tempo-
rary files;
6 S. BONESSO ET AL.
• community data, which is unstructured data, usually in text format,

that can be distilled into dynamic networks in order to infer social
trends. Some examples are consumer rankings and reviews or the
timeline of social media sites.
• self-quantification data, provided by individuals through personal
actions and behaviors, such as that collected through wristbands that
monitor fitness activities. The data can be uploaded in mobile device
applications, tracked, and aggregated.
To unlock the potential of this high volume of fast-moving and diverse

data, technologies and analytics methods have made a leap forward in
recent years, on the one hand, to capture, store, integrate, transform, and
retrieve data (data management) and, on the other hand, to select the
right model for analysis and to provide interpretations of the results (data
analysis).
1.2 Data Science: How to Extract Value from Big

Data
The scientific body of knowledge that provides methods, processes, and
systems to extract insights from data is defined as data science. It is an
interdisciplinary field that combines statistics, computer science, data min-
ing, machine learning, and analytics to understand and explain how we
can generate analytical insights and prediction models from structured and
unstructured big data. Ten years before the rise of the big data phenom-
enon, data science was defined by Cleveland as “an action plan to enlarge
the technical areas of statistics” (Cleveland 2001: 21). The advent of big
data and its related challenges in data management and analysis have pro-
gressively expanded the domain of data science beyond the statistics field,
assuming an increasing relevance. Figure 1.1 shows a Google Trends chart
that displays web searches for the term “data science,” highlighting the
dramatic increase in interest in data science in correspondence with the
interest in big data.
As the size of data (volume) is increasing at an exponential rate, scal-
ability represents a key aim for models and new technologies that allow the
storing and processing of a growing amount of data. Since data is gener-
ated at a high rate (velocity), its value decays over time. Thus, analytics
methods for streaming data are continually improved to provide timely
Fig. 1.1 Google trends for the keyword “data science” (July 2019)
answers to decision makers. Moreover, since big data can derive from het-
erogeneous sources (variety), current databases are severely susceptible to
inconsistent, incomplete, and noisy data. Improvements in big data sci-
ence techniques aim at guaranteeing a higher quality of the data in terms
of accuracy, completeness, and consistency.
Under the umbrella of data science can be included all the methods and
techniques which refer to the two main processes of extracting knowledge
from big data, namely data management and data analysis (Sivarajah
et al. 2016).
Data management encompasses the following three stages:
• Data acquisition and warehousing. According to the 2017 Big Data

Analytics Market Study released by Dresner Advisory Services, data
warehouse optimization is considered critical or very important by
70 percent of all respondents. Big data has changed the way to cap-
ture and store multiple data formats into a single format and consoli-
date them in one place, promoting the development of new data
storage devices and architectures. Due to their limited scalability,
traditional relational database systems have been replaced by unified
storage and processing environments for big data across multiple
servers like Hadoop/MapReduce/Spark, which have become
8 S. BONESSO ET AL.
s tandard tools for big data processing (Li et al. 2017). Contrary to a
data warehouse, which retains data from operational systems and is
meant to answer a pre-defined set of questions, a recent and different
storage repository is represented by data lakes, a new technology
which stores various data in its native, raw forms in a centralized
location from different sources, regardless of its use in the immediate
future, assuming that analysis will happen later, on demand
(Jain 2017).
• Data cleansing. Data quality represents a main issue in dealing with
big data. This stage encompasses all the procedures for correcting or
removing inaccurate and corrupt data.
• Data integration and aggregation. Through these procedures data
from separate sources is combined into meaningful and valuable
information, which are then gathered and expressed in a summary
form for subsequent analysis.
The second subprocess, data analysis, implies the use of techniques

which involve a number of different disciplines, including statistics, data
mining, machine learning, neural networks, social network analysis, signal
processing, pattern recognition, optimization methods, and visualization
approaches (Chen and Zhang 2014). It comprises two stages; on the one
hand, analysis and modeling that refers to the methods and techniques to
generate and experiment with algorithms for extracting insights from data
in order to get answers to specific problems, and on the other hand, data
interpretation, which encompasses the techniques for visualizing and pre-
senting data in an understandable form for decision makers.
Data analytical methods, which have seen a rapid development in recent
years, can be grouped into three main categories: descriptive, predictive,
and prescriptive analysis (Chen and Zhang 2014; Davenport and Kim
2013). A definition of these methods and of their related techniques are
provided in Table 1.1.
Findings from the BARC’s BI Trend Monitor 2019 show that predictive
and prescriptive analytics emerge among the most important trends.
Moreover, the variety of data sources has fostered the diffusion of dedicated
software and algorithms for analyzing specific types of data such as texts,
videos, and social media analytics (Granville 2014). Another scientific
domain, artificial intelligence and one of its subsets, machine learning, is
receiving increasing attention for its application in big data analysis (McKinsey
Global Institute 2016). The concept underpinning machine learning is to
Table 1.1 Classification of types of big data analytical methods

Data Aim of the analysis Techniques
analytics
methods
Descriptive Provide insight into the past in a way Descriptive statistics (such as
analytics that developments, patterns, and mean, median, mode, standard
exceptions become evident, in the form deviation, variance, and frequency
of producing standard reports, ad hoc measurement of specific events)
reports, and alerts. They answer to the and data-mining techniques.
question: What has happened?
Predictive Make predictions and forecasts about Statistical methods and data-
analytics future events using historical and mining techniques to identify
current data. They answer to the uncovered patterns and capture
question: What could happen? relationships in data. They are
categorized into two groups:
Regression techniques and
machine learning techniques.
Prescriptive Quantify the effect of future decisions Business rules, algorithms,
analytics in order to advise on possible outcomes experimental design,
before the decisions are actually made. optimization, machine learning,
They provide recommendations and computational modeling
regarding actions that will take procedures.
advantage of the predictions. They
answer to the question: What should
we do?
give the algorithm a massive number of “experiences” (training data) and a

generalized strategy for learning, then let it identify patterns, associations,
and insights from the data. Through machine learning it is possible to create
algorithms that “learn” from data without being explicitly programmed. A
cutting-edge area of research within the machine learning domain is deep
learning, which uses neural networks with many layers (hence the label
“deep”) to push the boundaries of machine capabilities. The two most pop-
ular programming tools for data analysis are the open source software Python
and R, which also provide deep learning and other machine learning librar-
ies. Another important stage of data analysis is visualization. As pointed out
by a survey conducted by McKinsey Global Institute (2016), visualization
skills show an increasing demand. Recent advancements in visualization
techniques and new software help make the results of complex data analyses
understandable for decision makers and also for those who are new to ana-
lytics, enabling them to turn data into valuable insights.
10 S. BONESSO ET AL.
1.3 Organizational Challenges for Leveraging

Big Data
A global survey administrated to 196 organizations by Gartner, Inc. in
2018 asked respondents to rate their organizations according to Gartner’s
five levels of maturity for data and analytics (for a summary of the levels
included in the model, see Table 1.2). It found that 60 percent of respon-
dents rated themselves in the lowest three levels, despite companies’ hav-
ing maintained that data management and analytics have represented a
priority in recent years. The survey also showed that, despite the increas-
ing attention devoted to advanced forms of analytics, 64 percent of firms
still consider enterprise reporting and dashboards their most business-
Table 1.2 A representation of the Gartner’s maturity model for data and
analytics
Level 1 Level 2 Level 3 Level 4 Level 5
Basic Opportunities Systematic Differentiating Transformational
Companies Companies Companies Companies Data is crucial for

operate in still operate in promote data promote the business strategy.
silos. Data is silos, with little harmonization internal sharing Chief data officer
used to collaboration and governance of best practices is member of the
generate single or knowledge so the analytics and appoint a board of directors.
metric within sharing. The can leverage chief data Data comes from
a particular data is used to end-to-end officer. Analytics public and
functional measure process data. are faster and unstructured
unit. The performance Executives more dynamic, sources and the
focus is on and provide champion data and data is used internet of things.
after-the-fact support for and analytics. for ROI. Data is used to
performance. decision Data is used to influence
Excel making establish visibility investments.
spreadsheets through excel and performance
dominate, spreadsheets, measurement
providing reports, and across processes.
limited dashboards. At this level
analytics. companies also
start to integrate
external data
sources, and
different types of
data are treated
in different ways.
critical applications for data and analytics. In the same vein, traditional
data sources such as transactional data and logs also continue to dominate,
although 46 percent of organizations now report using external data
(Gartner 2018).
What are the main barriers that prevent companies from benefitting
from the full potential of data and analytics?
According to recent surveys conducted worldwide across a variety of
industries, the biggest barriers companies face in extracting value from
data and analytics are organizational (Gartner 2018; McKinsey Global
Institute 2016). Specifically, the NewVantage Partners Executive Survey
(2019), administrated to 65 Fortune 1000 leading firms, reveals that
among the barriers mentioned by the companies, 95 percent stem from
organizational challenges and only 5 percent are related to technology.
Indeed, companies have responded to competitive pressure by making
investments in technology, without implementing the necessary organiza-
tional changes.
The first challenge is incorporating data and analytics into a core strate-
gic vision. Companies need to diffuse a supportive organizational culture
or mind-set to invest in data-driven initiatives (Pigni et al. 2016). The
implementation of the digital infrastructure should be tailored in line with
the specific uses of the data according to the company’s strategic objec-
tives. For this reason, companies need to appraise the value of big data in
providing a competitive advantage in their business, to understand the
type of problems that can be addressed by data analytics, to equip them-
selves with the adequate infrastructures and tools, and to change the con-
solidated decision-making habits, adopting a more test-and-learn culture
to measure the business impact of data analytics. Companies that embrace
a data-driven culture produce more innovative products and services, are
more competitive, and increase productivity by 5–10 percent more than
companies that do not (Davies 2016). However, among the 65 Fortune
1000 leading firms surveyed in the NewVantage Partners Executive Survey
(2019), only 31 percent have created a data-driven organization and only
28.3 percent have promoted a data culture. The diffusion of a data-driven
mind-set requires a high commitment from the top and middle manage-
ment, who are in charge of leading the digital transformation across the
different business units. This is also in line with the progressive shift from
a traditional model of organizing companies to an agile one. Indeed, big
data analytics is conceived as fundamental for firms aiming to adopt agile
principles (Rialti et al. 2018), since transparency of information, c ontinuous
learning, and quick, efficient, and continuous decision making are indi-
cated as among the main traits that distinguish agile organizations
(McKinsey and Company 2017).
The second related challenge refers to the design of the internal orga-
nizational structure and the external network to support data and analytics
activities. A company can decide to adopt a decentralized or centralized
organizational design for data management and analytics activities. In the
first case, a central department for data science, thus a dedicated center of
excellence, promotes economies of scale and specialization. A decentral-
ized structure means that the single business departments are made
accountable for the generation of value from data, with consequent needs
to promote coordination through cross-functional practices. Interestingly,
a recent McKinsey Analytics study (2018) revealed that leading compa-
nies, in comparison to laggard ones in the big data field, have structured
their analytics unit by adopting a hybrid model led by a center of excel-
lence. Moreover, companies through the design of external networks can
establish partnerships with platform providers to get access to advanced
tools or solutions, or with data providers to gain access to specific data sets.
The design of the organizational structure of the data analytics activities
is closely related to the definition of the set of competencies that the data
science/analytics teams should possess to effectively conduct data acquisi-
tion and preparation, model building, and data presentation. Another
challenge companies are facing is attracting and retaining appropriate big
data professional roles. A 2018 search on LinkedIn Jobs, using the key-
word “analytics,” resulted in 218,866 entries (Bowers et al. 2018), high-
lighting the shortage of qualified and competent big data specialists, such
as data scientists, business analysts, data engineers, and data architects,
among others (McKinsey Global Institute 2016). To respond to this mar-
ket demand, in recent years there has been an exponential growth in the
number of master of science graduates in analytics and data science.
Educational programs in data science place more emphasis on computing,
data management, and data mining, thus they are more data-centric,
whereas programs in analytics are more problem-centric, including the
entire spectrum of analytics (descriptive, predictive, and prescriptive) and
engaging most often in client-based capstone experiences (Bowers et al.
2018). Despite the increasing academic offerings of dedicated educational
programs, the shortage of analytical and data science skills continues to
represent a critical constraint (LinkedIn Workforce Report 2018; McKinsey
Global Institute 2016). The skills gap of big data professional roles not
only refers to technical/analytics competencies but also encompasses soft

or behavioral skills that are considered a requisite for companies. Among
the most in-demand soft skills for big data roles are business acumen,
communication, interpersonal skills, curiosity, and interdisciplinary orien-
tation (Costa and Santos 2017; Davenport and Patil 2012). Even if higher
education institutions seem to have responded promptly to employers’
demand on the side of technical skills, educational programs in data sci-
ence and business analytics still view personal and interpersonal competen-
cies as a second priority (Bowers et al. 2018). Therefore, the development
of such skills for big data professional roles remains a critical issue, as
behavioral competencies are the major source of mismatch between com-
panies’ requests and candidates’ skill profiles.
1.4 The Emotional Intelligence Side of Big Data

In an article published in the Harvard Business Review in 2017, Beck and
Libert, both practitioners in the machine learning field, maintained that
the rise of artificial intelligence makes emotional intelligence more impor-
tant. They argued that “skills like persuasion, social understanding, and
empathy are going to become differentiators as artificial intelligence and
machine learning take over our other tasks.” These skills, which are usually
defined as “soft” in comparison to the technical/hard skills, refer to the
broader concept of emotional intelligence and its related behavioral com-
petencies, namely the ability to recognize, understand, and manage one’s
own emotions (emotional competencies); the ability to understand other
people’s concerns, feelings, and emotional states; the capacity to build and
maintain positive relationships and to behave appropriately with others
(social competencies); and the ability to analyze information and situa-
tions (cognitive competencies) (Boyatzis 2009; Goleman 1998).
From research conducted by MIT and Deloitte involving more than
3700 business executives, managers, and analysts from 131 countries, it
emerged that technical skills alone cannot guarantee the achievement of
individual performance in the digital age. Instead, the study revealed that
most successful companies are those that in their digital transformation
efforts put their focus on soft skills, such as the ability to develop a vision,
and to have a change-oriented mind-set and other leadership and
collaborative skills (Kane et al. 2016). Behavioral competencies also have
been recently found as among the most important abilities for big data pro-
fessional roles, with specific regard to business analysts and data scientists
who are primarily involved in data modeling, analysis, and presentation

(Costa and Santos 2017; Kim and Lee 2016; Vidgen et al. 2017).
Why do behavioral competencies make a difference for big data roles?
The ultimate aim of these roles is to help the business make better deci-
sions, and this implies continually putting themselves in the shoes of dif-
ferent stakeholders (such as senior executives, coworkers, and clients) to
enhance the understanding of their needs and deliver valuable insights
through persuasive and effective communication. But among others, they
also require organizational awareness for identifying the appropriate
sources of data, teamwork for promoting collaboration within the data
science team and coordination across different business units, and ques-
tioning and experimentation skills for identifying alternative methods,
analytics techniques, and algorithms.
Even if in recent years many efforts have been made to define the skills
set of big data professions on the technical side, by identifying the knowl-
edge and capabilities that distinguish the most in-demand professional
profiles, there is still not a clear understanding of which emotional and
social competencies companies should search and develop for promoting
high-performing data science teams.
1.5 The Aim of This Book and Its Structure

This book represents the first comprehensive and up-to-date description
of the big data professional roles that are emerging as the most in-demand
jobs in the labor market and are becoming central in every organization,
from for-profit to not-for-profit companies. The book offers an in-depth
investigation of the behavioral competencies that these professionals
require to achieve higher performance in companies that aim to turn
insights from data into competitive advantages. Shedding new light on the
human side of big data through the lenses of emotional and social intelli-
gence competencies, this book aims to i) advance the understanding of the
requirements of the different professions that deal with big data; ii) pro-
vide a comprehensive review of the competency profile required by these
professional roles, with a specific focus on the behavioral competencies
needed to achieve superior performance at work; and iii) provide an appli-
cation of the competency modeling process in the area of big data
professions and offer new empirical insights specifically on the job profiles
and related behavioral competencies of data scientists and business analysts.
The structure of the book is outlined as follows.
Chapter 2 – How Big Data Creates New Job Opportunities: Skill Profiles
of Emerging Professional Roles. Starting from the shortage of analytical and
data science skills necessary to make the most of big data, the chapter
offers a comprehensive review and classification of the job profiles of dif-
ferent big data roles. Recent surveys have identified more than 100 job
titles adopted in the labor market only referring to data science and analyt-
ics professionals. Studies also agree that there is an improper use of the
label of “data scientist” to define a plethora of different professions. Main
differences among several profiles (chief data officer, data architect, data-
base architect, database administrator, data engineer, data scientist, data
analyst, and business analyst) have been defined according to the volume
of data analyzed, the tools they use, and the educational background pos-
sessed. The chapter outlines that technical skills alone seem to be not
enough to succeed in the big data and data science field, whereas possess-
ing behavioral competencies or soft skills has become a mandatory require-
ment for big data profiles.
Chapter 3 – Emotional and Social Intelligence Competencies in the
Digital Era. The chapter is meant to provide a clear understanding of why
it is important to analyze behavioral competencies of big data professional
roles since they are called to understand data, interpret the data, and
transmit its meaning to the upper levels of the organizations. The chapter
explains the evolution of the emotional and social intelligence compe-
tency framework and provides insights on the large body of research that
has investigated the impact of behavioral competencies on individual per-
formance. It also introduces the competency framework that will be
adopted the empirical analysis described in Chap. 4 providing the classifi-
cation and the definition of thirty-three behavioral competencies. Finally,
it opens the discussion on the role of behavioral competencies in the big
data era, identifying the state of the art and the major gaps that need to
be addressed.
Chapter 4 – When Hard Skills Are Not Enough: Behavioral Competencies
of Data Scientists and Data Analysts. This chapter concentrates attention
on an analysis of the two most in-demand big data professional roles,
data scientists and data analysts. These two profiles have a direct impact
on business functions and decision-making processes, and therefore are
at the core of organizational changes. The behavioral competencies of
both professionals are investigated in order to emphasize the peculiari-
ties of each profession. The chapter illustrates the empirical evidence
collected through an in-depth qualitative exploratory study on a sample
of data scientists and data analysts operating in the Italian context. The
study adopts the competency-based methodology, and specifically, data
has been collected through behavioral event interview, a consolidated
technique that does not rely on perceptions of the main important com-
petencies for the professional roles under investigation but allows for the
detection of the behaviors that are actually enacted in the work environ-
ment. The chapter provides an in-depth description of the tasks and
responsibilities of the two roles under investigation, as well as of the
behavioral competencies manifested in critical events/incidents in which
individuals felt effective in performing their job in the organizational
context. In particular, the behavioral competency portfolio of data sci-
entists and data analysts is described through a competency framework
which encompasses thirty-three behavioral competencies grouped into
six areas: awareness, action, social, cognitive, exploratory, and organiza-
tional action competencies.
The fifth and last chapter – Managing Big Data Professionals through a
Competency-Based Approach – contributes to the current debate on how to
overcome the skill shortage that characterizes the demand for big data
professional roles. First, it offers managerial insights in describing how
organizations and specifically HR practitioners can benefit from the
competency-based approach to increase the effectiveness of the selection
and recruiting processes of candidates, achieving a better match between
job offers and demand. Second, it provides recommendations for the
higher education system to offer better designed curricula for entry-level
big data professions. There is increasing attention within different institu-
tions on developing and sponsoring educational programs on data science
and business analytics. However, there is still a need to design such pro-
grams carefully to provide adequate preparation, both in terms of techni-
cal and behavioral competencies.
References
BARC. (2015). Big data use cases. Getting real on data monetization. Retrieved
from https://www.sas.com/content/dam/SAS/bp_de/doc/studie/ba-st-
barc-bigdata-use-cases-de-2359583.pdf
BARC. (2019). BI Trend Monitor 2019. The world’s largest survey of BI trends.
Retrieved from https://www.sitsi.com › download
Beck, M., & Libert, B. (2017, February 15). The rise of AI makes emotional intel-
ligence more important. Harvard Business Review.
Berners-Lee, T., & Shadbolt, N. (2011, December 31). There’s gold to be mined
from all our data. The Times (pp. 1–2).
Bowers, M. R., Camm, J. D., & Chakraborty, G. (2018). The evolution of analyt-
ics and implications for industry and academic programs. INFORMS Journal
on Applied Analytics, 48(6), 487–499.
Boyatzis, R. E. (2009). Competencies as a behavioral approach to emotional intel-
ligence. Journal of Management Development, 28(9), 749–770.
Brynjolfsson E., Hitt L. M., & Kim H. H. (2011). Strength in numbers: How does
data-driven decision making affect firm performance? Working paper, 2011.
SSRN working paper. Available at SSRN: http://ssrn.com/abstract=1819486
Chen, C. L. P., & Zhang, C. Y. (2014). Data-intensive applications, challenges,
techniques and technologies: A survey on big data. Information Sciences,
275, 314–347.
Chen, H., Chiang, R. H. L., & Storey, V. C. (2012). Business intelligence and
analytics: From big data to big impact. MIS Quarterly, 36(4), 1165–1188.
Cleveland, W. S. (2001). Data science: An action plan for expanding the technical
areas of the field of statistics. International Statistical Review, 69(1), 21–26.
Costa, C., & Santos, M. Y. (2017). The data scientist profile and its representative-
ness in the European e-competency framework and the skills framework for the
information age. International Journal of Information Management,
37(6), 726–734.
Davenport, T. H. (2014). Big data at work: Dispelling the myths, uncovering the
opportunities. Boston: Harvard Business Review Press.
Davenport, T. H., & Kim, J. (2013). Keeping up with the quants. Your guide to
understanding and using analytics. Boston: Harvard Business Review Press.
Davenport, T. H., & Patil, D. J. (2012). Data scientist: The sexiest job of the 21st
century. Harvard Business Review, 90(October), 70–76.
Davies, R. (2016). Big data and data analytics. The potential for innovation and
growth. European Parliamentary Research Service.
De Mauro, A., Greco, M., & Grimaldi, M. (2016). A formal definition of big data
based on its essential features. Library Review, 65(3), 122–135.
Dresner Advisory Services Publishers. (2017, December 20). Big data Analytics
market study. Retrieved from https://www.microstrategy.com/getmedia/
cd052225-be60-49fd-ab1c-4984ebc3cde9/Dresner-Report-B ig_Data_
Analytic_Market_Study-WisdomofCrowdsSeries-2017.pdf
Gandomi, A., & Haider, T. M. (2015). Beyond the hype: Big data concepts, meth-
ods, and analytics. International Journal of Information Management,
35, 137–144.
Gartner. (2018, February 5). Survey shows organizations are slow to advance in
data and analytics. Retrieved from https://www.gartner.com/en/newsroom/
press-releases/2018-02-05-gartner-survey-shows-organizations-are-slow-to-
advance-in-data-and-analytics
George, G., Haas, M. R., & Pentland, A. (2014). Big data and management.
Academy of Management Journal, 57(2), 321–325.
Goleman, D. (1998). Working with emotional intelligence. New York: Bantam Dell
Pub Group.
Google Trends. (2019). Searches for the term Data Science. Retrieved 19 July 2019
from https://www.google.com/trends/explore#q=data%20science
Granville, V. (2014). Developing analytics talent. Becoming a data scientist.
Indianapolis: Wiley.
Günther, W. A., Mehrizi, M. H. R., Huysman, M., & Feldberg, F. (2017).
Debating big data: A literature review on realizing value from big data. Journal
of Strategic Innovation Information Systems, 26(3), 191–209.
Jain, V. K. (2017). Big Data and Hadoop. New Delhi: Khanna Publishing.
Kane, G. C., Palmer, D., Phillips, A. N., Kiron, D., & Buckley, N. (2016). Aligning
the organization for its digital future. Cambridge, Massachusetts, United States:
MIT Sloan management review and Deloitte University Press.
Kim, J. Y., & Lee, C. K. (2016). An empirical analysis of requirements for data
scientists using online job postings. International Journal of Software
Engineering and Its Applications, 10(4), 61–172.
Kwon, O., Lee, N., & Shin, B. (2014). Data quality management, data usage
experience and acquisition intention of big data analytics. International Journal
of Information Management, 34(3), 387–394.
Lee, I. (2017). Big data: Dimensions, evolution, impacts, and challenges. Business
Horizons, 60(3), 293–303.
Li, K. C., Hai, J., & Zomaya, A. (2017). Big data management and processing.
New York: Imprint Chapman and Hall/CRC.
LinkedIn. (2018). LinkedIn Workforce Report. Retrieved from https://economic-
graph.linkedin.com/resources/linkedin-workforce-report-december-2018
Marr, B. (2017, January 23). Really big data at Walmart: Real-time insights from
their 40+ petabyte data cloud. Forbes.
McKinsey Analytics. (2018, January). Analytics comes of age. Retrieved from
https://www.mckinsey.com/~/media/McKinsey/Business%20Functions/
McKinsey%20Analytics/Our%20Insights/Analytics%20comes%20of%20age/
Analytics-comes-of-age.ashx
McKinsey and Company. (2017, December). The 5 trademarks of agile organiza-
tions. Retrieved from https://www.mckinsey.com/business-functions/organi-
zation/our-insights/the-five-trademarks-of-agile-organizations
McKinsey Global Institute. (2016, December). The age of analytics: Competing in
a data-driven world. Retrieved from https://www.mckinsey.com/business-
functions/mckinsey-analytics/our-insights/the-age-of-analytics-competing-
in-a-data-driven-world
Mishra, D., Luo, Z., Jiang, S., Papadopoulos, T., & Dubey, R. (2017). A biblio-
graphic study on big data: Concepts, trends and challenges. Business Process
Management Journal, 23(3), 555–573.
NewVantage Partners. (2019). Big data and AI executive survey 2019. Executive
Summary of Findings. Retrieved from https://newvantage.com/wp-content/
uploads/2018/12/Big-Data-Executive-Sur vey-2019-Findings-
Updated-010219-1.pdf
Pigni, F., Piccoli, G., & Watson, R. (2016). Digital data streams: Creating value
from the real-time flow of big data. California Management Review,
58(3), 5–25.
Rialti, R., Marzi, G., Ciappei, C., & Caputo, A. (2018). Reframing agile organi-
zation. Do big data analytics capabilities matter? Academy of Management
Global Proceedings.
Sanders, N. R. (2016). How to use big data to drive your supply chain. California
Management Review, 58(3), 26–48.
Sivarajah, U., Kamal, M. M., Irani, Z., & Weerakkody, V. (2016). Critical analysis
of big data challenges and analytical methods. Journal of Business Research,
70, 263–286.
Vidgen, R., Shaw, S., & Grant, D. B. (2017). Management challenges in creating
value from business analytics. European Journal of Operational Research,
261(2), 626–639.
World Economic Forum. (2018). The Future of Jobs Report 2018. Retrieved from
http://www3.weforum.org/docs/WEF_Future_of_Jobs_2018.pdf
CHAPTER 2
How Big Data Creates New Job

Opportunities: Skill Profiles of Emerging
Professional Roles
Abstract Big data jobs will increase in importance over the next years.
However, at the international level, the labor market for these profession-
als is characterized by a critical skill shortage. What are the big data spe-
cialist profiles that are most sought in the market? What are their main
differences in terms of tasks and skill requirements? This chapter provides
a snapshot of the most in-demand big data jobs, contributing to clarify
their boundaries. It also delves into the main characteristics of the specific
professional profiles that have received increasing attention in recent years,
namely data scientists and data/business analysts. The review of the con-
tributions provided by experts and scholars operating in the data science
and analytics domain clarifies the main differences between these roles on
the technical side. However, despite the increasing importance of soft
skills, the behavioral competency profile of big data jobs is still ill defined.
Keywords Big data professional profiles • Skill shortage • Soft skills •

Technical skillset
When people decide to pursue a career as an aircraft pilot, they are

embarking on a professional career in which there is a checklist for
literally every conceivable problem scenario. There are many professions
like that in today’s world. That’s not the world of a data professional.
Data professionals are much more like the creative classes of writers,

https://doi.org/10.1007/978-3-030-33578-6_2
artists, composers, and architects. There are many shades of gray in this
profession.
(Mahadevan 2018: X)
2.1 The Market for Big Data Jobs: Trends

and Skills Shortage
At the end of 2018, Gartner’s analysts identified the ten major technology
trends that will affect all industries in 2019 (Gartner 2019) and that can
be ascribed to the following three main areas:
• intelligent (advancement in artificial intelligence): autonomous

things, augmented analytics, artificial intelligence-driven
development;
• digital (virtual and real worlds are blended to create an immersive,
digitally enhanced and connected environment): digital twins,
empowered edge, immersive experience, digital ethics and privacy,
quantum computing;
• mesh (connections between an expanding set of people, business,
devices, contents, and services to deliver digital outcomes): block-
chain and smart spaces.
The diffusion of these technological breakthroughs is shifting the

boundaries between the work tasks performed by people and those per-
formed by machines and algorithms, with consequent changes in the labor
market structure. Some new jobs are emerging, while others are expected
to become redundant. However, a study conducted by the World
Economic Forum (2018) revealed that the increased demand for new
roles will offset the decreasing demand for others. Specifically, since 85
percent of the global companies surveyed indicated their intention to
increase their use of big data analytics, big data jobs are gaining even more
importance over the next years.
Recent studies and projections have been carried out in the US labor
market. Research conducted by Burning Glass Technologies in collabora-
tion with IBM and the Business-Higher Education Forum (Burning Glass
Technologies 2017) indicated that in 2015 the number of data science
and analytics jobs were over 2,350,000, and by 2020 this number was
expected to rise by 15 percent, with nearly 364,000 new job postings. A
2 HOW BIG DATA CREATES NEW JOB OPPORTUNITIES: SKILL PROFILES… 23
similar trend characterizes the European Union: in 2018 the number of

big data professionals in the EU28 reached 7.2 million, corresponding to
3.4 percent of the total workforce, with an increase of 8.4 percent over the
previous year (The Lisbon Council and International Data Corporation
2019). These professional profiles show a significant shortage (LinkedIn
2018) and are among the most difficult to recruit: in the US on average,
these jobs remain open for 45 days, five days longer than the market aver-
age (Burning Glass Technologies 2017). This skill gap in Europe grew in
2018 by 18 percent, reaching approximately 571,000 unfilled positions in
the EU28 (The Lisbon Council and International Data Corporation
2019). Moreover, a study by APEC showed that India, South Korea,
Japan, China, and Australia are the key countries that are investing in big
data in the Asia Pacific, but at the same time underlined the skills shortage
for data science and analytics jobs in this geographic area (APEC 2017).
The demand varies significantly across industries. The sectors that pres-
ent the highest demand for big data jobs are finance and insurance, profes-
sional services, IT, wholesale and retail, and manufacturing. The industries
with the lowest presence of big data professionals are construction, trans-
port, and health care (Burning Glass Technologies 2017; Datalandscape
2017; PwC 2017).
But what are the big data specialist profiles most sought in the market?
What are their main differences in terms of tasks and skills required? This
chapter provides a snapshot of the most in-demand big data jobs, contrib-
uting to clarify their boundaries. It also delves into the main characteristics
of the specific professional profiles that have received increasing attention
in recent years, namely data scientists and data/business analysts, due to
their high contribution in extracting value from data and consequently
due to the effectiveness of the decision-making process.
2.2 Professional Roles in the Data Science Field

In the last decade, both the academic and professional communities have
provided guidance to distinguish emerging data science and analytics jobs.
Even though in this fast-evolving field these professional profiles change
constantly with new roles and technical skills requirements, proving a
description of their evolution and of their main tasks helps to: (i) improve
the understanding of the differences among these jobs that should be part
of the data science team of the company, and consequently to better orient
recruiters’ search in the labor market; and (ii) promote their skills and
career development following their evolution over time.
Considering different contributions from experts and scholars operat-
ing in the data science and analytics domain (such as Burning Glass
Technologies 2017; Costa and Santos 2017; Davenport 2014; Marr 2018;
McKinsey Global Institute 2016; NewVantage Partners 2019), we have
identified the most critical jobs based on their frequency of citation in the
studies mentioned above and on their impact in supporting companies to
deal with the complexity of big data and to extract value from it.
One of the new executive roles created by the digital transformation is
that of chief data officer (CDO) – also defined as data analytics officer –
who has the responsibility to help the business understand the value of big
data; defining and implementing a big data strategy at the company level;
and ensuring that the data is correct, secure, and governed properly, espe-
cially with regard to privacy and ethics issues. Gartner (2018) defined
CDOs as “accountable and impactful change agents,” since they are asked
to lead their organizations toward data-driven transformation initiatives.
As the company progressively becomes aware of the opportunity offered
by big data for its business, the appointment of the CDO as a member of
the executive board, reporting directly to the chief executive office, suc-
cessfully contributes to the diffusion of a data-driven culture (see Sect.
1.4). In the last Big Data and AI Executive Survey (2019), it emerged that
67.9 percent of the companies involved have appointed a CDO, up from
just 12.0 percent in 2012, and that 48.1 percent have ascribed to this role
primary accountability for data, even though it is still ill defined. In this
regard, as shown in the last Gartner Chief Data Officer survey (Gartner
2018), the CDOs’ responsibilities include data management, analytics,
data science, ethics, and digital transformation. In summary, their priori-
ties are to: (i) identify, communicate, and pursue business opportunities
using available data; (ii) promote a data-driven culture, and a common
language and practices about data and analytics across the organization,
especially where data is in silos; (iii) increase the transparency about the
types of data collected and their use where data and information is in silos;
(iv) deal with the security and ethical implications of big data; and (v)
guarantee data quality standards. Experts in the big data labor market
maintain that CDOs are required to combine a technical background, in
terms of expertise or familiarity with the major big data technologies and
solutions as well as modeling techniques, with strong behavioral compe-
tencies necessary to lead the team and communicate effectively.
The other highly sought big data profiles can be classified according to
the two main processes described in Chap. 1, namely data management
and data analysis. The remainder of this section will illustrate the roles
dealing with data management that have responsibility for capturing, stor-
ing, integrating, transforming, and retrieving data, whereas the following
sections will provide a detailed description of the profiles more deeply
involved in data analysis, namely data scientist and data analyst/busi-
ness analyst.
Four key main roles are crucial for the development and maintenance
of infrastructure for a company’s data ecosystem: data architect, database
architect, database developer, and data engineer.
The data architect – also conceived as the contemporary data modeler –
creates blueprints for data management systems. After assessing the com-
pany’s potential data sources, both internal and external, architects design
a plan to integrate, centralize, protect, and maintain these sources. They
are requested to have expertise with requirement analysis, platform selec-
tion, technical architecture design, application design and development,
testing, and finally deployment. Among the most required skills are solu-
tions architecture, relational database management systems or founda-
tional database skills, cloud computing, software development, SQL,
NoSQL, software development life cycle, data governance, data visualiza-
tion, data mining, data analysis, and data migration experience. Concerning
behavioral competencies, this role should demonstrate primarily analytical
problem solving, communication, and leadership in directing and advising
the team of data modelers, data engineers, database administrators, and
junior architects. The role of data architect cannot be confused with
another professional role – database architect – who is in charge of the
design of the database architecture, meaning that he/she develops model-
ing strategies to ensure that the database is secure, scalable, and capable of
reliable performance. Among the main tasks of this role are developing
database solutions to store and retrieve company information; installing
and configuring information systems to ensure functionality; and analyz-
ing structural requirements for new software and applications. After the
database architecture is designed, a database architect works with other
information technology professionals such as programmers, system admin-
istrators, analysts, software engineers, and database administrators to
implement the database.
If the database architect is more responsible for the design of the data-
base’s architecture to meet an employer’s needs, the database developer,
also known as database administrator, is more responsible for day-to-day

operations and necessary infrastructure (Bayern 2019), namely creating
and implementing computer databases, developing new applications for
databases or modifying legacy applications to work with a database setup,
and maintaining and securing the system. Often in many organizations the
two roles of database architect and administrator are actually performed
by the same person. The main tasks of the database administrator can be
summarized as follows: (i) modify and update existing databases, expand-
ing their capacity; (ii) design and develop new databases meeting the
company’s needs; (iii) troubleshoot database issues, running perfor-
mance-testing procedures to ensure the proper operations of a database;
(iv) develop database documentation, ensuring that operational manuals
and supporting documentation includes information on changes added
to the database; and (v) assign accessibility for users and monitor usage.
The required technical skill set encompasses:
• applying technical design and development skills to the creation of

database programs;
• analyzing existing databases and data needs of the company to
develop effective systems;
• using knowledge of specific programming languages and codes to
perform specific tasks;
• following implementation processes for new databases; and
• troubleshooting and providing solutions for any bugs in new data-
base applications.
Structured Query Language is the primary language that database

developers use. In addition, they can be asked to adopt language program-
ming skills in C, C++, C#, or Java. Besides these technical skills, online job
posts highlight the importance of the soft skills “critical thinking,” to
translate business needs into database solutions, and social competencies,
since a database developer works with different organizational roles to
ensure that all databases are functioning as intended and to develop tech-
nical projects.
The primary activity of data engineers is data conversion and treatment
in order to keep data available and ready for data scientists and analysts, so
they can spend more of their time running actual analysis rather than
implementing infrastructure. Data engineers’ tasks include: i) gathering
and processing raw data at scale (including writing scripts, web scraping,
calling APIs, writing SQL queries, etc.); and ii) processing unstructured
data into usable data formats, performing data modeling and design. As
indicated by Granville (2014), their activities can be summarized into
three actions: extracting, loading, and transforming data. Companies look
for data engineers who have extensive experience in building and optimiz-
ing data pipelines, therefore in manipulating data with SQL, T-SQL, R,
Python, Spark Hadoop, Hive, Oozie, Apache Flume, and Pig (Burning
Glass Technologies 2017; Burns 2016).
Table 2.1 reports the key responsibility of the discussed professional
roles, highlighting some areas of overlapping especially between data
architect and data engineer.
From the description above, derived primarily by the community of
practitioners in this field, it emerges that these roles are clearly defined
concerning their main activities and technical requirements. However,
there is a lack of academic work in understanding the contribution pro-
vided by the data architect, database developer, and data engineer to the
business performance and limited attention devoted to the analysis of the
behavioral competencies that distinguish these professionals.
Table 2.1 Summary of the main responsibilities of data architect, database archi-
tect, database administrator, and data engineer
Responsibilities/role Data Database Database Data
architect architect administrator engineer
Data warehousing solutions ✔

Extraction, transformation, and ✔ ✔
loading
Data architecture development ✔
Data modeling ✔ ✔
System development ✔
Database architecture testing ✔
Installing data warehouse solutions ✔
Designing of database to meet ✔
scalability, security, performance,
and reliability requirements
Managing of data structure ✔
Maintaining the database’s security ✔
Organizing database recovery and ✔
backup procedures
Monitoring database performance ✔
2.3 Data Scientist: Is It Still the Twenty-First

century’s Sexiest Job?
When in 2008 the influential data scientists D.J. Patil and Jeff
Hammerbacher shared their experience respectively in LinkedIn and
Facebook and discussed what to call the members of their teams, coining
the term “data scientist,” they probably did not imagine that ten years
later that role would become the most sought-after profession in the world
(Costa and Santos 2017; Davenport and Patil 2012).
According to Glassdoor’s 50 Best Jobs in America for 2019, data scien-
tist is the best job in America for the fourth year in a row based on the
number of open positions, median base salary, and job satisfaction.
LinkedIn’s list of most promising jobs of 2019 confirmed this trend, indi-
cating data scientist as the most in-demand role in the US, and the job site
Indeed reported in 2019 that job postings have increased by 256 percent
since 2013. Similarly, in Europe and the Asia Pacific region, the demand
for data scientists is characterized by continuous growth and, conse-
quently, by a high skills shortage (APEC 2017; Big Cloud 2019). This
evidence confirms the prediction made by Davenport and Patil in 2012
when they defined data scientist as “the sexiest job of the 21st century.”
Due to the rapid technological changes introduced in the data science
field in the last decade, the role of data scientist has evolved dramatically
in recent years, generating a lack of consensus about what exactly this
individual does and what skills are needed. Data scientists are conceived as
a hybrid of the following five traits (Davenport and Patil 2012;
Davenport 2014):
• Hacker: capability to code (the 2019 Big Cloud’s report on data

scientist salaries and jobs in Europe shows that Python is the most
popular modeling coding language, used by almost 60 percent of the
respondents, followed by R, SQL, JAVA, and Matlab) and familiarity
with big data technology architectures (e.g., Hadoop/MapReduce).
• Scientist: capability to design experiments and to collect, analyze,
and describe findings from data.
• Trusted adviser: capability to communicate and manage relationships
in order to support executives in the decision-making process.
• Quantitative analyst: capability to conduct statistical and visual analy-
sis, also of unstructured data, and to master the machine learn-
ing approach.
• Business expert: knowledge about the specific business enables the

data scientist to generate hypotheses and test them quickly and to
provide valuable solutions to key functional problems.
Deriving a definition of data scientists through a Delphi study that

involved several practitioners, Vidgen et al. (2017: 634) highlighted other
attributes of data scientists. Specifically they “must be curious, problem-
focused, able to work independently, and capable of co-creating and com-
municating stories to the business that form the basis for actionable insight
into data.”
Thus, a first distinguishing trait of data scientists that explains the use
of the term “scientists” is to have an inquisitive nature or an intense curi-
osity: they ask new open-ended questions and formulate hypotheses to be
tested to understand the meaning of data and generate innovative and
practical solutions to business problems. Research conducted by Harris
and Mehrotra (2014) highlighted the creative and entrepreneurial mind-
set that characterizes data scientists: they often address original problems
and require a high level of autonomy in testing their hypotheses and
implementing improvements derived from their discoveries. When in
2006 Jonathan Goldman, former data scientist at LinkedIn, created
People You May Know, he demonstrated his inquisitive nature starting
from a question: How do users build their network of contacts in LinkedIn?
Studying data on social networks already available by the company, he
formulated theories and tested some hypotheses, trying to discover mod-
els that could predict which type of network a specific user profile would
have built based on his/her features. He was in charge of the implementa-
tion of a recommendation engine that enabled LinkedIn to support users
to get in contact with profiles that the model suggested as of potential
interest for the user. The case illustrates how data scientists are motivated
in their job by the opportunity to be engaged in meaningful projects that
enable them to work with interesting data and original problems
(Davenport and Patil 2012; Vidgen et al. 2017). However, recent surveys
reveal that data scientists spend most of their time cleaning and organizing
data rather than mining or modeling it, and that this activity of data prepa-
ration is viewed as the least enjoyable part of their work (CrowdFlower
2017). This explains the low company tenure of data scientists who on
average stay one year or less in the same organization (Big Cloud 2019).
From the “data” side, Granville (2014) defined their work based on
three main activities: discovering good data sources, accessing data, and
distilling information from data to provide valuable insights for the

decision-making process. This latter activity implies data exploration,
cleaning, summarization and analysis, the development of algorithms that
process data automatically, and data presentation. This role is also labelled
“advanced analyst” (Burning Glass Technologies 2017) in order to dif-
ferentiate it from that of the business or data analyst. Data scientists not
only clean, analyze, and visualize data, just like data analysts, but they have
a deeper expertise in these activities and are able to train and optimize
machine learning models.
Harris and Mehrotra (2014) illustrated the main differences between
these two profiles considering different criteria, such as:
• the type of data they work with: analysts use more structured and
semi-structured data, whereas scientists also deal with unstruc-
tured data;
• the tools deployed: analysts use statistical and modeling techniques,
whereas scientists adopt mathematical languages, machine learning,
natural language processing, and open-source tools that access and
manipulate data on multiple servers like Hadoop; and
• the nature of the work: report, predict, prescribe, and optimize ver-
sus explore, discover, and investigate.
Thus, while an analyst may be able to describe trends and translate

those results into business terms, the scientist will raise new questions and
will be able to build models to make predictions based on new data. In
order to do so, the knowledge base expected from a data scientist exceeds
that for a computer scientist or a statistician. Their role is different from
that of computer scientists since they have a much stronger background in
computational statistics, experimental design, sampling, and Monte Carlo
simulations. It also differs from statisticians since data scientists’ activity is
not limited to data analysis but also encompasses the implementation of
predictive algorithms that process data automatically and requires them to
master sophisticated statistical techniques that will be applied to manipu-
late and extract value from large-volume, fast-flowing, and unstructured
data (Granville 2014). Thus, data scientists in comparison to data analysts
are more technical in nature, dealing with programming, computational
complexity, algorithm design, data mining, distributed architecture, and
artificial intelligence (Verma et al. 2019).
This skill set sought in data scientists is confirmed by LinkedIn (2019),

which indicates data mining, data analysis, Python, R, and machine learn-
ing as the most requested skills. LinkedIn profiles have been used by aca-
demic researchers to explore how data scientists define their knowledge
base and skill profile. The top ten self-reported featured skills of data sci-
entists confirm those illustrated above: data analysis, Python, R, SQL,
machine learning, data mining, research/statistics, SPSS, SAS, and statisti-
cal modeling (Ecleo and Galido 2017). Since big data requires the analysis
of a huge amount of data in text, video, or image formats, data scientists
are requested to be familiar with at least some of the analytical approaches
for unstructured data, such as natural language processing for extracting
meaning from text. Its application is particularly valuable in investigating
customers’ opinion on products or brands (Davenport 2014).
Table 2.2 reports a summary of the technical skills of data scientists
most frequently mentioned in academic studies.
Besides the technical knowledge and skills, the different studies on data
scientists’ profiles are devoting attention to those soft or behavioral com-
petencies that they should manifest to work collaboratively within the dif-
ferent parts of the organization and to deliver the results of their work
effectively. Table 2.3 reports the main studies that have described the soft
skills of data scientists.
Table 2.2 Technical skills of data scientists identified in scholarly articles

Data scientists’ technical skill set Sources
Query databases (e.g., SQL) De Mauro et al. (2016); Harris et al.

(2013); Kandel et al. (2012)
Work with and manage big data (e.g., data Costa and Santos (2017); De Mauro et al.
mining process, extract information, and (2016); Harris et al. (2013); Kim and Lee
identify patterns) (2016)
Data cleaning, transformation, and Costa and Santos (2017); Kandel et al.
processing (e.g., apache Hadoop) (2012)
Identify correlation and causation among Dhar (2013)
data
Identify patterns, trends, and opportunities Costa and Santos (2017); De Mauro et al.
(2016); Harris et al. (2013)
Apply statistical methods to analyze data Costa and Santos (2017); De Mauro et al.
(e.g., Bayesian/Monte Carlo statistics, (2016); Harris et al. (2013); Kim and Lee
classical statistics) (2016); Lee and Han (2016)
Validate conclusions Costa and Santos (2017)
Table 2.3 Soft skills of data scientists identified in scholarly articles

Data scientist’s soft skill set Sources
Curiosity Costa and Santos (2017); Davenport and Patil (2012);

Vidgen et al. (2017)
Creative thinking Davenport and Patil (2012); Kim and Lee (2016)
Exploring new ideas Harris and Mehrotra (2014)
Problem solving Kim and Lee (2016); Shirani (2016)
Critical thinking Shirani (2016)
Entrepreneurial attitude Costa and Santos (2017); Harris and Mehrotra (2014)
Working independently Vidgen et al. (2017)
Self-motivation Kim and Lee (2016)
Adaptability Kim and Lee (2016)
Business acumen Costa and Santos (2017); Davenport and Patil (2012)
Understanding business De Mauro et al. (2016)
context
Strategic thinking Kim and Lee (2016)
Communication Costa and Santos (2017); Davenport and Patil (2012); Kim
and Lee (2016); Shirani (2016); Verma et al. (2019)
Teamwork Shirani (2016); Verma et al. (2019)
Customer orientation Kim and Lee (2016)
Leadership Kim and Lee (2016); Verma et al. (2019)
As previously highlighted, personal capabilities frequently associated

with this role are those related to the generation of innovative solutions,
such as curiosity, creative thinking, problem solving, and critical thinking.
Moreover, as discussed above, data scientists are characterized by a growth
mind-set; thus they are motivated by interesting and original problems,
which enable them to set challenging goals and undertake a learning path.
They are also required to be adaptable. This means being able to embrace
change and find alternative ways to address the problem, being comfort-
able in dealing with uncertainty and unstructured activities, and learning
from failure. Another set of capabilities that seems to distinguish data sci-
entists is that related to business acumen: understanding the business envi-
ronment and strategic thinking. Data scientists should wear the hat of
business experts and advisors, reporting the metrics, such as the return on
investment, of the identified solutions in order to quantify their impact on
the business. In doing so, they cannot only sit in front of their computers
immersed in data in the hope of finding something interesting. They
should be fully engaged in the business and work closely with executives
to acquire a deep domain knowledge and to provide insight on a strategic
level (Kim and Lee 2016). They also require autonomy and an entrepre-
neurial mind-set to explore and take action for business opportunities, to
ask the right questions for the specific business, and to be responsible for
the implementation of the analytical solutions (Stadelmann et al. 2019).
Moreover, data scientists should not carry out their activity being isolated
in silos; instead they should be connected with important stakeholders and
the other big data roles, specifically data engineers and analysts (Redman
2019). Consequently, among the skills aimed at favoring relationship
management, the different studies have indicated teamwork, leadership,
and customer orientation. Among the relational competencies identified,
experts and scholars acknowledge that data scientists should communicate
their findings in a way that can be understood by people outside their
field. This competency is usually associated with the capability of using
data to tell a story supported by visualization tools. Big data analyses often
require reporting of data in visual formats. More sophisticated technolo-
gies for displaying data in dashboards and visual analytics have been devel-
oped in recent years, with a consequent need to support decision makers
in their interpretation. Insufficient or ineffective communication has been
identified as one of the main factors that explains the failure of a big data
project (Davenport 2014).
2.4 Data Analyst Versus Business Analyst

Demand for knowledgeable data analytics professionals will show a trend
of dramatic growth (World Economic Forum 2018). The main differences
between data scientist and business/data analyst have been delineated in
the previous section. However, there is still confusion on what distin-
guishes a data analyst from a business analyst. They both work with data
analytics to help others make better data-driven decisions, and often these
terms are used interchangeably; however, their job requirements present
some differences.
Business analysts are more concerned with the business implications of
the data and the actions that should be implemented based on their analy-
sis. They usually leverage the work of data scientists to orient the decision-
making processes toward solutions. Thus, their main responsibility falls in
the area of business advisory (driving decision making, influencing busi-
ness units’ strategies, reporting strategic insights to partners) and project
management (analyzing business needs, communicating the results
achieved) (De Mauro et al. 2016). Daily tasks usually performed by busi-
ness analysts can be summarized as follows (Kunis 2019):
• analyze and elicit business needs about products, services, and proj-
ect requirements, often through conversations with stakeholders;
• define a business case;
• analyze large amounts of complex data to provide the business with
fact-based insights;
• model and specify through documentation the requirements to vali-
date solutions, obtaining the approval of all relevant stakeholders
and ensuring that they meet essential quality standards.
A recent study that explored the skill sets required for analytics posi-
tions underlined that business analysts require domain-specific knowledge,
and the statistical package most frequently adopted consists primarily of
Excel (Verma et al. 2019).
Data analysts are instead more focused on data, and they create reports
and visualizations to explain what insights the data is hiding (De Mauro
et al. 2016). Thus, they turn numbers into stories, but spend more time in
a silo in comparison to the business analyst. Their activity includes:
• writing SQL queries to extract data from the data warehouse, clean-
ing and organizing raw data;
• performing recurring and ad hoc quantitative analysis to find trends
in the data and to support day-to-day decision making. For instance,
they work with customer-centric algorithm models and tailor them
to each customer;
• translating data into visualizations and metrics, generating reports,
and creating and improving dashboards to help the company inter-
pret and make decisions with the data;
• presenting the results of a technical analysis to external clients or
internal teams.
A restricted group of studies, both from academia and practitioner

fields, has attempted to detect behavioral competencies of data/business
analysts. A study conducted on job ads from Fortune 500 corporate web-
sites suggested that data/business analysts are required to possess inter-
personal skills such as communication and self-motivation, as well as
flexibility, customer orientation, planning, and leadership (Lee and Han
2016). Another study conducted on a sample of business analysts identi-

fied as relevant soft skills the ability to challenge, communication, confi-
dence, negotiation, and problem solving (Paul and Tan 2015).
Communication skills are often emphasized as important abilities (De
Mauro et al. 2016; Verma et al. 2019) since these individuals are required
to explain results to a non-expert audience, and they play an intermediate
role between organizations and clients. Similar competencies are empha-
sized by practitioners, who suggest that data/business analysts need to
score high levels of communication skills, such as negotiation, since they
constantly interact with different organizational areas and clients and raise
any potential risks that could affect the organization (Egeland 2015).
2.5 Future Challenges for the Big Data

Profession
This chapter has provided a description of the big data professional pro-
files most currently sought by employers, delineating their jobs and their
skill profile not only in terms of technical requirements but also illustrating
the behavioral competencies relevant to successfully perform in their role.
Besides the misalignment between the demand and the availability of these
roles in the labor market, companies will face several challenges in hiring
and retaining these professionals.
The first issue concerns the proliferation of new job titles in the big data
labor market that can be explained by a deeper specialization of the knowl-
edge and skills necessary to use the new technologies, especially in the
artificial intelligence domain and the more sophisticated analytics tech-
niques. Tasks that were usually included in the job description of data
scientists and analysts are currently performed by a specialized role. This is
the case for instance with data visualization skills, whose importance in
supporting the decision-making process has led to the definition of a dedi-
cated role, the data visualization specialist, or the case of machine learning,
where specific professionals, like the machine learning engineer and scien-
tist, are requested to perform sophisticated programming and work with
complex data sets and algorithms to train intelligent systems. To build
effective big data teams, companies should become aware of the complex-
ity of big data professions and understand the main differences in order to
orient their search, identifying the key roles they need to put together in
the big data team, rather than looking for a single role that is expected to
embrace several knowledge domains.
A second challenge related to the increasing variety of specialized big

data jobs is to identify metrics aimed at measuring their individual perfor-
mance and the contribution they generate for the business. Thus, how
does the professional role improve the decision making in the organiza-
tion? For each profile specific key performance indicators should be devel-
oped in order to monitor the value it is generating for the company, and
in so doing supporting a data-driven culture. For instance, some metrics
can be related to the assessment of the cost savings, the increased revenue
obtained thanks to the algorithm generated, the extent, time, and access
to data sources, the number of solutions proposed, or amount of fraud
eliminated.
Another relevant issue concerns the continuous updating of the com-
petency profiles of these roles that are subject to continuous change and
the sources used to identify the skill set. The analysis conducted in this
chapter shows that – especially in the case of behavioral competencies –
the profile is still ill defined, with a focus primarily on relationship manage-
ment skills such as communication and teamwork. The complexity of
these roles may require a broader set of emotional, social, and cognitive
competencies to be performed effectively, as anticipated in Chap. 1.
Moreover, academic experts have mainly focused their attention on the
analysis of behavioral competencies of data scientists, devoting less atten-
tion to the other professional profiles. As a consequence, soft skills of the
other job families are derived from practitioners mainly considering the
employers’ job requirements that are limited to a few soft skills, primarily
the social ones. Finally, the research conducted by professional and aca-
demic experts has primarily relied on opinions, which are subject to poten-
tial bias. Indeed, data on the skill set of professionals in the big data field
is collected through practitioners, for instance adopting Delphi methodol-
ogy (Vidgen et al. 2017) or analyzing web-based job postings (De Mauro
et al. 2016; Verma et al. 2019), or consulting the profiles published in
LinkedIn (Ecleo and Galido 2017). A more in-depth investigation of the
actual competencies activated by these role holders while they are per-
forming their jobs is required to provide a more complete and concrete
representation of the set of behaviors that are necessary to achieve superior
performance. The following chapters will provide insights on this spe-
cific issue.
References
APEC Human Resource Development Working Group. (2017, July). Data science
and analytics skills shortage: Equipping the APEC workforce with the competencies
demanded by employers. Retrieved from https://www.apec.org/Publications/
2017/11/Data-Science-and-Analytics-Skills-Shortage
Bayern, M. (2019, July 12). Database administrator vs. database architect: What’s
the difference?. Retrieved from https://www.techrepublic.com/article/
database-administrator-vs-database-architect-whats-the-difference/
Big Cloud. (2019, January). Data science salary report 2019 Europe. Retrieved
from https://bigcloud.io/salary-survey-apac/
Burning Glass Technologies. (2017). The quant crunch: How the demand for data
science skills is disrupting the job market. Retrieved from https://www.bhef.
com/publications/quant-crunch-how-demand-data-science-skills-disrupting-
job-market
Burns, E. (2016, August 30). The data engineer is one of the most important jobs in
big data. Search Business Analytics.
ness in the European e-competency framework and the skills framework for the
information age. International Journal of Information Management,
37(6), 726–734.
CrowdFlower. (2017). 2017 Data scientist report. Retrieved from https://www.
figure-eight.com/download-2017-data-scientist-report/
Datalandscape. (2017). The European data market study: Final report. Retrieved
from http://datalandscape.eu/study-reports/european-data-market-study-
final-report
Davenport, T. H. (2014). Big data at work: Dispelling the myths, uncovering the
opportunities. Boston: Harvard Business Review Press.
century. Harvard Business Review, 90(October), 70–76.
De Mauro, A., Greco, M., Grimaldi, M., & Nobili, G. (2016, June 15–17). Beyond
data scientists: A review of big data skills and job families. International Forum
on Knowledge Asset Dynamics 11th, Proceedings of IFKAD, 2016, Towards a
New Architecture of Knowledge: Big Data Culture and Creativity, Dresden
(Germany), 1844–1857.
Dhar, V. (2013). Data science and prediction. Communications of the ACM,
56(12), 64–73.
Ecleo, J. J., & Galido, A. (2017). Surveying LinkedIn profiles of data scientists:
The case of the Philippines. Procedia Computer Science, 124, 53–60.
Egeland, B. (2015, August 27). 5 key soft skills for IT business analysts. BAtimes.
Gartner. (2018). 3 top take-aways from the Gartner chief data officer survey.
Gartner, January 2019.
Gartner. (2019, February 18). Gartner identifies top 10 data and analytics technol-
ogy trends for 2019. Gartner press releases.
Glassdoor. (2019). 50 best jobs in America for 2019. Retrieved from https://www.
glassdoor.com/about-us/bestjobs2019/
Granville, V. (2014). Developing analytics talent. Becoming a data scientist.
Indianapolis: Wiley.
Harris, J. G., & Mehrotra, V. (2014). Getting value from your data scientists. MIT
Sloan Management Review, 56(1), 15–18.
Harris, H. D., Murphy, S. P., & Vaisman, M. (2013). Analyzing the Analyzers. An
introspective survey of data scientists and their work. Sebastopol: O’Reilly Media.
Indeed. (2019, March 14). Top 5 tech skills for data scientists. Retrieved from
https://www.beseen.com/blog/talent/data-scientist-skills/
Kandel, S., Paepcke, A., Hellerstein, J. M., & Heer, J. (2012). Enterprise data
analysis and visualization: An interview study. IEEE Transactions on
Visualization and Computer Graphics, A18(12 December), 2917–2926.
Kunis, L. (2019, January). Business analyst vs. data analyst. Springboard.
Lee, C. K., & Han, H. J. (2016). Analysis of skills requirement for entry-level
programmer/analysts in fortune 500 corporations. Journal of Information
Systems Education, 19(1), 17–27.
LinkedIn. (2018). LinkedIn Workforce Report. Retrieved from https://economic-
graph.linkedin.com/resources/linkedin-workforce-report-december-2018
LinkedIn. (2019, January 10). The most promising jobs of 2019. Retrieved from
https://blog.linkedin.com/2019/january/10/linkedins-most-promising-jobs-
of-2019
Mahadevan, M. (2018). Data professional at work. New York: Apress.
Marr, B. (2018, May 9). The 6 top data jobs in 2018. Forbes.
McKinsey Global Institute. (2016, December). The age of analytics: Competing in
a data-driven world. Retrieved from https://www.mckinsey.com/business-
functions/mckinsey-analytics/our-insights/the-age-of-analytics-competing-in-a-data-
driven-world
NewVantage Partners. (2019). Big Data and AI Executive Survey 2019. Executive
Summary of Findings.
Paul, D., & Tan, Y.L. (2015). An investigation of the role of business analysts in IS
development. ECIS 2015 Completed Research Papers. Paper 142.
PwC. (2017). Investing in America’s data science and analytics talent. Retrieved
from https://www.pwc.com/us/en/library/data-science-and-analytics-
skills.html
Redman, T.C. (2019, May 16). Do your data scientists know the “why” behind
their work? Harvard Business Review.
Shirani, A. (2016). Identifying data science and analytics competencies based on

industry demand. Issues in Information Systems, 17(4), 137–144.
Stadelmann, T., Stockinger, K., Bürki, G. H., & Braschler, M. (2019). Data
Scientists. In M. Braschler, T. Stadelmann, & K. Stockinger (Eds.), Applied
data science: Lessons learned for the data-driven business (pp. 31–45).
Cham: Springer.
The Lisbon Council and International Data Corporation (IDC). (2019). Data as
the Engine of Europe’s Digital Future: The European Data Market Monitoring
Tool Report Update 2018 – Second Report on Policy Conclusions.
Verma, A., Yurov, K. M., Lane, P. L., & Yurova, Y. V. (2019). An investigation of
skill requirements for business and data analytics positions: A content analysis
of job advertisements. Journal of Education for Business, 94(4), 243–250.
261(2), 626–639.
https://www.weforum.org/reports/the-future-of-jobs-report-2018
CHAPTER 3
Emotional and Social Intelligence

Competencies in the Digital Era
Abstract It is widely acknowledged that emotional intelligence is a crucial

component in organizations. It has been proved that leaders and employees
who are emotionally intelligent are more efficient, creative, and make better
decisions. Although decades of studies in different settings have analyzed
emotional intelligence in a variety of roles, there is still a limited application
concerning new job profiles such as data professionals. In a world which is
rapidly changing because of technological and digital innovation, it is timely
to analyze not only the technical skills of these new emerging job profiles but
also their behavioral competencies. This chapter aims to first delineate the
main characteristics of emotional intelligence. Second, it provides an in-
depth clarification of the developed competency framework used to detect
the behavioral competencies of big data professionals. Finally, it offers
insights about how soft skills are considered to be as crucial as technical skills
by the labor market despite the difference between machine and human skills
that seem to draw attention to technological and data driven competencies.
Keywords Emotional, Social, and Cognitive competencies (ESCs) •

Competency-based human resource management • Competency model •
Digitalization
The study of competencies opens the door to insights about humans and
human talent, and potential applications for their development.
(Boyatzis 2009: 764)

https://doi.org/10.1007/978-3-030-33578-6_3
3.1 Emotional Intelligence and Data Driven

Organizations
Every day we experience nearly 500 emotional experiences. We usually
perceive only a fraction of them, but they influence how we interact with
other people, how we perceive events, and how we make decisions, and
they eventually impact our job performance. Intense discussions with col-
leagues, personal loss, expectations or frustrations, and even favorable
events could alter our own feelings and those of others.
Since the beginning of the 2000s, Google has been concerned about
the well-being of its employees. Why was a data-mining giant like Google
interested in whether its employees were happy at work or unhappy?
Chade-Meng Tan was an engineer hired by Google in 2000, employee
number 107. He was one of the first to improve the quality of the site’s
search results and he played a key role in the launch of mobile search. In
2007, with a team of leading experts in mindfulness, neuroscience and
emotional intelligence, he developed an internal course for fellow Google
employees called “Search Inside Yourself” (SIY). In those days, Google
allowed its employees to spend 20% of their working time on whatever
side project they wanted. It is because of this 20% that Gmail and Maps
were developed. Chade-Meng Tan had the idea to offer a curriculum for
emotional intelligence because “emotional intelligence can help people
succeed” (Meng Tan 2012). Rapidly, the course became so popular that
was attended by more than 1000 employees and was featured on the front
page of the Sunday Business section of The New York Times. The main
purpose of the course was to help individuals find a balanced awareness of
what’s happening around them in a way that diminishes stress and frustra-
tion both at work and in their personal lives. SIY evolved from a medita-
tion program into a course on emotional intelligence and that was the
turning point: “emotional intelligence even affects the work effectiveness
of engineers. Among the top six characteristics that distinguish top engi-
neers from average engineers, only two are cognitive; four have to do with
emotional competencies” (Meng Tan 2012). To see whether Meng Tan’s
words matched with reality, Google collected feedback and psychometric
data. They found that some of the outcomes experienced by the partici-
pants were more retention, more promotions, a better work environment,
more empathy, less self-rumination and self-perceived stress, and more
constructive self-criticism.
3 EMOTIONAL AND SOCIAL INTELLIGENCE COMPETENCIES… 43
In this chapter, we will briefly describe the model of Emotional

Intelligence based on the competencies that enable individuals to demon-
strate an intelligent use of their emotions in managing themselves and
working with others to be effective at work. A brief history of the concept
of Emotional Intelligence will be presented and the development of the
competency framework adopted in the following chapter (Chap. 4) will be
explained. Finally, implications for a theory of behavioral competencies in
a changing digitalized work environment will be discussed.
3.2 Competency-Based Approach and Emotional

Intelligence: The Importance in the Workplace
It was David McClelland who in 1973 advanced the concept of compe-
tence as a basis for identifying what differentiates outstanding performers
from average performers at work. By collecting data from more than thirty
organizations and interviewing executive roles in different sectors, he
showed that a wide range of emotional competencies, rather than a few
sets of cognitive competencies, distinguished top performers from average
ones (McClelland 1998). McClelland paved the way for scholars and prac-
titioners to use a variety of forms of competency-based human resource
management (Boyatzis 2009; Campion et al. 2011; Vos et al. 2015). This
was drawn from the main assumption that competencies are essential dif-
ferentiators of performance. According to McClelland (1973), compe-
tence was more important for success in work and in life than was
intelligence, as traditionally defined and measured by IQ tests. In particu-
lar, McClelland (1994) maintained that “there are alternative combina-
tions of characteristics that lead to success in a particular job” (McClelland
1994, cited by Jacobs 2001: 161). Richard Boyatzis (1982) and Daniel
Goleman (1995) were strongly influenced by the work of McClelland, and
they developed a theory of performance in the work setting. They sug-
gested that “emotional intelligence is observed when a person demon-
strates the competencies that constitute self-awareness, self-management,
social awareness, and social skills at appropriate times and ways in suffi-
cient frequency to be effective in the situation” (Boyatzis et al. 2000).
This stream of research, based on the theory of performance, pays
attention to explaining and predicting the outcome of effectiveness in
different workplace contexts, usually by observing managers and leaders
(Boyatzis 1982; Boyatzis et al. 1995; McClelland 1973; Spencer and
Spencer 1993). This is the so-called “competency” approach.
According to this approach, a behavioral competency is an underlying
characteristic of a person that leads to or causes effective or superior per-
formance. Competencies are hence learned capabilities that lead to effec-
tive or superior performance and are reflected by a set of behaviors that
share a common underlying intent (Boyatzis 2006, 2009). Indeed, the
concept of competency comprehends both action (how an individual
behaves according to a specific situation) and intent (how much effort an
individual has towards something) (Boyatzis 2009).
Specific competencies are identified in four domains: self-awareness,
self-management, social awareness, and relationship management
(Boyatzis 2008; Boyatzis et al. 2000; Goleman 1995, 1998). Self-
awareness is the foundational component, because it is the ability to iden-
tify our own emotions and the effect they have on us and others (emotional
self-awareness). It implies a profound knowledge about our own strengths
and weaknesses that is needed in order to constantly motivate ourselves.
Self-awareness is achieved through an accurate self-assessment (Boyatzis
1982) which allows individuals to see their own personal abilities and limi-
tations through a constant search for feedback, and the capacity to learn
from mistakes (Boyatzis 1982; Goleman 2001). Self-management refers
to “managing one’s internal states, impulses, and resources” (Boyatzis
2016: 288) and it allows individuals to regulate their own emotions and
to identify and prevent emotional triggers. In particular, individuals with
high self-management competencies are able to avoid distress and disrup-
tive feelings such as rumination. Social awareness collects competencies
that refer to knowing and managing emotions in others. Namely, these are
competencies that enable individuals to accurately read situations and
empathize with the emotions of others. It allows individuals to handle
relationships and other’s feelings. On the other hand, relationship man-
agement includes competencies that involve the relationships with others
and the capacity to induce desirable responses in others. The effectiveness
of the relationship skills also depends on the ability to attune ourselves to
the emotions of another person.
Decades of studies conducted by Boyatzis (1982), Goleman (1998,
2001) and colleagues have shown how emotional self-awareness is a pre-
requisite for effective self-management, which conversely could predict
greater social competencies. Figure 3.1 presents the emotional and social
intelligence competency framework.
Fig. 3.1 The emotional and social intelligence competency framework
The contingency model advanced by Richard Boyatzis (1982) main-

tained that performance effectiveness derives from the best fit between the
individual, his or her job demands, and the organizational environment.
Moreover, a bundle of competencies rather than a single one in itself
would have a more significant impact on performance, echoing the insight
that “In life—and particularly on the job—people exhibit these competen-
cies in groupings, often across clusters, that allow competencies to support
one another” (Goleman 2001: 39).
This is not the only approach. Another significant line of research,
opened by Mayer and Salovey, put into question the way to conceive and
derive traditional standards of intelligence (Mayer and Salovey 1997;
Mayer et al. 2000a, b; Salovey and Mayer 1990). Individuals with a high
Emotional Intelligence score were considered to be able to perceive emo-
tions in themselves and others, but also to regulate them in order to reach
a positive status (Salovey and Mayer 1990). Salovey and Mayer’s original
model (1990: 189) described emotional intelligence as the “ability to
monitor one’s own and others’ feelings and emotions, to discriminate
among them, and to use this information to guide one’s thinking and
action”. Hence, their framework (Mayer and Salovey 1997; Salovey and
Mayer 1990) was cognitive oriented, because it was mainly focused on the
mental attitude needed to recognize and interpret emotions (see Goleman
2001 for an in-depth description of the model and its evolution).
So far, studies have tried to delineate the set of competencies that lead
to higher performance in different settings and in a variety of jobs. The
following section will describe the empirical evidence found by extant
literature.
3.3 Emotional Intelligence and Work

Environment
The data documenting the importance of emotional intelligence compe-
tencies for outstanding performance have been collected for more than
two decades. Individuals with a higher emotional intelligence are more
effective at work; they are more creative and better leaders (Fernández-
Berrocal and Extremera 2006; Goleman et al. 2002; Newman et al. 2010;
Joseph et al. 2015; O’Boyle et al. 2011). Moreover, a longitudinal study
on a large sample of graduates showed that emotional intelligence compe-
tencies such as adaptability and teamwork predict career satisfaction and
success (Amdurer et al. 2014). Furthermore, a study conducted by food
service employees from nine different locations of the same restaurant
franchise showed that job satisfaction and performance were highly cor-
related with employees’ emotional intelligence (Sy et al. 2006).
Research has provided empirical insights on those competencies needed
to achieve superior performance at individual and organizational levels
(Dreyfus 2008; Koman and Wolff 2008; Ryan et al. 2009; Zhang and Fan
2013). Indeed, over the last two decades, emotional intelligence compe-
tencies have been found to significantly impact different aspects of the
workplace, such as job performance, job satisfaction and job outcomes (Sy
et al. 2006), low turnover (Wong and Law 2002), company rank (Lopes
et al. 2006), and group performance as well (Koman and Wolff 2008).
Depending on the organizational context and situational factors that
characterize specific jobs, certain combinations of competencies are more
important than others in order to pursue positive performance. For instance,
managing groups and interpersonal sensitivity were two competencies
found among the most effective scientists and engineers of a major US gov-
ernment research center (Dreyfus 2008). The distinguishing competencies
demonstrated by the physician leaders who were part of a top-rated US
academic healthcare institution were empathy, initiative, emotional self-
awareness and organizational awareness (Hopkins et al. 2015). A study on
different sized Italian companies showed that outstanding executives
showed more efficiency orientation, initiative, self-confidence, networking,

oral communications, persuasiveness, systems thinking, and pattern recog-
nition (Boyatzis and Ratti 2009).
Measures of the observed behaviors have been tested and validated by
a rich set of research in the last decades and several approaches have been
found: such as informants’ ratings (360-degree assessments), behavioral
event interviews (Boyatzis 1982; Spencer and Spencer 1993), direct cod-
ing of behaviors from audio tapes of critical incidents or videotapes of
simulations (Boyatzis 2009; Cherniss 2010), and assessment centers
(Thornton and Byham 1982). These methods share the core hypothesis
that behaviors are observed and rated by knowledgeable informants rather
than by oneself (Boyatzis 2016).
Therefore, the literature on behavioral competencies and emotional
intelligence has provided empirical insights into the distinctive competen-
cies that explain outstanding performance in specific professional roles
(see Table 3.1 for some examples).
Despite the large body of research over the last decades that have inves-
tigated the distinguishing behavioral competencies of the best performers
in a variety of roles, so far, limited attention has been devoted to the job
profiles that are emerging in the big data field. As highlighted in Chap. 2,
the competency profile of these roles is still ill defined. Studies have relied
primarily on methodological approaches subject to potential bias such as
asking practitioners’ opinions on the most relevant competencies they
think big data professionals should demonstrate or analyzing employer’s
web-based job postings. The following section will introduce a competency-
based framework that can be adopted to the exploration of the behav-
ioural competencies that characterize professionals in the data science field.
3.4 The Behavioral Competencies Necessary

in Today’s Workplace: The Development
of a Competency Framework
The models and studies presented in the previous section have deeply con-
tributed to an understanding of how emotional intelligence competencies
represent a crucial determinant of better performance in the workplace.
Likewise, the identification of the behavioral competencies needed for a
job allows companies to better orient their search in the labor market.
However, new job profiles have started to emerge within a business
Table 3.1 Examples of professional roles analyzed in terms of performance out-

comes and related behavioral competencies
Authors Professional roles Performance indicators Distinctive behavioral
competencies
Boyatzis Leaders in a large Higher revenue from the Planning, achievement

(2006) US consulting clients and gross margin orientation, self-confidence,
company taking a risky stand, self-
control, adaptability,
conscientiousness, values
learning, networking,
leadership, coaching, empathy,
facilitating learning,
Systems thinking.
Dreyfus Scientists and Higher performance Managing groups and
(2008) engineers working collected through interpersonal sensitivity
at a major US nominations from peers,
government supervisor, and
research center subordinates
Williams School principals Higher performance Self-confidence, achievement
(2008) in a large collected through peers, orientation, initiative,
Midwestern supervisor and teachers’ organizational awareness,
United States nominations. Use of a leadership, teamwork
urban school broader repertoire of
district environmental spanning
strategies
Boyatzis Executives in an Higher performance Efficiency orientation,
and Ratti Italian division of collected through self-confidence, networking,
(2009) a large nomination from peers pattern recognition, systems
multinational and supervisors thinking
Boyatzis Managers in Higher performance Empathy, group management,
and Ratti Italian collected through developing others, oral
(2009) cooperatives nomination from peers communications, and use of
and supervisors concepts.
Ryan European Higher performance Achievement orientation,
et al. managers collected through initiative, teamwork and
(2009) nomination from clients cooperation, and team
leadership
Boyatzis Divisional Number of financial Adaptability and influence
et al. executives in a advisor recruiters whose
(2012) financial services total compensation
company package is entirely based
on commissions for new
cash invested by clients
(continued)
Table 3.1 (continued)
Authors Professional roles Performance indicators Distinctive behavioral

competencies
Zhang Chinese project Meeting project’s overall Emotional self-awareness,

and Fan managers working performance, meeting emotional self-control,
(2013) on large and owner’s requirements, empathy, organizational
complex meeting project’s awareness, cultural
construction multiple goals, understanding and
projects stakeholders’ satisfaction communication
e nvironment that have posed different challenges. New technologies, dif-

ferent innovation processes, and more importantly, Industrial Revolution
4.0 have required organizations to change their structures and hire new
employees. Consequently, existing competency models must be integrated
and take into consideration those individual behaviors that are even more
necessary for operating in todays’ more complex organizational
environment.
Drawing on the extensive contributions provided by the competency-
based approach (Boyatzis 1982; Boyatzis et al. 2000; Goleman 1998;
Goleman et al. 2002; Spencer and Spencer 1993) but also considering the
following:
• the evidence that emerged from several interviews we administrated

(adopting the Behavioural Events Interview technique) (Boyatzis
1998; McClelland 1998) in the period 2015–2017 to several effec-
tive managerial and entrepreneurial roles operating in different
industries;
• the soft skills most needed by employers in today’s workplace (e.g.
Azevedo et al. 2012; LinkedIn 2019; McKinsey Global Institute
2018; QS 2019; Robles 2012; The World Economic Forum 2018);
• the most advanced studies that investigate the effective behaviors
that big data professionals should manifest to perform their job
effectively and to assume a more innovative and entrepreneurial
mindset (see Chap. 2 for a review);
we defined a framework that encompasses thirty-three competencies

grouped into six thematic areas that are defined as follows:
• Awareness. Competencies that allow individuals to understand them-

selves, other people, and the organizational relationships;
• Action. Competencies that allow individuals to realize ideas, plans
and solutions, and to work methodically and with initiative;
• Social. Competencies that allow individuals to interact positively
with other people and that help them to work with others effectively;
• Cognitive. Competencies that allow individuals to analyze and use
information effectively to interpret phenomena or situations;
• Exploratory. Competencies related to activating the processes of
innovation generation;
• Organizational Action. Competencies related to the interpretation
of the competitive environment, the identification of business
opportunities, and the alignment of the individual behaviors to the
organizational goals and priorities.
The Competency Hexagon – as we labeled the framework – is graphi-

cally represented in Fig. 3.2., while Table 3.2 reports in detail the behav-
ioral competencies that contribute to each of the six areas of the Hexagon
with the related definition.
Fig. 3.2 The competency Hexagon

Table 3.2 Competency Hexagon: The thirty-three competencies and related

definition
Area Competency Definition
Awareness Self-awareness Capacity to be in tune with your inner self and being
able to evaluate the impact of emotions on your actions
and work performance, always keeping in mind the
guiding values. It is also the capacity to evaluate your
inner abilities and limits. It is based on the desire to
receive feedback and new perspectives about yourself
and to be motivated by continuous learning and
self-development
Empathy Capacity to sense and accurately understand others’
feelings and perspectives and take an active interest in
their concerns
Organizational Capacity to locate and decipher social networks and
awareness power relations and the ability to understand the
“political” balance in any organization and the guiding
values and unspoken rules that govern the behavior of
its members
Action Efficiency Capacity to perceive input and output relationships and
orientation include the concern for increasing the efficiency of
actions
Achievement Capacity to require high quality standards to try to
orientation constantly improve your results, setting challenging and
measurable goals, and measuring the progress made
Resilience Capacity to recover from adversity and respond to it
positively by using personal resources
Initiative Capacity to act to accomplish something and to take this
action prior to being asked or forced or provoked into it
Change agent Capacity to recognize the need for change, to promote
and manage it
Flexibility Capacity to adapt oneself by modifying one’s behavior
in the face of changes, unexpected circumstances or
different situations
Self-control Capacity to dominate emotions and impulses even in
situations of stress or difficulty
Accuracy Capacity to develop the activities with precision and to
check several times
Risk taking Capacity to take a risk or to carry out an activity with an
uncertain outcome
Risk The capacity to identify in advance possible negative
management impacts of uncertain activities and contain losses
Collection of Capacity to look for the correct information
information
(continued)
Social Persuasion The capacity to convince other people of the value of

your point of view and to get their support
Conflict The capacity to induce the parties in conflict to have a
management dialogue and identify solutions in which everyone can
recognize themselves
Teamwork Capacity to be collaborative and available to the group,
to induce others to engage actively and enthusiastically
in the common cause, to reinforce the team spirit and
encourage the participation of all members
Developing Capacity to stimulate, support and provide resources for
others the improvement and growth of other people
Networking Capacity to create, maintain, and use personal
relationships to achieve goals
Leadership Capacity to lead others and trigger phenomena
involving emotional resonance, to instill a sense of pride
and inspire people through a compelling vision, and to
bring out their best aspects
Customer Capacity to understand other people’s needs and pay
focus attention to their satisfaction
Cognitive Systems The capacity to break down complex problems and
thinking understand cause-and-effect relationships between the
parties
Diagnostic Capacity to conduct an accurate examination of the
thinking situation and describe the nature of the problem
Pattern Capacity to recognize similarities among issues and
recognition make logical connections between concepts of different
domains
Lateral Capacity to try new ways of looking at problems and
thinking adopt unconventional perspectives
Exploratory Questioning Capacity to formulate questions in order to gather
information and challenge the current situation
Observing Capacity to observe the environment around you in
different contexts with the aim of finding new ideas
Experimenting Capacity to explore new ideas through experiments and
trials
(continued)
Organizational Visionary Capacity to create and articulate a vivid future image of

action thinking your group and/or organization and to define the
actions and objectives necessary to achieve it
Strategic Capacity to understand the strategic and competitive
thinking environment of the company
Opportunity Capacity to perceive the opportunities emerging from
recognition the environment
Commitment Capacity to be responsible and to act for the good of
toward the the group
group
Integrity Capacity to be consistent with yourself
The first four areas of competencies (Awareness, Action, Social, and

Cognitive) included in the Hexagon have been defined by the competency-
based literature. However, within each of these four areas of competency,
we have added further skills that are considered to be relevant in the cur-
rent organizational contexts.
Awareness encompasses all of the competencies that in the emotional
and social competency framework illustrated in Fig. 3.1 are grouped
within the Self-awareness cluster, namely Emotional Self-awareness, and
within the Social-awareness cluster are Empathy and Organizational
Awareness. These competencies also seem to be relevant for big data pro-
fessionals. Self-awareness is the fundamental competency of the emotional
and social framework because it represents the premise for managing one’s
emotions and understanding and managing others (Goleman et al. 2002).
With specific regard to empathy and organizational awareness, it has been
shown that big data professionals need to have a deep understanding of
their business context and of the stakeholders’ needs in order to effectively
perform their job (De Mauro et al. 2016).
The Action group includes the competencies present in the Self-
Management cluster (Fig. 3.1), namely Achievement Orientation, Self-
control, and Flexibility. Moreover, we included Efficiency Orientation,
Initiative, and Accuracy as originally described in the codebook of Boyatzis
(1982), Change Agent as defined in the Emotional Competency Inventory
(ECI) (Wolff 2005), and Collecting Information derived from the work of
Spencer and Spencer (1993). In this group, we also added Risk Taking
and Risk Management, which are competencies that characterize the
entrepreneurial mindset (Kyndt and Baert 2015; Morris et al. 2011) and
that are expected in the skillset profile of big data professionals (Costa and
Santos 2017; Harris and Mehrotra 2014). Indeed, these roles require
awareness of the potential returns and losses that a specific proposed solu-
tion can bring to the company and accordingly support decision making
to maximize the business opportunities. A final competency added to the
Action group is Resilience. This competency enables individuals to bounce
back from adversity and to retain a sense of hopefulness about the future
even in the face of adversity and stress (Dulewicz and Higgs 2005). This
is assuming an increasing importance in the labor market. Indeed, in the
last QS’s Global Skills Gap (QS 2019), resilience was reported to have the
highest deficiency (rated as very important by employers but limitedly
present in the profile of the graduate hired) among the skills analyzed in
the survey.
The Social group encompasses all of the competencies of the relation-
ship management cluster (Fig. 3.1), namely, Conflict Management,
Teamwork, Developing Others, Leadership, and Persuasion. In this area
of competencies, we added Networking, which was derived from Boyatzis
(1982) and from more recent contributions that analyze its role in mana-
gerial and entrepreneurial roles (Kyndt and Baert 2015; Snell et al. 2014).
Finally, customer focus was included by adopting Spencer and Spencer’s
framework (1993). This group of competencies is expected to be relevant
in determining the performance outcomes of big data professionals, as
discussed in Chap. 2. For instance, data scientists leverage on Developing
Others in mentoring junior roles within organizations or need Leadership
and Teamwork to work in synergy with others (Kim and Lee 2016; Shirani
2016; Verma et al. 2018). Several studies have emphasized the importance
of customer focus for data scientists, since this competency enables them
to effectively interpret and satisfy stakeholders’ needs (Kim and Lee 2016).
Also persuasion is required to capture the attention of the stakeholders
with a compelling data storytelling in order to obtain their support for the
proposed solution.
The Cognitive area of the Hexagon was derived from the original
work of Boyatzis’s (1982) and the subsequent definition of the cognitive
intelligence competencies (Boyatzis 2009). Specifically, two competen-
cies, namely Systems Thinking and Pattern Recognition, have been
demonstrated to predict outstanding performance across professional
roles and organizational contexts. Systems thinking is the ability to rec-

ognize the factors that have an impact on a complex situation by identi-
fying cause- effect relationships among different elements or events.
Pattern recognition allows individuals to connect the dots, namely, to
identify similarities among different and often distant conceptual
domains and to derive new business opportunities. Indeed, pattern rec-
ognition – also defined as analogical thinking or associational thinking –
is a way to combine previous experiences with new ones and derive
potential solutions (Baron 2006; Baron and Ensley 2006). This compe-
tency represents the cognitive process of “connecting concepts (ideas,
problems, fields of study, events, and trends) that appear, at first glance,
to be unconnected” (Dyer et al. 2008: 320). It appears to be central for
data professionals as well, because they need to capture meanings or
trends in data which are usually unclear at a first glance. We added two
further skills to this group of competencies, Diagnostic Thinking and
Lateral Thinking. The former is used when individuals make a careful
examination of the nature and the causes of a problem and consider why
the problem exists, why it is necessary to solve it, who is involved, and
how much time is needed (Puccio et al. 2011). Lateral thinking expresses
the ability to explore different ways of examining a challenging task
instead of accepting what appears to be the solution (De Bono 1992).
Whereas vertical thinking builds on existing patterns, lateral thinking
seeks to restructure existing patterns or move across patterns by identify-
ing different or unconventional ways and directions to solve a problem
(Hernandez and Varkey 2008). Lateral thinking is of great interest in
this study, because an unstructured and large volume of data require
digging into it and looking at it from different perspectives to find trends
and patterns in the data and turn the observations into business solu-
tions (Costa and Santos 2017; De Mauro et al. 2016; Harris et al. 2013).
In the Competency Hexagon we added two more areas of competency:
Exploratory and Organizational Action. The exploratory competencies
describe the behaviors that individuals adopt to scan the world around
them and to explore novel ideas. Dyer, Gregersen and Christensen (2008)
identified three main exploratory competencies used by entrepreneurs to
make sense of different situations: Questioning, Observing, and
Experimenting. These three competencies refer to the ability to explore
the world and find new ideas by challenging the status quo, by observing
and paying attention to what happens, or by experimenting and using a
hypothesis testing mindset. These three competencies are consistent with
previous research that describes big data professionals as individuals who

are curious to explore new paths (Costa and Santos 2017; Davenport and
Patil 2012; Harris and Mehrotra 2014; Vidgen et al. 2017), and they criti-
cally think about a certain situation (Shirani 2016).
Finally, the Organizational Action competencies allow individuals to
interpret the competitive environment, to sense business opportunities, to
think about the future of the organization, and to promote the alignment
between individual behaviors and organizational goals. This group
includes Visionary Thinking through which the individual is able to create
a vivid image of the future of an organization that he or she wants to build
in the long term and to share it with other organizational members. Also,
it includes Strategic Thinking, which is the ability to understand the stra-
tegic and competitive environment of the company (Moon 2013; Puccio
et al. 2011). As suggested by previous studies, data scientists need to be
driven by business curiosity (Costa and Santos 2017; Davenport and Patil
2012) and oriented by strategic thinking (Kim and Lee 2016). Another
competency included in this group is Opportunity Recognition. This is
the capacity to perceive opportunities emerging from the environment
and to perceive changed conditions or potential resources (Morris et al.
2011). Finally, the framework included Integrity, which is a competency
usually linked to authentic leadership and is expected to be found among
decision makers (Palanski and Yammarino 2007). Studies have also proved
that integrity has a positive relationship with the intentions of the follow-
ers and their performance (Dineen et al. 2006; Peterson 2004). This com-
petency has also been found by Robles (2012), who maintained that
business executives perceive integrity and work ethic to be among the ten
most important soft skills. Moreover, this competency is crucial for big
data analytics professionals, because they have daily access to personal and
sensitive data about clients, users, and employees.
3.5 The Application of a Behavioral Competency

Framework to a Changing Digitalized World
A recent article in The Financial Times (Nilsson 2018) reported the story
of a young graduate student from a top ranked university. She was just
hired when she was asked to use Python, a programming language that
she was not familiar with. Rather than giving up, she leveraged on what
she learned during her graduate studies that is, to be unafraid. She found
a way to learn how to master the software and she turned a threat into an
opportunity.
The interviews of five hundred famous innovators revealed that a “dis-
proportionate number of them” went to Montessori schools, where “they
learned to follow their curiosity” (Dyer and Gregersen cited by Brynjolfsson
and McAfee 2014: 313). Larry Page, the founder of Google, was one of
the Montessori school children and he recalled how “part of that training
[was] not following rules and orders [but] being self-motivated, question-
ing what’s going on in the world, doing things a little bit differently”
(Brynjolfsson and McAfee 2014: 314).
Both the young students and the outstanding innovators were highly
specialized and prepared. Their technical skills were strong and they spent
years in refining their knowledge. But the elements that distinguished
both the students and the innovators was the capacity to leverage on emo-
tional, social, and cognitive competencies over the years.
Therefore, what reports, academia, and policy makers are claiming
loudly is that it is imperative to nurture an emotionally intelligent work-
force, especially in a fast-changing environment where digitalization is
pushing the frontiers between humans and machines. The new workforce
needs to be equipped with both hard skills and soft skills. Determining the
most important soft skills is the other requirement. As shown in the previ-
ous section, different competencies, usually a bundle of competencies, are
needed for specific tasks and roles. It is still an open question about the
competencies needed for the new emerging job profiles such as data ana-
lysts and scientists (World Economic Forum 2018). Likewise, what the
new workforce will look like is unclear to policy-makers, business leaders,
individual workers, and academics. The following chapter (Chap. 4) will
contribute to shed lights on behavioral competencies most frequently
manifested by data analysts and data scientists while they actually perform
their job.
References
Amdurer, E., Boyatzis, R. E., Saatcioglu, A., Smith, M. L., & Taylor, S. N. (2014).
Long term impact of emotional, social and cognitive intelligence competencies
and GMAT on career and life satisfaction and career success. Frontiers in
Psychology, 5, 1447.
Azevedo, A., Apfelthaler, G., & Hurst, D. (2012). Competency development in
business graduates: An industry-driven approach for examining the alignment
of undergraduate business education with industry requirements. The

International Journal of Management Education, 10, 12–28.
Baron, R. A. (2006). Opportunity recognition as pattern recognition: How entre-
preneurs “connect the dots” to identify new business opportunities. Academy
of Management Perspectives, 20(1), 104–119.
Baron, R. A., & Ensley, M. D. (2006). Opportunity recognition as the detection
of meaningful patterns: Evidence from comparisons of novice and experienced
entrepreneurs. Management Science, 52(9), 1331–1344.
Boyatzis, R. E. (1982). The competent manager: A model for effective performance.
New York: Wiley.
Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis
and code development. Thousand Oaks: Sage Publications, Incorporated.
Boyatzis, R. E. (2006). Using tipping points of emotional intelligence and cogni-
tive competencies to predict financial performance of leaders. Psicothema,
18, 124–131.
Boyatzis, R. E. (2009). Competencies as a behavioural approach to emotional
intelligence. Journal of Management Development, 28(9), 749–770.
Boyatzis, R. E. (2016). Commentary on Ackley (2016): Updates on the ESCI as
the behavioral level of emotional intelligence. Consulting Psychology Journal:
Practice and Research, 68(4), 287–293.
Boyatzis, R. E., & Ratti, F. (2009). Emotional, social and cognitive intelligence
competencies distinguishing effective Italian managers and leaders in a private
company and cooperatives. Journal of Management Development,
28(9), 821–838.
Boyatzis, R. E., Cowen, S. S., & Kolb, D. A. (1995). Innovation in professional
education: Steps on a journey from teaching to learning. San Francisco:
Jossey-Bass.
Boyatzis, R. E., Goleman, D., & Rhee, K. (2000). Clustering competence in emo-
tional intelligence: Insights from the emotional competence inventory (ECI)s.
In R. Bar-On & J. D. A. Parker (Eds.), Handbook of emotional intelligence
(pp. 343–362). San Francisco: Jossey-Bass.
Boyatzis, R. E., Good, D., & Massa, R. (2012). Emotional, social, and cognitive
intelligence and personality as predictors of sales leadership performance.
Journal of Leadership & Organizational Studies, 19(2), 191–201.
Brynjolfsson, E., & McAfee, A. (2014). The second machine age: Work, Progress,
and prosperity in a time of brilliant technologies. New York/London: W
Norton & Co.
Campion, M. A., Fink, A. A., Ruggeberg, B. J., Carr, L., Phillips, G. M., &
Odman, R. B. (2011). Doing competencies well: Best practices in competency
modelling. Personnel Psychology, 64, 225–262.
Cherniss, C. (2010). Emotional intelligence: Toward clarification of a concept.
Industrial and Organizational Psychology, 3, 110–126. https://doi.org/
10.1111/j.1754-9434.2010.01231.x.
Cherniss, C., & Boyatzis, R. E. (2013). Leader interpersonal and influence skills:
The soft skills of leadership. In R. E. Riggio & S. J. Tan (Eds.), Building inter-
personal skills in management programs (pp. 53–72). New York: Routledge.
ness in the European e-Competency framework and the skills framework for
the information age. International Journal of Information Management,
37(6), 726–734.
century. Harvard Business Review, 90(10), 70–76. 128.
De Bono, E. (1992). Serious creativity: Using the power of lateral thinking to create
new ideas. New York: Harper Business.
De Mauro, A., Greco, M. Grimaldi, M., & Nobili, G. (2016). Beyond data scien-
tists: A review of big data skills and job families. International forum on knowl-
edge asset dynamics 11th, Proceedings of IFKAD, 2016, Towards a new architecture
of knowledge: Big data culture and creativity, June 15–17, 2016, Dresden,
Germany, 1844–1857.
De Vos, A., De Hauw, S., & Willemse, I. (2015). An integrative model for com-
petency development in organizations: The Flemish case. The International
Journal of Human Resource Management, 26(20), 2543–2568. https://doi.
org/10.1080/09585192.2014.1003078.
Dineen, B. R., Lewicki, R., & Tomlinson, E. C. (2006). Supervisory guidance and
behavioral integrity: Relationships with employee citizenship and deviant
behavior. Journal of Applied Psychology, 91(3), 622–635.
Dreyfus, C. R. (2008). Identifying competencies that predict effectiveness of
R&D managers. Journal of Management Development, 27(1), 76–91.
Dulewicz, V., & Higgs, M. (2005). Assessing leadership styles and organisational
context. Journal of Managerial Psychology, 20(2), 105–123.
Dyer, J. H., Gregersen, H. B., & Christensen, C. (2008). Entrepreneur behav-
iours, opportunity recognition, and the origins of innovative ventures. Strategic
Entrepreneurship Journal, 2, 317–338.
Fernández-Berrocal, P., & Extremera, N. (2006). Emotional intelligence: A theo-
retical and empirical review of its first 15 years of history. Psicothema, 18, 7–12.
Global Skills Gap Report. (2019). Retrieved in https://www.qs.com/portfolio-
items/the-global-skills-gap-report-2019/
Goleman, D. (1995). Emotional intelligence. New York/England: Bantam Books.
Goleman, D. (1998). Working with emotional Intelligence. New York:
Bantam Books.
Goleman, D. (2001). An EI-based theory of performance. In C. Cherniss, R. E.
Boyatzis, & M. Elias (Eds.), Emotionally intelligent workplace (pp. 27–44). The
Jossey-Bass Wiley Company: San Francisco.
Goleman, D., Boyatzis, R. E., & McKee, A. (2002). Primal leadership: Realizing
the power of emotional Intelligence. Cambridge, MA: Harvard Business
School Press.
Harris, L. R., & Brown, G. T. L. (2013). Opportunities and obstacles to consider
when using peer- and self-assessment to improve student learning: Case studies
into teachers’ implementation. Teaching and Teacher Education, 36, 101–111.
https://doi.org/10.1016/j.tate.2013.07.008.
Hernandez, J. S., & Varkey, P. (2008). Vertical versus lateral thinking. Physician
Executive, 34(3), 26–28.
Hopkins, M. M., O’Neil, D. A., & Stoller, J. K. (2015). Distinguishing competen-
cies of effective physician leaders. Journal of Management Development,
34(5), 566–584.
Jacobs, R. (2001). Managing employee competence in global organizations. In
J. Kidd, X. Li, & F. Richter (Eds.), Maximizing human intelligence deployment
in Asian business (pp. 44–56). New York: Palgrave.
Joseph, D. L., Jin, J., & Newman, D. A. (2010). Emotional Intelligence: An inte-
grative meta-analysis and cascading model. Journal of Applied Psychology,
95(1), 54–78.
Joseph, D. L., Jin, J., Newman, D. A., & O’Boyle, E. H. (2015). Why does self-
reported emotional Intelligence predict job performance? A meta-analytic
investigation of mixed EI. Journal of Applied Psychology, 100(2), 298–342.
Koman, L., & Wolff, S. (2008). Emotional intelligence competencies in the team
and team leader. Journal of Management Development, 27(1), 55–75.
Kyndt, E., & Baert, H. (2015). Entrepreneurial competences: Assessment and
predictive value for entrepreneurship. Journal of Vocational Behavior,
90, 13–25.
LinkedIn. (2019) Global talent trends. The 3 trends transforming your workplace.
Retrieved from https://business.linkedin.com/content/dam/me/business/
en-us/talent-solutions/resources/pdfs/global_talent_trends_2019_
emea.pdf
Lopes, P., Grewal, D., Pepper, J., Gall, M., & Salovey, P. (2006). Evidence that
emotional intelligence is related to job performance and affect and attitudes at
work. Psicothema, 18(Suppl), 132–138.
Mayer, J. D., & Salovey, P. (1997). What is emotional intelligence? In P. Salovey
& D. Sluyter (Eds.), Emotional development and emotional intelligence:
Implications for educators (pp. 3–31). New York: Basic Books.
Mayer, J. D., Salovey, P., & Caruso, D. R. (2000a). Emotional intelligence as zeit-
geist, as personality, and as a standard intelligence. In R. Bar-On & J. D.
A. Parker (Eds.), Handbook of emotional intelligence (pp. 92–117). San
Francisco: Jossey-Bass.
Mayer, J. D., Salovey, P., & Caruso, D. R. (2000b). Models of emotional intelli-
gence. In R. J. Sternberg (Ed.), Handbook of intelligence (pp. 396–420).
Cambridge: Cambridge University Press.
McClelland, D. C. (1973). Testing for competence rather than for intelligence.
American Psychologist, 28(1), 1–14.
McClelland, D. C. (1994). The knowledge-testing-educational complex strikes
back. American Psychologist, 49(1), 66–69. https://doi.
org/10.1037/0003-066X.49.1.66.
McClelland, D. C. (1998). Identifying competencies with behavioral event inter-
views. Psychological Science, 9, 331–339.
McKinsey & Company. (2018). Skill shift: Automation and the future of the work-
force. Discussion Paper, McKinsey Global Institute (MGI). Retrieved from
https://www.mckinsey.com/~/media/McKinsey/Featured%20Insights/
Future%20of%20Organizations/Skill%20shift%20Automation%20and%20
the%20future%20of%20the%20workforce/MGI-Skill-Shift-Automation-and-
future-of-the-workforce-May-2018.ashx
Meng Tan. (2012). Retrieved in https://knowledge.wharton.upenn.edu/article/
googles-chade-meng-tan-wants-you-to-search-inside-yourself-for-inner-and-
world-peace/
Moon, B. J. (2013). Antecedents and outcomes of strategic thinking. Journal of
Business Research, 66(10), 1698–1708.
Morris, M. H., Webb, J. W., & Franklin, R. (2011). Understanding the manifesta-
tion of entrepreneurial orientation in the nonprofit context. Entrepreneurship
Theory and Practice, 35(5), 947–971.
Newman, D. A., Joseph, D. L., & MacCann, C. (2010). Emotional intelligence
and job performance: The importance of emotion regulation and emotional
labor context. Industrial and Organizational Psychology, 3, 159–164. https://
doi.org/10.1111/j.1754-9434.2010.01218.x.
Nilsson, P. (2018, Sept 3). What top employers want from MBA graduates. The
Financial Times. Retrieved 15 Feb 2019. https://www.ft.com/
content/64b19e8e-aaa5-11e8-89a1-e5de165fa6191
O’Boyle, E. H., Humphrey, R. H., Pollack, J. M., Hawver, T. H., & Story, P. A.
(2011). The relation between emotional intelligence and job performance: A
meta-analysis. Journal of Organizational Behavior, 32, 788–818.
Palanski, M. E., & Yammarino, F. J. (2007). Integrity and leadership: Clearing the
conceptual confusion. European Management Journal, 25(3), 171–184.
Peterson, D. (2004). Perceived leader integrity and ethical intentions of subordi-
nates. Leadership & Organization Development Journal, 25(1), 7–23.
Puccio, G. J., Mance, M., & Murdock, M. C. (2011). Creative leadership: Skills
that drive change. New York: Sage.
Robles, M. M. (2012). Executive perceptions of the top 10 soft skills needed in
today’s workplace. Business Communication Quarterly, 75(4), 453–465.
Ryan, G., Emmerling, R. J., & Spencer, L. M. (2009). Distinguishing high‐per-
forming European executives. Journal of Management Development,
28(9), 859–875.
Salovey, P., & Mayer, J. D. (1990). Emotional intelligence. Imagination, Cognition
and Personality, 9, 185–211.
Snell, S. J., Tonidandel, S., Braddy, P. W., & Fleenor, J. W. (2014). The relative
importance of political skill dimensions for predicting managerial effectiveness.
European Journal of Work and Organizational Psychology, 23(6), 915–929.
Spencer, L. M., & Spencer, S. M. (1993). Competence at work: Models for superior
performance. New York: Wiley.
Sy, T., Tram, S., & O’Hara, L. A. (2006). Relation of employee and manager
emotional intelligence to job satisfaction and performance. Journal of Vocational
Behavior, 68, 461–473.
Thornton, G. C., & Byham, W. C. (1982). Assessment centers and managerial
performance. University of California: Academic.
Verma, B., Zhang, L., & Stockwell, D. (2018). Roadside video data analysis: Deep
learning (Studies in computational intelligence). Singapore: Springer.
261(2), 626–639.
Williams, H. (2008). Characteristics that distinguish outstanding urban principals:
Emotional intelligence, social intelligence and environmental adaptation.
Journal of Management Development, 27, 36–54. https://doi.
org/10.1108/02621710810840758.
Wolff, S. B. (2005). Emotional competence inventory (ECI) technical manual. Hay
GroupMcClelland Center for Research and Innovation.
Wong, C. S., & Law, K. S. (2002). The effects of leader and follower emotional
intelligence on performance and attitude: An exploratory study. The Leadership
Quarterly, 13, 243–274.
http://www3.weforum.org/docs/WEF_Future_of_Jobs_2018.pdf
Zhang, L., & Fan, W. (2013). Improving performance of construction projects. A
project manager’s emotional intelligence approach. Engineering, Construction
and Architectural Management, 20(2), 195–207.
CHAPTER 4
When Hard Skills Are Not Enough:

Behavioral Competencies of Data Scientists
and Data Analysts
Abstract This chapter contributes to the current debate on how to over-

come the skill shortage that characterizes the demand of big data profes-
sions in the labor market. If the technical competencies expected for these
responsibilities have been defined, their behavioral skills are still under-
explored. The chapter addresses this void through an exploratory study
providing empirical evidence collected through the Behavioral Event
Interview (BEI) technique. Drawing on an Italian sample of data scientists
and data analysts, the study provides a description of the competency port-
folio manifested by the two professional roles. The results show that both
data scientists and data analysts manifest a wide repertoire of behavioral
competencies that are needed to attain successful performance and to face
the challenges of the digital transformation.
Keywords Behavioral competencies • Competency-model • Data

analyst • Data scientist
What makes analytical organizations so interesting, in our view, is the

needed combination of human and computational perspectives.
Analytical decision-making is at the intersection of individual and
organizational capabilities.
(Davenport et al. 2010: 17)

https://doi.org/10.1007/978-3-030-33578-6_4
4.1 Emotional Intelligence and Behavioral

Competencies of Big Data Professionals
With a master’s degree in computer engineering and expertise in real time
analysis of sensor data, when Leonardo started to work in the Research &
Development Department of his current company he was involved in a
project that already had been started by two other colleagues who left the
firm in the same period that he had arrived. He was in charge of finalizing
a project that required the generation of algorithms and data analysis of a
telemetry system. After a very fast inception in which he collected all the
information necessary to take charge of the project, he started an in-depth
analysis of the work previously done.
I found that the system was fallacious, the way in which it was designed
prevented the implementation of crucial features of the company’s product.
I realized that system was too complex in relation to the actual firm’s needs.
I asked myself, what had happened? Why didn’t the others do something
else? So, on one hand you have a bit of a feeling, I do not mean resentment,
which makes you say what did the other people do before me? I decided to
apply another solution to a signal at the analysis and algorithm level, and I
found that it generated better outcomes. I brought this solution to mind
after having read a paper on dynamics and physical models. I thought to
keep the model much simpler from the physics side and to add some changes
in terms of signals that had been not considered in the model yet. A thing
that is very important in data analysis is the capacity to visually communicate
the results, trying to include all of the relevant information in a visual out-
come that helps the audience to understand what is happening. For this
reason, during an internal weekly meeting I decided to present to the tech-
nical team what I was designing and the expected outcomes. I highlighted
the coherence of the data and its reliability, pointing out that the solution I
proposed was easier to test and to develop. The team was positively impressed
by my presentation and the colleagues asked me to explain to the other non-
technical departments (marketing and design) what this new solution
will generate.
What can we learn from this story? Leonardo is a technically prepared

data scientist, but what enabled him to achieve a higher performance for
his company was his capacity to combine his technical expertise with a
repertoire of emotional, social, and cognitive competencies. Specifically,
he manifested the capacity to conduct an accurate examination of the
situation and describe the nature of the problem (diagnostic thinking).
4 WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 65
Then he asked himself questions to challenge the actual model (question-

ing). He recognized technical similarities between the paper that he read
and the problem that he was addressing (pattern recognition). He started
to elaborate and test alternatives (experimenting) and demonstrated a
concern for increasing the efficiency of action (efficiency orientation).
Also, Leonardo relied on a set of social competencies to better understand
the actual needs of the company (customer focus) and to communicate
effectively his solution to the organization in order to gain the necessary
support to implement the solution proposed (persuasion).
As discussed in the previous chapter, behavioral competencies have
been found to be a critical component in predicting workplace perfor-
mance across a variety of settings and for different job profiles (Boyatzis
1982, 2006, 2009; Boyatzis and Ratti 2009; Boyatzis and Sala 2004;
Hopkins et al. 2015; McClelland 1973, 1998; Spencer 2001; Spencer
et al. 2008; Spencer and Spencer 1993; Ryan et al. 2009). Contemporary
research acknowledges that behavioral competencies are as essential as
technical skills (Boyatzis 2008; Goleman 1998), because they account for
predicting superior individual and team performance (Boyatzis 2009;
Druskat et al. 2013; O’Boyle Jr. et al. 2011; Stubbs Koman and
Wolff 2008).
If emotional intelligence is taken for granted, what about its application
in a digitalized organizational environment? In particular, what are the
behavioral competencies needed by big data professionals who contribute
to the generation and manipulation of a large volume of data in our orga-
nizations on a daily basis?
Despite the increasing effort, mainly from scholars, practitioners and
policy makers, to define the characteristics of data workers, a standard and
precise definition of the different roles has not yet been reached, as dis-
cussed in Chap. 2. If the activities and responsibilities of data analytics’
roles are still in a grey area, their behavioral competencies look even more
unclear. The empirical evidence is limited and confusing. As an example,
Joseph et al. (2010) suggested that data analytics’ roles need to possess a
practical intelligence, which is made up of a set of skills (managerial, intra-
personal, and interpersonal). Costa and Santos (2017) suggested a con-
ceptual model that includes competencies such as business acumen,
communication, entrepreneurship, curiosity, and interdisciplinary orienta-
tion. Davenport and Patil (2012) added some cognitive competencies like
associative thinking to the discussion. This is considered to be an essential
ability because it helps data scientists to find a pattern from diverse formats
of data. In addition, creativity has been found to be a key characteristic

because it helps to organize data in visual representations (Harris and
Mehrotra 2014; Davenport and Patil 2012). Despite these fragmented
insights that attempt to provide a more complete description of data pro-
fessionals by including these soft skills, a clear comprehension of these
roles is still missing.
This chapter offers an in-depth description of behavioral competencies
of two big data professionals – data scientists and data analysts – in order
to provide more clarity about the soft skills needed by these two relevant
roles. This study was conducted in the Italian context, where in 2018 the
analytics market reached a value of 1393 billion euros with a growth rate
of 26 percent (Vercellis 2018). According to the observations of Big Data
Analytics & Business Intelligence based in Politecnico of Milan, 56 per-
cent of companies employed data analysts, 46 percent data scientists, and
42 percent data engineers (Vercellis 2018). Organizations are increasingly
embracing a data driven culture in order to be successful. However, the
majority of the Italian companies are still anchored to a traditional model
with underdeveloped analytical skills that, if present, are mostly repre-
sented by data analysts or business analysts and by a restricted group of
data scientists (Di Deo 2019). In this study, data scientists and data ana-
lysts operate in Northern Italy and in different industries. This geographi-
cal area is particularly suitable for this research because an impressive
digital transformation has involved the majority of its companies, which
has required them to hire people in analytical roles (Istat 2018).
Different from the studies considered in Chap. 2, which derived com-
petencies from content analysis of job advertisements or from focus
groups, this study identifies the competencies by using a consolidated
technique, the Behavioral Event Interview (BEI), which allows one to
detect when a competency has actually been enacted by an individual who
obtains a result (Boyatzis 1982; Gorman et al. 2017; Spencer and Spencer
1993). The competency framework shown in Chap. 3 is adopted to ana-
lyze the behavioral competency profile for both of these two roles.
4.2 Data Scientists and Data Analysts

in the Italian Context: An Empirical Study
This section provides a description of the different phases of the empirical

research conducted in the Italian context that involved a sample of data
scientists and data analysts.
The first step was the definition of the sample. Participants were selected
by searching their job title on professional networks like LinkedIn and
through snowball sampling, that is by asking each selected participant to
identify other potential subjects to involve. Therefore, the following con-
ditions were adopted to include a participant in the sample: (i) docu-
mented experience in the data analytics area, (ii) active in gathering and
analyzing data on a daily basis; (iii) experience in delivering reports for
strategic analysis and business choices; (iv) development of statistical anal-
yses on large data sets to define trends and derive business insights used by
organizations to set future goals. Drawing on the characteristics of both
two profiles described in Chap. 2, we developed a list of criteria to set the
boundaries of data scientists and data analysts and to distinguish them
from other big data professionals. A check list was then used to select the
participants and exclude the ones who did not fit with these criteria. A
database of potential interviewees was created and used to contact them
and ask their voluntary participation. Twenty-four professionals accepted
the invitation to participate in the study, specifically 11 data scientists and
13 data analysts.
In terms of sample characteristics, we obtained an heterogenous sam-
ple. First, participants had different seniority. On average, they worked for
their current organization for eight years with a minimum of two months
and a maximum of twenty-five years. Their role seniority was on average
6 years with a minimum of 5 months to a maximum of 20 years. They
were employed in 20 different organizations that belong to a variety of
sectors: IT and consulting (five companies); software production (five
companies); business intelligence and training (three companies); web and
marketing (two companies); retail (one company); human resources and
ICT (one company); R&D (one company); consulting (one company);
and insurance (one company). The average revenue of these companies in
2017 was about 3 million euros. Concerning the number of employees,
eleven firms had fewer than 50 employees, two had from 50 to 149
employees, one had from 150 to 249; and six had more than 250 employ-
ees. This variety was also reflected at the individual level. The average age
of respondents was about 38 years old (with a minimum of 24 and a maxi-
mum of 56 years) and 16 percent of the sample was represented by women.
Regarding their education, 10 percent obtained a PhD, 50 percent
received a master’s degree and 35 percent a bachelor’s degree; only 5 per-
cent did not hold any academic degree. The educational background of
the respondents was heterogeneous, encompassing statistics, mathematics,
management and economics, computer science, physics, biology, cogni-

tive science, psychology, and communication. It is common to have het-
erogeneous educational background: “some of the best and brightest data
scientists are PhDs in esoteric fields like ecology and systems biology”
(Davenport and Patil 2012: 6).
In addition, data scientists and analysts of our sample worked in differ-
ent organizational departments like R&D, business intelligence, adminis-
tration and control, or IT.
In contrast to extant studies that mapped the soft skills of data analytics’
roles and that mainly used secondary sources such as survey questionnaires
(Aasheim et al. 2012) or content analysis of job announcements (Shirani
2016; De Mauro et al. 2016), we adopted a consolidated methodology,
namely the Behavioral Event Interview (BEI), that does not rely on per-
ceptions of the main important competencies for the professional roles
under investigation, but it detects the actual competencies enacted by the
role holders in their work environment (Boyatzis 2009; Emmerling and
Boyatzis 2012; Scapolan et al. 2017). Data was collected between May
and June 2018 through semi-structured interviews. The interview proto-
col comprised an introductory section in which respondents were asked to
provide a detailed description of their daily activities and main responsi-
bilities in order to obtain a more in depth understanding of their job char-
acteristics. The core of the interview was concentrated on the BEI protocol
(Boyatzis 1998; McClelland 1973) which was developed from the critical
incident interview technique (Flanagan 1954). BEI was used to collect
actual critical events or incidents in which each respondent was effective in
performing his or her tasks in the company. In particular, each respondent
was asked to recall five critical situations that had occurred during the last
12 months and to describe the context, the people involved, what he or
she thought, felt, or said, and how he or she behaved. In order to avoid
potential bias in detecting successful episodes, a description of the final
outcome of a certain event was constantly asked and checked by the inter-
viewer. For instance, positive episodes were about winning a new client,
solving an issue in data analytics, or the development of new software. On
the opposite side, minor issues or revisions in their daily activities were
excluded from the analysis.
The interviewers detected the intent of the specific behaviors guiding
the interviewee though a set of probing questions (Boyatzis 2009). By
gathering actions, reactions, and decisions, the interviews on the behav-
ioral episodes made it possible to discern the main behaviors and conse-
quently the related competencies expressed by the interviewees while

performing their job (Spencer and Spencer 1993). In total, we collected
120 episodes. Each interview, which lasted on average 1.5 hours, was
recorded and transcribed for subsequent coding.
The first step of the analysis was aimed at creating a comprehensive
description of the respondents’ duties and responsibilities. We integrated
our own understanding of the respondents’ activities within the frame-
work already established in the literature (De Mauro et al. 2016; Harris
and Mehrotra 2014). The second step of analysis concentrated on detect-
ing the main behavioral competencies expressed by respondents and clas-
sifying them into a portfolio of competencies. To identify the behavioral
competencies manifested by the interviewees, we adopted a competency
codebook that included 33 competencies clustered into six areas: aware-
ness, action, social, cognitive, exploratory, and organizational action compe-
tencies which are illustrated in Chap. 3 (Fig. 4.1). Each competency was
measured by analyzing its frequency (Ryan et al. 2009), which was com-
puted by the recurrence with which one specific competency was activated.
4.3 The Behavioral Competency Profiles of Data

Scientists and Data Analysts
This section aims to discuss the main similarities and differences that
emerged from the analysis of the behavioral competencies that were most
frequently activated by the two professional roles in their work environ-
ment. Considering the six areas of competencies (Fig. 4.1), the profile of
the overall sample is illustrated as follows: action (33 percent), social (21
percent), awareness (20 percent), and cognitive (15 percent), followed by
exploratory (7 percent) and organizational action competencies (4 percent).
A more detailed comparison between the two roles for each of the
thirty-three competencies is provided in Fig. 4.2.
As reported in Figs. 4.3 and 4.4 and considering the first ten most fre-
quently manifested competencies by the two distinct roles, data scientists
and data analysts show the same competencies even though there are some
differences in terms of the frequency of activation. These competencies are
the following:
• self-awareness, empathy, and organizational awareness for the

awareness area;
Fig. 4.1 Behavioral competencies framework: the competency hexagon
• achievement orientation, initiative, and efficiency orientation for the

action area;
• customer focus and persuasion for the social area;
• diagnostic thinking and pattern recognition for the cognitive area.
The following sections will provide more evidence and concrete exam-
ples on how data scientists and data analysts activate behavioral competen-
cies in performing their job.
4.4 Action Competencies

A young data scientist who recently joined one of the largest public Italian
companies recalled an episode in which his manager asked him to develop
new software: “I decided to anticipate the timeline of the project, doing a
Fig. 4.2 Competencies of data scientists and data analysts according to the fre-
quency of manifestation
Fig. 4.3 Competencies of data scientists according to their frequency of

manifestation
migration of the analysis from R to Python. My colleagues were thinking

about that, but they did not consider it as a priority. I instead thought
that, since we had to do the analysis, it was meaningful to start immedi-
ately with something that was as close as possible to industrialization”.
The most frequent competency area manifested by the sample was
Action, which collects competencies that express how individuals realize
ideas, plans, and solutions, then work methodically and with initiative. In
particular, respondents show high achievement orientation, initiative, and
efficiency orientation.
Fig. 4.4 Competencies of data analysts according to their frequency of

manifestation
An expert data scientist in a large software production company revealed

that during the long and complex implementation of a software program
that aggregated information about the financial exposure of bank accoun-
tants, he first set a tight meeting schedule, sharing the calendar with his
staff and all the people involved in the project [Achievement Orientation].
Also, by monitoring each step and driving the situation towards a specific
direction: “I tried to convey the situation towards an intermediate solu-
tion for different reasons. The first reason was that I know my company
and I know that certain things must be dealt with, let’s say, gradually. The
second reason was that, in my opinion, as a cost-benefit ratio it was much
more advantageous [Efficiency Orientation] and allowed colleagues to
continue to consolidate their experiences”.
A young business analyst in a large company explained how the team
used a data quality architecture developed before his arrival in the organi-
zation. After a few months, he realized the inaccuracy of the system and
the inefficiency that it was causing to the entire organization. Despite his
junior position, he describes “I went to the manager and I offered him the
solution: I could work on that project for a month and repeat everything
over and over again, by doing at the same time the project I was hired for
and eventually assessing whether my solution was better than the old one
[Initiative].” His intuition was correct, and the manager decided to change
the system in order to avoid a further waste of time and resources. He
demonstrated the capacity to take action first, not reacting to events or
being forced by them.
Initiative and willingness to achieve challenging goals are usually men-
tioned in the skillset of data scientists (as discussed in Chap. 2). The nar-
ratives collected demonstrated that these competences also enabled data
scientists to attain effective outcomes. With specific regards to achieve-
ment orientation, the two profiles show “to be confident in their ability to
learn and to adopt a growth mindset, setting them up in a path to greater
performance” (Truninger et al. 2018: 4). Moreover, different from the
literature that defined the skillset of big data professional profiles, effi-
ciency orientation emerged in both profiles as the tenth most frequently
activated competency. This can be associated to a pragmatic approach that
is typical of project-based jobs like those of data scientists and data analysts
who are spurred to provide quick answers to external clients and internal
business teams, with a continuous concern for assessing the costs and ben-
efits of the proposed solutions.
A competency that differentiates the skill profiles of data scientists is

flexibility. A young data scientist with a doctoral degree in physics described
a difficult project in which he was asked to provide several months’ fore-
casts, whereas he could return only one-week predictions. After several
failing trails, he attempted to consider a different perspective. “At that
point, I decided to start over with something else in the sense that I
changed my model to manage data. Instead of inserting data for the last N
days, I thought to insert data adding information about each day, year,
month, and date (whether people were on vacation or if they were unex-
pected people) and I assigned to each day the variables I had or the vari-
ables I could create. A different world opened to me and other different
potential models.” This narrative highlights the ability of this data scientist
to adapt himself by modifying his behavior in case of required changes or
unexpected circumstances.
4.5 Social Competencies

A young data scientist working for one of the biggest retail companies
described how he explained the entire process to implement a software
program to a difficult client and how he put attention on the customer’s
needs. According to the data scientist, it was crucial to use visual represen-
tations and to rely on “different techniques to present the information you
are analyzing and what you are looking for in the data and packaging the
results aesthetically by supporting the data with visual messages for the
client. In this way, it prevents the client from getting bored while he is
waiting for the results [Customer Focus].”
Social competencies include capabilities that allow a positive interaction
with other people and help to work with others effectively. In particular,
data scientists and analysts manifest high customer focus and persuasion.
A senior data scientist shared the challenges encountered while negoti-
ating with an important client: “when you go to meet a client and the
client thought to spend 1,000 and you ask for 10,000, you need to explain
why. All of the ability is there if you can make it clear about the techno-
logical complexity and all of the study needed to develop a certain product
and the added value the technology will give them. So, if you can explain
all these aspects, then you can convince the client and find a compromise,
and maybe go from 10 to 8; but for sure it is not going to be 2 [Persuasion]”.
Within the social competencies, leadership is the third most mani-
fested competency. A data analyst who was also a project manager of a
small start-up company, described an episode in which some competing

stakeholders were involved: “During the implementation of an applica-
tion that tracked the movements of the animals, I was responsible to
coordinate the team development, which is part of the company. I took
care to give to a junior profile the right instructions about what to install
inside the platform and how it worked in terms of usability. The funda-
mental problem was that the thing was managed but no one was making
decisions. Therefore, I announced: “Ok, someone needs to make deci-
sions here, we need to make decisions, we need to be clear, we need to
be categoric, and I will take full responsibility for making these deci-
sions.” I asked for a mandate to make these decisions and I systematically
began to arrange all of the different aspects that were in front of me
[Leadership]”.
What differentiates data scientists from data analysts for this specific
area of competencies are teamwork and networking. A data scientist
who works for a company that develops video games based in London
recalled an episode in which, working in synergy with the other mem-
bers of her team and combining their individual expertise, they could
solve a computational issue: “Everyone in the group has his own tech-
nique, but we were able to understand an issue by working together that
we could not understand otherwise. One member of the group recalled
an article and she was able to tell us how things were done, but I was
more interested in understanding why this was done and in understand-
ing the mathematical reasons. Another member of the group was much
more interested in “Eventually, why does it work in this way?” We have
three very different personalities, but we started talking and everyone
brought his own perspective and eventually we knew something new
[Teamwork]”.
A senior data scientist with a leading role in a big Italian company
described how she implemented a new instrument for an important client
whose collaboration was difficult. She recalled how she managed to bind
a good relationship with the client and, at the same time, to implement
the new complex instrument and sell it successfully to the client. In par-
ticular, the implementation stage required more effort than expected, and
she used external resources to be acquainted with the instrument: “I had
a colleague from another company who already knew this tool. We
organized a couple of meetings with him and he gave us some informa-
tion. I asked him how he would do some things” [Networking].
4.6 Awareness Competencies

A data analyst working for a web marketing company recalled one of her
most difficult events of managing a client who “lives in a sort of techno-
logical illiteracy. After several discussions with the client, I mediated with
the client by getting into the client’s mindset so that I could adapt my
analysis and my research and guide the other person to acquire a new
awareness. I needed patience, empathy, and control of impulsiveness”.
The awareness competencies seem to play a crucial role for both pro-
files. Specifically, emotional self-awareness, namely the capacity to under-
stand one’s own emotions and their effects but also to know one’s inner
values, strengths and limits, emerged respectively as the first and second
most manifested competencies for data scientist and data analysts (see
Figs. 4.3 and 4.4). There are several reasons why this competency is rec-
ognized as extremely important to manifest. First, it allows us to under-
stand the impact of our emotions on our decisions and actions. This makes
it possible to analyze and correct our decision making processes. Second,
being aware of our strengths and weaknesses is the first and necessary step
of every effective individual change process that also needs to consider the
real starting point in terms of personal characteristics. In addition, the
awareness of our individual limits and points of mastery allows us to avoid
risky situations and to face issues with an appropriate self-confidence.
Finally, understanding ourselves also helps us to manage ourselves and to
understand others. Consequently, it is a fundamental component of the
manifestation and of the development of other groups of competencies.
A young data scientist who was implementing a digital agenda for one
of the largest computer companies in the Italian Chamber of Commerce
described the first stages of the project and the relationship with different
stakeholders involved. In particular, he observed the following: “I started
writing the first points and then I tried to leave more room for my col-
leagues in the subsequent steps, because I knew they were better or more
experienced than me” [Self-Awareness].
Another data scientist working for an incubator explained: “In this cir-
cumstance I said to my manager that this customer was crazy, that I didn’t
want to work with him anymore and that if you came to me it’s because
than you are sure that you will have the project finalized within the
deadline. And he told me that’s true. We have this compromise by which
I can tell him all I want, and he knows that I will reach the result”
[Self-Awareness].
Besides self-awareness, the profile of both roles encompasses the capac-

ity to sense others’ emotions (empathy) and the capacity to recognize the
values and the culture of an organization but also to understand its infor-
mal processes and structure, the unspoken rules and the key power rela-
tionships (organizational awareness).
During an organizational change process which also involved the tran-
sition to a new software system, one of the interviewees described the
following: “That company decided to adopt a new software, SAS. The
majority of the employees were not familiar with this software and they
looked scared; they were resistant to the change. How did I understand
that they were skeptical? Well, it was from their attitudes, their behaviors,
how they talked to give you some information, how they said uhm”
[Empathy].
If data scientists and analysts are good at reading others’ signals, they
also manifest the ability to interpret the organizational ecosystem. This
was described by a senior data scientist who described a complex negotia-
tion process composed of several “meetings with various subordinates, in
order to find the functional and technical needs that were impacted the
most. I was looking for the people in the company who could support me
and had the power and the skills to do so. I noticed that during the meet-
ings that all of the participants looked at one woman when some uncertain
issues emerged and waited for her opinion. I understood that she was
informally recognized as the leader and hence that all her doubts had to be
dispelled” [Organizational Awareness].
4.7 Cognitive Competencies

“I opened the program and I checked it. Controlling programs also means
doing a mental process and noticing where there could be a duplication.
And the duplication was there. I had the aggregate data and I went back-
wards, so I looked at the step before the one that generated that data.
Then, if the data was duplicated, I had to intervene. If the data was not yet
duplicated but the numbers did not convince me, I would have to go back
until I got to the input and then to the data that the user gave me in order
to see if the data were actually correct. For example, if a file has 10 records
three months before and 10 the month after, it has 20 records. This means
that something was wrong at that point, which means that I have to check
the historical series too [Diagnostic Thinking].” This is how a data analyst
described the step by step cognitive process through which he was able to
detect an error. Even though diagnostic thinking emerges among the ten
most activated competencies of both professional roles, it mainly seems to
characterize the profile of data analysts.
Cognitive competencies are the ability to use information and analyze
it effectively to interpret phenomena or situations and to analyze problems
with a scientific mindset.
Another data analyst recalled a critical event in which she realized that
a consistent amount of money was missing from the database: “After sev-
eral rounds of checks, I realized that money was spread through three
different projects. I found the value that did not sound right, and it was
quite simple to find it. First, I checked everything because it is easy to
make mistakes (I checked twice). Then when I found that the results were
correct, I checked every project that I had recorded as critic with the team
leaders. So, I did the same for all the projects that were considered as pri-
ority and critical. At the end when I found it, I analyzed it again; I ana-
lyzed all the indicators of the project. I realized that the project could go
out of control, and then I reported it to my boss” [Diagnostic Thinking].
Likewise, for diagnostic thinking, another cognitive competency, pat-
tern recognition is manifested more by data analysts than by data scien-
tists. A data analyst was required to collect data about the behavior of a
large sample covering a long period of time. To do so, he adopted a system
by which he was able to find an underlying ratio: “I analyzed several key-
words that could be linked to the industry, reference keywords. I associ-
ated the keywords to various markets linked to the users and therefore to
potential markets from which users could arrive. I then matched the infor-
mation of the various markets and the various keywords, I extrapolated
graphs, Excel data, and a document that I eventually presented during an
event” [Pattern Recognition].
In this area of competencies, there are two competencies most fre-
quently activated by data scientists in comparison to data analysts: systems
thinking and lateral thinking.
Systems thinking is the ability to break down complex problems and
understand the cause–effect relationships. A data analyst in charge of ana-
lyzing several consumers’ data, was asked by the company’s director to
understand how a specific parameter changed based on specific character-
istics of the clients. He explained “obviously I cannot look at just one
customer’s variable and see the impact on the parameter. I have to con-
sider different combinations of these variables to see how they contribute
to determine the parameter” [Systems Thinking].
Lateral thinking is the ability to think outside the box when looking at
problems, and it is particularly crucial for generating non-conventional
solutions. A senior data scientist explained: “In marketing there is the
concept of a funnel, a funnel of people who are contacted at the beginning
and gradually there is a skimming up until arriving at the bottom of the
funnel, the purchase moment. A client asked: Every week I contact people
who are at different stages of this funnel and every week I would like to
understand how many of them are going down through the funnel. I then
turned around the problem and approached what was the final solution.
Let’s see the problem in this way. Every week we understand how many of
those who bought were people who were in the funnel, and you had con-
tacted them for the first time 7 days before, or if they were very slow
people, you had contacted them more than two months before. So, we
will do a histogram every week with a slice that are the hot ones and a slice
that are the blue ones, which are the coldest ones. In this way we can see
how the sales change over the weeks” [Lateral Thinking].
4.8 Exploratory Competencies

A data scientist narrated an episode in which he had to select the data
source that would allow him to get access to a high volume of new exter-
nal data. He critically started to ask to himself what he could have done
with such data. “I evaluated the proposed data sources and I created a list
of technical questions to address to the providers in order to have more
detailed information about the features of this data, such as how frequently
the data is updated or which characteristics it includes. In this way, I had a
new approach to the problem that highlighted the need to get further
information” [Questioning].
“Questioning” is the capability to challenge the current situation, to
attempt to understand the nature of the problem and use a different
approach to it.
As highlighted in Chap. 2 (Davenport and Patil 2012: 5), the ability to
scan the environment with a curios attitude allows data scientists to see
new opportunities, “a desire to go beneath the surface of a problem and
to find the questions at its heart.” In so doing, they create value for orga-
nizations, and they are a crucial asset to generate innovation. Despite the
fact that exploratory competencies (observing, questioning, and experi-
menting) are more frequently demonstrated by data scientists than by data
analysts, confirming the job requirements described in the academic and
practitioners’ communities, these competencies have been manifested in

our sample less than expected.
New ideas for the generation of new solutions are collected by data
scientists keeping themselves updated on the evolution of the big data
field, especially by reading articles and books or consulting experts on
specific technologies and techniques. As a data scientist narrated: “many
books have been published on this issue, you have to read and assimilate
them and then generate hypotheses. While I read a book on big data, I
think about my clients, and I start to imagine for whom I can adapt the
technique that I am learning” [Observing].
To explore new solutions and try new things by adopting a trial and
error approach, all behaviors are related to the “experimenting” compe-
tency. “I had a series of numbers distributed over time, and I was wonder-
ing whether these numbers followed a seasonal trajectory as they were
involved in a parking lot project. Maybe people preferred to park on
Wednesdays and leave on Sundays. It was a classic case, a typical case from
a textbook. Then, I tried to apply this idea to the data, and I noticed that
it was a great idea [Experimenting], but I realized I could not go further
to make forecasts because historical series usually have a problem; unless
you have a very long history, the inaccuracy of this data tends to explode
in the long run.” This data analyst, in addition to showing pattern recog-
nition by discovering a recurring users’ behavior, also stretched data itera-
tively and adopted an experimental approach.
4.9 Organizational Action Competencies

A data analyst describes how he improved an e-commerce platform and
achieved a remarkable growth of turnover. He first monitored the users’
behavior of the platform and consequently adapted its usability accord-
ingly. “Using the database of Italian companies and the ATECO code [a
classification of economic activities adopted by the Italian National
Statistical Institute], therefore using the sector of the companies belong-
ing to the publishing sector, I extrapolated a list of companies according
to turnover, geographical location, and number of employees. Why geo-
graphical location and number of employees? Because, actually, the num-
ber of employees outlines the size of the publishing house and I was
interested in discovering the smaller publishing houses, because the smaller
ones are those that print the most interesting books. And then the geo-
graphical location, because obviously if you are a publisher based in Milan
with the headquarters in Rome, which are commercially strategic points,

it means that you are hardly a small company that does small activities. In
fact, I found several interesting publishing houses in other cities such as
M. and C., smaller places on the map but which would have allowed me
to obtain interesting results for this type of activity [Strategic Thinking]”.
The organizational action area of competencies, which encompasses
strategic and visionary thinking, opportunity recognition, commitment
toward the group and integrity, includes capabilities that has been expressed
by respondents with a frequency of manifestation of about 6 percent.
Data scientists and data analysts manifested only one of these compe-
tencies at the same frequency level, that is “Commitment Towards the
Group”. This is the ability to sacrifice personal targets and needs in favor
of the team or of the organization. A young data scientist working for a
small company based in London narrated “I worked very hard, and during
two weekends I was at work from morning to night because I wanted to
finish the project and because that was the turning point for my group as
well. If I had missed it, it also would have been bad for the data group”
[Commitment Toward the Group].
Within this area, two competencies were demonstrated at a higher fre-
quency of manifestation by data analysts, namely strategic thinking, which
is the ability to understand the competitive environment of the company,
and opportunity recognition, which is the ability to perceive new business
possibilities emerging from the environment. A data analyst operating in a
web marketing agency explained how she had to address the need of a
high demanding client operating in the food & beverage industry. “The
client demonstrated a very negative approach toward e-commerce; he
needed to regain confidence on it. The sector is characterized by a high
competitiveness; the food & beverage industry is blowing up in the web.
The solution I proposed was to work in a market niche such as PDO-
Protected Designation of Origin products, focusing on the added value
offered by the client’s offerings [Opportunity Recognition]. I started an
analysis of the food products sold on line considering their geolocation.
The analysis was conducted in order to position the products that were
more appreciated by the final customers in different geographical markets.
In other words, I tried to understand the best positioning of the products
for each foreign geographical market [Strategic Thinking]”.
The presence of these competencies in the repertoire of data analysts
can be explained by the fact that this professional role in some organiza-
tional contexts works more closely with the sales department. They
frequently meet with clients in order to address their requests that rely on
analytics techniques. Thus, strategic thinking and opportunity recognition
are often manifested in relation to the businesses of the clients and indi-
rectly to the company’s business.
Despite the very low frequency, data scientists were shown to activate
slightly more of the remaining two competencies of this area, visionary
thinking and integrity. Visionary thinking is the capacity to create a vivid
image of a desired future. Managers and leaders who share a vision with
employees and make it explicit affect organizational engagement and pro-
ductivity (Goleman et al. 2002). The structure of Italian companies is
highly centralized and the power is mainly in the hands of the top manage-
ment; this partially explains the shortage of this competency. The recent
drive to embrace digitalization among Italian companies is a further expla-
nation. Indeed, despite the common attempt to adopt a data-driven cul-
ture, the top management is still struggling to abandon their traditional
role and delegate the decision-making process to other roles such as data
scientists or data analysts. Integrity is considered a key trait of profession-
als who manage a large volume of data on a daily basis; for this reason, it
is often taken for granted. This is why it was not manifested explicitly as
the specific determinant of role effectiveness, but it was considered as an
underlying behavior strictly integrated in the daily activities. If analytics
and big data poses a number of questions for policy makers about data
protection, privacy, and surveillance, the same issues are expected to
receive similar attention from the individuals who work in the field (big
data professionals have manipulated and used data pertaining to informa-
tion about clients, internet users, and retailers, excluding data about inter-
nal movement of employees, surveillance, and human resources. However,
the recent EU General Data Protection Regulation (GDPR) introduced
in the past May of 2019 has impacted both public and private organiza-
tions. As the ethical issue in data analytics is becoming even more critical,
behavior related to the integrity competency are expected to further
increase in terms of frequency of activation.
4.10 The Competency Profile of Data Scientists

and Data Analysts: Concluding Remarks
The interviews offered a multifaceted picture of the two analytics’ roles.

They further enriched the soft skills profile defined by the academic and
practitioner communities discussed in Chap. 2. To obtain effective results
in their job, data scientists and data analysts rely on a broader repertoire of
behavioral competencies. They are not confined exclusively to cognitive
and social behaviors as job postings in the labor market continue to
highlight.
The competency hexagon with its thirty-three behavioral competencies
has provided a fine-grained framework to understand the complexity of
these two roles. Considering the specific area of competencies, the narra-
tives collected directly by the real-life experiences of the role holders
pointed out the importance of the action, social and awareness competen-
cies. They not only identified the main commonalties but also the distinc-
tive features of the two profiles. Even though the narratives reported in
this chapter exemplified the manifestation of single competencies, data
scientists and data analysts simultaneously activated more than one com-
petency in each specific situation in order to achieve a positive outcome.
This means that different areas of competencies interact with each other in
combinations or bundles.
From the interviews, it also emerged that some competencies consid-
ered crucial for the big data analytics’ roles, especially in the area of cogni-
tive, exploratory and organizational action competencies, were activated
less frequently than expected. Even though these competencies present a
lower frequency of manifestation in comparison to others, it does not
mean that their impact on the final result is less relevant. In order to attain
positive outcomes, data scientists and data analysts are required to activate
the aforementioned competencies coherently with the characteristics of
the situations that they face in the workplace. In its Data Science Salary
Survey, O’Reilly Media (King and Magoulas 2016; Suda 2018) asked data
scientists about how much time they spent on specific analytics tasks dur-
ing their workday. The survey revealed that these roles do not work in silos
but are continually exposed to social interaction. Roughly 42 percent of
the respondents indicated that they spend between 4 and 8 hours a week
in meetings and 6 percent of them spend more than half of their work
week in them. Moreover, nearly half of them spent 1–4 hours per week
presenting analysis, with 6 percent spending four hours or more per day
sharing findings with management. This explains the higher frequency for
manifestation of the awareness competencies like empathy and organiza-
tional awareness but also for social competencies such as persuasion and
teamwork. However, most of their time is devoted to basic exploratory
data analysis and data cleaning. These activities are also labeled with terms
like data munging or wrangling rather than modelling data. As data
reparation accounts for a high percentage of the work of data scientists,

p
less time remains for other relevant tasks such as the following:
• identifying the exact business problem and then converting it into an

analytic problem that can be solved with data;
• suggesting machine learning and predictive modelling that explains
discrepancies and helps in understanding what went wrong and
where, if the previous models do not deliver the outcomes in correla-
tion with the business requirements;
• framing open-ended questions on the business;
• examining data from a variety of angles to determine hidden weak-
nesses, trends, and opportunities.
From this evidence, it has emerged that the boundaries of the job of
these professional roles seem to need further specification in order to
redistribute in a more effective way their time toward more highly valued
activities that can lead to the actions of specific competencies such as lat-
eral thinking, questioning, observing, experimenting, strategic thinking,
and opportunity recognition. Even though the work situations in which
data scientists and data analysts can deploy the aforementioned competen-
cies are less frequent than situations in which other areas (such as action or
social) can be manifested, when exploratory and organizational action
competencies are activated, they may generate even a higher impact in
comparison to other behaviors. Consequently, not all competencies are
required to be activated at the same level, but they need to be present in
the behavioral repertoire of data scientists and data analysts, so that they
can manifest them when they are required. As the companies adopt a more
data-driven culture and enlarge and empower their data science team, they
will be able to exploit the full potential of their data scientists and
data analysts.
References
Aasheim, C., Shropshire, J., Li, L., & Kadlec, C. (2012). Knowledge and skill
requirements for entry-level IT workers: A longitudinal study. Journal of
Information Systems Education, 23, 193–204.
Boyatzis, R. (1982). The competent manager: A model for effective performance.
New York: Wiley Interscience.
Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis
and code development. Thousand Oaks: Sage.
Boyatzis, R. E. (2006). An overview of intentional change from a complexity per-

spective. Journal of Management Development, 25(7), 607–623.
Boyatzis, R. E. (2008). Competencies in the 21st century. Journal of Management
Development, 27(1), 5–12.
Boyatzis, R. E. (2009). Competencies as a behavioral approach to emotional intel-
ligence. Journal of Management Development, 28(9), 749–770.
Boyatzis, R. E., & Ratti, F. (2009). Emotional, social and cognitive intelligence
competencies distinguishing effective Italian managers and leaders in a private
company and cooperatives. Journal of Management Development,
28(9), 821–838.
Boyatzis, R. E., & Sala, F. (2004). Assessing emotional intelligence competencies.
In G. Geher (Ed.), The Measurement of Emotional Intelligence (pp. 147–180).
Novas: Hauppauge, Science Publishers.
Costa, C., & Santos, M. Y. (2017). The data scientist profile and its representativeness
in the European e-competency framework and the skills framework for the infor-
mation age. International Journal of Information Management, 37(6), 726–734.
century. Harvard Business Review, 90(October 2012), 70–76.
Davenport, T. H., Harris, J. G., & Morison, R. (2010). Analytics at work: Smarter
decisions, better results. Boston, MA: Harvard Business Review Press.
De Mauro, A., Greco, M. Grimaldi, M., and Nobili, G. (2016, June 15–17).
Beyond data scientists: A review of big data skills and job families. International
Forum on Knowledge Asset Dynamics 11th, Proceedings of IFKAD, 2016,
Towards a New Architecture of Knowledge: Big Data Culture and Creativity,
Dresden (1844–1857).
Di Deo, I. (2019). Data Analyst: chi è, cosa fa e quanto guadagna un analista dei
dati. Retrieved July 1, 2019, https://blog.osservatori.net/it_it/data-analyst-
cosa-fa-stipendio
Druskat, V. U., Mount, G., & Sala, F. (2013). Linking emotional intelligence and
performance at work: Current research evidence with individuals and groups.
New York: Psychology Press.
Emmerling, R. J., & Boyatzis, R. E. (2012). Emotional and social intelligence
competencies: Cross cultural implications. Cross Cultural Management: An
International Journal, 19(1), 4–18.
Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin,
51(4), 327–358.
Goleman, D. (1998). Working with emotional intelligence. New York:
Bantam Books.
Goleman, D., Boyatzis, R., & McKee, A. (2002). Primal leadership: Realizing the
power of emotional intelligence. Boston, MA: Harvard Business School Press.
Gorman, J. C., Dunbar, T. A., Grimm, D., & Gipson, C. L. (2017). Understanding
and modeling teams as dynamical systems. Frontiers of Psychology, 8, 1053.
Hopkins, M. M., O’Neil, D. A., & Stoller, J. K. (2015). Distinguishing competen-
cies of effective physician leaders. Journal of Management Development,
34(5), 566–584.
Istat. (2018). Indicatori Industria 4.0-Fattori di digitalizzazione, Istituto
Nazionale di Statistica, http://dati.istat.it. Accessed 1 May 2017.
Joseph, D., Ang, S., Chang, R. H. L., & Slaughter, S. (2010). Practical intelli-
gence in IT: Assessing soft skills of IT professionals. Communications of the
ACM, 53(2), 149–154.
King, J., & Magoulas, R. (2016). 2016 Data science salary survey tools, trends, what
pays (and what doesn’t) for data professionals. Sebastopol: O’Reilly Media.
McClelland, D. C. (1973). Testing for competency rather than for ‘intelligence’.
American Psychologist, 28, 1–14.
McClelland, D. C. (1998). Identifying competencies with behavioural-event
interviews. Psychological Science, 9, 331–339.
O’Boyle, E. H., Jr., Humphrey, R. H., Pollack, J. M., Hawver, T. H., & Story,
P. A. (2011). The relation between emotional intelligence and job perfor-
mance: A meta-analysis. Journal of Organizational Behavior, 32, 788–818.
Ryan, G., Emmerling, R. J., & Spencer, L. M. (2009). Distinguishing high-perform-
ing European executives. Journal of Management Development, 28(9), 859–875.
Scapolan, A., Montanari, F., Bonesso, S., Gerli, F., & Mizzau, L. (2017).
Behavioural competencies and organizational performance in Italian perform-
ing arts. Academia Revista Latinoamericana de Administración, 30(2), 192–214.
Spencer, L. M. (2001). The economic value of emotional intelligence competen-
cies and EIC-based HR programs. In C. Cherniss, R. E. Boyatzis, & M. Elias
(Eds.), Emotionally intelligent workplace (pp. 45–82). San Francisco: The
Jossey-Bass Wiley Company.
Spencer, L. M., & Spencer, S. M. (1993). Competency at work. Models for superior
performance. New York: John Wiley & Sons.
Spencer, L. M., Ryan, G., & Bernhard, U. (2008). Cross-cultural competencies in
a major multinational industrial firm. In R. J. Emmerling, V. Shanwal, &
M. Mandal (Eds.), Emotional intelligence: Theoretical and cultural perspectives
(pp. 191–208). New York: Nova Science.
Stubbs Koman, E., & Wolff, S. B. (2008). Emotional intelligence competencies in
the team and team leader. Journal of Management Development, 27(1), 55–75.
Suda, B. (2018). 2017 data science salary survey. Sebastopol: O’Reilly Media.
Truninger, M., Fernández-i-Marín, X., Batista-Foguet, J. M., Boyatzis, R. E., &
Serlavós, R. (2018). The power of EI competencies over intelligence and indi-
vidual performance: A task-dependent model. Frontiers of Psychology, 9, 1532.
Vercellis, C. (2018). Big Data Analytics in Italia: un mercato in evoluzione.
Retrieved July 1, 2019, https://blog.osservatori.net/it_it/mercato-big-data-
analytics-in-italia
CHAPTER 5
Managing Big Data Professionals through

a Competency-Based Approach
Abstract Behavioral competencies are more complex and more difficult

to assess and to develop than technical skills. On one hand, companies still
struggle to accurately assess behavioral competencies of big data profes-
sionals during the recruiting process. On the other hand, higher education
institutions are still not adequately equipping students enrolled in data
science and analytics degree programs with the soft skills requested by
employers. The chapter provides methodological recommendations to
design courses for competency development based on the Intentional
Change Theory. Also, it offers insights for human resource management
specialists to revise the way they search and assess behavioral competencies
of big data profiles to improve the effectiveness of the hiring process.
Keywords Competency development • Intentional change theory •

Higher education • Human resource management • Data science
While hard skills may get a candidate’s foot in the door, it’s soft skills
that ultimately open it.
(Lydia Liu, Head of HR, Home Credit Consumer Finance Co. Ltd.,
quoted in LinkedIn, 2019 Global Talent Trends)

https://doi.org/10.1007/978-3-030-33578-6_5
5.1 Behavioral Competencies as a Decisive Factor

in Hiring Outstanding Big Data Professionals
When Lorenzo became involved in his first data project in his new com-
pany, he had just successfully completed his Master’s degree in data sci-
ence. He also had a Bachelor’s degree in computer science, had served
prior internships where he strengthened his coding and programming
skills and improved his knowledge of Python and R, and had developed a
project in the field of machine learning in collaboration with the univer-
sity from which he graduated. He was asked to work on raw data from an
external source, documenting the data cleaning and the subsequent
implementation of an algorithm. However, the outcomes he presented
did not meet the company’s needs. How did not Lorenzo live up to the
hype? What we have learnt in the previous chapters is that big data profiles
require multiple skills, from technical to behavioral competencies, to
attain outstanding performance. Lorenzo failed in his job for three
main reasons.
Firstly, he was not aware that his ultimate mission was to solve a busi-
ness problem, not simply to analyze data or build a great model. He was
too focused on the technology part of his job: he acted as a data geek
instead of asking himself questions about how the company would benefit
from the data, interacting with the executives to better frame the problem,
highlighting the output of the model that stakeholders cared about most,
removing technical jargon when explaining the work done to non-
technical people, simplifying the analytical solution and tailoring it to the
company’s needs, changing his approach after the preliminary feedback
received, and paying more attention to the replicability of the solution in
order to pursue efficiency. In other words, he did not manifest the behav-
ioral competencies considered critical for big data professions like strategic
thinking, customer focus, empathy, adaptability, and efficiency orienta-
tion, as discussed in Chap. 3.
The second reason for the failure is that the recruiter probably did not
adequately consider and assess the behavioral competencies of the appli-
cants for this position. This can be attributed to the bias that affects big
data professionals, who are often seen only as numerically-minded indi-
viduals. Big data roles have not long existed: consequently, companies do
not yet have a clear understanding of the responsibilities, skillset, and ulti-
mate contribution to the firm’s performance, especially in those organiza-
tional contexts that are only now embarking upon a data-driven culture.
5 MANAGING BIG DATA PROFESSIONALS… 91
Key findings from the last LinkedIn Global Talent Report (LinkedIn
2019) revealed that 89 percent of the firms interviewed said bad hires
typically lack soft skills. Companies seem still to struggle to accurately
assess behavioral competencies, and if coding or analytics skills are easier
to evaluate during the recruiting process, identifying soft skills seems
much harder. This explains why this gap emerged too late.
The last reason can be ascribed to the misalignment between graduates’
skills and employer’s expectations in today’s labor market. The skills short-
age refers not only to data science and analytics competencies, as discussed
in Chap. 2, but also to behavioral competencies. Soft skills have become
even more difficult to hire, especially since the rise of automation and
artificial intelligence that makes emotional intelligence even more valuable
(Beck and Libert 2017; Gustein and Sviokla 2018; McKinsey Global
Institute 2018; World Economic Forum 2019). For instance, creativity
(QS Intelligence Unit 2019; LinkedIn 2019) – that is one of the distinc-
tive characteristics of data scientists – represents the most in-demand skill
at the global level and encompasses the cognitive and the exploration
competencies included in the Competency Hexagon (Chaps. 3 and 4),
like lateral thinking, pattern recognition, questioning, observing, and
experimenting, all competencies that machines cannot easily replicate. A
recent study from McKinsey predicts that as automation is progressively
introduced, the demand for such competencies will sharply increase by
2030 (McKinsey Global Institute 2018). According to the QS report ‘The
Global Skills Gap in the 21st Century’ (QS Intelligence Unit 2019),
employers also show a low level of satisfaction with graduates’ profiles in
other behavioral competencies, such as resilience, adaptability, leadership,
and persuasion. Although there is a general agreement that higher educa-
tion institutions (HEIs) should equip students with these competencies
before they enter the labor market, contributing to fill the gap and meet
companies’ expectations, effective initiatives in this direction are still lim-
ited (Ritter et al. 2018).
The next sections of this chapter will address these issues. Firstly, some
methodological recommendations for HEIs will be introduced on how to
implement competency-based courses within their degree programs in
data science and big data. Subsequently, the chapter will provide insights
for human resource management specialists on how to revise the way they
search and assess big data profiles by adopting a competency-
based approach.
5.2 Developing Behavioral Competencies in Data

Science and Big Data Academic Programs
Behavioral competencies are more complex and difficult to develop than
technical skills; thus pursuing their development can face obstacles, from
the different roles instructors have to assume, to the active learning strate-
gies and the different training tools they need to adopt (Bedwell et al.
2014). These elements may discourage the introduction of dedicated aca-
demic courses in Bachelor and Master’s programs and in the field of data
science and big data, as highlighted in a recent study that investigated how
academic curricula are meeting the industry’s analytics needs (Bowers
et al. 2018).
As demand for big data professionals has seen exponential growth in
recent years and this is expected to increase in the future, as discussed in
Chap. 2, higher education institutions are still trying to provide an ade-
quate answer to the skills shortage, including in their offerings dedicated
programs in data science, analytics, and big data. The introduction of spe-
cific courses aimed at developing behavioral competencies within these
educational programs seems to be hampered by several factors. One is the
lack of awareness among the faculty members in charge of designing the
degree programs of the skills necessary for big data jobs to achieve positive
performance at work. Another factor is the rapid technological changes
that continuously affect the skillset of such professionals. The need to
equip students with the most recent advancements in tools and techniques
has led universities to adopt a more technologically-oriented approach in
designing educational programs. Moreover, even where faculty members
acknowledge that aspiring big data professionals should learn emotional
intelligence competencies as part of their academic experience, they face
credit hours limits, with the consequent difficulty of motivating at the
institutional level the decision to leave a technical course out of the pro-
gram in favor of a non-technical one (Bowers et al. 2018).
Since the primary objectives of HEIs include to prepare people to
become outstanding professionals (Boyatzis and Saatcioglu 2008), to fully
meet the labor market requirements, and to increase students’ employ-
ability, more emphasis needs to be placed on emotional intelligence in the
degree programs. Dedicated learning experiences should be introduced to
allow students who want to pursue a career in the big data field: (i) to
become aware of the behavioral competency profile that their future pro-
fessional role is expected to manifest in the workplace; and (ii) to acquire
the methodology and related techniques to develop the necessary behav-

ioral competencies.
An effective methodological approach to competency development has
been proposed by those academic programs that have espoused the pro-
cess of intentional change to skills development (Boyatzis 2006; Boyatzis
and Saatcioglu 2008; Boyatzis et al. 2002). The intentional change theory
(ICT) is based on the ‘whole person’ pedagogical approach, wherein indi-
viduals are engaged in self-directed development, since “learning does not
occur until the learner makes it happen” (Hoover et al. 2010: 194), espe-
cially when the subject of learning is new effective behaviors that aim to
substitute established ineffective habits.
The two fundamental aspects of effective application of the intentional
change process are desired and sustainable change. Change – and conse-
quently the development of behavioral competencies – is desired by the
person, since learning goals are set intentionally by individuals in line with
their desired personal and professional life. Differently from the majority
of training experiences, the change is also sustainable, in that it lasts for a
relatively long time, in contrast to the majority of soft skills training pro-
grams in which individuals experience the so called ‘honeymoon effect’ –
that is, a short-term improvement followed by a decline within a few
months (Boyatzis 2006).
The process of intentional change has been successfully applied for
competency development in international higher education contexts
(Boyatzis et al. 2002, 2010; Boyatzis and Saatcioglu 2008). It has shown
its positive learning outcomes in the improvement of students’ compe-
tency portfolio in longitudinal studies, and in terms of career satisfaction
and success (Amdurer et al. 2014). The learning process involves five dis-
coveries or discontinuities – also defined as epiphanies, which are effec-
tively wake-up calls, or moments that awaken the person to the need to
consider a change – that lead to successive improvements in behavior and
subsequent competency development (Boyatzis 2006; Kolb and Boyatzis
1970; Leonard 2008). Table 5.1 offers a representation of the process of
intentional change.
The first discovery – understanding the ideal self and creating a per-
sonal vision – leads individuals toward a mindful reflection on what mat-
ters most to them and on who they want to be (their ideal self) in terms
of their passions, purpose, desired future, and core values. These compo-
nents are integrated and expressed in a personal vision statement. In so
doing, the change process becomes grounded in intrinsic motivations
Table 5.1 A representation of the five discoveries in Intentional Change Theory
Discontinuity 1. Discontinuity 2. Discontinuity 3. Discontinuity 4.

My Ideal Self My Real Self My personal learning Experimenting and
Who do I want to Who am I? What agenda practicing the new
be? my Ideal and Real Which capabilities behavior until mastery
Self are similar / should I develop to How and where I can
different? attain the desired experience the new
future? behavior in actual safe
settings?
Discontinuity 5. Trusting relationships that encourage each stage in the process

Who can help me in my learning path?
and beliefs in possibility, spurring individuals to be more resilient and

increasing in them a feeling of hope. The second discovery – assessing the
real self and comparing it to the ideal self – supports individuals in increas-
ing their awareness of their current self. Usually during this phase indi-
viduals are involved in a behavioral competency assessment that allows
them to understand the actual level of mastery of the behavioral compe-
tencies. The interpretation of feedback is crucial for identifying individual
strengths and weaknesses, respectively those areas in which the ideal and
real self overlap, and those areas in which the desired future is not consis-
tent with the current self. These first two discoveries, stimulating indi-
viduals to ask themselves “Who do I want to be?” (ideal self) and “Who
am I?’ (real self), help to nurture individual self-awareness which, as dis-
cussed in the previous chapters, is a behavioral competency considered a
cornerstone of the emotional intelligence framework and the premise for
subsequent development of the other behavioral competencies
(Goleman 1998).
The third discovery – creating a learning agenda or plan to close the
gap between the real and ideal selves – leads individuals to set learning
goals and actions that they enthusiastically look forward to testing. This
phase concretely moves a person closer to his or her vision by identifying
ways to foster strengths and reduce the identified gap. The fourth discon-
tinuity – practicing new competencies and behaviors – is dedicated to
actual experimentation with the new behaviors in different personal and
professional contexts. The last discovery is focused on identifying and
building trusted relationships that support and encourage each step in the
change process.
The following section will discuss in more detail how the different dis-
coveries can be tailored in dedicated academic courses to nurture the
behavioral competencies of big data professionals.
5.3 Introducing Wake-Up Calls in Data Science

and Big Data Educational Programs
The story narrated at the beginning of this chapter is an exemplar case of

a graduate in the early stages of his career who demonstrates a strong
vocational orientation toward the field of big data but who has not yet
developed the necessary self-awareness to become attuned to himself, oth-
ers, and the organizational environment, a state of mind also known as
mindfulness. Recent research shows that mindful engineers are more able
to generate new ideas, think outside the box, and find better solutions to
problems (Rieken et al. 2019). Moreover, mindful executives make more
effective choices about how to respond to people and situations, have
stronger working relationships, achieve better project outcomes, increase
budgets and team headcount, and attain better career outcomes (Goleman
and Lippincott 2017).
The Intentional Change Theory offers insight into how to practice
mindfulness, helping individuals, on the one hand, to keep in mind why
they are doing what they do, to discover the calling in their career, and
to develop an image of their desired future (the ideal self discontinuity),
and, on the other hand, to reflect on their inner capabilities and areas
for improvement to effectively direct personal change (the real self
discontinuity).
Within a course dedicated to the development of behavioral competen-
cies for big data roles, self-awareness can be stimulated through several
experiential learning activities, summarized in Table 5.2. During the first
wake-up call, ‘Who do I want to be?’, students should be introduced to
deep self-reflection on their future professional dreams. The process of
visioning helps the individual to imagine tasks in which he/she will be
involved in the job, the main responsibilities and interactions within the
work environment, and the ultimate purpose of his/her activity. For
instance, an aspiring data scientist or data engineer can ask him/herself:
What kind of business problems do I want to address? What kind of data
do I prefer to work with? In which industry do I wish to operate? In what
way does this job help me put into practice my inner values and my pas-
sions? What will be my contribution to the performance of the company?
Table 5.2 Experiential activities for developing self-awareness of the inner iden-
tity and future work self, and the necessary behavioral competency profile
Discovery and key Experiential learning activities
question
Ideal self who do I Self-reflection on values, passions, and dreams, and behavioral
want to be in the competencies necessary to achieve the future
future? Writing ‘my personal vision’ essay
Feedback from the facilitator
Coaching and peer coaching
Vicarious learning through narratives of work experiences and
hiring process from professionals, employers, and recruiters in the
data science field
Real self Assessment of behavioral competencies (multisource assessment or
Who am I? behavioral event interviews)
Feedback from the facilitator
Coaching and peer coaching
The more detailed the description of the desired professional life, the
easier it will be for the students to think about the behavioral competen-
cies necessary to perform the job outstandingly independently of their
actual level of mastery. An important role in this phase of discovery of the
ideal self – namely, linking the vision of the desired profession with the
behavioral skillset necessary to achieve it – is played by the instructor, who
assumes the role of a learning facilitator and can activate the five discover-
ies of Intentional Change Theory. Indeed, trusting relationships have an
important role in helping and supporting the individual during each dis-
covery of the intentional change process.
Specifically within ideal self discovery, this discontinuity consists of
leveraging the person’s key relationships in order better to envision the
desired future. By talking with other people, sharing their views of the
future, and receiving feedback from them, the individual gains help and
encouragement to really look inside themselves and find the motivation to
pursue the change process. Facilitators may stimulate participants toward
self- and group reflection, helping them to become more aware of the job
opportunities in the big data field, delineating the boundaries among dif-
ferent roles, and favoring dialogue with role models who can inspire a
more concrete visualization of the future. Indeed, role modeling is often
mentioned as one of the techniques to improve mindfulness (Goleman
and Lippincott 2017) and, in the specific case of ideal self discontinuity,
interaction with individuals who already perform big data roles may pro-
vide further opportunities to gather insights into the job’s main character-
istics. Through these professionals’ narratives, students may be involved in
a form of vicarious learning, getting acquainted with the situations in
which they will probably be most frequently involved in their future career
and with the behavioral competencies they will be required to manifest.
During the narration of role models’ experiences, the facilitator may help
the students to read the behaviors activated by the professionals through
the lens of the behavioral competency framework. To increase students’
awareness of the competency profile expected in the desired professional
roles, facilitators should engage participants in a conversation with labor
market operators, such as participating in recruiting events dedicated to
big data professions and consulting online job advertisements in the area
of data science.
Besides activating trusting relationships with the instructor or with big
data professionals, potential employers, and recruiters interested in data
science profiles, the participants can be invited to share their image of their
professional future with the other participants on the course through a
peer coaching experience. Peer coaching is a helpful, mutually beneficial
relationship with the goal of personal or professional development. It is
based on qualities such as unconditional positive regard, authenticity,
mutual trust, and reciprocity of the process (Parker et al. 2008, 2015).
Past evidence has shown positive outcomes of coaching and peer coaching
in educational programs in terms of increased self-confidence, empower-
ment, self-understanding, success in dealing with change, and develop-
ment of soft skills (Boyatzis et al. 2006; Gerli et al. 2019; Goldman et al.
2011; Parker et al. 2008). In a peer coaching session, participants support
each other’s development by listening, asking what may sometimes be
provocative questions, and giving critiques, in order to promote clarity
from the narrator (Kotlyar et al. 2015). The attention is on the whole
person and, through a process of reflection, peers reciprocally build aware-
ness of the cognitive and affective aspects of their desired future. The
ultimate aim of the coaching session is to help each peer to achieve a
deeper level of analysis and reflection on personal values, passions, and
future dreams, to make hidden assumptions explicit. As suggested by
Parker et al. (2008: 491) “the peers engage in shared sense-making of
each other’s worldview.”
After this deep exploration of one’s inner identity and of the desired
future, the reflections developed during the first discovery can be
incorporated in a written essay, ‘My Personal Vision’, in which the partici-

pants describe their compelling path to their desired career five or ten
years in the future (McKee et al. 2008). A prominent part of the personal
vision is devoted to narration of the future work self, or to the individual’s
representation of their hopes and aspirations for their future working life
(Strauss et al. 2012). As prior studies suggest, the clearer and easier it is for
a person to imagine their future work self, the more this image represents
a motivational driver for proactive career behaviors, such as career plan-
ning, competency development, career consultation, and network build-
ing (Strauss et al. 2012).
To orient the learning path toward competency development, individu-
als should become aware of their strengths and areas for improvement.
For this reason, in the second discovery, students are stimulated to com-
pare their current level of mastery of behavioral competencies with the
competency profile expected in their desired professional profile. Analysis
of the actual behavioral profile can be conducted with reference to two
main methodological approaches to competency assessment. The first
consists of multisource feedback, in which their own and external raters’
evaluations are combined to provide a comprehensive view of the most
and least frequently manifested behaviors of the student. For instance, the
Emotional and Social Competency Inventory – University Version
(ESCI-U) is one of the most effective 360-degree competency assessment
tools widely adopted in both academic and organizational settings
(Boyatzis et al. 2015; Boyatzis and Goleman 2007). One of the primary
benefits of using 360-degree assessment is to collect different perspectives
on the person from both the personal and professional contexts. Identifying
“if the person uses the competency behavior at home or in leisure settings
provides a more thorough review of the person’s range of action. It also
suggests different tactics in helping the person change their behavior in
either work or personal settings” (Boyatzis 2018: 9).
The second approach refers to a third-party evaluation by administering
behavioral event (Camuffo and Gerli 2004; Spencer and Spencer 1993)
through which the interviewee reconstructs how the person activated spe-
cific behavioral competencies in past concrete situations (see Chap. 4 for a
more detailed description of this methodology). The behavioral compe-
tencies included in the competency hexagon, illustrated in Chap. 3 and
used to explore the behavioral profiles of data scientists and analysts, may
represent another tool that can be adopted in the assessment process.
Through in-class and individual discussion of this assessment, guided

by the facilitator and integrated with peer coaching and coaching sessions,
students become aware of their current behavioral portfolio and compare
their ideal and real selves, reflecting on their strengths (competencies that
are indicated as necessary if they are to obtain their personal vision and
that they demonstrate most) and areas of improvement (competencies
that are indicated as necessary for their personal vision but that they dem-
onstrate least).
The core activity in which students are involved in the third discovery
is drafting a learning plan. Therefore, for each competency chosen as a
learning goal, the students define timeframes and concrete sets of actions
they aim to practice in everyday contexts. In order to be effective and
realizable, the learning plan has to be customized based on personal char-
acteristics, needs, and learning styles (Kolb and Kolb 2005; Goleman et al.
2002). The definition of the plan represents a form of active learning,
since it “helps participants move rapidly between theoretical ideas and
efforts to apply those ideas in the context of their work organizations and
within their projects” (Waddock and Lozano 2013: 276). In this stage of
the process, the fifth discovery can be activated by means of social learning
through interaction with peers. The instructor can support students in the
writing of their plan, offering the opportunity to discuss in small teams the
learning actions to be implemented. Specifically, facilitators can invite stu-
dents to form small groups in which participants who aim to learn the
same competencies but may prefer different learning styles can discuss
possible concrete actions that they can implement to experience the new
behaviors. Trusting relationships with those whom the new behaviors are
practiced can also be specified by the student in the plan. The learning
plan can be revised by the instructor, who can provide valuable insights
into how to make it more concrete and actionable.
Once the learning plan has been defined, it can be put into practice in
the fourth discovery. This discontinuity represents the step in which the
person, in line with his/her learning plan, experiments with and practices
new behaviors. Changing requires long-lasting practice, reflection, and
repetition of new behaviors, allowing new habits to be mastered and
applied in all contexts (Boyatzis 2006). Every personal context can be
conceived as a learning laboratory where, in safe environments, students
can become familiar with the new behaviors without being afraid of nega-
tive feedback. However, in order to practice behavioral competencies in
situations similar to those that students will face in their data science
rofession, they can use a capstone project or an internship experience

p
during their academic path as a learning context. A capstone project, as a
form of project-based learning, is frequently adopted in data analytics
degree programs (Bowers et al. 2018) and has been shown to be effective
for facilitating practice of behavioral competencies (English and Kitsantas
2013) through the involvement of students in real data science problems,
defined by companies, that should be solved cooperatively.
Finally, in order to monitor students’ progression along their learning
path, a follow-up session can be organized some months after the end of
the course. A few days before the follow-up, the facilitator can ask students
to share with the class a presentation in which they describe how they have
put into practice the learning plan created during the course. Specifically,
the content of the presentation may include: an indication of the compe-
tencies that the students aimed to learn and the relevance of these for
attaining their personal vision; a brief presentation of their learning plan;
the contexts in which they experimented with new behaviors; the way in
which they evaluated their learning progress and their level of mastery of
the competencies; the difficulties they encountered in practicing the new
behaviors; and how their trusted relationships supported them in attaining
their results. The follow-up session may provide an opportunity for vicari-
ous learning, since students who aimed to develop the same emotional or
social competency can: i) compare their experience with that of their peers,
understanding how to remove the obstacles they encountered during their
experimentation with new behaviors; ii) learn new techniques to practice
the competency; and iii) identify new actions that are related to the same
competency but adopt a different learning style. At the end of the presen-
tation of each experience, the instructor can provide further advice that
helps to direct students’ efforts toward effective practice of the competen-
cies. Moreover, students can be involved in further experiential learning
initiatives (courses, seminars, laboratories, etc.) dedicated to the develop-
ment of specific behavioral competencies in order to acquire techniques
that allow them to put the new behaviors into practice in their daily
activities.
This section has provided some methodological and didactic insights
into how to design a learning experience within academic data science
programs through which students may increase their awareness of the
importance of behavioral competencies for big data professionals and
undertake a path toward their development.
As highlighted by A. Charles Thomas, PhD in sociology and Chief

Data & Analytics Officer at General Motors, during his keynote speech at
the 2019 Wharton Customer Analytics conference, “academic programs
that train data scientists should include a focus on the whole person, not
just the science. Otherwise the result is data scientists who can get too
wrapped up in the breadth of data they’re handling and lose focus on
actionable insights” (Knowledge@Wharton 2019). The recommendations
offered in this section go in this direction. They can also be taken into
consideration by corporate academies in their training courses, and by
training managers when identifying the training needs and the training
programs to offer employees.
5.4 How to Make Hiring more Emotional

and Successful
When every candidate has the same expertise in Hadoop, or demonstrates

the same familiarity with NoSQL databases, how can you distinguish
between them?
A growing number of international companies, like Google, L’Oréal,
and AT&T, have introduced emotional intelligence as a key element of
assessment in their hiring processes, attaining positive outcomes in terms
of higher productivity, employee engagement, and profit. If, at the educa-
tional level, universities play a central role in equipping students with both
the technical and the behavioral skills necessary to achieve outstanding
performance in big data jobs, at the employer level, companies can reduce
failures in the hiring process, recruiting for soft skills and designing human
resource management practices according to a competency-based approach.
LinkedIn’s Global Talent Trends (LinkedIn 2019) reveals that 57 per-
cent of companies still struggle accurately to assess soft skills in their can-
didates, and that they continue to adopt unstructured approaches during
the selection process based on recruiters’ personal feelings. But how can
companies improve their ability to recruit for behavioral competencies?
Firstly, they should take into consideration the compatibility or fit
between the competencies valued most by the company itself and the
competency profile of the applicants. Executives, in collaboration with the
HRM department, can benefit from defining the behavioral competencies
that the data science team needs to manifest in order to promote a data-
driven culture inside the organization and to succeed in its digital
t ransformation. Moreover, these competencies have to be indicated explic-

itly in the job description and in job advertisements to orient candidates’
applications. As illustrated in Chaps. 2 and 4, the different big data profiles
(data architect, data engineer, data scientist, etc.) are characterized by dif-
ferent technical and soft skillsets.
Secondly, recruiters can be trained in administering and codifying
behavioral event interviews in order to assess candidates’ level of mastery
of the key competencies for that specific position, addressing questions
such as: “Tell me about a time when you had to deal with a highly demand-
ing customer. How did you manage the relationship?” “Have you ever
been in a situation in which you needed to adjust your behavior? How did
you know and what did you do?” “Tell me the last time you introduced an
innovation in your company.” For instance, the Chief Data & Analytics
Officer at General Motors spurred the introduction of questions into GM’s
interview process that would help determine whether candidates had an
aptitude for insights. As he said, “Every single person should have an apti-
tude for insights. This is no longer a back-office activity. It’s at the table
and it should be part of how we make decisions” (Knowledge@Wharton
2019). Moreover, gamification can be integrated in the recruitment pro-
cess to attract prospective big data profiles and assess their soft skills while
candidates are engaged in a simulated work environment. Gamification has
been applied during the hiring process to make assessment methods more
game-like, improving candidates’ reactions and consequently increasing
the accuracy of the prediction of future work behaviors. Recent evidence
shows that game elements, such as storylines, feedback, avatars, visuals,
and voiceovers, effectively assess candidates’ soft skills such as resilience,
adaptability, and decision-making (Georgiou et al. 2019). Assessment of
behavioral competencies during the hiring process represents an opportu-
nity for the newly hired to understand their level of mastery of the key
behavioral competencies for their job and the organizational culture.
A relevant role in promoting individual behaviors coherent with the
organization’s goals is played by the induction program, which should be
structured in a way that allows the new employee to gain familiarity with
the organizational environment and the job requirements. During that
period, the skillset of the professional profile should be made explicit, and
periodic feedbacks should be provided to support the individual to redi-
rect his/her behaviors to better meet the job and company expectations.
Specifically, feedback should be used: i) to help the newly hired to reflect
on the behaviors activated and the outcomes attained; ii) to provide exam-
ples of how to activate the appropriate combination of behavioral compe-

tencies relevant to face specific work situations effectively; and iii) to
provide guidance on how to improve areas of strength further and at the
same time work on areas for improvement as assessed during the hir-
ing process.
All the aforementioned practices aim to introduce into the organiza-
tional context a more fine-grained approach of the analysis of behavioral
competencies in order to improve the recruitment and assessment of suit-
able big data profiles.
Despite their relevance, behavioral competencies are still ill-defined for
these professions, and further work is required by both the academic and
the practitioner communities to understand their effective combination
for the different big data job families. The practical insights provided here
may help companies not to lose time searching for ‘unicorns’, but to hire
the skills that can help address actual business problems and meet spe-
cific needs.
References
Amdurer, E., Boyatzis, R. E., Saatcioglu, A., Smith, M. L., & Taylor, S. N. (2014).
Long term impact of emotional, social and cognitive intelligence competencies
and GMAT on career and life satisfaction and career success. Frontiers in
Psychology, 5, 1447.
Beck, M., & Libert, B. (2017, February 15). The rise of AI makes emotional intel-
ligence more important. Harvard Business Review.
Bedwell, W. L., Fiore, S. M., & Salas, E. (2014). Developing the future workforce:
An approach for integrating interpersonal skills into the MBA classroom.
Academy of Management Learning & Education, 13, 171–186.
Bowers, M. R., Camm, J. D., & Chakraborty, G. (2018). The evolution of analyt-
ics and implications for industry and academic programs. INFORMS Journal
on Applied Analytics, 48(6), 487–499.
Boyatzis, R. E. (2006). Intentional change theory from a complexity perspective.
Journal of Management Development, 25(7), 607–623.
Boyatzis, R. E. (2018). The behavioral level of emotional intelligence and its mea-
surement. Frontiers in Psychology, 9, 1–12.
Boyatzis, R. E., & Goleman, D. (2007). Emotional and social competency inven-
tory. Boston, MA: The Hay Group.
Boyatzis, R. E., & Saatcioglu, A. (2008). A 20-year view of trying to develop
emotional, social and cognitive intelligence competencies in graduate manage-
ment education. Journal of Management Development, 27, 92–108.
Boyatzis, R. E., Stubbs, E. C., & Taylor, S. N. (2002). Learning cognitive and
emotional intelligence competencies through graduate management educa-
tion. Academy of Management Learning and Education, 1, 150–162.
Boyatzis, R. E., Smith, M. L., & Blaize, N. (2006). Developing sustainable leaders
through coaching and compassion. Academy of Management Learning &
Education, 5, 8–24.
Boyatzis, R. E., Lingham, T., & Passarelli, A. (2010). Inspiring the development
of emotional, social and cognitive intelligence competencies in managers. In M.
G. Rothstein & R. J. Burke (Eds.), Self-management and leadership develop-
ment (pp. 62–90). Cheltenham: Edward Elgar Publishing.
Boyatzis, R. E., Gaskin, J., & Wei, H. (2015). Emotional and social intelligence
and behavior. In S. Goldstein, D. Princiotta, & J. A. Naglieri (Eds.), Handbook
of intelligence. Evolutionary theory, historical perspective, and current concepts
(pp. 243–262). New York: Springer.
Camuffo, A., & Gerli, F. (2004). An integrated competency-based approach to
management education: An Italian MBA case study. International Journal of
Training and Development, 8(4), 240–257.
English, M. C., & Kitsantas, A. (2013). Supporting student self-regulated learning
in problem- and project-based learning. Interdisciplinary Journal of Problem-
Based Learning, 7(2), 128–150.
Georgiou, K., Gouras, A., & Nikolaou, I. (2019). Gamification in employee selec-
tion: The development of a gamified assessment. International Journal of
Selection and Assessment, 27(2), 91–103.
Gerli, F., Bonesso, S., & Cortellazzo, L. (2019). Nurturing students’ awareness of
their behavioral competencies: The Competency Lab experience. 5th International
Conference on Higher Education Advances (HEAd’19). Valencia, Spain:
Universitat Politecnica de Valencia.
Goldman, E., Wesner, M., & Karnchanomai, O. (2011). Reciprocal peer coach-
ing: A critical contributor to implementing individual leadership plans. Human
Resource Development Quarterly, 24(1), 63–87.
Goleman, D. (1998). Working with emotional intelligence. New York:
Bantam Books.
Goleman, D., & Lippincott, M. (2017, September 08). Without emotional intel-
ligence, mindfulness doesn’t work. Harvard Business Review.
Goleman, D., Boyatzis, R. E., & McKee, A. (2002). Primal leadership: Realizing
the power of emotional intelligence. Cambridge, MA: Harvard Business
School Press.
Gustein, A. J., & Sviokla, J. (2018, July 17). 7 skills that aren’t about to be auto-
mated. Harvard Business Review.
Hoover, D. J., Giambatista, R. C., Sorenson, R. L., & Bommer, W. H. (2010).
Assessing the effectiveness of whole person learning pedagogy in skill acquisi-
tion. Academy of Management Learning and Education, 9, 192–203.
Knowledge@Wharton. (2019, July 24). Why future jobs will require data analytics
skills. Retrieved from https://knowledge.wharton.upenn.edu/article/
evolution-of-chief-data-officer/
Kolb, D. A., & Boyatzis, R. E. (1970). Goal setting and self-directed behavior
change. Human Relations, 23(5), 439–457.
Kolb, A. Y., & Kolb, D. A. (2005). Learning styles and learning spaces: Enhancing
experiential learning in higher education. Academy of Management Learning &
Education, 4(2), 193–212.
Kotlyar, I., Richardson, J., & Karakowsky, L. (2015). Learning to lead from out-
siders. The value of career communities as a source of external peer coaching.
Journal of Management Development, 34(10), 1262–1127.
Leonard, D. C. (2008). The impact of learning goals on emotional, social, and
cognitive intelligence competency development. Journal of Management
Development, 27(1), 109–128.
Linkedin. (2019). Global talent trends. The 3 trends transforming your workplace.
Retrieved from https://business.linkedin.com/content/dam/me/business/
en-us/talent-solutions/resources/pdfs/global_talent_trends_2019_emea.pdf
McKee, A., Boyatzis, R., & Johnston, F. (2008). Becoming a resonant leader:
Develop your emotional intelligence, renew your relationships, sustain your effec-
tiveness. Boston, MA: Harvard Business School Press.
McKinsey Global Institute. (2018). Skill shift: Automation and the future of the
workforce. Retrieved from https://www.mckinsey.com/~/media/mckinsey/
featured%20insights/future%20of%20organizations/skill%20shift%20automa-
tion%20and%20the%20future%20of%20the%20workforce/mgi-skill-shift-
automation-and-future-of-the-workforce-may-2018.ashx
Parker, P., Hall, D. T., & Kram, K. E. (2008). Peer coaching: A relational process
for accelerating career learning. Academy of Management Learning and
Education, 7(4), 487–503.
Parker, P., Wasserman, I., Kram, K. E., & Hall, D. (2015). Relational communica-
tion approach to peer coaching. Journal of Applied Behavioral Science,
51(2), 231–252.
QS Intelligence Unit. (2019). The global skills gap in the 21st century. Retrieved
from https://www.qs.com/portfolio-items/the-global-skills-gap-in-the-21st-
century
Rieken, B., Shapiro, S., Gilmartin, S., & Sheppard, S. D. (2019, January 04). How
mindfulness can help engineers solve problems. Harvard Business Review.
Ritter, B. A., Small, E. E., Mortimer, J. W., & Doll, J. L. (2018). Designing man-
agement curriculum for workplace readiness: Developing students’ soft skills.
Journal of Management Education, 42(1), 80–103.
Spencer, L. M., & Spencer, S. M. (1993). Competence at work: Models for superior
performance. New York: Wiley.
Strauss, K., Griffin, M. A., & Parker, S. K. (2012). Future work selves: How salient
hoped-for identities motivate proactive career behaviors. Journal of Applied
Psychology, 97(3), 580–598.
Waddock, S., & Lozano, J. M. (2013). Developing more holistic management
education: Lessons learned from two programs. Academy of Management
Learning & Education, 12, 265–284.
World Economic Forum. (2019). The future of jobs report 2018. Retrieved from
https://www.weforum.org/reports/the-future-of-jobs-report-2018
Index
A B
Algorithm design, 30 BARC’s BI Trend Monitor 2019, 8
Analytical tools Behavioral competencies
Apache Flume, 27 data analysts and, 14, 57, 64–85
Apache Pig, 27 data scientists and, 14, 57
Hadoop, 7, 27, 28, 30, 101 definition of, 101
Hive, 27 methods of, 16, 93, 98
MapReduce, 7, 28 Behavioral Event Interview (BEI), 16,
Oozie, 27 47, 66, 68, 102
Python, 9, 27, 28, 31, 56, 72, 90 Big data, 11, 24, 36, 66, 83, 85, 90,
R, 9, 27, 28, 31, 72, 90 101
Spark Hadoop, 27 challenges, 2–16
SQL, 5, 25, 27, 28, 31, 34 data driven culture, 66
T-SQL, 27 definition of, 4
Analytics Big Data Analytics & Business
descriptive, 8, 12 Intelligence (University of
predictive, 8, 12 Politecnico di Milano), 66
prescriptive, 8, 12 Big Data Analytics Market, 7
what they are, 8 Big Data and AI Executive Survey,
APEC, 23, 28 24
Artificial intelligence, 3, 4, 8, 13, 22, Big data professionals, 13, 16
30, 35, 91 business analysts, 13, 14
Awareness current workforce of, 23
data analysts of, 77, 78 data analysts, 15, 66
data scientists of, 77, 78 data scientists, 13–15, 66
definition of, 50 future scenario of, 35–36

https://doi.org/10.1007/978-3-030-33578-6
108 INDEX
Big data technology architectures, see Contingency theory approach, see

Analytical tools Boyatzis, Richard E.
Boyatzis, Richard E., 13, 43–45, 47, Costa, Carlos, 13, 14, 24, 28, 54–56, 65
49, 53, 54, 65, 66, 68, 92, 93,
97–99
Brynjolfsson, Eric, 2, 57 D
Burning Glass Technologies, 22–24, Data analysts
27, 30 behavioral competencies possessed, 15
Business analytics, see Big data education and background, 15
professionals job requirements, 33, 80
main tools used, 9, 11, 15, 92
Data analytics
C data mining, 6, 8, 12, 25, 30, 31, 42
Chade-Meng Tan (Meng), 42 data visualization, 25, 35
Cherniss, Cary, 47 data warehousing, 7
Chief data officer (CDO), 10, 15, Data analytics officer, 24
24 Data cleaning, 84, 90
Cluster analysis/clustering, 69 Data-driven culture, see Big data
Cognitive competencies, see Data professionals
Competency hexagon database administrator, 15, 25–27
Competency-based approach database architect, 15, 25–27
Boyatzis, R.E., 49 data engineer, 12, 15, 25–27, 33,
development, 93 66, 95, 102
phases, 66, 94 Data Science Salary Survey, 84
Competency codebook, see Data scientists
Competency hexagon behavioral competencies, 14, 16,
Competency hexagon, ix, 13, 36, 43, 36, 57, 64–85
57, 64, 65, 69, 78–80 education, 68
action competencies, 16, 50, 53, job requirements, 80
69–75, 84 main tools used, 15
awareness competencies, 16, 50, 53, profile, 14, 15, 57, 75, 83–85, 98
69, 77–78, 84, 97 Data scientists: the sexiest job of the
cognitive competencies, ix, 13, 16, 21st Century (Davenport and
36, 43, 50, 53, 57, 64, 65, 69, Patil), 28
78–80 Data workers, see Data professionals
exploration competencies, 47, Davenport, Thomas H., 2, 4, 8, 13,
91 24, 28, 29, 31, 33, 56, 65, 66,
organizational action competencies, 68, 80
16, 50, 55, 56, 69, 81–85 Deloitte, 13
social competencies, ix, 13, 14, 16, Delphy methodology, 36
26, 36, 44, 50, 53, 57, 64, 65, De Mauro, Andrea, 4, 34–36, 53, 55,
69, 75–76, 84, 100 68, 69
INDEX 109
E Human resource management, 43, 91,

Emmerling, Robert J., 68 101
Emotional and social intelligence key performance indicators, 36
competencies
relationship management
competencies, 33, 36, 44, 54 I
self-awareness competencies, 43, 44, Innovation, 49, 50, 80, 102
46, 53, 69, 77, 78, 94–96 Istat (Istituto nazionale di statistica),
self-management competencies, 43, 66
44, 53
social-awareness competencies, 43,
44, 53 J
Emotional intelligence Java, 26, 28
application, 65
definition of, 45
development, 43 L
Entry-level big data professions, 16 LinkedIn, 12, 23, 28, 29, 31, 36, 49,
EU General Data Protection 67, 91, 101
Regulation (GDPR), The, 83 Lisbon Council and International Data
Corporation (IDC), The, 23
F
Facebook, 28 M
Flanagan, John C., 68 Machine learning, 4, 6, 8, 9, 13, 28,
30, 31, 35, 85, 90
Manipulation of data, 5, 65
G McAfee, Andrew, 57
Gartner’s maturity model, 10 McClelland, David C., 43, 49, 65
Glassdoor’s 50 Best Jobs in America, McKinsey Global Institute, 2, 8, 9, 11,
28 12, 24, 49, 91
Goleman, Daniel, 13, 43–46, 49, 53, Mehrotra, Vijay, 29, 30, 54, 56, 66, 69
65, 83, 94–96, 98, 99 Modelling coding language
Google, 42 JAVA, 26, 28
Page, Larry, 57 Matlab, 28
Search Inside Yourself (SIY), 42 R, 9, 27, 28, 31, 90
trends, 6, 7 SQL, 5, 25, 27, 28, 31, 34
Granville, Vincent, 8, 27, 29, 30 Monte Carlo simulations, 30
H N
Hadoop, see Analytical tools Natural language processing, 30, 31
Hard skills, see Soft skills NewVantage Partners Executive
Harris, Jeanne G., 29, 30, 54–56, 66, 69 Survey, 11
110 INDEX
O Soft skills, 13, 15, 26, 31, 32, 35, 36,

Open-source tools, 30 49, 56, 57, 66, 68, 83, 91, 93,
Organizational structure 97, 101, 102
big data technology, 4 Spencer, Lyle M., 43, 47, 49, 53, 54,
data driven culture, 11, 24, 36, 66, 65, 66, 69, 98
85, 90, 101 SPSS predictive analytics software, see
manager roles in, 16, 49 Statistical software
Statistical analysis, 8, 28, 67
Statistical software
P R, 9, 27, 28, 31, 72, 90
Patil, Dhanurjay DJ, 4, 13, 28, 29, 56, SAS, 5, 31, 78
65, 66, 68, 80 SPSS, 31
Program language and tools, 8, 9, 11, Strategy
12, 15, 24, 26, 28, 30, 31, 33, data scientists and, 24, 32, 82
56, 76, 92, 98 strategic thinking, 32, 56, 82, 83,
See also Analytical tools 85, 90
PwC, 23 Structure data, 6, 30
Python, see Analytical tools Structured Query Language (SQL), 5,
25–28, 31, 34
Q
Quantitative analysis, 34 T
Training program
for data scientists, 30, 101
R for managers, 101
Relationship management, see Emotional
and social intelligence competencies
U
Unstructured data
S analysis of, 26, 28, 30
Sala, fabio, 65 processing, 27
Santos, Maribel Yasmina, 13, 14, 24, report, 26
28, 54–56, 65
Search Inside Yourself (SIY), see Google
Self-awareness, see Emotional and V
social intelligence competencies Visualization of data, 5, 25, 35
Self-management, see Emotional and
social intelligence competencies
Shirani, Ashraf, 54, 56, 68 W
Social-awareness, see Emotional and World Economic Forum, 3, 4, 22, 33,
social intelligence competencies 49, 57, 91

Behavioral Competencies of Digital Professionals

Uploaded by

Copyright:

Available Formats

You might also like

Behavioral Competencies of Digital Professionals

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Behavioral Competencies of Digital Professionals

Uploaded by

Copyright:

Available Formats

Behavioral Competencies

ISBN 978-3-030-33577-9 ISBN 978-3-030-33578-6 (eBook)

are the deeper, relatively stable characteristics we have, like openness or

Cleveland, OH, USA Richard E. Boyatzis

on their behavioral competencies. This book is a journey into the still

Venice, Italy Sara Bonesso

We are also grateful to the Scientific Committee of the Ca’ Foscari

1 The Organizational Challenges of Big Data 1

2 How Big Data Creates New Job Opportunities: Skill

3 Emotional and Social Intelligence Competencies in the

4 When Hard Skills Are Not Enough: Behavioral

5 Managing Big Data Professionals through a Competency-­

Sara Bonesso is Associate Professor of Business Organization and Human

(International Workshop on Human Resources Management), ICEI

Consortium for Research on Emotional Intelligence in Organizations. His

Table 1.1 Classification of types of big data analytical methods 9

The Organizational Challenges of Big Data

Keywords Big data • Data science • Data analytics • Organizational

The point is not to be dazzled by the volume of data, but rather to

© The Author(s) 2020 1

1.1 Competing in a Data-Driven Age

• Improvements in decision-making. Companies can use sophisticated

order to optimize the merchandising process and detect trends as

markets. Likewise, machine learning and cloud computing are receiving

• public data, held by government institutions and local communities,

• community data, which is unstructured data, usually in text format,

To unlock the potential of this high volume of fast-moving and diverse

1.2 Data Science: How to Extract Value from Big

• Data acquisition and warehousing. According to the 2017 Big Data

The second subprocess, data analysis, implies the use of techniques

Table 1.1 Classification of types of big data analytical methods

give the algorithm a massive number of “experiences” (training data) and a

1.3 Organizational Challenges for Leveraging

Companies Companies Companies Companies Data is crucial for

only refers to technical/analytics competencies but also encompasses soft

1.4 The Emotional Intelligence Side of Big Data

who are primarily involved in data modeling, analysis, and presentation

1.5 The Aim of This Book and Its Structure

How Big Data Creates New Job

Keywords Big data professional profiles • Skill shortage • Soft skills •

When people decide to pursue a career as an aircraft pilot, they are

© The Author(s) 2020 21

2.1 The Market for Big Data Jobs: Trends

• intelligent (advancement in artificial intelligence): autonomous

The diffusion of these technological breakthroughs is shifting the

similar trend characterizes the European Union: in 2018 the number of

2.2 Professional Roles in the Data Science Field

also known as database administrator, is more responsible for day-to-day

• applying technical design and development skills to the creation of

Structured Query Language is the primary language that database

Data warehousing solutions ✔

2.3 Data Scientist: Is It Still the Twenty-First

• Hacker: capability to code (the 2019 Big Cloud’s report on data

• Business expert: knowledge about the specific business enables the

Deriving a definition of data scientists through a Delphi study that

distilling information from data to provide valuable insights for the

Thus, while an analyst may be able to describe trends and translate

This skill set sought in data scientists is confirmed by LinkedIn (2019),

Cleveland, OH, USA Richard E. Boyatzis

Venice, Italy Sara Bonesso

1 The Organizational Challenges of Big Data 1

2 How Big Data Creates New Job Opportunities: Skill

3 Emotional and Social Intelligence Competencies in the

4 When Hard Skills Are Not Enough: Behavioral

5 Managing Big Data Professionals through a Competency-

Table 1.1 Classification of types of big data analytical methods 9

showed more efficiency orientation, initiative, self-confidence, networking,

e nvironment that have posed different challenges. New technologies, dif-