Challenges to the Organisational Adoption of Big Data
Analytics: A Case Study in the South African
Telecommunications Industry
Iman Malaka
Department of Information Systems
University of Cape Town
Private Bag Rondebosch, 7701
imzamalaka@gmail.com
ABSTRACT
The purpose of this interpretive study was to explore the
challenges to the adoption of Big Data Analytics (BDA) in
organisations. The Technology-Organisation-Environment (TOE)
model was used to guide the study. Data was collected from a
large telecommunication organization in South Africa. Seven
participants, from both Information Technology (IT) and business
were interviewed to gain a holistic overview of challenges
towards the adoption of BDA. An inductive approach was used
for analysis. Findings revealed technological challenges to the
adoption of BDA as being Data Integration; Data Privacy; Return
on Investment; Data Quality; Cost; Data Integrity; and
Performance and Scalability. From the organizational perspective,
the major challenges were Ownership and Control; Skills
Shortages; Business Focus and Prioritisation; Training and
Exposure; Silos; and Unclear Processes. From the environmental
context there were no major challenges highlighted.
Organisational challenges were deemed to be the major inhibitors
to adoption of BDA
Categories and Subject Descriptors
• Information systems~Data analytics • Social and
professional topics~Management of computing and
information systems
Keywords
Big Data; Big Data Analytics; Technology Adoption; South
Africa.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or
distributed for profit or commercial advantage and that copies bear this notice and
the full citation on the first page. Copyrights for components of this work owned
by others than ACM must be honored. Abstracting with credit is permitted. To
copy otherwise, or republish, to post on servers or to redistribute to lists, requires
prior specific permission and/or a fee. Request permissions from
Permissions@acm.org.
SAICSIT '15, September 28-30, 2015, Stellenbosch, South Africa
© 2015 ACM. ISBN 978-1-4503-3683-3/15/09…$15.00
DOI: http://dx.doi.org/10.1145/2815782.2815793
Irwin Brown
Department of Information Systems
University of Cape Town
Private Bag Rondebosch, 7701
irwin.brown@uct.ac.za
1. INTRODUCTION
The effective use of advanced analytics is considered one of the
most important functions across different industries today,
providing not only daily critical decision-making support, but also
data useful for operational efficiency, development of innovative
products and gaining of competitive advantage [1]. The
introduction of electronic communication technologies and
platforms such as Skype and WhatsApp (commonly referred to as
‘Over The Top’ services) that deliver audio, video and other
media over the Internet protocol have created extremely
competitive markets that have forced many companies across
most industries to explore and implement innovative and smarter
products and services to the customer [2]. However, these may
also be disruptive, with Big Data Analytics (BDA) being one such
technology that is both innovative and disruptive [2].
Not completely new, the concept of Big Data is believed to have
its roots in progressive Internet corporations such as Amazon,
Google, Netflix, Yahoo and others, which analyse large sets of
Internet traffic data in real time to accelerate the organisational
decision-making process. The greatest hurdle to organisations
successfully deploying Big Data strategies is not simply the lack
of understanding of Big Data concepts, or the value that the
technology can provide to organisations. Rather, the gap lies
between insightful data prospects and an organisation's capability
to exploit the technology [3, 4].
A recent study by Gartner [5] on global adoption of Big Data,
indicates that an increasing number of organisations from varied
industries, such as retailing, pharmaceuticals, and insurance, to
financial services and telecommunications have either embarked
on or are in the process of launching Big Data projects. Some
54% of the companies surveyed cited improved customer
experience as the main driver for embarking on these projects, and
Big Data was recently listed among the top 10 innovative strategic
trends, together with technologies such as Cloud Computing, Data
Analytics and Mobile Technology [6]. Research has shown that
BDA can provide significant opportunities for the
telecommunication industry to leverage available infrastructures
[7]. A recent survey conducted by Strategy Worx on
implementation and use of Big Data Analytics (BDA) in South
Africa indicates that although online companies have made
advances in the adoption of data gathering and analytics using Big
Data technologies, telecommunication operators, banks and other
retail companies in South Africa are still lagging behind [8].
With organisations across different industries in South Africa
having understood the value of advanced analytics and the
opportunities presented by technologies such as Big Data,
questions arise as to why its adoption and use in South African
industries such as telecommunications remains relatively low [9],
and what challenges are presented by BDA? The purpose of this
study is to gain insight into challenges to the adoption of BDA in
the telecommunications industry in South Africa. The research
question posed is: What are the main challenges to the adoption
of BDA in telecommunications organisations in South Africa?
In the next section we review the literature on BDA. This is
followed by an outline of the research methodology before the
results are discussed and implications drawn. Finally the paper is
concluded.
2. LITERATURE REVIEW
2.1 Defining Big Data and Big Data Analytics
The recent boom in social computing has added to the exponential
growth in amounts of data from which insights can be mined.
Widely used in the Information and Communication Technology
(ICT) sector, among organisations, academia and vendors, the
term Big Data has become synonymous with very sizeable
volumes of data used to produce smarter and more insightful
analytics that traditional data analytics cannot provide [10, 11].
Manovich [12] defines Big Data as a relatively inadequate term
that is used to depict large sets of data as determined by an
organisation’s history with data management, and its inability to
process seemingly large volumes of data using conventional data
analytics or Business Intelligence (BI) technologies. Consensus
has generally been reached on the appropriateness of Gartner’s
[13] definition, i.e. “Big Data is high-volume, velocity and variety
of information assets that demand cost-effective, innovative forms
of information processing for enhanced insight and decision
making”. This means that Big Data is data not only located in
relational databases but also in different, unrelated sources such as
files, documents, e-mail, web traffic, Call Data Records (CDR),
audio, video, and social media [10].
Big Data in the telecommunications industry is not only attributed
to structured and unstructured data from the Internet, social media
or web logs, but also to the transactional and operational data
streams from traditional telecom data sources such as CDRs
(made up of real time call details, usage and spend, calling
patterns and location of a customer) together with other analytical
data from legacy billing and Customer Relationship Management
(CRM) systems [14].
Big Data can hence be defined through characteristics used by
organisations in understanding behavioural patterns and decision
making processes. These are briefly described as follows:
Volume: Prediction shows that by the year 2020, human beings as
well as industries will generate on average approximately 43
trillion gigabytes of data, which is three hundred times more than
was generated in 2005. The large scale of data collected has
resulted in modern database management tools being outdated
[15].
Velocity: This refers to analysis of the streaming of data, from the
point of collection to aggregation and eventually reporting. Real
time or near-real time processing of data provides organisations
with up-to-date competitive information [15, 16].
Variety: Different forms of data from structured and unstructured
source systems are collected from devices such as smart phones
and other digital and electronic devices. For example, current
Internet consumption statistics indicate that people consume well
over four billion hours of video on YouTube and exchange thirty
billion bits of material on Facebook monthly [15].
BDA provides sufficient prospects to determine previously
unthinkable insights from data sets, however, this comes with
significant challenges in terms of both computational and storage
expense. Volume, velocity, and variability in Big Data
repositories demand advanced analytics beyond operational
reporting and dashboards [17].
2.2 Evolution of Analytics
The era of Big Data has brought with it advancement in analytics.
BDA consists of advanced analytical techniques that operate on
Big Data sets, and comprises both Big Data and Analytics to
generate insightful trends in business intelligence that facilitate
advanced and competitive decision-making processes [1, 18].
The evolution of BDA is deeply rooted in the historical database
management and warehousing field, which uses distinctive modes
of information accumulation, extraction and analysis technologies
[19]. The analytical capabilities are grounded in statistical
methods from the 1970s, after which the capability evolved from
structured database-managed content to the unstructured contents
from Web 2.0, and eventually to the recent mobile- and sensor-
based content [18]. Analysis that makes use of Big Data mining
technologies and quantitative methods such as mathematical
algorithms, statistical computing and stochastic process are
generally categorized as “advanced” forms of analysis [20].
Davenport [21] classifies advanced analytics into three
dimensions, as follows:
Descriptive Analytics: Descriptive analytics is the most used
analytic technique that summarises Big Data into reduced, more
useful pieces of information. The information is often presented in
a reporting tool, for ease of use and availability [21].
Predictive Analytics: This form of analysis uses historical data to
predict the future, the purpose being to summarise what happened
in order to forecast the likelihood of an action or event happening
in the future. This type of analysis is often used to predict
consumers’ spend, based on past buying history and other
variables [21].
Prescriptive Analytics: Prescriptive analytics is often used to
prescribe an action and establish the cause so as to enable the
business decision-maker to take action based on information
provided. It uses a prognostic statistical prototype made up of two
variables: “actionable data and a feedback system that tracks the
outcome produced by an action taken” [21].
2.3 Comparison of Big Data Analytics and
Traditional Analytics
Data Analytics typically refers to Business Intelligence and
Analytics (BI&A) tools that are used for data quarrying and
statistical analysis [18]. Many of the techniques used rely on
advanced and viable technologies such as Relational Database
Management Systems (RDBMS), enterprise data warehousing,
and Business Process Management (BPM) [19]. The difference
between authentic Big Data methods and traditional data
warehousing methods often referred to as Business Analytics can
be defined by the characteristics of Big Data (volume, variety and
velocity) as mentioned above. A similarity is that both seek to
gather intelligence and insights from data of varying size and
nature, transforming it into business gain [22].
2.4 Adoption of Big Data Analytics
The adoption and use of BDA provides an opportunity for
organisations to outperform competitors [23]. The implementation
of BDA might be time-consuming and costly, but the superior
competitive advantage it could provide may outweigh its barriers
in the long run, as demonstrated in various cases, such as that of
eBay and Google [24]. In 2011, MIT Sloan Management Review
teamed up with IBM to conduct a survey of 3,000 analysts,
managers and executives working across thirty industries in over a
hundred countries [2]. The study revealed that high-performance
organisations had implemented and were making use of BDA five
times more than the low-performing organisations.
Research has shown that BDA can provide great opportunities for
the telecommunication industry to leverage available
infrastructures and different data facets that are increasing in size
exponentially [7]. A recent survey on adoption of Big Data
Analytics in South Africa by Strategy Worx indicates that large
corporations such as banks, retailers and telecommunication
operators have the necessary technology infrastructure in place,
but do not currently use BDA in a substantial way [8].
2.5 Challenges of Adoption and Use of Big
Data Analytics
As a means of framing the challenges to BDA adoption identified
in literature, Tornatzky and Fleischer’s Technology-Organisation-
Environment (TOE) Framework will be used [25]. The TOE
suggests three major groups of factors that impact on the adoption
of a new technology – i.e. technological, organisational and
environmental [25].
2.5.1 Technology Issues
Time and cost issues affect BDA projects as these projects are
rarely a once-off investment with a fixed start and end point [26].
Given the nature of such projects, involving multiple data sources
and cross-organisational groups, it is inevitable that such projects
proceed on a step-by-step basis. However, such an incremental
approach, using a Big Data platform to support multiple
initiatives, can make it difficult to get buy-in from senior
management, given that they typically demand a clear and
immediate business case [26].
The data integration capabilities of vendors are typically cited
as an important criterion when deciding on a vendor with which to
partner. Telecommunication operators usually have a legacy of
multiple systems and databases serving different departments and
functions, and supporting multiple products and services.
Addressing these fragmented data sources and structures cannot
be taken lightly, so vendors should have extensive experience in
data integration and management skills [7, 14]. The complex
integration touch points presented in the telecommunication
industry make it difficult to present a single interaction due to
fragmented data [7].
In terms of veracity, uncertainty of data keeps data scientists busy
trying to organise data, with more people creating it. Poor or bad
quality data can cost companies large annual losses if not properly
managed [15].
The analytics on Big Data requires massive performance and
scalability. Most users have cited issues such as slowness
experienced in processing large data volumes, slow query
response time and the lack of processing (CPU) capacity for
analytics as some of the challenges of adoption of BDA [1]. These
challenges have resulted in replacement of analytics platforms to
gain better performance and increased productivity.
2.5.2 Organisational issues
Poor communication processes are often cited as contributing to
project failure. BDA projects cannot be seen purely as
technology-driven projects. Their value arises when they are
utilized within key decision-making processes [11]. As such BDA
is unlikely to deliver value without participation from business.
The general ownership and control of BDA initiatives is often at
departmental level rather than at a strategic level [1]. The
department that runs business intelligence (BI) and data
warehouse (DW) initiatives tends to own the BDA process. This
is seen as a challenge, as BI and DW technologies are not often
designed for BDA [1].
A skills shortage of IT and analytical professionals is an issue
with BDA, given the novel nature of Big Data and the fact that so
many disparate sectors are now seeking to hire people with data
analytics expertise. Skills shortages are an issue with regards to
roles such as data scientists, programmers, modellers and analysts.
Many academic institutions are yet to include BDA as part of their
curricula [14].
2.5.3 Environmental issues
Competition has been cited as contributing to the adoption of
novel technologies [25]. Telecommunications markets have
become saturated, which has lead to increased competitive
pressure as organizations strive to reduce customer churn and
increase revenues [7]. This has resulted in telecommunications
operators turning to BDA to obtain insights that could lead to
more compelling services than competitors [7]. Other studies
however suggest that competitive pressure has no direct impact on
the adoption of technologies [27].
Vendor reliance is an important consideration for BDA adoption
[7]. While there are a number of obstacles to broadening the
adoption of analytics, model complexity is one of the greatest.
Developing these models requires highly trained statisticians, who
are presently in short supply [1]. BDA vendors try to address this
matter by providing off the shelf and customised pre-built models
that are easier and faster to deploy [1]. Marketers are also trying
to make it easier for business users to construct algorithms and
analyse the results themselves [7].
Regarding data security and privacy, an absence of policies,
standards and regulations concerning Big Data is a critical
challenge [1, 7, 28]. Regulations, standards, and policies are
lagging in the monitoring, development, usage, ethics, and
management of Big Data [1, 28]. Data security and privacy are of
significant concern for telecommunications as they house
sensitive data of customers. Alongside development of BDA
tools, it is important to create shields to anticipate data misuse
[29]. Policies such as the South African Protection of Personal
Information (PoPI) Act of 2013, and other such acts might not
foresee the speedy development of these technologies, and
provide only limited guidance on how an organisation should
collect and use data to predict or respond to issues such as natural
disasters, national security threats or emergencies. Further, law
with regards to the governance of Big Data is still nascent [30].
Figure 1 below illustrates these sets of challenges to BDA
adoption as identified in the literature.
 of Figure 1 were used to construct interview questions. All
interviews conducted were recorded then transcribed.
Table 1: Respondents and Roles
Participan BDA
t Skill Code Position Group
A Specialist IA Snr IS
Specialist:
Technical
BA
B Specialist IB Snr IS
Specialist:
Architect
C User IC Snr Business/ Marketing
Specialist:
Insights &
Analytics
Figure 1: TOE Framework Reflecting Big Data Analytics (BDA)
Challenges- Adapted from Tornatzky & Fleischer [25] D User ID Snr Mgr: Business/ Marketing
Insights &
Analytics
3. RESEARCH METHODOLOGY E Strategist IE Snr Mgr: Business/ Marketing
An interpretive philosophy was considered as being ideal for this Product
research study, to understand the challenges to adoption from the Develop.
perspective of different role players within an organisation (e.g.,
F Strategist IF Snr Finance
Big Data users and Big Data specialists) [31]. A single Specialist:
organization case study approach was chosen to gain an in-depth
understanding of the challenges in deploying a contemporary Finance
technology such as BDA in the context of real-life practice by G User IG Exec Head: Business/
gathering evidence from multiple sources [31]. Single CVM Marketing
organization case study research is useful for gaining rich insight,
where the aim is analytic generalizability, rather than statistically
generalizing to a population [32]. A multinational
telecommunication operator in South Africa was selected for this
3.2 Data Analysis
Thematic analysis was used to unearth recurring themes. The
research, as it met the criteria of having been engaged with BDA
following process was followed [33]:
and having encountered challenges.
Understanding of the data: Familiarity with the body of data
3.1 Data Collection was achieved through reading and re-reading of transcripts.
In order to gain an organisation-wide perspective on the Rough notes were made to highlight early impressions.
challenges of BDA adoption, a variety of stakeholders were
interviewed. These included users (senior managers or executives Generation of initial codes: Coding was used to reduce a large
who use data analytics extensively), business strategists (senior body of data into smaller pieces of meaning.
managers who use data analytics to build strategic business Search for global themes: Global themes were based on the TOE
models and products), and the technical specialists (analysts, data framework in Figure 1.
scientists and data architects) involved in the day-to-day
operational design and mining of Big Data. The researchers’ Reviewing of themes: Each emergent theme was classified as
knowledge of the organization, and professional relationships either a challenge or non-challenge. Only the challenges were
established with managing executives were used to identify key given further consideration, in line with the purpose of this study -
participants from different divisions and roles within the to identify the challenges to BDA adoption.
organisation.
Identification of basic theme groupings: Based on participants'
A total of seven respondents were interviewed as in Table 1. responses to questions themes were finalized, and appropriate
Three were highly recommended by the executives in charge of quotations were identified that conveyed the essence of the
relevant divisions, due to their knowledge of the subject and themes.
involvement in other similar research studies. Snowball sampling
was employed to select other participants based on their role and 3.3 Validity of Findings
area of specialisation within the organisation. The decision to use To validate the findings, observation, supporting documentation
a small sample allowed for more detailed and in-depth interviews and the researchers’ own experience were used. These were used
within a limited timeframe. as a supplement to interviews, to corroborate the findings [31].
Permission was obtained to use the results of a survey conducted
Interviews were semi-structured, and both face-to-face and online by the company group earlier in the year. The survey was
methods (using Microsoft Lync for participants outside Cape experimental in nature, as its primary objective was to assess the
Town) were employed. Themes drawn from the TOE framework current status of BDA initiatives and identify challenges
encountered during the implementation of such initiatives within
the organisation. The survey not only covered questions around
the technology aspects but also included portions of analytics,
which were covered more extensively in this research study. This
supporting documentation aided in validating the findings of this
research study.
4. RESULTS OF DATA ANALYSIS
4.1 Adoption and BDA Landscape
From the analysis it was apparent that the organisation was at the
evaluation or early implementation stage of BDA adoption. It was
also evident from the interviews that the organisation had
embarked on a few Big Data initiatives in the past and was
presently re-evaluating current data assets with a view to defining
a BDA strategy for the near future. When asked if BDA was
considered as mostly an opportunity or a problem within the
organisation, all respondents indicated the former, due to the
value that had already been realised from current BI and data
analytics initiatives. As stated: “From products perspective we
need insights from Big Data to survey correct customers”
(Participant E).
The participants used the three characteristics; volume, velocity
and variety of Big Data to describe the current landscape of BDA.
The MIT and IBM classification of capabilities of analytics as
defined in three stages, aspirational, experienced and transformed
[2] was also used to assess the current and future landscape of BI
and BDA within the organisation. The consensus was that BI was
at an experienced stage and that BDA was at an aspirational stage.
4.2 Challenges to BDA Adoption
To get to the crux of which of the challenges were more critical
than others, the participants were asked what they saw as major
challenges, from a technology, business and environmental
perspective. It is important to note that some of the factors
highlighted were seen as non-challenges by some participants due
to perception being influenced by the role or function as
performed by the participant within the organisation. For example,
data privacy was seen as a non-challenge by a business strategist,
as the concern of privacy is usually addressed by another division
within the business. The sections that follow will discuss in detail
the analysis and findings of each identified challenge to the
adoption of BDA.
4.2.1 Organisational Challenges
Six major organizational challenges to BDA adoption were
identified, these being: (1) Ownership and Control; (2) Skills
Shortage; (3) Business Focus and Prioritisation; (4) Unclear
Processes; (5) Training and Exposure, and (6) Silos. Each will be
discussed in turn.
4.2.1.1 Ownership and Control
Analysis of data based on all the participants interviewed reveal
that ownership, control and overall responsibility for data and
analytics mostly resides with the BI and DW teams under the
supervision of IT/IS: “Traditionally it’s been IT, I suppose. I’d
probably argue that it should be sitting in the commercial space
in terms of BI” (Participant C). Participant B also indicated that
there were other small pockets within the organisation currently
running with their own platforms for data and analytics: “Like
Customer Value Management [CVM] is doing some analytics,
finance is doing some analytics … [BICC] division is going to do
some cross company analytics”.
Participant A saw this factor as one of the major business
challenges towards the adoption of BDA within the organisation:
“… data ownership, to me, is one of the challenges as to why we
are not moving forward on this”. Most participants indicated that
in order to realise benefits of the adoption of BDA, there had to be
“…. collaboration between different areas within the business”
(Participant E), with the roles and functions of each entity
involved clearly defined. Participant E saw the benefits of
collaboration between different business units being realised from
the utilisation of different skill sets: “…. I don’t think that it
should fall under any one part of the business. So you should
combine different parts of the business, you may need key people
from a technology point of view, view from finance, from
commercial or from enterprise”. Participants D and G went as far
as identifying their department (CVM) as being one of the
business units to own and control companywide execution of
BDA, with IT/IS being the custodians of the data and architecture:
“… we are probably the best team to monetise that information.
So I wouldn’t want to own the information from IT architecture or
gathering information” (Participant G). “If I think it should
belong anywhere, then it should belong to us. We are a big team;
we are ten people, consisting of data modellers and analysts and
people that specialize in segmentation. I think we are the right mix
of people” (Participant D). Supporting documentation regarding
ownership indicated that, “Big data projects have good business
representation. There are occasional tensions between business
and IT objectives, but these are generally quickly resolved.”
4.2.1.2 Skills Shortage
The participants being interviewed indicated a shortage of skills
as one of the key challenges faced by the organisation in the
adoption of BDA. This can be attributed to BDA still being a new
phenomenon compared to BI and analytics [14], which most
organisations have built over the years. As stated: “There is a big
difference between BI team and your kind of Big Data Analytics
team or person one would need…” (Participant F). Participant E
(and supporting documentation) differed from the assertion above,
as on the contrary, it was believed that the current BI resource
skill would be able to execute BDA if provided with adequate
training and exposure: “For sure, there are a lot of very clever
people within… whose brilliant ideas are sometimes overlooked”
(Participant E). Participant C also saw the BDA skill shortage as a
challenge that could be addressed with adequate training of the
existing data analysts. He further indicated that comparing the
skill sets of a Big Data scientist with those of BI were: “…. like I
suppose asking a soccer player to play rugby. They are both ball
games, but they are slightly different. You can find that the one
guy can be good at the other, but they haven’t been trained”.
Participant A believed that the skills shortage “is definitely a
challenge in terms of Big Data practitioners and Data Scientist,”
with “Data Scientist also being the next hype word. In terms of the
skill, analytically, you must have a PhD, and be mathematically
minded. You definitely need to because the variety is so different
from normal analytics”. It was believed that the challenge is not
only confined to the organisation or the telecommunications
industry, “but rather it is a skill that we are mostly short and
there’s very few people in South Africa and when I use Vendors it
is to supplement my team” (Participant G).
4.2.1.3 Business Focus and Prioritisation
Analysis of interview data and supporting documentation linked
the lack of an overall coordinated business strategy to the low
prioritisation and lack of focus of initiatives such as BDA by the
organisation: “I mean this, from my point of view, might be a
priority for me to build this model with the best effort I can give
with the information that’s available. Will this then be taken
forward and looked at in the future? I don’t think so” (Participant
F). Participant A agreed with this assertion: “Not everyone is
concerned about this Big Data; we have business as usual and
this is white noise”.
Lack of business focus and prioritisation of such initiatives can
also be attributed to the historical background of the
telecommunications environment, whereby focus has been on the
connectivity of a customer. There is also a corporate culture built
on always being first to market: “We wait very long for things to
happen [with Big Data] and that is an issue. Getting the
technologies incorporated, functional and working is an issue for
us and being able to support them in-house. I mean once the data
is there it’s there, right?” (Participant D). Significantly,
Participant G did not share the sentiments of Participant D, but
rather identified this theme as a challenge due to lack of business
priority experienced from the fast-paced nature of the
telecommunications environment: “The challenges are pretty
much business priorities. In an organisation where we have to
chase the next target it is difficult to find the appropriate time”.
The lack of business prioritisation and focus on BDA can be
linked to the failure of similar initiatives conducted in the past in
showing a quantifiable return on investment or commercial value.
4.2.1.4 Unclear Processes
According to Participant A, “you need to have business process
mindset change in terms of Big Data”. People and processes are
often cited as factors contributing to the failure of most IS
initiatives [10]. Participant E saw the main business challenge to
the adoption of BDA arising from unclear communication
channels, i.e. “lack of efficiency in the process” (Participant F)
and perhaps the silos associated with the distribution of data
within the organisation, which make the “entire process of getting
our data into a Big Data solution [cumbersome]” (Participant C).
4.2.1.5 Training and Exposure
Training and exposure to BDA technology was highlighted by
most interviewees. Analysis revealed a direct link between the
shortage of skills and adequate training and exposure. Adoption of
BDA within the organisation was possibly being hindered by the
lack of appropriate training of staff. Participant D’s perception
was that the organisation had the correct mindset to execute and
run BDA models, with the limitation being attributed to lack of
training and exposure:
“… the reality right now is that we are in the zone of what we call
‘makeshift’. We are learning on the ground as we go as a lot of us
come from different environments, like I’m a geneticist by trade.
That’s different. We have analysts from banks and GIS people. I
mean we are all here and we use Teradata, we use SQL. and with
the tools we currently have, we basically up-skilled ourselves.
Having said that, if we get a better car to drive, absolutely the
best use of the data we have and tools, we will have to go for
further training. Over and above that, be exposed to anyone using
Big Data because right now we are not as exposed as other
companies are” (Participant D).
Participant A indicated that “the complexity comes through where
you are trolling through social media, e-mails, customer surveys,
logs, tweets... but with education and training that shouldn’t be a
problem”.
4.2.1.6 Silos
Two sub-themes emerged from this theme, i.e., the challenge of
silos of disparate and inconsistent information due to non-
integration of data sources and legacy systems, as well as the
challenge of silos as presented by the widely used
telecommunications business process framework, which has a
functional split between Engineering, IS and Business [26].
Participant F attributed this challenge to the
“...silo type nature of any telecoms or any business that can cause
inefficiencies within an organisation and that’s a key point to
highlight because the more we work in silos and not try to
integrate these projects or into different information hubs, the
more we will have a pitfall or a short fall in anything we try to do.
That’s the key point to having a one stop shop, where everybody
comes to a point where you can extract the information and share
the information with other parts of the business.”
When asked what the main business challenges to the adoption of
BDA were, Participant A and Participant E mentioned silos: “The
challenges are due to the silos, so you have data sitting in one
part of the business and you have another important set of data
sitting in another part of the business” (Participant A). Participant
E indicated that the organisation “should break silos” in order to
resolve data integration issues, lack of ownership and other
challenges faced by this organisation.
4.2.2 Technological Challenges
Seven major technological challenges were identified, these
being: (1) Data Integration; (2) Data Privacy; (3) Return on
Investment; (4) Data Quality; (5) Cost; (6) Data Integrity; and (7)
Performance and Scalability. Each will be discussed in turn.
4.2.2.1 Data Integration
Participants saw the integration of different data facets as integral
to successful BDA adoption. They highlighted the challenges of
integration of a multiple legacy system as a potential inhibitor to
the adoption of BDA: “… we see that data sitting there and that
data sitting there and no one is integrating the data” (Participant
A). Participant B indicated that as part of the BDA strategy the
organisation is in the process of decomposing assets, but at the
same time needed to “…leverage of our key assets and the other
key word-reuse. The other thing you would be looking at is the
integration of the statistical tool with your environment. The
reason why we won’t be moving away very quickly from SAS is
that SAS, it’s got interface capability with Teradata”. A
predictive analytics tool such as SAS has the ability to support
BDA by using the data integration studio to integrate with
Hadoop, coupled with its packaged business models or solutions,
data management and visualisation capabilities [1].
Participant D indicated that the integration of multiple data facets
will save the organisation much time spent in trying to understand
customer’s behavioural patterns: “We spend a lot of time trying to
understand that particular customer [because] …there’s no one
source of information for me even though there’s all the sites. I
have tried different sources, different people, and different
information for all of the years” (Participant F). Participant D
referred to this one source as “the nerve centre”, currently being
built within the organisation: “What the nerve centre does, it takes
data from the warehouse…., they take some network stats, they
take some spatial exchange information, they take some census
data, and they run a little algorithm”. When asked to identify
major technological challenges of BDA, Participants E and G
highlighted integration as a major challenge: “Integration is also
important – if it’s not integrated you can’t use it, you can’t
campaign off from it” (Participant G).
The supporting documentation provided did not highlight data
integration as a challenge. As noted: “We integrate data across
most data sources using batch processes to make it available for
further analytical processing. We have shared master data /
indexes in place that aid data integration”.
4.2.2.2 Data Privacy
The IS and business participants’ views on the actual challenge
presented by BDA were aligned. Both entities highlighted data
and privacy as a challenge, indicating that the organisation was to
an extent “…pretty much compliant with everything that’s going
out there based on the PoPI Act and the Privacy Act because we
are looking at aggregated information across the base”
(Participant G). Participant B agreed that the main issue could be
attributed to the way IS “… sort of interpret the act and we think,
how we have to apply it to us”, with Participant G also having
highlighted interpretation of an Act as a hindrance on how the
business could use the data: “The constraint is on how the
business interprets the Acts and their understanding of my
environment” (Participant G). Participant C mentioned that “…
going down to Big Data is one of the most intrusive things you
could do”, whilst Participant A said “… but you have algorithms
and ways you can apply the data to work around that and still
make it anonymous.”
4.2.2.3 Return on Investment
Participants indicated that it was good to recognise the potential
benefits presented by the adoption of BDA, but not when these
initiatives failed to translate into actual monetary value: “There is
value, but it is difficult to put the Rands and Cents onto it and you
need to put the Rands and Cents onto it in order to justify
spending on the adoption” (Participant C). This challenge has a
direct link to the theme of lack of business focus and prioritisation
of initiatives such as BDA. As noted by Participant A: “…from a
cost perspective we have to do a solid business case to prove that
this initiative is going to give us a positive ROI”. Participant F,
from finance department, indicated a “… need to be careful when
we go into something like this. It may start small and expand
rather than go big, because it can hurt you; sometimes hundreds
of millions of Rands can go into this very easily. I mean, you can
justify all you want until the cows come home, but it has to yield
the results”.
4.2.2.4 Data Quality
The IS participant’s perspective on this challenge was that with a
technology such as Big Data there would definitely be issues of
data quality and the business users who
“… access the Big Data appliances will have to understand that it
is, - as you get it, that’s what it is! So would you need to get a
deeper understanding of what you are looking at?.... It is not a
nice neatly packaged environment where you have done your
transformation, you have applied your business rules, you have
actually integrated into few other sources and you have presented
it” (Participant B).
The business users’ perception, however, was that “It is quite
often not about being a hundred percent right all the time versus
getting the majority right most of the time” (Participant C). The
findings indicate that the perception of data quality or veracity is
often based on a user’s valuation of data's appropriateness to serve
its purpose in a given setting.
4.2.2.5 Cost
The challenges presented by cost in the adoption of BDA within
the organisation brought about a dichotomy in the perceptions
expressed by the participants. Available literature points to cost as
being a challenge to the adoption of BDA, especially as BDA
projects tend to run long term [26]. However, when asked if the
cost would be regarded as a major hindrance to implementing
BDA within the organisation, Participants B, C and E indicated
that it was not considered as a hindrance, as long as the initiative
could prove that it was “going to generate revenue in the long
run” (Participant E).
Participants B and C indicated that the cost of the implementation
would be minimal as the organisation did not “buy data, we
produce it” (Participant C), and that the quotations already
provided by some of the vendors did not “sound like this is going
to be very expensive to actually get the big cluster environment”
(Participant B). The perceptions presented by Participants B, C
and E were supported by the supporting documentation provided.
It was noted: “We are able to effectively fund and resource Big
Data opportunities. There are some roadblocks that may slow
progress, but can generally be overcome (e.g., procurement lead
times)”. This perception can be attributed to the organisation’s
culture when it comes to the implementation of initiatives that are
perceived as being of value, and perhaps also to the company’s
vision of consistently being the first to market.
The dichotomy came about from the views presented by
Participants A, D, F and G, who indicated that the
telecommunications industry, in particular the organisation under
study, was not immune to challenges faced by many other
organisations due to the volatile economic climate: “I think with
budget pressure as well, everyone is struggling across the world
and they are not performing as well as they would like to”
(Participant A). Participant D attributed this challenge to the lack
of prioritisation from IT: “The feedback we get from the data team
(IS/IT) is that it’s cost, that… it’s not prioritised as they are
paying for XYZ”. Participant A indicated that IT “cannot just go
spending money because even that portion of the budget that has
been allocated is not as big as we would like it to be”.
4.2.2.6 Data Integrity
Data integrity was highlighted in the analysis of supporting
documentation as having a direct link to data quality, as well as
data integration, as it offered assurance for the validity of data
provided: “Data integrity issues are caused by source system
problems. Not all data is integrated nor should it be” (extract
from supporting documentation). This challenge seemed to be a
burning issue with business users as understandably they were the
main users of this data: “I think there should be some sort of
governance that should validate the integrity of the data; I don’t
think it currently exists in that there is one point that can validate
the integrity of that data” (Participant F). Participant C
highlighted data integrity as the biggest technological challenge in
the adoption of BDA: “I think probably number one is data
integrity. No matter what technology you are using or you stick it
in, data integrity is always the one to mess with you the most and
it’s what you end up spending more time on”. Participant D
echoed the sentiments: “We constantly have integrity issues with
the data we receive and on our side that will definitely be high”
4.2.2.7 Performance and Scalability
The Teradata database, currently used within the organisation is
renowned for supporting centralised and mostly large DWs that
produce scalability and fast data performance [1]. Participant B
noted: “Currently we are running on Teradata , you could use the
Exalytics platform which OBIEE [Oracle Business Intelligence
Enterprise Edition] is going to be installed on and you can use an
Infiniband [high performance] connection to an Exadata
platform; those type of processing speeds you will get up there
will be forty Gigs and above” (Participant B). Despite this, many
participants still cited performance degradation as one of the
technological challenges to the adoption of BDA: “…when I run
some queries against it, will take five hundred hours to execute”
(Participant G). Participant B agreed that poor performance was a
hindrance to the adoption of BDA in that the user required:
“performance-data at his fingertips; if he runs a query now, two
seconds later it is there. Currently… there are challenges”. The
slow query response time and lack of processing capacity has
been mentioned in literature as one of the challenges to the
adoption of BDA due to high volumes of data being managed [1].
Participant B further indicated that the solution to this
phenomenon could come from in-memory capability provided by
Exalytics, which is “definitely on the cards as well”.
5. DISCUSSION AND IMPLICATIONS
The analysis of data indicates that the TOE framework [25] was
useful in framing the challenges to BDA adoption in
organisations. The analysis revealed that the major .challenges
experienced were technological and organizational, and not to do
with the external environment. This may be related to the state of
BDA adoption in the organisation, whereby during the early
stages of adoption, it is mainly internal factors that act as
hindrances.
All participants interviewed clearly identified and acknowledged
the opportunities presented by the effective use of BDA to “treat
every customer in a unique way” (Participant D). Although the
participants were able to give examples of different opportunities
that could be derived from the adoption of BDA, those from
business did not always recognise the value of BDA. Participant C
said that “the analysis needs to speak to a proposition”, so that he
“can see value in it”, but he did not “know how much it is” and
that he did not “see anybody coming and saying… what it’s
worth”. Participant B from IS attributed the lack of a properly
defined value proposition to business users’ inability to interpret
and familiarise themselves “with what they are looking at…
whether it is now Big Data or integrated data warehouse, where it
is nicely polished and transformed’ the business still struggles to
understand”.
Participants A and B indicated that “the technology is there, well,
almost there”, and that the organisation was in a process of
defining the strategy and “ out of the strategy” IS would
determine whether they “need Hadoop” or “… MapReduce just to
get that benefit out of the Big Data”. This finding is in line with
that of Strategy Worx who revealed that retailers and
telecommunication operators had the necessary technology
infrastructure in place, but did not currently use BDA in a
significant way [8].
Data analysis from the supporting documentation together with
participant interviews, uncovered a lack of ownership and control
as the major challenge to the adoption of BDA with most data
driven initiatives being controlled and managed at a departmental
level instead of at a strategic business level [1]. This theme can be
directly linked to the silo nature of this organisation, which in turn
links to the lack of integration of data into “one source of
information” (Participant F) or as Participant B calls it, a “nerve
centre”. According to Participant F, the organisation typically
captures and stores data in separate networks, and different and
unrelated systems, including operational support systems (OSS),
network management systems, business support systems (BSS),
and customer relationship management (CRM) systems. Business
value (ROI) can only be realised when these datasets are
combined and used to drive business decisions and actions: “For
example; if a consumer goes to the… (mall) and purchases a
ticket to the movies with the cell phone, why can’t we campaign
this very customer based on location? Possibly this might have
other privacy concerns, which is another challenge in the Big
Data space” (Participant A). Corporate culture in the organization
is still defined by its network technology and engineers, rather
than being informed by customer demand and agile business
processes.
Most participants cited skills shortage as one of the major
challenges faced by the organisation in adopting BDA. Participant
G indicated that “Analytics as a whole is a skill that we are mostly
short of and there are very few people in South Africa’ who
specialise in that trade”. Together with Participant B she
indicated that the organisation’s reliance on vendors was mainly
“to supplement” the lack of skill. Participant F attributed this
shortage of skilled BDA personnel to the lack of training.
Participant D added that being “exposed to practical examples of
how BDA [work] adds value to other businesses”. Available
literature suggests that the challenge of skills and training in BDA
will eventually be eliminated, once academic institutions
introduce BDA as part of their curricula [14].
Literature recommends that system engineers and architects must
guarantee that data is secured and utilised as per regulations [29].
Participant C said: “going down to Big Data is one of the most
intrusive things you could do”. Findings suggest data ought to be
used in an ethical and secure manner to overcome customers’
privacy concerns.
6. CONCLUSION
Telecommunications operators in South Africa collect petabytes
of pieces of diverse data, including call data records (rated and
unrated usage), web-based content and location-based data. They
have an advantage over other industries through continuous access
to customers on a daily basis via mobile devices. This is an
opportunity that Participant A referred to as a “gold rush”. The in-
depth case study conducted offers some insight as to why, despite
the opportunity presented, BDA adoption is still in its infancy in
the industry.
Technological challenges such as data integration, data privacy,
return on investment determination, data quality, cost, data
integrity and performance and scalability all present as major
impediments. From an organisational perspective, issues such as
ownership and control of BDA, skills shortages, business focus
and prioritisation, training and exposure, silos, and unclear
processes need also to be addressed. The challenges faced by the
telecommunications industry, specifically in South Africa, are not
markedly different from those faced by other industries, as
highlighted in previous empirical studies conducted, however a
key finding is that the organisational challenges are seen as being
greater hindrances than the technological.
Future research can build on this exploratory study by
investigating in-depth one or more of the key challenges, and how
organisations resolve the challenges. By investigating a number of
diverse organisations the effect of varying contextual conditions
on BDA adoption can be elucidated. Theory-building studies can
be conducted to better explain how the interrelated challenges
work together to inhibit BDA adoption.
7. REFERENCES
[1] Russom, P. (2011). Big data analytics. TDWI Best Practices
Report, Fourth Quarter.
[2] LaValle, S., Lesser, E., Shockley, R., Hopkins, M. S., &
Kruschwitz, N. (2011). Big data, analytics and the path from
 insights to value. MIT Sloan Management Review, 52(2), 21-
31.
[3] Bughin, J., Livingston, J., & Marwaha, S. (2011). Seizing the
potential of ‘Big data’. McKinsey Quarterly, 4, 103-109.
[4] Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R.,
Roxburgh, C., & Byers, A. H. (2011). Big data: The next
frontier for innovation, competition, and productivity.
Report: McKinsey Global Institute.
[5] Gartner, 2. (2013). Gartner survey reveals that 64 percent of
organisations have invested or plan to invest in Big data in
2013. Retrieved 10, 2013, Retrieved from
http://www.gartner.com/newsroom/id/2593815
[6] Savitz, E. (2012). Gartner: Top 10 strategic technology
trends for 2013. URL
Http://Www.Forbes.Com/Sites/Ericsavitz/2012/10/22/Gartne
r-10-Critical-Tech-Trends-for-the-Next-Five-Years,
[7] McCarthy, C., & Bali, S. (2013). Big data analytics and the
Telco: How Telcos can monetize customer data. (Research
No. TE010-000312). Europe: Ovum. Doi: 20 May 2013. (Big
Data Analytics and the Telco).
[8] BizTech Africa. (2013, Aug. 26, 2013, 8:50 a.m.) SA
corporates lag on Big Data Analytics.
[9] Puschel, J. (2013). Telco-big-data-building-a-future-
business-platform. (Research No. 3000012380). Europe:
Informa Telco and Media. Doi: 02 August 2013. (Big data)
[10] Davenport, T. H., Barth, P., & Bean, R. (2012). How ‘Big
Data’ is different. MIT Sloan Management Review, 54(1),
22-24.
[11] Gandomi, A., & Haider, M. (2015). Beyond the hype: Big
data concepts, methods, and analytics. International Journal
of Information Management, 35(2), 137-144.
[12] Manovich, L. (2011). Trending: The promises and the
challenges of big social data. Debates in the Digital
Humanities, 2, 460-475.
[13] Gartner. (2013). Big data. Retrieved 1001, 2013, Retrieved
from http://www.gartner.com/it-glossary/big-data/
[14] Szaniawski, K. (2013). Telco-big-data-new-tools-and-
strategies. (Research No. 3000012380). Europe: Informa
Telco and Media. Doi: 06 August 2013. (Big data).
[15] Gantz, J., & Reinsel, D. (2012). The digital universe in 2020:
Big data, bigger digital shadows, and biggest growth in the
Far East. IDC iView: IDC Analyse the Future,
[16] Bollier, D., & Firestone, C. M. (2010). The promise and peril
of Big data. Aspen Institute, Communications and Society
Program Washington, DC, USA.
[17] Fisher, D., DeLine, R., Czerwinski, M., & Drucker, S.
(2012). Interactions with Big data analytics. Interactions,
19(3), 50-59.
[18] Chen, H., Chiang, R. H. L., & Storey, V. C. (2012). Business
intelligence and analytics: From Big data to big impact. MIS
Quarterly, 36(4), 1165-1188.
[19] Chaudhuri, S., Dayal, U., & Narasayya, V. (2011). An
overview of business intelligence technology.
Communications of the ACM, 54(8), 88-98.
[20] Philpott, S., & CoE, T. B. (2010). Advanced analytics.
[21] Davenport, T. H. (2012). Enterprise analytics: Optimize
performance, process, and decisions through Big Data. FT
Press.
[22] McAfee, A., & Brynjolfsson, E. (2012). Big data: The
management revolution. Harvard Business Review, 90(10),
60-66.
[23] Brynjolfsson, E., Hitt, L., & Kim, H. (2011). Strength in
numbers: How does data-driven decision-making affect firm
performance? Available at SSRN 1819486.
[24] Brown, B., Chui, M., & Manyika, J. (2011). Are you ready
for the era of ‘Big data’? McKinsey Quarterly, 4, 24-35.
[25] Tornatzky, L., & Fleischer, M. (1990). The processes of
technological innovation. Lexington, MA: Lexington Books,
[26] Lambert, P. (2013). Telco-big-data-the-role-of-the-operators.
(Research No. 3000012380). Europe: Informa Telcom and
Media. Doi: 09 August 2013. (Big data)
[27] Thong, J. Y.L. (1999). An integrated model of information
systems adoption in small business. Journal of Management
Information Systems, 15(4), 187-214.
[28] Martin, K. E. (2015). Ethical issues in the Big Data industry.
MIS Quarterly Executive, Forthcoming.
[29] Bryant, R., Katz, R. H., & Lazowska, E. D. (2008). Big-Data
Computing: Creating Revolutionary Breakthroughs in
Commerce, Science and Society.
[30] Yan, J. (2013). Big Data, Bigger Opportunities. White Paper,
April, 9
[31] Saunders, M. N., Saunders, M., Lewis, P., & Thornhill, A.
(2011). Research methods for business students, 5/e Pearson
Education India.
[32] Walsham, G. (1995). Interpretive case studies in IS research:
Nature and method. European Journal of Information
Systems, 4(2), 74-81.
[33] Braun, V., & Clarke, V. (2006). Using thematic analysis in
psychology. Qualitative Research in Psychology, 3(2), 77-
101.