DATA ANALYTICS

CS-503
UNIT –II
BIGDATA:
The definition of big data is data that contains greater variety, arriving in
increasing volumes and with more velocity. This is also known as the three Vs.

Put simply, big data is larger, more complex data sets, especially from new data
sources. These data sets are so voluminous that traditional data processing software
just can’t manage them

Big data analytics examines large amounts of data to uncover hidden patterns,
correlations and other insights. With today’s technology, it’s possible to analyze
your data and get answers from it almost immediately – an effort that’s slower and
less efficient with more traditional business intelligence solutions.
BENEFITS:

1. Answering Important Customer Questions

The answers to all your most pressing business questions exist within the
right data sets. Using big data and analytics, companies can learn:

 What customers want

 Where they’re missing out on conversions
 Who their best customers are
 Why people choose different products

Every day, your organisation gathers more useful insights into your sales and
marketing questions, so you can begin to adjust and optimise your marketing
strategies. The more you learn about your customers, the more competitive
your company becomes.
For instance, the U.S. company Target uses data analytics to predict
customer pregnancies, to help them better offer promotions to their
customers. You can even use big data combined with artificial intelligence
to create marketing strategies based on predictions about your customer.

For instance, if you use big data to learn that your customers are more likely
to up-sell at certain times in the month, you can send lead nurturing emails to
prompt those conversions.

. Marketing Strategies & Making Confident Decisions

All companies need to make complex decisions as they grow.

Big data can help you to make those choices with confidence, based on an in-
depth analysis of what you know about your marketplace,
industry, and customers.

One of the biggest benefits of big data is its accuracy. Big data analytics
gives you a complete overview of everything you’ve learned so far as you’ve
developed your organisation. This means that you don’t have to guess
whether you should launch a new marketing campaign or try a new product.
Instead, you can look back over the information you have, and make focused
decisions designed to generate the highest possible ROI.

Add machine learning and AI to the mix, and your big data collections can
form neural networks that help your artificial intelligence to suggest positive
changes for your company.

3. Optimising and Understanding Business Processes

Knowledge is power. That concept is at the heart of big data analytics.

Big data technologies like cloud computing and machine learning help you to
stay ahead of the curve by identifying inefficiencies and opportunities in your
company practices.
For instance, your big data analytics can tell you that your email and
marketing strategies is working, but your social media profiles aren’t
reaching the right people. On the other hand, if you use big data internally,
you can find out which parts of your company culture are having the right
impact, and which may be causing a turnover.

Using existing evidence to make quick decisions ensure that you spend more
of your budget on the things that are helping your business grow, and less on
strategies that don’t work.

For instance, one area that is seeing a lot of growth with big data analytics is
supply chain optimisation. Geographic sensors can track goods and delivery
vehicles and optimise routes by providing live data and traffic information.

4. Empowering the Next Generation

Finally, as a new generation of technology leaders enter the marketplace,
big data delivers the agility and innovation top-tier talent needs from their
employer.

For instance, millennials are natural technology natives. The younger people
in your team will expect access to technology that allows them to make
useful decisions rapidly. By constantly collecting and analysing information,
you can create an agile culture that’s ready to evolve to suit the latest
trends.

In the past, limited data sets, poor analytics processes and a lack of the right
skill meant that businesses could only access a small amount of the
information available to them. Now, companies can not only answer critical
questions faster but empower their teams to accomplish more with the
information they collect.
Advantages Of Big Data Processing
Ability to process Big Data in DBMS brings in multiple benefits, such as-

 Businesses can utilize outside intelligence while taking decisions

Access to social data from search engines and sites like facebook, twitter are
enabling organizations to fine tune their business strategies.

 Improved customer service

Traditional customer feedback systems are getting replaced by new systems

designed with Big Data technologies. In these new systems, Big Data and natural
language processing technologies are being used to read and evaluate consumer
responses.

 Early identification of risk to the product/services, if any

 Better operational efficiency

Big Data technologies can be used for creating a staging area or landing zone for
new data before identifying what data should be moved to the data warehouse. In
addition, such integration of Big Data technologies and data warehouse helps an
organization to offload infrequently accessed data.
CHALLENGES:
The challenges in Big Data are the real implementation hurdles. These require
immediate attention and need to be handled because if not handled then the
failure of the technology may take place which can also lead to some unpleasant
result. Big data challenges include the storing, analyzing the extremely large and
fast-growing data.
Some of the Big Data challenges are:
1. Sharing and Accessing Data:
 Perhaps the most frequent challenge in big data efforts is the inaccessibility
of data sets from external sources.
 Sharing data can cause substantial challenges.
 It include the need for inter and intra- institutional legal documents.
 Accessing data from public repositories leads to multiple difficulties.
 It is necessary for the data to be available in an accurate, complete and
timely manner because if data in the companies information system is to be
used to make accurate decisions in time then it becomes necessary for data
to be available in this manner.

2. Privacy and Security:

 It is another most important challenge with Big Data. This challenge
includes sensitive, conceptual, technical as well as legal significance.
 Most of the organizations are unable to maintain regular checks due to
large amounts of data generation. However, it should be necessary to
perform security checks and observation in real time because it is most
beneficial.
 There is some information of a person which when combined with external
large data may lead to some facts of a person which may be secretive and
he might not want the owner to know this information about that person.
 Some of the organization collects information of the people in order to add
value to their business. This is done by making insights into their lives that
they’re unaware of.

3. Analytical Challenges:
 There are some huge analytical challenges in big data which arise some
main challenges questions like how to deal with a problem if data volume
gets too large?
 Or how to find out the important data points?
 Or how to use data to the best advantage?
  These large amount of data on which these type of analysis is to be done
can be structured (organized data), semi-structured (Semi-organized data)
or unstructured (unorganized data). There are two techniques through
which decision making can be done:
 Either incorporate massive data volumes in the analysis.
 Or determine upfront which Big data is relevant.

4. Technical challenges:
 Quality of data:
 When there is a collection of a large amount of data and storage
of this data, it comes at a cost. Big companies, business leaders
and IT leaders always want large data storage.
 For better results and conclusions, Big data rather than having
irrelevant data, focuses on quality data storage.
 This further arise a question that how it can be ensured that data
is relevant, how much data would be enough for decision making
and whether the stored data is accurate or not.
 Fault tolerance:
 Fault tolerance is another technical challenge and fault tolerance
computing is extremely hard, involving intricate algorithms.
 Nowadays some of the new technologies like cloud computing
and big data always intended that whenever the failure occurs the
damage done should be within the acceptable threshold that is the
whole task should not begin from the scratch.
 Scalability:
 Big data projects can grow and evolve rapidly. The scalability
issue of Big Data has lead towards cloud computing.
 It leads to various challenges like how to run and execute various
jobs so that goal of each workload can be achieved cost-
effectively.
 It also requires dealing with the system failures in an efficient
manner. This leads to a big question again that what kinds of
storage devices are to be used.
CHARACTERISTICS OF BIG DATA
Big data can be described by the following characteristics:

 Volume
 Variety
 Velocity
 Variability

(i) Volume – The name Big Data itself is related to a size which is enormous. Size
of data plays a very crucial role in determining value out of data. Also, whether a
particular data can actually be considered as a Big Data or not, is dependent upon
the volume of data. Hence, ‘Volume’ is one characteristic which needs to be
considered while dealing with Big Data solutions.

(ii) Variety – The next aspect of Big Data is its variety.

Variety refers to heterogeneous sources and the nature of data, both structured and
unstructured. During earlier days, spreadsheets and databases were the only
sources of data considered by most of the applications. Nowadays, data in the form
of emails, photos, videos, monitoring devices, PDFs, audio, etc. are also being
considered in the analysis applications. This variety of unstructured data poses
certain issues for storage, mining and analyzing data.

(iii) Velocity – The term ‘velocity’ refers to the speed of generation of data. How
fast the data is generated and processed to meet the demands, determines real
potential in the data.

Big Data Velocity deals with the speed at which data flows in from sources like
business processes, application logs, networks, and social media sites,
sensors, Mobile devices, etc. The flow of data is massive and continuous.

(iv) Variability – This refers to the inconsistency which can be shown by the data
at times, thus hampering the process of being able to handle and manage the data
effectively.