Professional Documents
Culture Documents
Unit 5 Bi
Unit 5 Bi
Unit 5 Bi
Unit -5
Emerging trends in BI
1. SaaS and Cloud Adoption
The COVID-19 pandemic put companies and industries in emergency mode as they
scrambled to make sense of the situation. With on-premise solutions unable to measure up to
the challenge of a largely remote workforce, many businesses were forced to look hard at
their current BI strategies. More organizations seek to migrate to cloud-based BI this year,
whether on a private or public cloud or SaaS solution.
Many companies are realigning their budgets post-pandemic to make room for adopting
cloud infrastructure in the move towards remote and disparate
workforces. Gartner predicts that by 2023, 40% of all enterprise workloads will be
deployed in the cloud, up from 20% in 2020. Businesses now consider analytics a mission-
critical capability, and companies aren’t shying away from adopting data solutions.
The global NLP market, valued at $10.72 billion in 2020, is projected to grow to $48.46
billion by 2026, at a CAGR of 26.84%. NLP bridges the gap between computers and humans
by eliminating the need for any programming language. It allows users to interact with data
by asking questions in a conversational format, such as, “What was the revenue for the last
quarter for Zone X?”
The next best thing after text-based queries is voice-based searches. By integrating this
capability with voice-activated digital assistants on mobile devices, software vendors make
data discovery even more user-friendly. Natural language generation (NLG) enables the
software to generate answers that are easy to understand, irrespective of technical skills.
In business terms, NLG is one of the latest trends in business intelligence. It outputs the
essential takeaways of a data visualization in conversational language, facilitating quick
insight interpretation.
3.Data Literacy
But to do so, they have first to establish and build up a firm foundation of data literacy. What
is data literacy? According to Gartner, it’s the ability to read, write and communicate data in
context – to put it simply, the ability to “speak data.” It’s key to increasing user adoption and
maximizing the effectiveness of BI tools. Data literacy is important for all individuals,
irrespective of their work profile and businesses.
Being able to interpret data, discover insights and ask the right questions are skills that can
drive change in any role at any level of any organization. Data-driven business owners have
to eliminate the data literacy gap between data analysts and non-technical users, a process
known as data democratization.
The democratization of data is critical,” Brody said. “The design and execution of reports,
data visualizations and analytics should not require a computer science degree.”
Businesses must promote a data-first culture to drive data literacy and encourage employees
to prioritize data. Teams across organizations need to learn how to analyze data and apply
insights to their tasks through training and initiatives.
While it can be time consuming and difficult, investing in upskilling employees pays off.
Dashboard software is evolving from simple KPI monitoring to in-depth data analysis
through interactivity and augmented analytics.
5.The API Economy and Automation
Though building a software solution seems more cost-effective than purchasing one,
enterprises often struggle with development logistics. Add in the pressure to pull
information from on-premise platforms, the cloud, streaming applications and more,
and it becomes overwhelmingly complex. Business applications should integrate
seamlessly across interfaces, networks and clouds for faster time to insight.
Businesses can extend software’s functionality by integrating it with other
applications. Application integration is ubiquitous; we use it even now, for instance,
when we move from Facebook to an eCommerce site without exiting the application.
It’s the API economy at work, and it’s shaping the present and future of analytics,
with BI solutions bringing insights to users within their business applications.
According to a study, by 2023, 60% of organizations will be able to build tailored
business solutions by integrating components from three or more disparate analytics
solutions. Applications in a business ecosystem can also trigger workflows based on
data changes through another trend in business intelligence – automation.
As analytics scales up to increasingly complex data volumes, automation is essential
to eliminating manual data processes. Rapid data analysis and decision-making define
business intelligence, and automation speeds up business processes, making it easier
to get answers.
Machine Learning
What is Machine Learning?
Machine learning is a modern innovation that has enhanced many industrial and professional
processes as well as our daily lives. It's a subset of artificial intelligence (AI), which focuses
on using statistical techniques to build intelligent computer systems to learn from available
databases.
As explained, machine learning algorithms have the ability to improve themselves through
training. Today, ML algorithms are trained using three prominent methods. These are two
types of machine learning: supervised learning, unsupervised learning.
Supervised learning
Supervised learning is the types of machine learning in which machines are trained using well
"labelled" training data, and on basis of that data, machines predict the output. The labelled
data means some input data is already tagged with the correct output.In supervised learning,
the training data provided to the machines work as the supervisor that teaches the machines to
predict the output correctly. It applies the same concept as a student learns in the supervision
of the teacher.
Supervised learning is a process of providing input data as well as correct output data to the
machine learning model. The aim of a supervised learning algorithm is to find a mapping
10
Unsupervised learning
Unsupervised learning is a type of algorithm that learns patterns from untagged data. It uses
machine learning algorithms to analyze and cluster unlabeled datasets. These algorithms
discover hidden patterns or data groupings without the need for human intervention.
•
•Its ability to discover similarities and differences in information make it the ideal solution
for exploratory data analysis, cross-selling strategies, customer segmentation, and image
recognition.
The of the biggest gaps in BI tools is that they don’t align well with the way most
businesses are structured.
BI tools are generally designed for data scientists and analysts. In a way, this
pervasive choice makes sense— after all, analysts and data scientists are best
equipped to understand the data, iterate on insights, and ask targeted follow-up
questions to build a more complete understanding of a data landscape.
And yet, marketers, salespeople, category managers, and people typically falling
under the “business” umbrella are on the frontlines of the decision-making
process. BI tools can be needlessly complex and cumbersome for these employees.
As such, BI tools often facilitate a cycle of dependency. Business people who want
to make data-driven decisions must rely on data scientists to use BI tools. In turn, data
scientists spend time building routine reports and answering marketing questions, for
example, instead of leveraging their advanced degrees and skillsets.
This cycle can easily lead to a backlog of questions, fatigue in your data scientists,
and/or reluctance in your business people, who may decide that data-driven decision-
making simply isn’t worth the frustration of relying on a third party every time a
question comes up.
Though BI tools have the potential to orchestrate incredible gains, they can be
incredibly inefficient.
Machine learning is uniquely poised to close the gap in BI tools because it can
perform important analysis and adapt to different data sets.
The kind of useful information that business intelligence should provide encompasses:
· Where your business has the most opportunity to edge out competitors and
gain market share
These questions — the broad ones that get to the core of your performance — have
historically been piled onto data analysts. Now, machine learning has the ability to
perform the same research and generate fast, accurate results.
It’s this automation that’s key. Machine learning is not positioned to replace data
scientists and analysts — rather, it can free up their time so that they can focus on
tasks that carry more value for your business. When data analysts aren’t shackled with
routine reports, they can take their research to the next level.
As such, machine learning is the necessary piece for truly self-service BI tools.
BI tools with machine learning implementations not only enable deeper insights into
data, but they also empower business people to take analysis into their own hands.
It’s common for people to ask us what is the difference between Business
Intelligence and Machine Learning. I also asked myself that question when I started
in this exciting world of data-based predictions.
The first step in any type of Business Intelligence is to collect raw data. Once stored,
data engineers use what are called ETL (Extract, Transform and Load) tools to
manipulate, transform and classify data in a structured database. These structured
databases are usually called data warehouses.
Business analysts use data visualization techniques to explore data stored in structured
databases. With this type of tool they create visual panels (or dashboards) to make
information accessible to non-data specialists. The panels help to analyze and
understand past performance and is used to adapt future strategy to improve KPIs
(Key Business Indicators
In short, traditional Business Intelligence allows us to have a descriptive vision of the
company’s activity, very visual and based on data. It mainly uses aggregated data to
describe future trends.
The mechanism that does this detects patterns in millions of data. This is an important
first difference from traditional BI, to which we could add these three aspects:
In contrast to the use of aggregated data, Machine Learning uses individual data with
defining characteristics of each of the instances. This way, thousands of variables can
be used to detect patterns.
For many companies, predictive analytics is nothing new. But it is increasingly used
by various industries to improve everyday business operations and achieve a
competitive differentiation.
In practice, predictive analytics can take a number of different forms. Take these
scenarios for example.
Identify customers that are likely to abandon a service or product. Consider a yoga
studio that has implemented a predictive analytics model. The system may identify
that ‘Jane’ will most likely not renew her membership and suggest an incentive that is
likely to get her to renew based on historical data. The next time Jane comes into the
studio, the system will prompt an alert to the membership relations staff to offer her
an incentive or talk with her about continuing her membership. In this example,
predictive analytics can be used in real time to remedy customer churn before it takes
place.
Send marketing campaigns to customers who are most likely to buy. If your business
only has a $5,000 budget for an upsell marketing campaign and you have three
million customers, you obviously can’t extend a 10 percent discount to each customer.
Predictive analytics and business intelligence can help forecast the customers who
have the highest probability of buying your product, then send the coupon to only
those people to optimize revenue.
An accurate and effective predictive analytics takes some upfront work to set up. Done right,
predictive analytics requires people who understand there is a business problem to be solved,
data that needs to be prepped for analysis, models that need to be built and refined, and
leadership to put the predictions into action for positive outcomes.
Text Analytics
Text analytics combines a set of machine learning, statistical and linguistic techniques to
process large volumes of unstructured text or text that does not have a predefined format, to
derive insights and patterns.
It enables businesses, governments, researchers, and media to exploit the enormous content at
their disposal for making crucial decisions. Text analytics uses a variety of techniques –
sentiment analysis, topic modelling, named entity recognition, term frequency, and event
extraction.
There are several techniques related to analyzing the unstructured text. Each of these
techniques is used for different use case scenarios.
Sentiment analysis
Sentiment analysis is used to identify the emotions conveyed by the unstructured text. The
input text includes product reviews, customer interactions, social media posts, forum
discussions, or blogs. There are different types of sentiment analysis. Polarity analysis is used
to identify if the text expresses positive or negative sentiment. The categorization technique is
used for a more fine-grained analysis of emotions - confused, disappointed, or angry.
Use cases of sentiment analysis:
Topic Modeling
Large amounts of data are collected everyday. As more information becomes available, it
becomes difficult to access what we are looking for. So, we need tools and techniques to
organize, search and understand vast quantities of information.
Topic modelling provides us with methods to organize, understand and summarize large
collections of textual information. It helps in:
Discovering hidden topical patterns that are present across the collection
Annotating documents according to these topics
Using these annotations to organize, search and summarize texts
Topic modelling can be described as a method for finding a group of words (i.e topic) from a
collection of documents that best represents the information in the collection. It can also be
thought of as a form of text mining – a way to obtain recurring patterns of words in textual
material.
There are many techniques that are used to obtain topic models. This post aims to explain the
Latent Dirichlet Allocation (LDA): a widely used topic modelling technique and the
TextRank process: a graph-based algorithm to extract relevant key phrases.
In the LDA model, each document is viewed as a mixture of topics that are present in the
corpus. The model proposes that each word in the document is attributable to one of the
document’s topics.
The LDA model discovers the different topics that the documents represent and how much of
each topic is present in a document. For example, LDA may produce the following results:
Topic 1: 30% peanuts, 15% almonds, 10% breakfast… (you can interpret that this topic deals
with food)
Topic 2: 20% dogs, 10% cats, 5% peanuts… ( you can interpret that this topic deals with pets
or animals)
This technique is used to find the major themes or topics in a massive volume of text or a set
of documents. Topic modeling identifies the keywords used in text to identify the subject of
the article.
Large law firms use topic modeling to examine hundreds of documents during large
litigations.
Online media uses topic modeling to pick up trending topics across the web.
Researchers use topic modeling for exploratory literature review.
Businesses can determine which of their products are successful.
Topic modeling helps anthropologists to determine the emergent issues and trends in a
society based on the content people share on the web.
NER is a text analytics technique used for identifying named entities like people, places,
organizations, and events in unstructured text. Named entity recognition (NER) helps you
easily identify the key elements in a text, like names of people, places, brands, monetary
values, and more. Extracting the main entities in a text helps sort unstructured data and
detect important information, which is crucial if you have to deal with large datasets.
Online reviews are a great source of customer feedback: they can provide rich insights about
what clients like and dislike about your products, and the aspects of your business that need
improving.
NER systems can be used to organize all this customer feedback and pinpoint recurring
problems. For example, you could use NER to detect locations that are mentioned most often
in negative customer feedback, which might lead you to focus on a particular office branch.
Many modern applications (like Netflix and YouTube) rely on recommendation systems to
create optimal customer experiences. A lot of these systems rely on named entity recognition,
which is able to make suggestions based on user search history.
For example, if you watch a lot of comedies on Netflix, you’ll get more recommendations
that have been classified as the entity Comedy.
NER is used to classify news content based on people, places, and organizations
featured in them.
Search and recommendation engines use NER for information retrieval.
For large chain companies, NER is used to sort customer service requests and assign
them to a specific city, or outlet.
Hospitals can use NER to automate the analysis of lab reports.
TF- used to determine how often a term appears in a large text or group of documents and
therefore that term’s importance to the document.
Event extraction
This is a text analytics technique that is an advancement over the named entity extraction.
Event extraction recognizes events mentioned in text content, for example, mergers,
acquisitions, political moves, or important meetings. Event extraction requires an advanced
understanding of the semantics of text content. Advanced algorithms strive to recognize not
only events but the venue, participants, date, and time wherever applicable. Event extraction
is a beneficial technique that has multiple uses across fields.
Link analysis: This is a technique to understand “who met whom and when” through
event extraction from communication over social media. This is used by law
enforcement agencies to predict possible threats to national security.
Geospatial analysis: When events are extracted along with their locations, the insights
can be used to overlay them on a map. This is helpful in the geospatial analysis of the
events.
Business risk monitoring: Large organizations deal with multiple partner companies
and suppliers. Event extraction techniques allow businesses to monitor the web to find
out if any of their partners, like suppliers or vendors, are dealing with adverse events
like lawsuits or bankruptcy.
1. Data gathering - Text data is often scattered around the internal databases of an
organization, including in customer chats, emails, product reviews, service tickets and
Net Promoter Score surveys. Users also generate external data in the form of blog
posts, news, reviews, social media posts and web forum discussions. While the
internal data is readily available for analytics, the external data needs to be gathered.
2. Preparation of data - Once the unstructured text data is available, it needs to go
through several preparatory steps before machine learning algorithms can analyze it.
In most of the text analytics software, this step happens automatically
3. Text analytics - After the preparation of unstructured text data, text analytics
techniques can now be performed to derive insights. There are several techniques
used for text analytics. Prominent among them are text classification and text
extraction.
Machine learning systems use past examples or training data to assign tags to a new set of
data. The training data and its volume are crucial, as larger sets of data helps the machine
learning algorithms to give accurate tagging results. The main algorithms used in text
classification are Support Vector Machines (SVM), Naive Bayes family of algorithms (NB),
and deep learning algorithms.
Advanced Visualization
Advanced Data Visualization gives a new meaning on how pictures can simplify information
needed to comprehend complex questions. Big Data does not mean much if the people who
control change can’t understand or have to spend too much time deciphering the Great Data
that is presented. In addition, Big Data speeds across the Internet, captured from people and
the Internet of Things (IoT) including items such as appliances, GPS, and building
maintenance. This Big Data constantly updates, second by second, providing not a static
picture, but a dynamic movie.
Organizations, need to find ways in keeping up with this Big Data in order to
understand their customers better and to move much more quickly, smoothly, and
efficiently. Take health insurance coverage in the United States. How has it changed over the
course of time? An Excel line or bar graph may show the details that more Americans have
gained health care coverage between 2008 and 2015. But the Advanced
Data Visualization, provided by the US Census bureau, gives a clearer sense of the trends,
through an animated U.S. map. Advanced Data Visualization provides a tool to keep up with
and make sense of Big Data in timely manner.
Advanced Data Visualization displays data through interactive data visualization, multiple
dimension views, animation, and auto focus.
Advanced Data Visualization fills the need when 2-dimensional graphics and one screen just
does not handle the information as well or results in slower comprehension of the data. For
example, an interactive mind map of the Internet of Things, allows users to visualize what
kinds of things, how IoT will be used, and what technologies are involved. Advanced Data
Visualization becomes useful only when it is the simplest way to describe the business
problem at hand and obtain Business Intelligence.
Users need to present their business needs and to see what story Big Data tells. Advanced
Data Visualization does this through interaction. For example, an Advanced Data
Visualization tool, called Linkurious, helped the International Consortium of Investigative
Journalists (ICIJ) investigators identify people who have been involved in tax fraud, by
hiding their money in Switzerland. Anybody can go to the ICIJ site and explore countries
or people who were involved with the Swiss leaks, through a simple interface. As a different
example, Information is Beautiful presents an Advanced Data Visual map describing the
major player in the IoT. People can click on different bubbles, in this map, to zoom into
details about specific IoT businesses. Advanced Data Visualization provides an interface for
people to search through and integrate Big Data to get meaningful results.
Evelson states in his blog that “Even with the smallest reasonably readable font, single line
spacing and no grid, you can’t fit more than a few hundred numbers on the screen.”
Advanced Data Visualization allows a person to fit more Big Data points by stacking the
data. For example, DrasticData uses the MBO-Scanner to create an interactive site for the
Dutch Ministry of Education. This tool allows user to “browse through a huge amount of data
on educational institutes and the many different courses organised by these institutes.”
To see multiple data points on one screen corresponding to different geographical regions,
Advanced Data Visualization, helps. Reuters is aware of this advantage and so partnered with
Graphiq to create Open Media Express. An example, from Open Media Express shows the
impact of the new drive to seek big budget, staff cuts at the Environmental Protection Agency
(EPA), through an interactive map of the United States.
Advanced Data Visualization only works when helping users understand how Big Data
addresses a business need. If done poorly, Advanced Data Visualization results in
information overload, increased expense and unnecessary complexity. Edward Tufte, a leader
in Information Visualization would agree. Tufte states, “The minimum we should hope for
with any display technology is that it should do no harm.” Applying Advanced Data
Visualization to some problems does more harm in presenting information. For example, a
company needs to track pharmaceutical approvals by the Federal Drug Administration to
figure out market potential. Since, it can take up to 10 months to approve a drug, an
application providing multiple dashboards or that allows the user to zoom into multiple data
points may be overdoing it. A simple Excel pie chart or pivot graph would present the
information more simply.
Business Intelligence (BI) - has undergone significant transformation in recent years and
certainly is not being superseded by artificial intelligence. In fact, it has become more
efficient and user-friendly since moving to the cloud, utilizing AI and machine learning, and
its embedded implementation.
Today's business leaders know that data is a valuable asset that needs to be used effectively to
ensure efficient company processes. They require powerful data analysis tools to help them in
the decision-making process. Businesses are embracing sophisticated analytics and data
science to gain insights and make more informed decisions, the result; turning your BI
department into a profit center.
BI is a valuable tool for both small and large businesses, it is evolving at a rapid pace and will
determine how companies will work with data in the future. It therefore needs to be mobile,
flexible and user-friendly. Here are 6 key trends that will shape the future of BI
One of the key business intelligence future trends many experts are predicting is a growth of
the digital business intelligence world into a space where tools and platforms will become
more broad-spectrum and eventually, more collaborative.
Many of today’s tools are siloed and independently operated by users — unconnected to a
broader network. However, there is a consensus that the next generation of business
intelligence will be geared toward larger sets of users and more connected to greater
systems.BI will provide shared, immersive analytic experiences in the future.BI development
has been focused on small form-factor devices, but the focus will now shift to very large
touch devices.This type of trend is now evident, with the expansion of some BI platforms
toward more evolved collaborative and machine learning systems.
A data analyst may observe “The marketing spend for a certain advertising company is very
high to acquire a new customer, and so you may want to decrease the budget on that
marketing company, or you may want to stop the ad altogether.”
BI software is expected to become increasingly intuitive in the coming years. The predictive
capabilities of these systems are projected to expand into identification features that provide
insight based on the context of the proposition.
These predictive functions will streamline decision making, accounting for compliance in the
process.“A manager will ask a question about data or relationships she hasn’t previously
considered, request data not yet accessible, or otherwise attempt to extend past the hard-set
information boundaries.”
In the traditional process, that means the investigation comes to a sudden stop…An ML
system can significantly speed that process, using rules and experience to quickly find new
data, see if existing data fits within compliance rules and grant immediate access.”
Artificial intelligence can analyze trends and past patterns to make educated guesses about
your data inquiries.
Eventually, you get to a place where business intelligence work doesn’t have to be jump-
started by human users anymore. Instead, you’re more likely to passively receive this
intelligence than you are to go looking for it in a report or even on a dashboard. This is data
proactivity: information brought to you. This may be as simple as important data points being
more prominent in a visualization or as advanced as notifications providing direct answers.
Up to this point, companies have been widely trumpeting innovation in their visual dashboard
designs. Features like more sophisticated charting and graphing have become paramount, and
data visualization has become the watchword.