Data Analytics in IoT

Unit – I

Internet of Things (IoT) analytics is a data analysis tool that assesses the wide range of
data collected from IoT devices. IoT analytics assesses vast quantities of data and produces
useful information from it.

IoT analytics are usually discussed in tandem with Industrial IoT (IIoT). Data is collected from a
wide range of sensors on manufacturing infrastructure, weather stations, smart meters, delivery
trucks, and all forms of machinery. IoT analytics can be applied to managing data centers and
applications handling retail and healthcare.
Devices that power IoT Analytics
There are a wide range of IoT devices that collect data:

Wearables

Dedicated trackers such as Fitbit or other smartwatches have gone beyond tracking steps. You
can track your friends' fitness activities, compete with them, message, and even answer the
phone by connecting your devices through the Internet. This information is tracked by fitness
companies, enabling them to create customized packages if you sign up. This can include
exercise routines, diet, goals, and more. The newest smart watches even monitor heart rates and
rhythms and have accurately diagnosed heart problems in their wearers.

Smart Home

Smart homes have security systems you can access and control when you are away from home,
to appliances you can turn on and off with digital assistance. There is a wide range of devices
that you can incorporate into your home and a wide range of data that can be collected to assess
usage patterns, the efficacy of systems, and more.

Healthcare

Healthcare has a wide range of IoT devices. Bluetooth technology creates hearing aids, records
heart and blood pressure, and monitors pulse-based alarm systems that can call for help. This has
helped enhance healthcare to a large extent. The data collected is invaluable in terms of creating
newer and better technology.

Voice-Activated Everything

Digital assistants are a form of IoT devices. Alexa, Siri, and Google take notes, find information,
play music, order cabs, tell the weather, set alarms, and everything else. The internet regularly
updates these digital assistants to improve functionality. Their data helps companies tailor their
services for you, based on your everyday interaction with digital assistants.
Types of IoT Data Sources
Most IoT applications include passive, active, dynamic data, structured data and unstructured data.

So the IoT is “all that” now, and a bag of chips. Perhaps it’s time we look at the emerging patterns of
IoT, which will provide better opportunities to define and solve problems. These patterns surround the
use of data, which is the real core of IoT.The patterns are:

Data from passive,

which is the ability to consume data from sensors that don’t actively communicate. These are
sensors that must be activated before they can transmit data, and they only produce data when
asked to do so. For example, a sensor that measures ground water saturation only produces
current data when the API is invoked.

Passive data does not mean a passive data/IoT application. Indeed, since the sensors need to be
managed, the application must take on the logic needed to do so. These are typically sensors that
are low power and exist in remote locations.

Data from active

Means that the sensor is typically streaming data, such as from a jet engine. Unlike passive data,
where we have to ask, these sensors constantly toss off data. Thus, you need to absorb the data
in near real-time, and the applications need sophisticated IoT data communications capabilities.
Data can’t be lost, it must be correctly parsed from the stream, and placed in the correct format
for both storage and processing.
Data from dynamic
Is the most sophisticated and the most useful. These are devices with sensors that communicate
dynamically (bidirectional) with IoT applications, such as a smart thermostat. These types of
sensors carry out a conversation of sorts with IoT applications. This allows a full range of
capabilities, including the ability to change the data that’s produced, change the format of the
data, change the frequency, and even deal with security issues and provide automated software
updates to dynamically deal with issues.

In short, data from dynamic is like an IoT application talking to another application. They do not
just deal with data points that devices produce, but can alter the data produced to meet the needs
of the target IoT application. These are also self- and auto-configurable, and these dynamic
capabilities really provide the best IoT capabilities.

A smart thermostat is an example of a device that provides dynamic IoT data.

Structured Data
The term structured data refers to data that resides in a fixed field within a file or record.
Structured data is typically stored in a relational database (RDBMS) and can consist of numbers
and text. Sourcing can happen automatically or manually, as long as it's within an RDBMS
structure. It depends on the creation of a data model, defining what types of data to include, and
how to store and process it.
The programming language used for structured data is SQL (Structured Query Language).
Developed by IBM in 1974, SQL handles relational databases and doesn’t require advanced
coding skills. Typical examples of structured data are names, addresses, credit card numbers,
numerical data, Microsoft Excel files, text files, and so on.
Unstructured Data
Unstructured data is more or less all the data that is not structured. Even though unstructured
data may have a native, internal structure, it's not structured in a predefined way. There is no data
model; the data is stored in its native format.
Typical examples of unstructured data are rich media, text, social media activity, video files,
audio files, surveillance imagery, and various other file formats.

The amount of unstructured data is much larger than that of structured data. Unstructured
data makes up a whopping 80% or more of all enterprise data, and the percentage keeps growing.
This means that companies not taking unstructured data into account are missing out on a lot of
valuable business intelligence.
Understanding Of Big Data
Data which are very large in size is called Big Data. Normally we work on data of size MB(WordDoc
,Excel) or maximum GB(Movies, Codes) but data in Peta bytes i.e. 10^15 byte size is called Big Data.

Big data refers to the large and complex sets of data that cannot be easily processed using traditional data
processing techniques. The size of these datasets can range from terabytes to petabytes and beyond.

The primary characteristics of big data are known as the "three Vs": volume, velocity, and variety.

Volume refers to the large amount of data.

Velocity refers to the speed at which data is generated and needs to be processed.

Variety refers to the diversity of data types and sources.

To understand big data, it is important to have a solid understanding of data analysis techniques
and tools such as data mining, machine learning, and statistical analysis. These techniques can
help to extract useful insights and information from large and complex datasets.
Applications of Big Data Analytics

Big data analytics has a wide range of applications across different industries and fields. Some of
the most common applications of big data analytics include:

1. Business intelligence: Big data analytics can be used to help organizations gain
insights into their customers, operations, and market trends. This can help organizations
to make better business decisions and optimize their processes.
2. Healthcare: Big data analytics can be used to analyze large volumes of patient data to
identify patterns and trends in disease and treatment outcomes. This can help healthcare
providers to improve patient care and reduce costs.
3. Finance: Big data analytics can be used in the finance industry to identify fraud, detect
market trends, and optimize investment strategies. The financial and banking sectors use
big data technology extensively. Big data analytics help banks and customer behaviour
on the basis of investment patterns, shopping trends, motivation to invest, and inputs that
are obtained from personal or financial backgrounds.
4. Marketing: Big data analytics can help marketers to understand customer behavior and
preferences, allowing them to create more targeted and effective marketing campaigns.
5. Transportation: Big data analytics can be used in the transportation industry to
optimize routes, reduce fuel consumption, and improve overall efficiency.
6. Manufacturing: Big data analytics can be used in manufacturing to optimize
production processes, reduce waste, and improve quality control.
7. Energy: Big data analytics can be used in the energy industry to optimize energy
consumption and reduce costs.

8. The government and military: also used technology at high rates. We see the
figures that the government makes on the record. In the military, a fighter plane requires
to process petabytes of data.

Government agencies use Big Data and run many agencies, managing utilities, dealing
with traffic jams, and the effect of crime like hacking and online fraud.

Aadhar Card: The government has a record of 1.21 billion citizens. This vast data is
analyzed and store to find things like the number of youth in the country. Some schemes
are built to target the maximum population. Big data cannot store in a traditional
database, so it stores and analyze data by using the Big Data Analytics tools.

9. E-commerce: E-commerce is also an application of Big data. It maintains

relationships with customers that is essential for the e-commerce industry. E-commerce
websites have many marketing ideas to retail merchandise customers, manage
transactions, and implement better strategies of innovative ideas to improve businesses
with Big data.
 Amazon: Amazon is a tremendous e-commerce website dealing with lots of traffic daily.
But, when there is a pre-announced sale on Amazon, traffic increase rapidly that may
crash the website. So, to handle this type of traffic and data, it uses Big Data. Big Data
help in organizing and analyzing the data for far use.

10. Social Media: Social Media is the largest data generator. The statistics have shown
that around 500+ terabytes of fresh data generated from social media daily, particularly
on Facebook. The data mainly contains videos, photos, message exchanges, etc. A
single activity on the social media site generates many stored data and gets processed
when required. The data stored is in terabytes (TB); it takes a lot of time for processing.
Big Data is a solution to the problem.

Overall, the applications of big data analytics are vast and varied, and can help organizations in a
wide range of industries to gain insights, make better decisions, and optimize their operations.
IoT Data Vs Big Data

IoT data and big data are two different types of data, although they are related and can be used
together to provide valuable insights.

IoT Data Big Data

IoT Collect and Process Data Generated by Big Data processes data that was created by
Machines, which is then Aggregated and humans, such as emails, posts on social
Compressed by Sensors Such as a Steam Iron. networking platforms, and other information
that users contribute.

In order to arrive to useful conclusions, The The processing of large amounts of data does
Internet of Things makes use of data that is not occur in real time. The gathering of data
collected, processed, and analysed in real time. comes first, followed by the analysis of it after
some time has passed.

The Internet of Things’ primary objective is to With the help of Big Data analysis, we may get
identify and resolve problems that may be to the bottom of an issue by delving into the
present in assets. mountains of data that are already collected.

IoT data refers to data generated by Internet of
Things (IoT) devices, which are physical
objects that are connected to the internet and
can collect and transmit data. IoT data includes
information such as sensor data, location data,
and usage data, and is often generated in real-
time.
Big data, on the other hand, refers to large and
complex data sets that cannot be easily
processed using traditional data processing
tools. Big data often includes both structured
and unstructured data from various sources,
including social media, transaction records,
and other sources.

While IoT data is a subset of big data, the main
difference between the two is the source and
nature of the data. IoT data is generated by
specific devices, whereas big data is typically
collected from multiple sources.
Additionally, IoT data is often real-time data
that requires quick processing, whereas big
data can be processed over a longer period of
time.

However, combining IoT data with other Big data is not a subset of IOT data, the main
sources of big data can provide valuable difference between the two is the source and
insights and enable organizations to make nature of the data.
better decisions. For example, combining IoT
data from sensors with big data from customer
interactions and social media can provide a
more comprehensive understanding of
customer behavior and preferences.

Defining IOT Analytics

Internet of Things (IoT) analytics is a data analysis tool that assesses the wide range of data
collected from IoT devices. IoT analytics assesses vast quantities of data and produces useful
information from it.

IoT analytics are usually discussed in tandem with Industrial IoT (IIoT). Data is collected from a
wide range of sensors on manufacturing infrastructure, weather stations, smart meters, delivery
trucks, and all forms of machinery. IoT analytics can be applied to managing data centers and
applications handling retail and healthcare.
Challenges of IoT Analytics

1. Data Diversity

IoT data streams are multi-model and heterogeneous in nature. So, IoT analytics
applications expose variety and veracity. Bigdata solutions are meant to deal with this data
diversity. However, accessing data from disparate sources needs drivers and connectors. Dealing
with this challenge requires technology that supports interoperability.

2. Data Quality

Most IoT streams are noisy and incomplete. This poses uncertainty to IoT analytics. Probabilistic
and statistical approaches must be employed to take the noisy nature of IoT into account.
Different IoT data streams should be associated with different reliabilities. This is the scope of
data integration in IoT analytics.

3. Real-Time Data

IoT streams feature a high volume of data. The data of such volume from multiple applications
should be processed in real-time. Hence, analytics can benefit from data streaming platforms
associated with the big data ecosystem. The high velocity of data can be in several cases
controlled by different data patterns and reports.

4. Time & Location Dependencies

IoT data comprises temporal and spatial information. The data is directly associated with the
business value. IoT applications in several cases process data promptly from respective locations.
Today, cloud computing facilitates the timely processing of data from required locations.

5. Cyber Security

The data from different IoT sensors are typically associated with cybersecurity requirements and
privacy sensitivities. Especially in the case of IoT analytics that involve the processing of
personal data, cybersecurity is mandatory to safeguard the data. Hence, IoT analytics need to be
supported by modern cybersecurity solutions.
IoT Analytics Lifecycle and Techniques

1. 1. Generate

The first step for any IoT project is to generate some data. For equipment manufacturers, this is
likely telemetry data from your machines, like voltage, temperature, rpm, flow rate, or fuel level.
For smart building applications, this is probably data like room occupancy, motion, office
temperature, and air quality.
Many companies are much further in this stage than they may realize. Especially companies in
the industrial space. Many machines already have sophisticated controllers that expose all of the
sensor data that’s needed. The company only needs to tap into that existing stream of
information. In other cases, the data may not yet exist and new sensors and devices need to be
incorporated in order to generate the data needed to eventually create business value. For
example, if your problem is to understand utilization rates of your office space, you may need to
purchase and install motion sensors into each room to generate the required data.

2. 2. Collect

Once the data is generated, it must be collected in a central repository that can be accessed and
queried by your team. This repository is sometimes called a data warehouse or data lake.
Implementing your own data warehouse can be a challenge, especially for companies that may
not have existing internal expertise in managing a large data infrastructure. Each of the major
cloud providers do have a data warehouse service that can be used.
Azure has an SQL Data Warehouse, Amazon Web Services has Redshift, and Google Cloud
Platform has BiQuery.

3. 3. Analyze

Now that the data is stored, it can be analyzed. This is also an inflection point on the analytics
lifecycle when it comes to the value of your data. While there is some amount of value in
generating and collecting data, this is the point at which you can start delivering real ROI.
Analyzing data is about understanding what happened in the past. The first step in this stage is
typically visualizing your historical IoT data on a dashboard. There’s an enormous amount of
value in being able to see your information for the first time. Field technicians can view this data
to better understand the past behavior of a piece of equipment being repaired. Facilities managers
can view historical occupancy rates to better plan office environments and services.
Analyzing data is also about processing information in different ways in order to derive insights
from raw data.

4. 4. React

If your company can reach the React stage in the analytics lifecycle, you’ll begin to see
significant value being added to your organization. The React stage is about automatically
making real-time decisions from your data that feed back into a business process.
For industrial companies, reacting to data can be used to implement more efficient condition-
based maintenance techniques to reduce cost. When faults are detected by your equipment, your
environment can react by automatically generating a support ticket in another system. In office
environments, conference rooms can be automatically freed up if no motion is detected in a
booked room. In construction environments, water can be automatically shut off if flow is
detected after work hours. No matter the industry, reacting to data is a fundamental part of IoT’s
potential value.

5. 5. Predict

Predicting the future can be the end-result of successfully traversing all stages of the IoT
Analytics Lifecycle. The Predict stage is about identifying the predictive indicators that lead to
eventual failures.
Predictive analytics and machine learning are very misunderstood and over-hyped terms. They’re
often viewed as a “cure all” to solve any problem. The reality is that it’s not magic and it requires
a lot of time, data, and effort.