Module 5

Data Analytics for IOT

Syllabus
 Introduction
 Apache Hadoop
 Using Hadoop Map Reduce for Batch Data Analysis
 Apache Oozie
 Apache Spark
 Apache Storm
 Using Apache Storm for Real-time Data Analysis
 Structural Health Monitoring Case Study.
Apache Hadoop

 It is a open source framework for distributed batch

processing of big data.

 MapReduce is a parallel programming model suitable

for analysis of big data

◦ Map Reducing Programming Model

◦ Hadoop MapReduce Job Execution.
◦ MapReduce Job Execution Overflow
Hadoop Cluster Setup
 Hadoop filesystem HDFS is highly fault
tolerant.

 Preferred OS- LINUX

 OTHER OS_ Cygwin Environment.
STEPS INVOVLVED IN SETTTING UP A
HADOOP CLUSTER:

 Install Java.

 Install Hadoop.

 Configure Network.

 Configure Hadoop

 Starting and Stopping Hadoop Cluster

 Install Java
 Hadoop requires Java 6 or later version.

 Commands for installing java:

#set the properties

sudo apt-get –q –y install python-software-properties

sudo add-apt-repository –y ppa:webupd8team/java
sudo apt-get –q –y update
#state that you accepted the license

echo debconf shared/accepted-oracle-license-v1-1 select true |

sudo debconf-set-selections
echo debconf shared/accepted-oracle-license-v1-1 seen true |
sudo debconf-set-selections
Using Hadoop Map Reduce for Batch
Data Analysis
Apache Oozie
 Apache Oozie is a scheduler system to manage &
execute Hadoop jobs in a distributed environment.

 Apache Oozie is a workflow scheduler for Hadoop. It

is a system which runs the workflow of dependent jobs.
Here, users are permitted to create Directed Acyclic
Graphs of workflows, which can be run in parallel and
sequentially in Hadoop.
Apache Spark
Apache Storm
Using Apache Storm for Real-time Data
Analysis
Structural Health Monitoring Case
Study.