Hadoop Ecosystems - Quick Reference.

02/03/2016

Master Planning ERP-NonSAP

Microsoft Mobile Oy

Karthika Janardhanan
Hadoop

j.karthika@tcs.com
TCS Experience Certainty branding will be added by ePublishing team after all reviews and before final upload
 Confidentiality Statement
Include the confidentiality statement within the box provided. This has to be legally
approved
Confidentiality and Non-Disclosure Notice
The information contained in this document is confidential and proprietary to TATA
Consultancy Services. This information may not be disclosed, duplicated or used for any
other purposes. The information contained in this document may not be released in
whole or in part outside TCS for any purpose without the express written permission of
TATA Consultancy Services.

Tata Code of Conduct

We, in our dealings, are self-regulated by a Code of Conduct as enshrined in the Tata
Code of Conduct. We request your support in helping us adhere to the Code in letter and
spirit. We request that any violation or potential violation of the Code by any person be
promptly brought to the notice of the Local Ethics Counselor or the Principal Ethics
Counselor or the CEO of TCS. All communication received in this regard will be treated
and kept as confidential.
 Table of Content

1. Introduction .............................................................................................................................................................. 5
2. Hadoop Ecosystems Quick Reference..................................................................................................................... 6
1. Introduction
The world of Hadoop comprises of a number of ecosystems associated with it. This document would help the
readers to understand the various Hadoop ecosystems, when to use the appropriate Hadoop ecosystem with an
example.

This would be handy to design the overall solution approach using Hadoop ecosystems for the given problem
statement.
2. Hadoop Ecosystems Quick Reference
S.No. Ecosystem What is it
1 HDFS This is the underlying data
(Hadoop architecture used in Hadoop
Distributed File eco system.
System)
2 Hbase It is a NoSQL DB where data
stored as columns. It is
inspired by Googles Big
table.
3 Map Reduce(MR) MR is the algorithm to apply
logical rules in the raw data
and derive the expected data
outcome.
4 Pig It is a Scripting language to
process data into Hadoop
cluster. Uses PIG-LATIN
language. Executes as step-by
step.
5 Hive An SQL-like interface to
Hadoop by building replica of
Relational Database Tables in
Hadoop cluster.
Data Warehouse
infrastructure that provides
data summarization and ad
hoc querying on top of
Hadoop
6 Sqoop
It has clear set of instructions
to source in the structured
static data from conventional
When to use Example
1. To store such huge data, Hadoop cluster
the files are stored across
multiple machines.
2. These files are stored in
redundant fashion to rescue
the system from possible
data losses in case of failure.
3.HDFS also makes
applications available to
parallel processing.
1.Huge volume of data Employee details
stored with column
2.Need for High write families such as
throughput personal data,
contact details,
employment details
etc.click below for
the link.
1.Huge volume of data Creating a weather
report with
2.Complex logic to derive the maximum average
required output temperature for
each city in the
country from the
raw weather data.
1.Raw data could be mapped Deriving maximum
to a schema and minimum mark
from student
2.Expected output could be academic data.
derived by applying scalar
functions on the data
Raw data present in Processing oracle
Relational database table or table data in
could be presented in a Hadoop cluster.
relational fashion.
Raw data present in Import from my SQL
conventional database which databases straight
has to be imported/exported into your Hive data
 database into Hadoop cluster.
A suite of tools that connect
Hadoop and database
systems
7 Flume
Flume is one of the eco
systems in Hadoop which is
used to connect into the
streaming data and get the
data loaded into Hadoop
cluster.
8 Oozie
A java Web Application which
is a workflow scheduler for
Hadoop
9 HUE(Hadoop
It is developed by Cloudera. It
User Experience)
is a web UI for Hadoop
10 Mahout
Machine-learning tool ,with
distributed and scalable
machine learning algorithms
on Hadoop platform
to and from Hadoop cluster warehouse
To source in streaming data To load twitter
into Hadoop cluster. tweets into Hadoop
cluster.
Scheduled execution of Batch processing
Hadoop data processing
steps with one or more
ecosystems
Enabling users to interact Custom Applications
with Hadoop cluster with with nice UI library
Web UI
Shopping cart
1.Recommendation Mining-
content in online
takes users behaviour and
Shopping sites.
find items said specified user
might like
Click on below icon
2.Clustering-takes e.g. text
documents and groups them
mahout. jpg
based on related document
topics
3.Classification-learns from
existing categorized
documents what specific
category documents look
like and is able to assign
unlabeled documents to
appropriate category
4.Frequent item set mining
:takes a set of item groups
and identifies which
individual items typically
appear together
Thank You

Contact

For more information, contact gsl.cdsfiodg@tcs.com (Email Id of ISU)

About Tata Consultancy Services (TCS)

Tata Consultancy Services is an IT services, consulting and business solutions

organization that delivers real results to global business, ensuring a level of certainty no
other firm can match. TCS offers a consulting-led, integrated portfolio of IT and IT-
enabled infrastructure, engineering and assurance services. This is delivered through its
TM
unique Global Network Delivery Model , recognized as the benchmark of excellence in
software development. A part of the Tata Group, Indias largest industrial conglomerate,
TCS has a global footprint and is listed on the National Stock Exchange and Bombay
Stock Exchange in India.

For more information, visit us at www.tcs.com.

IT Services
Business Solutions
Consulting

All content / information present here is the exclusive property of Tata Consultancy Services Limited (TCS). The content /
information contained here is correct at the time of publishing. No material from here may be copied, modified, reproduced,
republished, uploaded, transmitted, posted or distributed in any form without prior written permission from TCS.
Unauthorized use of the content / information appearing here may violate copyright, trademark and other applicable laws,
and could result in criminal or civil penalties. Copyright 2011 Tata Consultancy Services Limited