HADOOP USE CASES

PRAMOD KUNJUKUNJU SAJI

HADOOP

• Being the backbone technology for Big Data storage and analytics
• Companies across the globe have started to migrate their data into Hadoop
WHY LOOK INTO BIGDATA?

• Hadoop is still a complex technology with several limitations and complications

• All Data is not Big Data and might not require a Hadoop solution
• Companies that use Hadoop, often fall into the error of not using the right Hadoop
architecture and tools, thereby reducing Hadoop’s efficiency
STUDYING HADOOP USE CASES WILL HELP TO:

• Understand what kind of big data problems need Hadoop

• What sort of infrastructure should one have in order to set up and work on the
Hadoop framework
BENEFITS OF HADOOP

• provides storage for any kind of data from various sources

• provides a platform for proficient analytics of the data with low latency
• a distributed, scalable and fault-tolerant system
• can store petabytes with relatively low infrastructure investment
HADOOP IN THE FINANCIAL SECTOR

• Risk modelling - to solve the question for banks to evaluate customers and
markets
• Many banks compile separate data warehouses into a single repository backed
by Hadoop for quick and easy analysis
• To create more accurate risk analysis models for the customers in its portfolio
• Risk analysis helps banks to manage their financial security and offer customized
products and services
HADOOP IN THE FINANCIAL SECTOR

• In the aftermath of the 2008 economic downturn, Hadoop helped maintain a better risk record
• Before that, every regional branch of the bank maintained a legacy data warehouse framework isolated
from a global entity
• After the economic recession, a single Hadoop Cluster containing more than petabytes of financial data
aggregated from multiple enterprise and legacy database systems.
• pulling in other data sources - such as customer call records, chat and web logs, email correspondence
and others
• techniques like sentiment analysis, text processing, pattern matching, graph creating
• Banks as well as government financial institutions use HDFS and MapReduce to commence anti-money
laundering practices, asset valuation and portfolio analysis
HADOOP IN THE FINANCIAL SECTOR

• Predicting market behaviour is another classic problem in the financial sector

HADOOP IN THE FINANCIAL SECTOR -
MORGAN STANLEY
• with assets over 350 billion is one of the world’s biggest financial services
organizations
• relies on the Hadoop framework to make industry critical investment decisions
• Hadoop provides scalability and better results through it administrator and can
manage petabytes of data which is not possible with traditional database
systems.
HADOOP IN THE FINANCIAL SECTOR -
JPMORGAN CHASE
• provides services in more than 100 countries
• leverage big data analytics more effectively by using frameworks like Hadoop on
massive volumes of structured and unstructured data
• has mentioned it on various channels that they prefer to use HDFS to support the
exponentially growing size of data as well as for low latency processing of
complex unstructured data
HADOOP IN THE FINANCIAL SECTOR

• analyse world economy

• fraud detection
• derive value-based products for customers
• analyse credit market data
• effective cash and credit management
• providing an enriching customer experience
HADOOP IN HEALTHCARE SECTOR

• leverages Big Data for curing diseases

• reducing medical cost
• predicting and managing epidemics
• maintaining the quality of human life by keeping track of large-scale health
indexes and metrics
HADOOP IN HEALTHCARE SECTOR

• Patient record keeping and regulatory requirements

• Data generated by electronic health devices – legacy systems are incapable of
processing this data
• The complexity and volume of the healthcare data is the primary driving force
behind the transition
• Using Hadoop, helps in easy and quick data representation, database design,
clinical decision analytics, data querying and fault tolerance
HADOOP IN HEALTHCARE SECTOR

• Hadoop allows the storage of unstructured healthcare data in its native form
• HDFS and MapReduce have the ability for processing terabytes of data, makes
Hadoop indispensable for the healthcare sector’s big data problems
• assists medical insurance companies, hospitals and beneficiaries to increase
their product value by devising smart business solutions
HADOOP IN HEALTHCARE SECTOR - HEALTHCARE
INFORMATION TECHNOLOGY COMPANY

• required to save seven years’ worth of historical claims and remittance data
• Had trouble using a traditional database system to store this data
• The complex processes of normalizing data, logging terabytes of information and
data querying for analytics was done smoothly thereafter
HADOOP IN HEALTHCARE SECTOR -
HEALTHCARE INSURANCE BUSINESS
• Medical insurance companies can create profitable policies suitable for different
parts of the population.
• Such an estimation needs the processing of massive amounts of data including
medicines, geographic regions, patient care records, diseases, symptoms, etc.
• Hadoop and MapReduce will prove to be economical in processing such massive
unstructured information.
HADOOP FOR TELECOM INDUSTRY

• Call Data Records Management

• Servicing of Telecom Data Equipment
• Advanced Telecom infrastructure planning
• Creating new products and services
• Network traffic analytics
HADOOP FOR TELECOM INDUSTRY

• Telecom industry generates high velocity and volume of data

• Companies target customers by catering specific products to stand out.
• Insights like location, interest, activities, and preferences of customers are being
used to create marketing and promotional plans for Telecom Industry.
HADOOP FOR TELECOM INDUSTRY

• Call Data Records Management

• Servicing of Telecom Data Equipment
• Advanced Telecom infrastructure planning
• Creating new products and services
• Network traffic analytics
HADOOP FOR TELECOM INDUSTRY - CHINA
MOBIL GUANGDONG 
• has an extensive network of customers
• they have to maintain billions of mobile call records, real-time access to call
records and billing information of the customers
• used HBase to store billions of call record details
• nearly 30 Terabytes of data is added to their database on a monthly basis
HADOOP FOR TELECOM INDUSTRY - NOKIA

• store and analyse massive volume of data from their manufactured mobile
phones
• Nokia’s Big Data, manages 100 TB of structured data along with 500+ TB of
semi-structured data
HADOOP IN RETAIL SECTOR

• Predicting demand
• Increasing profit
• Creating targeted marketing
• Promotional campaigns
HADOOP IN RETAIL SECTOR

• Retail analytics - major consumers of the data warehousing industry and is

responsible for its innovation and growth
• Due to the ability to collect and store far more data about consumers, their
behaviour and consumption - both online and in stores
• The historical sales record along with the help of Hadoop and MapReduce are
used for increasing profit margins and sales
HADOOP IN RETAIL SECTOR

• Etsy is an online marketplace

• Sears has both online and, brick-and-mortar stores
• Both these companies needed to analyse large volumes of log data, for marketing
campaigns, sales management, inventory management, etc.
• Used Amazon Elastic MapReduce to create a Hadoop cluster.
• Stored and analysed the data to estimate consumer behaviour, search
recommendation, product placement, inventory management, targeted marketing,
product promotion, etc.
HADOOP IN RETAIL SECTOR - RETAIL
ANALYTICS
• Inventory forecasting
• Dynamic pricing of products
• Supply chain efficiency
• Targeted and customized promotion and marketing
• Fraud detection and prevention
HADOOP FOR BUILDING RECOMMENDATION
SYSTEM
• Online retailers
• Online newsfeeds
• Online videos and movies
HADOOP FOR BUILDING RECOMMENDATION
SYSTEM
• Needs to measure compatibility between individual profiles in order to make quality matches for a potential
relationship
• Data acquired by the company is taken from the users during their signup and during their online behaviour
• People are required to fill surveys in order to describe their attributes and characteristics they look for in a partner
• The company blends this information with demographics knowledge and their online activities to build a
comprehensive image of the respective user as well as to create a list of recommended set of partners
• The data included is a combination of structured information, matching stats, which is used for scoring and
recommendation algorithms
• Hadoop was brought in to replace the data storage and the analytics system, because of its fast analytic support
and low cost storage
WHEN TO USE BIGDATA?

• Companies use big data in their systems to improve operations

• Provide better customer service
• Create personalized marketing campaigns
• Take other actions that, ultimately, can increase revenue and profits