Hadoop

Assignmnet2
1. Explain the importance of below 4 demons in job execution with minimum of 5 points

1. Name node:

Name node is the master node which collects and maintains the metadata.

Whenever client wants to store the data it contacts the Name node.

Name node gives the available list of data nodes to the client to store the data.

Collects the heart beat and block report from the data nodes.

Stores the metadata in the fs_image and edit_log files.

Belongs to Hadoop 1.x

2. Data node:

Data node is responsible to store the physical data in its block.

Sends heartbeat for every 3 seconds to Name node and block report for every 6 hours.

Every data node in the cluster has a Task Tracker which is responsible to perform the
actual computation on the data.

Data node replicates each input split into two copies and stores in different nodes.

Belongs to Hadoop 1.x

3. Resource Manager

Resource manager is the master node in YARN(MR2) which is responsible for the
resource management of the cluster.
 Resource manager selects the node with best hardware configuration and creates
application master.

Resource manager sends the list of available resources to the Application Master so that
Application master starts the job in the available Data Nodes.

Whenever Resource Manager gets the application request from client it checks and
confirms with the Name Node whether the file is available in the metadata or not.

If Resource Manager finds the name of existing file it confirms with the client by sending
APP_ID.

4. Node Manager:

Node Manager is the slave daemon and sends ping to the Resource Manager.

Node manager is so similar like a Task Tracker which is responsible to perform the actual
computations on the data stored in the Data Nodes.

Node manager will be controlled by Application Master for job executions.

Once job is started, Node Manager constantly sends the ping to Application Master.

2.

import java.util.*;

class HadoopString

public static void main(String args[])

{
 String line = "Hadoop is open source frame work, We are learning Hadoop
framework with Helpism";

// To split the given string using split method

String mylist[] = line.split(" ");

int count =0;

// To check how many time a word has repeated

for (String i: mylist)

if (i.equals("Hadoop"))

count = count+1;

System.out.println("Number of times Hadoop repeated: " + count);

// To replace ',' with ';' using replace method

 String line2 = line.replace(",",";");

System.out.println(line2);

//array list with generic string and store the spitted string inside it.

ArrayList<String> list = new ArrayList<String>();

for(int j=0; j<mylist.length; j++)

list.add(mylist[j]);

System.out.println("The generic list: ");

for(String n: list)

System.out.println(n);