Big Data

Run Hive Job on EMR– Demo

Table of Contents

Steps to create EMR Cluster – Demo ............................................................................................. 2

Step 1: Select the S3 service. ................................................................ Error! Bookmark not defined.
Step 2: Click Create bucket.................................................................... Error! Bookmark not defined.
Step 3: Write the Bucket name. Click Create. ...................................... Error! Bookmark not defined.
Step 4: Click the bucket name. .............................................................. Error! Bookmark not defined.
Step 5: Click Create folder. .................................................................... Error! Bookmark not defined.
Step 6: Type the folder name. Click Save. ........................................... Error! Bookmark not defined.
Step 7: Select the EMR service. ............................................................ Error! Bookmark not defined.
Step 8: Click Create clusters. ................................................................. Error! Bookmark not defined.
Step 9: Type the Cluster name. Click the folder icon........................... Error! Bookmark not defined.
Step 10: Select the S3 bucket created earlier. Click Select. ............... Error! Bookmark not defined.
Step 11: Choose the latest version. ...................................................... Error! Bookmark not defined.
Step 12: Choose the instance type as “m4.large”. Choose the number of instances as per your
requirement. Enter the EC2 key pair. Click Create cluster. ................. Error! Bookmark not defined.
Step 13: Check the cluster status. ......................................................... Error! Bookmark not defined.
Step 14: Go to EC2 service. Three instances are created automatically. ....... Error! Bookmark not
defined.
Step 15: Click the master node Security group. ................................... Error! Bookmark not defined.
Step 16: Click the Inbound tab. Click Edit. ............................................ Error! Bookmark not defined.
Step 17: Click Add Rule button. ............................................................. Error! Bookmark not defined.
Step 18: Add “SSH” and make it anywhere. Click Save. ..................... Error! Bookmark not defined.
Step 19: SSH your instance. .................................................................. Error! Bookmark not defined.

1
Big Data

Steps to run Hive Job on EMR – Demo

Pre-requisite: -------- Commented [SKS[1]: Please add.

Step 1: Click the cluster you created earlier.

Step 2: Click Steps tab. Click “Add Step” button.

2
Big Data

Step 3: Select the step type “Hive program”. Give it a name. Enter the Script S3 location and
Input S3 location.
Script location: S3://us-east-1.elasticmapreduce.samples/cloudfront/code/Hive_CloudFront.q

Input location: s3://us-east-1.elasticmapreduce.samples

Step 4: Simultaneously, Go to S3 service on a new tab and click the bucket you created earlier.

3
Big Data

Step 5: Click Create folder.

Step 6: Type the folder name. Click Save.

4
Big Data

Step 7: Go to EMR service tab again. Click the folder icon.

Step 8: Select the folder “outputs” from the bucket. Click Select.

5
Big Data

Step 9: Check the cluster status.

Step 10: Select the S3 bucket you created earlier. Select the outputs folder.

6
Big Data

Step 11: Choose the os_requests.

Step 12: Download it. Open it in a notepad.

7
Big Data

Step 13: Check the file.