200410116027

TY-lT-2-A BDA
Practical-3
Aim: Installation of Hadoop on Ubuntu os and study of hdfs Hadoop
commands.
l)start-all.sh:
Description: This command starts namenode,datanodes,secondary namenodes and
resourcemanager.

Syntax:
sbin/start-atl.sh
WARNING: Attempting to start all Apache Hadoop daemons as hadoop in 10 seconds.
WARNING: This is not a recommended production deployment configuration.
WARNING: Use CTRL-C to abort.
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [vboxuser]
Starting resourcemanager Starting
nodemanagers

Execution:

Overview 'localhost:9000' Vactive)

Tue Feb 21 +0530 2023

Version: 3.3.1. ra3b9C37a397ad4188041dd80621bdeefc4688Sf2

Jun 15 +0530 2021 by ubuntu trom (HEAD detached at rejease-3.3.1-RC3)

Cluster ID: CID-c2584a82-7043-47b6-ai3e.80821363ea55

Block pool ID: Be-580821346-127.0.1.1-1676916642716

Summary
Security is off
Safernode is off.
4 files and directories. 2 blocks (2 replicated blocks. O erasure coded block groups) 6 total hlesystem object(s).
Heap Memory used 54.42 MB of 84.59 MB Heap Mernory. Max Heap Memory is 997.63 MB.
Non Heap Memory used 51.13 MB of 54.63 MB Commited Non Heap Memory Max Non Heap Memory is eunbounded>.
Configured Capacity: 23.32 6B
Configured Remote Capacity:
OFS used: 28.03 KB (0%)

Non OFS Used: 13.87 (3B

OFS Remaining: 824 GB (35.34%)

Block Pool used: 28.03 KB (0%)

2) mkdir:
Description: make a directory on Hadoop localhost with name SVIT
Syntax:

1
200410116026
TY-lT-2-A BDA-3161607
WARNING: Use of this script to bin/hadoop /SVIT dfs is
deprecated.
execute "hdfs dfs" instead.
WARNING: Attempting to execute
replacement
dfs -mkdtr

Execution:

avinay supergroup 0B Feb 15 0B SVIT

3)put:
Description: Put the file named 'FIRST.txt' to Hadoop storage from the local
storage

Syntax:
bin/hdfs dfs -put
/usr/share/hadoop /FIRST.txt /SVIT/
2023-02-15 INFO sast.Sas1DataTransferC1ient: SASL encryption trust check:
locatHostTrusted = false, remoteHostTrusted = false

Execution:

Last Block
Owner Group Size Modified Replication Size Name

11:12

4) copyFromLocaI:
Description: Copy a file named 'SECOND.txt' to Hadoop storage system at '/SVIT/'
folder.

SVIT
200410116026
TY-lT-2-A BDA-3161607
Syntax:
are/ bin/hdfs dfs -copyFromLocal /usr/sh
hadoop/SECOND.txt /SVIT/
sasl.SaslDataTransferClient: SASL encryption
2023-02-15 11:20:34, 151 INFO
check: tocaIHostTrusted = false, trust remoteHostTrusted = false
Execution:

5) moveFromLocaI:
Description: Move the file 'THIRD.txt' to Hadoop storage,means it will be deleted from
locat storage.

Before:

Syntax:
btn/hdfs dfs -moveFromLocal /usr/sh
are/hadoop/THIRD.txt /SVIT/
2023-02-15 INFO sas1.Sas1DataTransferC1ient: SASL encryption trust check:
localHostTrusted = false, remoteHostTrusted = false
Execution:

SVIT
200410116026
TY-lT-2-A BDA-3161607
Last DIOCK

6) get:

Description: copy files from Hadoop storage to the local storage.

Syntax:
bin/hdfs dfs -get /SVIT/THIRD.txt /
usr/share/hadoop/
2023-02-15 11:35:01, 325 INFO sas1.sas1DataTransferc1ient: SASL encryption trust
check: localHostTrusted = false, remoteHostTrusted = false
Execution:

7) getmerge:
Description: Merge to files and save the content of them in third files.
Syntax:
bin/hdfs dfs -getmerge /SVIT/THIRD.
txt /SVIT/SECOND. txt /usr/share/hadoop/merged.txt
2023-02-15 11:43: 15,941 INFO sast.SaslDataTransferClient: SASL encryption trust check:
localHostTrusted = false, remoteHostTrusted = false
Execution:

SVIT
200410116026
TY-lT-2-A BDA-3161607
matrtx.Ja
r mergel.
txt
merged.
txt merge
txt
8) copyToLocaI:
Description: Simply copy the file from Hadoop storage to local storage
Syntax:

bin/hdfs dfs -copyToLocat /SVIT /usr/tocal/hadoop/

Execution:

9) rmdir:
Description: Remove whole directory from Hadoop storage.
Syntax:
hadoop@vboxuser:/usr/ coca L/hoöoop$ bin/hadoop dfs -rmdir /SVIT
WARNING: Use of this script to execute dfs is deprecated.
WARNING: Attempting to execute replacement "hdfs dfs" instead.
Before:
Go!

Show 25 v entries
O Permission Owner Group Size Last Modified Replication Block Size Name
O drwxr-xr-x hadppp sypergrpyp 0B Feb 20 23:47 0B SVIT
Showing I to I of I entries
Previous Next

Execution:

No data available in table

SVIT
200410116026
TY-lT-2-A BDA-3161607
Showing O to O of O entries Previous Next

10)rm:

Descrption: Remove a file from hadoop storage.

Syntax:
coco unaö001S bin/hdfs dfs -rm /SVIT/FIRST.txt
Deleted /SVIT/FIRST.txt
Before:

Permission Owner Group Size Last Modified Replication 11 Block Size Name

o -rw-r—r" h?dpop supergroup 21B Feb 21 00:04 1 128 MB FIRST.txt

Showing I to I of I entries Previous Next

Execution:
/SVIT G
o
!
Show 25 v entries
.1 1
Permission Owner Group Size Last Modified Replication Block Size Name
No data available in table

Showing O to O of O entries Previous Next

11) sbin/hadoop/stop-all.sh:
Description: Stop all the Hadoop components started by 'start-all.sh' command
Syntax:
sbin/stop-all.sh
WARNING: Stopping alt Apache Hadoop daemons as malavinay in 10
seconds.
WARNING : Use CTRL-C to abort.
Stopping namenodes on [0.0.0.0]
Stopping datanodes
Stopping secondary namenodes [bda-virtualbox]
Stopping nodemanagers
Stopping resourcemanager
Execution:

SVIT
200410116026
TY-lT-2-A BDA-3161607

A Problem loading page X

C¯_) @ localhost:9870/explorer.html#/SVIT 0 01

Unable to connect
Firefox can't establish a connection to the server at
localhost:9870.

• The site could be temporarily

unavailable or too busy. Try again in a
few moments.
• Ifyou are unable to load any pages,
check your computer's network
connection.

• If your computer or network is protected by a firewall or proxy, make sure that

Firefox is permitted to access the Web.
Try Again