HDFS Command

Create a directory in HDFS

• mkdir
• Usage:
– hadoop fs -mkdir <paths>
• Example:
– hadoop fs -mkdir /user/saurzcode/dir1
/user/saurzcode/dir2
 List the contents of a directory
• ls
• Usage :
– hadoop fs -ls <args>
• Example:
– hadoop fs -ls /user/saurzcode
 Upload a file in HDFS
• put
• Usage:
– hadoop fs -put <localsrc> ... <HDFS_dest_Path>
• Example:
– hadoop fs -put /home/saurzcode/Samplefile.txt
/user/saurzcode/dir3/
 Download a file in HDFS
• get
• Usage:
– hadoop fs -get <hdfs_src> <localdst>
• Example:
– hadoop fs -get /user/saurzcode/dir3/
Samplefile.txt /home/
 See contents of a file
• cat
• Usage:
– hadoop fs -cat <path[filename]>
• Example:
– hadoop fs -cat /user/saurzcode/dir1/abc.txt
Copy a file from source to destination

• cp
• Usage:
– hadoop fs -cp <source> <dest>
• Example:
– hadoop fs -cp /user/saurzcode/dir1/abc.txt
/user/saurzcode/dir2
 Copy a file from/To Local file system
to HDFS
• copyfromlocal
• Usage:
– hadoop fs -copyFromLocal <localsrc> URI
• Example:
– hadoop fs -copyFromLocal /home/saurzcode/
abc.txt /user/saurzcode/abc.txt
Move file from source to destination

• mv
• Usage :
– hadoop fs -mv <src> <dest>
• Example:
– hadoop fs -mv /user/saurzcode/dir1/abc.txt
/user/saurzcode/dir2
Remove a file or directory in HDFS
• rm
• Usage :
– hadoop fs -rm <arg>
• Example:
– hadoop fs -rm /user/saurzcode/dir1/abc.txt
Display the aggregate length of a file.

• du
• Usage :
– hadoop fs -du <path>
• Example:
– hadoop fs -du /user/saurzcode/dir1/abc.txt
 Change Replication Factor

• Setrep
• Usage:
• hadoop fs -setrep [-R] [-w] <numReplicas>
<path>
• Example:
• hadoop fs -setrep -w 3 /user/hadoop/dir1
 Searching a file

• find
• Usage:
– hadoop fs -find <path> ... <expression> ...
• Example:
– hadoop fs -find / -name test –print
 Displays free space

• df
• Usage:
– hadoop fs -df [-h] URI [URI ...]
• Example:
– hadoop dfs -df /user/hadoop/dir1
 Count no. of directories, files and
bytes
• count
• Usage:
– hadoop fs -count [-q] [-h] [-v] <paths>
• -count –q options show:
– QUOTA, REMAINING_QUATA, SPACE_QUOTA, DIR_COUNT,
PATHNAME REMAINING_SPACE_QUOTA, FILE_COUNT,
CONTENT_SIZE
• The -h option shows sizes in human readable format.
• The -v option displays a header line.
• Example:
– hadoop fs -count -q hdfs://nn1.example.com/file1
– hadoop fs -count -q -h hdfs://nn1.example.com/file1
– hdfs dfs -count -q -h -v hdfs://nn1.example.com/file1