Professional Documents
Culture Documents
Hadoop Prac Commands
Hadoop Prac Commands
Hadoop Prac Commands
Contents
Prerequisites before Initiate HDFS file operation.
$start-dfs.sh (To Start all Daemons)
Jps (To check for all Daemons working)
2)Sqoop Commands
Sqoop import command.
Sqoop import with Where clause command.
Sqoop export command
Sqoop Incremental append.
3) Hive Commands
Internal/Managed table Creation in Hive
External table Creation in Hive
Loading data from Local file system to Hive
Static partitioning in Hive
Dynamic partitioning in Hive
Bucketing in Hive
get Command
-getcommand is used to copy data from hadoop system to local file
system, it will copy the data from hdfs stored directories to local file
system we can do the same by using copyToLocal command .
cp Command
It will copy file from one directory of HDFS file system to destination
directory in HDFS file system itself.
mv command
By using move command the File1.txt in wep directory will be moved to the new directory
/user/sumit .
Sqoop Command
Sqoop import command.
RDBMS-HDFS
The above Sqoop command will copy the file from local database which is in
localhost and in table student will be copied into Sqoop data directories in hdfs,
here first we create connections with jdbc and mysql and then data will be copied
to hdfs part file in directories.
Sqoop import with Where clause command.
Sqoop export command
HDFS-RDBMS
The command is used to load data from local database to hdfs in incremental
manner that means by looking at the last value of check column data will be
loaded to hdfs , the data after the specified value of column will be loaded
Hive Commads:
Internal table creation
External table creation
By creating the external table it helps in the way as if we will drop the
external table then the table will be deleted (Metadata) but the data
associated with the table (Actual data) will remain there in the
warehouse directories of hive .
Bucketing in Hive
Based on the resulting value, store the data into the corresponding bucket.
Data is distributed evenly between corresponding buckets.