Exercise #7: Configuring Hadoop on Ubunto (https://www.youtube.com/watch?

v=I7BKFKurA2c)

Objectives: At the end of this exercise, you will be able to:

❖ install VMWare, and

❖ perform setting up and installing Hadoop in Linux environment

1. Use the tar commands to extract files:

✓ Open Terminal then keye in “tar xvf nameOfTarFile”
✓ or right click -> extract here
2. Next, in your Hadoop folder -> etc folder there are some files that are important. These are:
✓ core-site.xml
✓ hdfs-site.xml
✓ hadoop-env.sh
✓ yarn-site.xml
3. in between configuration for:
A. core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
B. hdfs-site.xml (NOTE: username is equal to your Ubuntu username)
<configuration>
<property>
<name>dfs.name.dir</name>
<value>/home/username/metadata_nn/dfs/name</value>
</property>

<property>
<name>dfs.data.dir</name>
<value>/home/username/metadata_nn/dfs/name/data</value>
</property>

<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
 C. yarn-site.xml
<configuration>

<!-- Site specific YARN configuration properties -->

<property>
<name>yarn.nodemanager.aux-service</name>
<value>mapreduce_shuffle</values>
</property>
</configuration>
D. hadoop-env.sh
NOTE:
✓ find your “java home” by going to setup folder->jdk,
✓ then right click “Open Terminal” type “pwd” or present working directory
✓ Java home is line 54
Example:

4. Next step is to install “ssh” or core shell

✓ Open Terminal -> sudo apt-get install ssh

5. Once install make your “keygen” to login directly (password less)

✓ ssh-keygen -t rsa -P ””
✓ hit “enter” for the key

6. You need to copy the actual key shell file (NOTE: ~/.ssh/authorized_keys are keys of other
computers that you connected to/trust, not your own key)
7. Format name node.
✓ Go to ->Setup->Hadoop folder ->bin.
✓ Then open Terminal. Key in ./hadoop namenode - format
8. Now, we have to let our Linux system where our file is located. Next step is setting up the bashrc
file. Bashrc is how you set things in Linux environment. So open Terminal (Desktop)
✓ In Terminal key in “Gedit ~/.bashrc” (to open Bashrc file)
✓ (in the last part of your bashrc file) type:
export HADOOP_HOME=/home/velinda/Setup/hadoop-3.3.5
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin

export JAVA_HOME=/home/velinda/Setup/jdk-20
export PATH=$PATH:$JAVA_HOME/bin
✓ Save your file and close bashrc. To save, go to the terminal (bashrc) “source ~/.bashrc”
✓ Then “clear”
✓ Then finally “start-all.sh” to initiate your Hadoop shell
9. Once all changes are done, now you can start hadoop