2023

Vignan University: HPC User Guide

Krishna Mouli & Umma Reddy Chirra

Boston IT Solutions (India) PVT LTD
3/13/2023
Table of Contents
I. Single Node Cluster Architecture .................................................................................................... 2
A. OS Version ................................................................................................................................... 2
B. OS User information.................................................................................................................... 2
1. User Access Details .................................................................. Error! Bookmark not defined.
II. Sample program to run the MPI program. ...................................................................................... 2
A. The following command runs the program in only master node. .............................................. 3
B. Run the “hello world” program in compute. .............................................................................. 3
C. Run the “hello world” program in master & compute. ............................................................... 3
III. Running Workloads using PBS Job Scheduler ............................................................................. 3
A. Basic Commands to work with openPBS for an End User........................................................... 3
B. Resource Configuration in openPBS interpretation .................................................................... 4
C. Sample Program to Run in openPBS ........................................................................................... 4
1. Output of the Sample script .................................................................................................... 4
D. A Sample example to run on Nvidia GPU .................................................................................... 5
E. Output of the GPU flags .............................................................................................................. 5
IV. How to access the windows VM through Compute Node ............... Error! Bookmark not defined.
V. Docker Usage .................................................................................................................................. 6
A. Background ................................................................................................................................. 6
B. Containers ................................................................................................................................... 6
VI. Docker Usage for a Web Development Lab use case. ................................................................. 7

1
I. Single Node Cluster Architecture

A. OS Version
NAME="Ubuntu"
VERSION="20.04.5 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.5 LTS"
VERSION_ID="20.04"

B. OS User information

Master Compute

The vfstr user is an MPI cluster where the end user can run programs with openMPI by
compiling program and run in the cluster.

II. Sample program to run the MPI program.

A sample program is located in /home/vfstr/data/openmpi-4.1.5/examples/ directory
which is a common shared directory between master and compute.

2
The sample program is a simple C program which prints “hello world”

A. The following command runs the program in only master node.

mpirun -np 1 --hostfile /etc/hosts --host master ./outputfile

B. Run the “hello world” program in compute.

mpirun -np 2 --hostfile /etc/hosts --host compute ./outputfile

C. Run the “hello world” program in master & compute.

mpirun -np 2 --hostfile /etc/hosts --host master,compute ./outputfile

III. Running Workloads using PBS Job Scheduler

OpenPBS software optimizes job scheduling and workload management in high-
performance computing (HPC) environments – clusters, clouds, and supercomputers –
improving system efficiency and people’s productivity. Built by HPC people for HPC people,
OpenPBS is fast, scalable, secure, and resilient, and supports all modern infrastructure,
middleware, and applications.

A. Basic Commands to work with openPBS for an End User

COMMAND DESCRIPTION USAGE EXAMPLE EFFECT
qsub Submit a job qsub myjob Submits the job "myjob" for the
execution.
qstat Show status of qstat -u abc Show the status of all the jobs
batch jobs submitted by the user abc
qdel Delete a job qdel 3221.pbsserver Deletes the job with job id
3221.
pbsnodes -a gives an overview over the status of all nodes of the compute
cluster.

3
B. Resource Configuration in openPBS interpretation

Status of the Job Queue

No. of Computational
Threads available

No. of GPU Assigned for

a Job

C. Sample Program to Run in openPBS

A sample script to Running Jobs in openPBS:
#!/bin/bash
#PBS -l nodes=1:ppn=12
#PBS -l walltime=2:00:00
#PBS -V
cd $PBS_O_WORKDIR
python3 /home/vfstr/pbs/sample.py

The job will run on 1 node with 12

cores of the CPU and the job will wait
for 2Hrs if the exiting job is running

Path of the file to run with

command
1. Output of the Sample script
The command will run the above sample file on the 12 core of CPU and the
submitted job name will be 2.compute the PBS will create two file
run_python_example.o2 & run_python_example.e2 respectively with O & E file
symbolic meaning Output file and Error file .
qsub run_python_example

4
The output file has the following can be viewed using the cat command.

D. A Sample example to run on Nvidia GPU

The user needs to activate the respective conda virtual environment before running program
in GPU with Python programming.

qsub run_python_example_gpu

E. Output of the GPU flags

5
 Remote Desktop Access Server
1. Check the communication between your personal PC and the windows 10 VM.
Open command prompt and type ping 192.168.10.125

2. Open run command on your personal PC and type mstsc and click Ok.

3. Remote desktop windows will come and enter the windows 10 VM details and access the
VM.

4. Once you connected to the windows 10 you can use the same login credentials.
Username: vignan
Password: Vignan@2023

IV. Docker Usage

A. Background
Docker is an open platform for developing, shipping, and running applications. Docker
enables you to separate your applications from your infrastructure so you can deliver
software quickly. With Docker, you can manage your infrastructure in the same ways you
manage your applications. By taking advantage of Docker’s methodologies for shipping,
testing, and deploying code quickly, you can significantly reduce the delay between writing
code and running it in production.

B. Containers
A container is a runnable instance of an image. You can create, start, stop, move, or delete a
container using the Docker API or CLI. You can connect a container to one or more networks,
attach storage to it, or even create a new image based on its current state.

6
 By default, a container is relatively well isolated from other containers and its host machine.
You can control how isolated a container’s network, storage, or other underlying subsystems
are from other containers or from the host machine.

A container is defined by its image as well as any configuration options you provide to it
when you create or start it. When a container is removed, any changes to its state that are
not stored in persistent storage disappear.

Example docker run command

The following command runs an ubuntu container, attaches interactively to your local
command-line session, and runs /bin/bash.

sudo docker run -i -t ubuntu /bin/bash

When you run this command, the following happens (assuming you are using the default registry
configuration):

1. If you do not have the ubuntu image locally, Docker pulls it from your configured registry, as
though you had run docker pull ubuntu manually.
2. Docker creates a new container, as though you had run a docker container create command
manually.
3. Docker allocates a read-write filesystem to the container, as its final layer. This allows a
running container to create or modify files and directories in its local filesystem.

4. Docker creates a network interface to connect the container to the default network, since
you did not specify any networking options. This includes assigning an IP address to the
container. By default, containers can connect to external networks using the host machine’s
network connection.

5. Docker starts the container and executes /bin/bash. Because the container is running
interactively and attached to your terminal (due to the -i and -t flags), you can provide input
using your keyboard while the output is logged to your terminal.
6. When you type exit to terminate the /bin/bash command, the container stops but is not
removed. You can start it again or remove it.

V. Docker Usage for a Web Development Lab use case.

Docker images can be downloaded from DockerHub using the following URL:https://hub.docker.com/

For the following usecase we will downloading and tomcat server from dockerhub by placing key
words in the search bar

7
Docker hub search engine will show a CatLog of docker image available in their repository, in this we
can see tomcat server is available.

Let Pull the image with the pull command in the Master node.

Once the docker image is pulled we can launch the docker image using the run command as an
isolated container in the master node so that we can the Tomcat services.

sudo docker run -it --rm tomcat:9.0

8
We can test the tomcat server by visiting http://container-ip:8080 in a browser or, if you need access
outside the host, on port 8888:

sudo docker run -it --rm -p 8888:8080 tomcat:9.0

Command to running the Tomcat servers with two cores of CPU and 4 GB of RAM.

sudo docker run -it --cpus="0.2" --memory="4g" --rm tomcat:9.0

The command sudo docker stats show the overall resource usage by the container launched.