Professional Documents
Culture Documents
High Performance Computing (Planing-Usage-Application)
High Performance Computing (Planing-Usage-Application)
Yudi Rosandi
Workshop on High Performance Computing: from design to application
Outline
Planing
Hardware components Infrastructures Operating system Basic knowledge
Usage
Cluster Application
Simulation, research and development Data processing, forecasting
Planing
Cluster
collection of computers interconected one and another via network working together on a single program or interdependent different programs
Planing of a cluster
What is the purpose? How much budget do we have? What is the expected outcome?
Load Balancing
multi-user environtment
Budget...LOW
build from scratch
chalenging precious experience troublesome but interresting freedom most problems comes from networking cable mesh high maintenance cost high power consumtion (depends on hardware selection)
Pitfalls
Budget...HIGH
high-end cluster
blade server ready built racks production ready integrated high-speed lan (infiny-band) good cooling system low power consumption
What we get
HPC Outcome
Research
General sciences
fast publication many ready to use programs ability to solve complex problems insight to unknown aspects in science parallel programming computer administration networking user management benchmarking and optimization
Computer science
HPC Outcome
Production
fast computation fast decission making increasing services high availability
Hardware components
Anything...
Hardware requirements
Computing nodes
fast computer fast network interface sufficient memory local scratch storage dedicated, reliable large storage sufficient memory (+8Gb) high-speed network connection (channel bonding, etc)
Storage server
Hardware requirements
Head server(s): reliable, not necessarily fast
dhcp nis login queuing note:
these servers may be separated individual servers, combined servers in one machine, or virtualized (e.g. Xen) Login server may need special consideration!
Needed infrastructure
Room temperature control Stabil electricity Power consumption networking backbone
Operating system
Things to consider
stability reliability managability accessibility security
Why????
Cluster Usage
don't do:
Remote login
Accesible from internet
static address dedicated
Remote login
Remote login
Files for ssh connection (under ~/.ssh/)
id_rsa (@client)
identity file, must be kept 'only' in the connecting computer SECRET! don't transfer via email, etc!
id_rsa.pub
public key, to be distributed to all servers
known_hosts (@client)
list of host keys of known servers
config (@client)
Shell
elements
variables program controls redirection pipes
customizing shell
make your self convenient personal commands, path, etc creating job-script building embarrassingly parallel programs automating processes
Shell: bash
Standard in Linux Complete Flexible Advanced Demo/example....
declaring variable simple program controls redirection and pipes
Application
Application
incomplete examples... Research:
modelling nano-sciences complex-system image rendering
Industry
Many others...
data processing
Thank you