Yachay

IBM.EC HPC and Big Data Y. Deng

Workshop on HPC, Big Data and Applications

for Yachay Technical University through IBM EC in 2017

Last Update: Tuesday, January 24, 2017
Workshop Plan 10:00-11:00 Lecture
11:00-11:30 Coffee break
11:30-12:30 Lecture
12:30-14:00 Lunch break
14:00 15:00 Lecture
15:00-15:30 Coffee break
15:30 16:30 Discussions or online experiments
Computer: Yachay Supercomputer
Projects Due: 23:59 on Fridays (view individual projects)

10:00-11:00 11:30-12:30 14:00-15:00 15:30 16:30
Week 1
Day 1 Introduction to Hardware: Past, Node Discussion
HPC and Big present, and architecture,
March 9 Data; future; central
Thursday Taxonomy Taxonomy processing
units,
accelerators, IO,
memory
architecture,
interconnection
networks
Day 2 Software: Message Performance Discussion
Programming passing measures:
March 10 models; protocols and Speedup;
Friday Taxonomy tools Efficiency

March 11 Saturday, March 12 Sunday: Informal discussions and weekend break.

Day 3 Overhead; Amdahls law Popular Online

strong and benchmarks practice
March 13 weak
Monday scalabilities
Day 4 MPI: process MPI: MPI: Collective Project 1
March 14 creation and Send/Receive operations
Tuesday management
Day 5 MPI: Task MPI lecture5: MPI lecture6: Project 1
mapping Examples Examples
March 15
Wednesday
Week 2
Day 6 Algorithms: Algorithms: Algorithms: Project 2

Yachay IBM.EC Workshop Page 1

Yachay IBM.EC HPC and Big Data Y. Deng

March 16 Problem Communication Load balance

Thursday decomposition; handling
Day 7 Algorithms: Linear Algebra: Linear Algebra: Project 2
March 17 Task mapping Matrix-vector Matrix-matrix
Friday operations operations
March 18 Saturday, March 19 Sunday: Informal discussions and weekend break.

Day 8 Linear Algebra: Linear Algebra: Iterative Project 3
Dense Ax=b Sparse Ax=b methods:
May 8 and Gaussian and iterative Jacobi,
Monday eliminations methods conjugate
gradient, Gauss-
Seidel, SOR etc
Day 9 Basic calculus: PDEs: Basics PDEs: Wave Project 3
Integration/Diff equations
May 9
Tuesday
Day 10 PDEs: Diffusion, PDEs: Maxwell PDEs: other Discussions
heat equations, equations equations of
May 10 Poison eqn choice
Wednesday
Week 3
Day 11 ODEs Molecular Molecular Project 4
dynamics dynamics: Drug
May 11 design
Thursday
Day 12 Monte Carlo Simulated Genetic Project 4
methods annealing algorithms
May 12
Friday
May 13 Saturday, May 14 Sunday: Informal discussions and weekend break.

Day 13 MC applications Fast Fourier FFT Final project
transforms applications
Optional
Day 14 Big data I Big data II Big data III Final project

Optional
Day 15 Cloud Cloud Cloud Final
computing I computing II computing III presentations
Optional

Yachay IBM.EC Workshop Page 2

Yachay IBM.EC HPC and Big Data Y. Deng

Prerequisites
1. Computing experiences with networked workstations;
2. Have written programs using languages such as FORTRAN or C or C++ or Java;
3. A major in mathematics or in any of the physical sciences or engineering.

Goals
To gain extensive practical experience in design and analysis of parallel algorithms for
scientific, engineering and commerce applications.

Workshop Expectations
I. Demonstrate knowledge of supercomputer architectures:
1. Node architecture, central processing units, and accelerators;
2. Distributed- and shared-memory;
3. Various interconnection networks;
4. Network architectures and performance;
5. IO subsystem.

II. Demonstrate mastery of software architecture and MPI:
1. Communication patterns and protocols;
2. Message passing interface (MPI);
3. Process creation and management;
4. Single-sided communications;
5. Collective operations.

III. Demonstrate understanding of performance measurements:
1. Speedup;
2. Parallel efficiency;
3. Overhead;
4. Load balance;
5. Scalability, strong or weak;
6. Amdahls law.

IV. Demonstrate mastery of parallel algorithms:
1. Algorithm classification according to complexities;
2. Problem decomposition;
3. Communication handling;

V. Demonstrate mastery of applications of parallel computers:
1. Linear algebra: matrix multiplication, solution of AX=b, eigenvalues and eigenvectors;
2. Basic calculus including high-dimensional integration;
3. Solutions of ODEs and systems including molecular dynamics;
4. Solutions of a few PDES including wave, diffusion, and Poison equations;
5. Fast Fourier transforms;
6. Markov chain Monte Carlo methods.

Workshop Overview
1. Architectures

Yachay IBM.EC Workshop Page 3

Yachay IBM.EC HPC and Big Data Y. Deng

Parallel computer architectures including node, network, and IO subsystems.

2. Algorithms
Design and analysis of basic parallel algorithms and software including MPI and
message passing protocols.
3. Applications
Parallel computing applications in linear algebra, calculus, basic differential equations,
Monte Carlo methods, and fast Fourier transforms.

Textbooks and Other Reading Material
HPC is a quickly evolving field and there is a massive amount of useful material online. I would distribute all
lecture notes and a copy of my book "Applied Parallel Computing". I will post most material including
project assignments, lecture notes, sample programs.

1. A. Grama et al, Introduction to Parallel Computing (2nd Edition), Addison Wesley (January,
2003); ISBN-10: 0201648652
2. V. Kumar et al, Introduction to Parallel computing, (Benjiamin/Cummings, Redwood City, CA,
1994)
3. G. E. Karniadakis and R. M. Kirby II, Parallel Scientific Computing in C++ and MPI: A Seamless
Approach to Parallel Algorithms and their Implementation (2003)
4. P. Pacheco, Parallel Programming with MPI (1st Edition), Morgan Kaufmann (October, 1996);
ISBN-10: 1558603395
5. Y. Deng, Applied Parallel Computing, World Scientific (Singapore, 2013):
http://www.worldscientific.com/worldscibooks/10.1142/7767
6. Eric F. van de Velde, Concurrent Scientific Computing, Springer-Verlag (New York, 1994):
http://www.springer.com/mathematics/analysis/book/978-0-387-94195-0
7. G. Fox et al, Parallel Computing Works! (Morgan Kaufmann Publishing (San Francisco, 1994):
http://www.amazon.com/Parallel-Computing-Works-Geoffrey-Fox/dp/1558602534
8. B. Barney, Introduction to Parallel Computing (Online tutorial):
https://computing.llnl.gov/tutorials/parallel_comp/

The Lecturer:
Yuefan Deng earned his BS (1983) in Physics from Nankai University and his Ph. D.
(1989) in Theoretical Physics from Columbia University. He is currently a
professor at the Stony Brook University in New York.

Prof. Dengs research covers parallel computing, molecular dynamics, Monte Carlo
methods, and biomedical engineering. He published more than 85 papers in these
areas and supervised 25 doctoral theses. He is the architect of Galaxy Beowulf Supercomputer at
Stony Brook built in 1997 and of NankaiStars Supercomputer which was China's fastest when it was
completed in 2004. He also built a supercomputer prototype called RedNeurons in 2007 with financial
support from Chinas Ministry of Science and Technology and Shanghais Commission of Science and
Technology. His research in US is supported by DOE, NSF, NIH, as well New York State. He lectured
widely in US, Germany, Russia, Brazil, Singapore, as well as the Greater China region.

Wiki: https://en.wikipedia.org/wiki/Yuefan_Deng

Yachay IBM.EC Workshop Page 4