See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/322203213

MIT Architecture of Complex Systems

Presentation · October 2016

CITATIONS READS

0 1,572

1 author:

Pradosh K. Roy
Independent Researcher
25 PUBLICATIONS   58 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Machine Learning & Fake News Identification View project

Multifractal Analysis , Chaotic Dynamics View project

All content following this page was uploaded by Pradosh K. Roy on 02 January 2018.

The user has requested enhancement of the downloaded file.

Architecture & Systems Engineering
Week 5: System Architect

Project Portfolio
Name
PRADOSH KUMAR ROY

Copyright © 2017. Massachusetts Institute of Technology. All rights reserved. 1

Architecture of Complex Systems

Instructions
Before you begin, you should save your Project Portfolio on Note: edX has a 10MB file size limit for
your local drive. We recommend the following format: document submission. If you have selected
large image(s), you may need to resize before
Lastname_Firstname_Course1_Week5
submitting, OR you may simply include a web
URL for the image in the image location. Be sure
Please note: You will not be able to re-download your file
after submission; therefore, please keep this file in a central to submit your assignment at least one hour
REQUIRED STEPS:
before the deadline to provide time for
location for future reference.
troubleshooting.
Step 1: Develop the system architecture.
While you will begin working in groups again this week,
the project deliverable is an individual submission. AStepOnce the deadline
2: Identify passes,
sources of you will not be able
ambiguity.
scoring rubric can be downloaded from the course in to upload the document and therefore will not
Stepbe
3: Identify
able to deliverables
submit and of the architect.
complete the
the Resources/Downloads tab on the top navigation.
Stepassignment.
4: Review and submit your project.
Like the previous two weeks, in Week 5 you will be self-
assessing your work as well as the work of three peers
in the class. If you have any questions, feel free to start Peer assessment is limited to 300 characters.
a thread in the Discussion Forum. Although work is
strictly individual, sharing ideas and concepts with other
students is encouraged.

Copyright © 2017. Massachusetts Institute of Technology. All rights reserved. 2

Architecture of Complex Systems

Week 5 Project

Overview
In the fifth and final project activity of this course,
your team will step up and view the system from REQUIRED STEPS:
the role of the architect. The steps to the right will
guide you through this process. Step 1: Develop the system architecture.

Note that some Scratch Pages are included at the

Step 2: Identify sources of ambiguity.
end of this document for you to capture any ideas, Step 3: Identify deliverables of the architect.
sketches, etc. that you have as you work through
the project. These will not be assessed and you Step 4: Review and submit your project.
are not required to submit them with your project
(but you may do so if you think they offer any
additional insight into your thinking process!).
NB. Scratch Pages Enclosed .

Copyright © 2017. Massachusetts Institute of Technology. All rights reserved. 3

Architecture of Complex Systems

Week 5 Project
System Overview : High Performance Computing System
System Complexity ~ 5.91 , de Weck’s Formula

Pleiades*, one of the world's most powerful supercomputers, represents NASA's state-of-the-art technology for meeting the agency's
supercomputing requirements, enabling NASA scientists and engineers to conduct modelling and simulation for NASA missions. This
distributed-memory SGI ICE cluster is connected with InfiniBand in a dual-plane hypercube technology.

Ranking : Top 500, 13 November 2016 , Legacy : Ranked Third in TOP500 LAPACK Rating at 487 TFlops November 2008
TPP : 7.25 Peta Flops [Peak] , Sustained : 5.95 Peta Flops

ABBREVIATIONS
HPCS : High Performance Computing System
PAC : Precision Air Conditioning System
IEEE : Institution of Electrical & Electronics Engineers
DMM : Distributed Memory Multicomputer
SMP : Shared Memory Multiprocessor
DSM : Distributed Shared Memory
GUI : Graphic User Interface e.g. X11R6
PBS : Portable Batch System
APE : Application Program Environment e.g. MPI , OpenMP, HPF ..
TPP : Theoretical Peak Performance
LAPACK : Linear Algebra Package http://www.netlib.org/benchmark/
NPB : NAS Parallel Benchmark 3.3.x https://www.nas.nasa.gov/publications/npb.html
NSF : National Science Foundation
ESP : Effective System Performance

https://www.hec.nasa.gov/about/overview.html

Copyright © 2017. Massachusetts Institute of Technology. All rights reserved. 4

Architecture of Complex Systems

STEP 1: ARCHITECTURAL REPRESENTATION

For your chosen system, develop the system architecture diagram and insert the diagram below. Remember: System
Architecture is the embodiment of a concept, the allocation of physical/informational function to the elements of
form, and the definition of relationships among the elements and with the surrounding context. Be sure to include all
information in a clean and understandable way.
Please remember the file size limit and resize or paste the image URL instead, as needed.

Name of System: High Performance Computing System

System Diagram/Schematic

Computing Processor Power Supply

Architecture
HPCS Parallelizing PAC System
Interconnect
IEEE Floating Point
Standards Storage
Visualizing Maintenance
OS Kernel
Job
top500.org Middleware
Scheduling
Compilers +
APE
Simulating
GUI
Scientist
Benchmarking PBS
LAPACK
Analyzing
Scalability NPB

PRODUCT SYSTEM BOUNDARY

Copyright © 2017. Massachusetts Institute of Technology. All rights reserved. 5

Architecture of Complex Systems

STEP 2: SOURCES OF AMBIGUITY

One of the primary roles of the architect is to reduce ambiguity. Fill in the first column of the following table with at least five sources of ambiguity, and then note
what data, analysis, or representations could be delivered by the architect to reduce ambiguity. Then, mark with an ’X’ in the corresponding cell to indicate the type
of ambiguity that particular data/analysis point is addressing.

Types of Ambiguity (Select all that apply)

Sources of Ambiguity Data, Analysis, or Representation to Reduce False Unknown Conflicting

Fuzziness Uncertainty
and Open Questions Ambiguity Information Information Information

DMM , SMP or Customer preferences in target market, x

DSM Architecture inputs from target applications and
current models
Double Precision Simulations based on 64 bit processor x x
Floating Point architecture , and extensive testing of
Accuracy prototype over various conditions.
http://www.davidhbailey.com/dhbpapers/high-prec-
arith.pdf

Scalability Analysis Extensive testing of Application x

Software in Stand–Alone mode ,
Theoretical Analysis.
TPP vs Sustained NSF Forecasting , x x
Performance Market forecasting studies , ESP
Analyss [System Utilization
Benchmark, LBL] http://crd-
legacy.lbl.gov/~oliker/papers/sc00_esp.pdf

Interconnect Simulation study based on usage x

patterns of current or similar
Infiniband vs Optical interconnects. Also projected needs for
Interconnect Exascale Computing.

Copyright © 2017. Massachusetts Institute of Technology. All rights reserved. 6

Architecture of Complex Systems

STEP 3: DELIVERABLES OF THE ARCHITECT

Over Weeks 3-5 of this course, you began to produce some of the deliverables of the architect. As a summary, the following table presents a
complete list of an architect’s deliverables. Mark the ones that have already been completed for your team’s system throughout this course as well
as the ones that you think haven’t yet been produced. (Note: You are not responsible for completing the deliverables “Not Yet Produced.”)
Are there any other additional deliverables you would add?

Not Yet
Completed
Produced

A clear, complete, consistent, and attainable set of goals x

Description of the broader context in which the system will sit, and the whole product context. x
Concept of the system. x
Concept of operations for the system, including contingency and emergency operations. x
Complete functional description of the system, with at least two layers of decomposition,
including description of primary and secondary externally delivered function; process flow with
internal operands and processes, including non-idealities, supporting processes, and interface x
processes with a process to ensure that the functional decomposition is followed.
The decomposition of form to two levels of detail, the allocation of function to form, and the
structure of form at this level. x

Details of all external interfaces and a process for interface control. x

A notion of the developmental cost, schedule and risks, and the design and implementation plan. x
Additional Deliverable of the Architect ? LAPACK Benchmark Results x
Additional Deliverable of the Architect ? NPB Benchmark Results x

Copyright © 2017. Massachusetts Institute of Technology. All rights reserved. 7

Architecture of Complex Systems

STEP 4: REVIEW & SUBMIT PROJECT

• Submit your completed Week 5 Project Portfolio
• Complete Self-Assessment of Project
• Complete Peer Assessments of Project (Peer assessment is limited to 300
characters)

• Note: The maximum file size that can be submitted is 10MB.

• A sample project submission and scoring rubric can be downloaded from
the course in the Resources/Downloads tab on the top navigation.
• Please remember that there are three steps to this assignment: Submission,
peer assessment, and self assessment. Please provide enough time by
each deadline to complete your assignment on time, as it is not possible to
submit once the submission window closes.

Copyright © 2017. Massachusetts Institute of Technology. All rights reserved. 8

Architecture of Complex Systems

Scratch Page*
Reminder: edX has a 10MB file size limit for document submission. If you have selected large image(s), you may need to resize
before submitting, OR you may simply include a web URL for the image in the image location. Be sure to submit your assignment at
least one hour before the deadline to provide time for troubleshooting.
Please remember the file size limit and resize or paste the image URL instead, as needed.

High Performance Computing System

Introduction : Parallel architectures have become the mainstay of scientific computing, including physics, chemistry, material science,
biology, astronomy, earth sciences, and others. The engineering application of these tools for modelling physical phenomena is now
essential to many industries, including petroleum (reservoir modelling), automotive (crash simulation, drag analysis, combustion
efficiency), aeronautics (airflow analysis, engine efficiency, structural mechanics, electromagnetism), pharmaceuticals (molecular
modelling), and others. In almost all of these applications, there is a large demand for visualization of the results, which is itself a
demanding application amenable to parallel computing. The Grand Challenge applications identified by the U.S. High Performance
Computing and Communications program in 1990s envisaged 3T performance. Development of RISC processors was a result of these
challenging demands for performance. Abstract architectures viz. SMP [Symmetric Multiprocessors] , DMM [Distributed Memory
Multicomputer] and DSM [Distributed Shared Memory] were developed to formalize the architectural trends. Explicit/Implicit programming
models were also developed simultaneously. However, as the programming models have become better understood and implementation
techniques have matured, compilers and run-time libraries have grown to provide an important bridge between the programming model
and the underlying hardware.

In 2000 AD , at GSFC NASA , through an innovative synergy of some old ideas from the parallel computing community and some new
low-cost technologies from the consumer digital electronics industry , emerged the Beowulf-class systems , the single most rapidly
growing class of high performance computing systems.

TeraFlops computing of 2000 is now revised to Petascale computing i.e. advanced computations required in fields such as weather and
climate simulation, nuclear simulations, cosmology, quantum chemistry, lower-level organism brain simulation, and fusion science. As on
2017 HPC systems are aspiring for Exascale computing. Benchmarking using LINPACK/LAPCK is now an industry standard.

Ref.
1.Ref. 1. Cueller, D.E.; Singh , Jaswinder P.; Gupta, A., Parallel Computer Architecture , Hardware/Software Approach , Morgan Kaufmann , San Francisco , 2002
2. Baker, Mark [Ed.] , Cluster Computing White paper, IEEE TFCC December, 2000.
3. http://www.top500.org
4. http://www.netlib.org

Copyright © 2017. Massachusetts Institute of Technology. All rights reserved. 9

Architecture of Complex Systems

Scratch Page*
Reminder: edX has a 10MB file size limit for document submission. If you have selected large image(s), you may need to resize
before submitting, OR you may simply include a web URL for the image in the image location. Be sure to submit your assignment at
least one hour before the deadline to provide time for troubleshooting.
Please remember the file size limit and resize or paste the image URL instead, as needed.

Pleiades, one of the world's most powerful

supercomputers, represents NASA's state-of-the-
art technology for meeting the agency's
supercomputing requirements, enabling NASA
scientists and engineers to conduct modeling and
simulation for NASA missions. This distributed-
memory SGI ICE cluster is connected with
InfiniBand in a dual-plane hypercube technology.
The system contains the following types of Intel
Xeon processors: E5-2680v4 (Broadwell), E5-
2680v3 (Haswell), E5-2680v2 (Ivy Bridge), and E5-
2670 (Sandy Bridge). Pleiades is named after the
astronomical open star cluster of the same name.

https://www.nas.nasa.gov/hecc/resources/pleiades.html

Copyright © 2017. Massachusetts Institute of Technology. All rights reserved. 10

Architecture of Complex Systems

Scratch Page*
Reminder: edX has a 10MB file size limit for document submission. If you have selected large image(s), you may need to resize
before submitting, OR you may simply include a web URL for the image in the image location. Be sure to submit your assignment at
least one hour before the deadline to provide time for troubleshooting.
Please remember the file size limit and resize or paste the image URL instead, as needed.

Architectural Decisions General Purpose HPCS . By general purpose system we mean that
the computer system must be able to be used to solve a range of
1. Processor scientific problems.
CISC
Nodes 11440 Intel Xeon E5 Series + 64 NVIDIA Tesla K40
Memory DDR3 FB-DIMMS (host); GDDR5 (GPU) , 932 TB
Parallel hardware systems now can be roughly divided into two
Total CPU cores: 246,048 groups: shared memory, where there is a single address space
Theoretical Peak 7.25 Pflop/s and physical memory system [SMP], and distributed memory
Sustained Performance 5.95 Pflop/s LINPACK Rating [DMM] , where each processor has its share of the system's
memory attached to it.
2. Architecture DMM
3. Interconnect : InfiniBand, with all nodes Hybrid systems mix the two flavors of memory. One form
4. Topology : Hypercube topology consists of an array of shared memory multiprocessors, tied
4. Storage : 29 PB of RAID 5 disk storage
together with a ultrafast network. Another flavor is to connect
5. Operating Environment SUSE Linux Enterprise Server 11
6. Parallel Development Environment MPI SGI MPT 2.14
shared memory multiprocessors together with a global memory
7.Math Library MKL 2017.0.0 system , separate from the different shared memories.
8. Job Scheduling : Altair PBS Professional
10. Compiler : Intel and GNU C, C++ and Fortran Distributed shared memory [DSM] systems have physically
distributed memory, but rely on a combination of operating
system and hardware to move address references where they
The system contains the following types of Intel Xeon processors: E5- are needed. Here the user has a single logically shared
2680v4 (Broadwell), E5-2680v3 (Haswell), E5-2680v2 (Ivy Bridge), address space, but accessing data belonging to another
and E5-2670 (Sandy Bridge).
processor can take significantly longer than accessing it from
the local memory - leading to the term NUMA, or non-uniform
memory access.

https://www.cs.indiana.edu/classes/b673/notes/architecture.html

Copyright © 2017. Massachusetts Institute of Technology. All rights reserved. 11

Architecture of Complex Systems

Scratch Page*
Reminder: edX has a 10MB file size limit for document submission. If you have selected large image(s), you may need to resize
before submitting, OR you may simply include a web URL for the image in the image location. Be sure to submit your assignment at
least one hour before the deadline to provide time for troubleshooting.
Please remember the file size limit and resize or paste the image URL instead, as needed.

Course1_Week5_Project_Selected_References

STEP 1: ARCHITECTUAL REPRESENTATION

Ref.
1. Cueller, D.E.; Singh , Jaswinder P.; Gupta, A., Parallel Computer Architecture , Hardware/Software Approach , Morgan
Kaufmann , San Francisco , 2002, Chapters 2,3 and 4 , pp. 74-261

STEP 2: SOURCES OF AMBIGUITY

Ref.
1. https://en.wikipedia.org/wiki/IEEE_754
2. David H. Bailey and Allan S. Snavely, "Performance modelling: Understanding the present and predicting the future,"
Proceedings of Euro-Par 2005, Lisbon, Portugal, Sep 2005.
3. Bo Einarsson [Ed] , Accuracy and Reliability in Scientific Computing, SIAM , Philadelphia, 2005
4. https://www.cse.wustl.edu/~dshook/cse566/lectures/Exascale.pdf

STEP3 : ADDITIONAL DELIVERABLES OF THE ARCHITECT

Ref
1. https://www.top500.org/system/177259
2. https://www.nas.nasa.gov/assets/pdf/papers/NAS_Technical_Report_NAS-2015-05.pdf
3. http://pubs.opengroup.org/architecture/togaf9-doc/arch/chap36.html

Copyright © 2017. Massachusetts Institute of Technology. All rights reserved. 12

View publication stats