Parallel Computer Architectures

Parallel Computer Architectures
Chapter 8
Tanenbaum, Structured Computer Organization, Fifth Edition, (c) 2006 Pearson Education, Inc. All rights reserved. 0-13-148521-0
Parallel Computer Architectures
(a) On-chip parallelism. (b) A coprocessor. (c) A multiprocessor. (d) A multicomputer. (e) A grid.
Instruction-Level Parallelism
(a) A CPU pipeline. (b) A sequence of VLIW instructions. (c) An instruction stream with bundles marked.
The TriMedia VLIW CPU (1)
A typical TriMedia instruction, showing five possible operations.
The TM3260 functional units, their quantity, latency, and which instruction slots they can use.
The major groups of TriMedia custom operations.

(a) An array of 8-bit elements. (b) The transposed array. (c) The original array fetched into four registers. (d) The transposed array in four registers.
On-Chip Multithreading (1)
(a) (c) Three threads. The empty boxes indicated that the thread has stalled waiting for memory. (d) Fine-grained multithreading. (e) Coarse-grained multithreading.
On-Chip Multithreading (2)
Multithreading with a dual-issue superscalar CPU. (a) Fine-grained multithreading. (b) Coarse-grained multithreading. (c) Simultaneous multithreading.
Hyperthreading on the Pentium 4
Resource sharing between threads in the Pentium 4 NetBurst microarchitecture.

Homogeneous Multiprocessors on a Chip
Single-chip multiprocessors. (a) A dual-pipeline chip. (b) A chip with two cores.
Heterogeneous Multiprocessors on a Chip (1)
The logical structure of a simple DVD player contains a heterogeneous

multiprocessor containing multiple cores for different functions .
Heterogeneous Multiprocessors on a Chip (2)
An example of the IBM CoreConnect architecture.

Introduction to Networking (1)
How users are connected to servers on the Internet.

Introduction to Networking (2)
A packet as it appears on the Ethernet.
Introduction to Network Processors
A typical network processor board and chip.

The Nexperia Media Processor
The Nexperia heterogeneous multiprocessor on a chip.

Multiprocessors
(a) A multiprocessor with 16 CPUs sharing a common memory. (b) An image partitioned into 16 sections, each being analyzed by a different CPU.
Multicomputers (1)
(a) A multicomputer with 16 CPUs, each with its own private memory.
(b) The bit-map image of Fig. 8-17 split up among the 16 memories.
Multicomputers (2)
Various layers where shared memory can be implemented. (a) The hardware. (b) The operating system. (c) The language runtime system.
Taxonomy of Parallel Computers (1)
Flynns taxonomy of parallel computers.
Taxonomy of Parallel Computers (2)
A taxonomy of parallel computers.

Sequential Consistency
(a) Two CPUs writing and two CPUs reading a common memory word. (b) - (d) Three possible ways the two writes and four reads might be interleaved in time.
Weak Consistency
Weakly consistent memory uses synchronization operations to divide time into sequential epochs.
UMA Symmetric Multiprocessor Architectures
Three bus-based multiprocessors. (a) Without caching. (b) With caching. (c) With caching and private memories.
Snooping Caches
The write through cache coherence protocol. The empty boxes indicate that no action is taken.
The MESI Cache Coherence Protocol
The MESI cache coherence protocol.

UMA Multiprocessors Using Crossbar Switches
(a) An 8 8 crossbar switch. (b) An open crosspoint. (c) A closed crosspoint.

UMA Multiprocessors Using Multistage Switching Networks (1)
(a) A 2 2 switch. (b) A message format.
UMA Multiprocessors Using Multistage Switching Networks (2)
An omega switching network.

NUMA Multiprocessors
A NUMA machine based on two levels of buses. The Cm* was the first multiprocessor to use this design.
Cache Coherent NUMA Multiprocessors
(a) A 256-node directory-based multiprocessor. (b) Division of a 32-bit memory address into fields. (c) The directory at node 36.
The Sun Fire E25K NUMA Multiprocessor (1)
The Sun Microsystems E25K multiprocessor.

The Sun Fire E25K NUMA Multiprocessor (2)
The SunFire E25K uses a four-level interconnect. Dashed lines are address paths. Solid lines are data paths.
Message-Passing Multicomputers
A generic multicomputer.
Topology
Various topologies. The heavy dots represent switches. The CPUs and memories are not shown. (a) A star. (b) A complete interconnect. (c) A tree. (d) A ring. (e) A grid. (f) A double torus. (g) A cube. (h) A 4D hypercube.
BlueGene (1)
The BlueGene/L custom processor chip.

BlueGene (2)
The BlueGene/L. (a) Chip. (b) Card. (c) Board. (d) Cabinet. (e) System.
Red Storm (1)
Packaging of the Red Storm components.

Red Storm (2)
The Red Storm system as viewed from above.

A Comparison of BlueGene/L and Red Storm
A comparison of BlueGene/L and Red Storm.
Google (1)
Processing of a Google query.

Google (2)
A typical Google cluster.
Scheduling
Scheduling a cluster. (a) FIFO. (b) Without head-of-line blocking. (c) Tiling. The shaded areas indicate idle CPUs.
Distributed Shared Memory (1)
A virtual address space consisting of 16 pages spread over four nodes of a multicomputer. (a) The initial situation. .
A virtual address space consisting of 16 pages spread over four nodes of a multicomputer. (b) After CPU 0 references page 10.
A virtual address space consisting of 16 pages spread over four nodes of a multicomputer. (c) After CPU 1 references page 10, here assumed to be a read-only page.
Linda
Three Linda tuples.
Orca
A simplified ORCA stack object, with internal data and two operations.
Software Metrics (1)
Real programs achieve less than the perfect speedup indicated by the dotted line.
Software Metrics (2)
(a) A program has a sequential part and a parallelizable part. (b) Effect of running part of the program in parallel.
Achieving High Performance
(a) A 4-CPU bus-based system. (b) A 16-CPU bus-based system. (c) A 4-CPU grid-based system. (d) A 16-CPU grid-based system.
Grid Computing
The grid layers.


Parallel Computer Architectures

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Parallel Computer Architectures

Uploaded by

Copyright:

Available Formats

Parallel Computer Architectures

Parallel Computer Architectures

The TriMedia VLIW CPU (1)

A typical TriMedia instruction, showing five possible operations.

The TriMedia VLIW CPU (2)

The TriMedia VLIW CPU (3)

The major groups of TriMedia custom operations.

The TriMedia VLIW CPU (4)

On-Chip Multithreading (1)

On-Chip Multithreading (2)

Hyperthreading on the Pentium 4

Resource sharing between threads in the Pentium 4 NetBurst microarchitecture.

Homogeneous Multiprocessors on a Chip

Heterogeneous Multiprocessors on a Chip (1)

The logical structure of a simple DVD player contains a heterogeneous

Heterogeneous Multiprocessors on a Chip (2)

An example of the IBM CoreConnect architecture.

Introduction to Networking (1)

How users are connected to servers on the Internet.

Introduction to Networking (2)

A packet as it appears on the Ethernet.

Introduction to Network Processors

A typical network processor board and chip.

The Nexperia Media Processor

The Nexperia heterogeneous multiprocessor on a chip.

Taxonomy of Parallel Computers (1)

Flynns taxonomy of parallel computers.

Taxonomy of Parallel Computers (2)

A taxonomy of parallel computers.

UMA Symmetric Multiprocessor Architectures

The MESI Cache Coherence Protocol

The MESI cache coherence protocol.

UMA Multiprocessors Using Crossbar Switches

(a) An 8 8 crossbar switch. (b) An open crosspoint. (c) A closed crosspoint.

UMA Multiprocessors Using Multistage Switching Networks (1)

(a) A 2 2 switch. (b) A message format.

UMA Multiprocessors Using Multistage Switching Networks (2)

An omega switching network.

Cache Coherent NUMA Multiprocessors

The Sun Fire E25K NUMA Multiprocessor (1)

The Sun Microsystems E25K multiprocessor.

The Sun Fire E25K NUMA Multiprocessor (2)

The BlueGene/L custom processor chip.

Red Storm (1)

Packaging of the Red Storm components.

Red Storm (2)

The Red Storm system as viewed from above.

A Comparison of BlueGene/L and Red Storm

A comparison of BlueGene/L and Red Storm.

Processing of a Google query.

A typical Google cluster.

Distributed Shared Memory (1)

Distributed Shared Memory (2)

Distributed Shared Memory (3)

Three Linda tuples.

Software Metrics (1)

Software Metrics (2)

Achieving High Performance

The grid layers.

You might also like