How Computer Architecture Trends

May Affect Future Distributed Systems

Mark D. Hill

Computer Sciences Department

University of Wisconsin--Madison
http://www.cs.wisc.edu/~markhill

PODC ‘00 Invited Talk

(C) 2000 Mark D. Hill University of Wisconsin-Madison

 Three Questions

• What is a System Area Network (SAN)

and how will it affect clusters?
– E.g., InfiniBand

• How fat will multiprocessor servers be

and how to we build larger ones?
– E.g. Wisconsin Multifacet’s Multicast & Timestamp Snooping

• Future of multiprocessor servers & clusters?

– A merging of both?

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Outline

• Motivation

• System Area Networks

• Designing Multiprocessor Servers

• Server & Cluster Trends

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Technology Push: Moore’s Law

• What do following intervals have in common?

– Prehistory to 2000
– 2001 to 2002
• Answer: Equal progress in absolute processor speed
(and more doubling 2003-4, 2005-6, etc.)
– Consider salary doubling

• Corollary: Cost halves every two years

– Jim Gray: In a decade you can buy a computer
for less than its sales tax today

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Application Pull

• Should use computers in currently wasteful ways

– Already computers in electric razors & greeting cards
• New business models
– B2C, B2B, C2B, C2C
– Mass customization
• More proactive (beyond interactive) [Tennenhouse]
– Today: P2C where P==Person & C==Computer
– More C2P: mattress adjusts to save your back
– More C2C: Agents surf the web for optimal deal
– More sensors (physical/logic worlds coupled)
– More hidden computers (c.f., electric motors)
• Furthermore, I am wrong
(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends
 The Internet Iceberg

• Internet Components
– Clients -- mobile, wireless
– “On Ramp” -- LANs/DSL/Cable Modems
– WAN Backbone -- IPv6, massive BW
– and ...

• SERVICES
– Scale Storage
– Scale Bandwidth
– Scale Computation
– High Availability

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Outline

• Motivation

• System Area Networks

– What is a SAN?
– InfiniBand
– Virtualizing I/O with Queue Pairs
– Predictions

• Designing Multiprocessor Servers

• Server & Cluster Trends

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Regarding Storage/Bandwidth

• Currently resides on I/O Bus (PCI)

– HW & SW protocol stacks
– Must add hosts to add storage/bandwidth

proc proc

memory interconnect

memory bridge

i/o bus

i/o slot 0 i/o slot n-1

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends
 Want System Area Network (SAN)

• SAN vs. Local Area Nework (LAN)

– Higher bandwidth (10 Gbps)
– Lower latency (few microseconds or less)
– More limited size
– Other (e.g., single administrative domain, short distance)
– Examples: Tandem Servernet & Myricom Myrinet

• Emerging Standard: InfiniBand

– www.inifinibandTA.org w/ spec 1.0 Summer 2000
– Compaq, Dell, HP, IBM, Intel, Microsoft, Sun, & others
– 2.5 Gbits/s times 1, 4, or 12 wires

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 InfiniBand Model (from website)

proc proc

memory interconnect

memory HCA (host channel adapter)

link
X T target
Other
router C switch C (disks)
networks
A A

Other switches, hosts, targets, etc.

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Inifiniband Advantages

• Storage/Network made orthogonal from Computation

• Reduce “hardware” stack -- no i/o bridge
• Reduce “software” stack; hardware support for
– Connected Reliable
– Connected Unreliable
– Datagram
– Reliable Datagram
– Raw Datagram

• Can eliminate system call for SAN use (next slide)

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Virtualizing InfiniBand

• I/O traditionally virtualized with system call

– System enforces isolation
– System permits authorized sharing
• Memory virtualized
– System trap/call for setup
– Virtual memory hardware for common-case translation

• Infiniband exploits “queue pairs” (QPs) in memory

– C.f., Intel Virtual Interface Architecture (VIA)
[IEEE Micro, Mar/Apr ‘98]
– Users issue sends, receives, & remote DMA reads/writes

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Queue Pair

proc
• QP setup system call
– Connect with process
Main – Connect with remote QP
dma-W4
Memory (not shown here)
dma-R3
• QP placed in “pinned”
send2 receive1
virtual memory
send1 receive2
• User directly access QP
– E.g., sends, receives &
HCA remote DMA reads/writes

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 InfiniBand, cont.

• Roadmap
– NGIO/FIO merger in ‘99
– Spec in ‘00
– Products in ‘03-’10

• My Assessment
– PCI needs successor
– InfiniBand has the necessary features (but also many others)
– InifiniBand has considerable industry buy-in (but it is recent)
– Gigabit Ethernet will be only competitor
• Good name with backing from Cisco et al.
• But TCP/IP is a killer
– Infiniband for storage will be key

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 InfiniBand Research Issues

• Software Wide Open

– Industry will do local optimization
(e.g., still have device driver virtualized with system calls)
– But what is the “right” way to do software?
– Is there a theoretical model for this software?

• Other SAN Issues

– A theoretical model of a service-providers site?
– How to trade performance and availability?
– Utility of broadcast or multicast support?
– Obtaining quasi-real-time performance?

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Outline

• Motivation

• System Area Networks

• Designing Multiprocessor Servers

– How Fat?
– Coherence for Servers
– E.g., Multicast Snooping
– E.g., Timestamp Snooping

• Server & Cluster Trends

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 How Fat Should Servers Be?

• Use
– PCs -- cheap but small
– Workgroup servers -- medium cost; medium size
– Large servers -- premium cost & size

• One answer: “yes”

PCs w/ Servers running

“soft” state databases for
“hard” state

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 How Do We Build the Big Servers?

• (Industry knows how to build the small ones)

• A key problem is the memory system

– Memory Wall: E.g., 100ns memory access =
400 instruction opportunities for 4-way 1GHz processor
• Use per-processor caches to reduce
– Effective Latency
– Effective Bandwidth Used

• But cache coherence problem ...

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Coherence 101
“4”
r0<-m[100] “4” r1<-m[100]
P0 P1 “?” “?” Pn-1
m[100]<-5 r2<-m[100] r3<-m[100]
cache cache cache
100 : X
45 100 : 4

interconnection network

memory memory
100 4

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Broadcast Snooping
P2:GETX
P2:GETX
P1:GETX
Ordered Address Network

P2:GETX P1:GETX
P1:GETX P2:GETX P1:GETX
P2:GETX P1:GETX
P2:GETX

Mem P0 P1 P2
data data data

data data

Data Network

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Broadcast Snooping

• Symmetric Multiprocessor (SMP)

– Most commercially-successful parallel computer architecture
– Performs well by finding data directly
– Scales poorly
• Improvements, e.g., Sun E10000
– Split address & data transactions
– Split address & data network (e.g., bus & crossbar)
– Multiple address buses (e.g., four multiplexed by address)
– Address bus is broadcast tree (not shared wires)
• But…
– Broadcast all address transactions (expensive)
– All processors must snoop all transactions

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Directories

Address Network
P2:GETXP2:GETX
P1:GETX
send send P1:GETX P2:GETX

Dir/Mem P0 P1 P2
data data data

data data

Data Network

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Directories

• Directory Based Cache Coherence

– E.g., SGI/Cray Origin2000
– Allows arbitrary point-to-point interconnection network
– Scales up well

• But
– Cache-to-cache transfers common in demanding apps
(55-62% sharing misses for OLTP [Barroso ISCA ‘98])
– Many applications can’t use 100s of processors
– Must also “scale down” well

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Wisconsin Multifacet: Big Picture

• Build Servers For Internet economy

– Moderate multiprocessor sizes: 2-8 then 16-64, but not 1K
– Optimize for these workloads (e.g. cache-to-cache transfers)

• Key Tool: Multiprocessor Prediction & Speculation

– Make a guess... verify it later
– Uniprocessor predecessors: branch & set predictors
– Recent multiprocessor work: [Mukherjee/Hill ISCA98],
[Kaxiras/Goodman HPCA99] & [Lai/Falsafi ISCA99]
– Multicast Snooping
– Timestamp Snooping

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Comparison of Coherence Methods

Coherence Snooping Multicast

Directories
Attribute Snooping
Find previous Yes Sometimes Usually
owner directly? (good)
Always Yes No No (good)
broadcast?
Ordering w/o Yes No Yes (good)
acks?
Stateless at Yes No No but
memory? simpler
Ordered Yes No Yes, a
network? challenge

Use prediction to improve on both?

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends
 Multicast Snooping

• On cache miss
– Predict "multicast mask" (e.g., bit vector of processors)
– Issue transaction on multicast address network
• Networks
– Address network that totally-orders address multicasts
– Separate point-to-point data network
• Processors snoop all incoming transactions
– If it's your own, it "occurs" now
– If another's, then invalidate and/or respond
• Simplified directory (at memory)
– Purpose: Allows masks to be wrong (explained later)

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Predicting Masks

block address predicted mask

Mask Predictor
feedback

• Performed at Requesting Processor

– Include owner (GETS/GETX) & all sharers (GETX only)
– Exclude most other processors

• Techniques
– Many straightforward cases (e.g., stack, code,
space-sharing)
– Many options (network load, PC, software, local/global)

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Implementing an Ordered Multicast Network

• Address Network
– Must create the illusion of total order of multicasts
– May deliver a multicast to destinations at different times
• Wish List
– High throughput for multicasts
– No centralized bottlenecks
– Low latency and cost (~ pipelined broadcast tree)
– ...
• Sample Solutions
– Isotach Networks [Reynolds et al., IEEE TPDS 4/97]
– Indirect Fat Tree [ISCA `99]
– Direct Torus

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Indirect Fat Tree [ISCA ‘99]

P$
DM
(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends
 Indirect Fat Tree, cont.

• Basic Idea
– Processors send transactions up to roots
– Roots send transactions down with logical timestamp
– Switches stall transactions to keep in order
– Null transaction sent to avoid deadlock
• Assessment
– Viable & high cross-section bandwidth
– Many "backplane" ASICs means higher cost
– Often stalls transactions
• Want
– Lower cost of direct connections
– Always delivery transactions as soon as possible (ASAP)
– Sacrifice some cross-section bandwidth
(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends
 Direct 2-D Torus (work in progress)
• Features
0 1
– Each processor is switch
– Switches directly connected
– E.g., network of Compaq 21364
14 15
• Network order?
– Broadcasts unordered
– Snooping needs total order

• Solution
– Create order with logical timestamps
instead of network delivery order
– Called Timestamp Snooping [ASPLOS ‘00]

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Timestamp Snooping

• Timestamp Snooping
– Snooping with order determined by logical timestamps
– Broadcast (not multicast) in ASPLOS ‘00

• Basic Idea
– Assign timestamp to coherence transactions at sender
– Broadcast transactions over unordered network ASAP
– Transaction carry timestamp (2 bits)
– Processors process transactions in timestamp order

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Timestamp Snooping Issues

• More address bandwidth

– For 16-processors, 4-ary butterfly, 64-byte blocks
– Directory: 3*8 + 3*72 + more = 240 + more
– Timestamp Snooping 21*8 + 3*72 = 384 (< 60% more)

• Network must guarantee timestamps

– Assert future transactions will have greater timestamps
(so processor can processor older transactions)
– Isotach [Reynolds IEEE TPDS 4/97] more aggressively

• Other
– Priority queue at processor to order transactions
– Flow control and buffering issues
(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends
 Initial Multifacet Results

• Multicast Snooping [ISCA ‘99]

– Ordered multicast of coherence transactions
– Find data directly from memory or caches
– Reduce bandwidth to permit some scaling
– 32-processor results show 2-6 destinations per multicast

• Timestamp Snooping [ASPLOS ‘00]

– Broadcast snooping with “order” determined by
logical timestamps carried by coherence transactions
– No bus: Allows arbitrary memory interconnects
– No directory or directory indirection
– 16-processor results show 25% faster for 25% more traffic

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Selected Issues

• Multicast Snooping
– What program property are mask predictors exploiting?
– Why is there no good model of locality
or the “90-10” rule in general?
– How does one build multicast networks?
– What about fault tolerance?

• Timestamp Snooping
– What is an optimal network topology?
– What about buffering, deadlock, etc.?
– Implementing switches and priority queues?

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Outline

• Motivation

• System Area Networks

• Designing Multiprocessor Servers

• Server & Cluster Trends

– Out-of-box and highly-available servers
– High-performance communication for clusters

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Multiprocessor Servers

• High-Performance Communication “within box”

– SMPs (e.g., Intel PentiumPro Quads)
– Directory-based (SGI Origin2000)

• Trend toward hierarchical “out of box” solutions

– Build bigger servers from smaller ones
– Intel Profusion, Sequent NUMA-Q, Sun WildFire (pictured)

SMP SMP

SMP SMP
(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends
 Multiprocessor Servers, cont.

• Traditionally had poor error isolation

– Double-bit ECC error crashes everything
– Kernel error crashes everything
– Poor match for highly available Internet infrastructure

• Improve error isolation

– IBM 370 “virtual machines”
– Stanford HIVE “cells”

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 Clusters

• Traditionally
– Good error isolation
– Poor communication performance (especially latency)
– LANs are not optimized for clusters

• Enter Early SANs

– Berkeley NOW w/ Myricom Myrinet
– IBM SP w/ proprietary network

• What now with InfiniBand SAN (or alternatives)?

(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends

 A Prediction

• Blurring of cluster & server boundaries

• Clusters
– High communication performance
• Servers
– Better error isolation
– Multi-box solutions
• Use same hardware & configure in the field

• Issues
– How do we model these hybrids?
– Should PODC & SPAA also converge?
(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends
 Three Questions

• What is a System Area Network (SAN)

and how will it affect clusters?
– E.g., InfiniBand
– Make computation, storage, & network orthogonal

• How fat will multiprocessor servers be

and how to we build larger ones?
– Varying sizes for soft & hard state
– E.g., Multicast Snooping & Timestamp Snooping

• Future of multiprocessor servers & clusters?

– Servers will support higher availability & extra-box solutions
– Clusters will get server communication performance
(C) 2000 Mark D. Hill PODC00: Computer Architecture Trends