07performance Cse130 Fa19 KV

© 2019 Ethan L.
Miller
Performance
Principles of Computer System Design
© 2019 Ethan L. Miller
Computer systems performance

❖ We want computers to perform “better”
‣ Complete tasks faster
‣ Use less power / energy
‣ Do more tasks
❖ How is performance measured?
❖ How can we design systems to improve performance?

‣ Intrinsic issues
‣ Technology-dependent issues
CSE 130: Principles of Computer System Design 2

Refresher: SI pre xes
Pre x Pre x Power (base 2) Size (base 10)
Kilo (k) milli (m) 210 / 2-10 103 / 10-3

Mega (M) micro (µ) 220 / 2-20 106 / 10-6
Giga (G) nano (n) 230 / 2-30 109 / 10-9
Tera (T) pico (p) 240 / 2-40 1012 / 10-12
Peta (P) femto (f) 250 / 2-50 1015 / 10-15
Exa (E) atto (a) 260 / 2-60 1018 / 10-18

fi
fi
fi
Measuring performance
❖ Computer systems can be measured in many aspects
❖ Capacity: amount of a resource that’s available (time, memory, etc.)

‣ Utilization: percentage of capacity being used
‣ Overhead: resource “wasted”
‣ Useful work: resource spent on actual work
❖ Time-based metrics
‣ Latency: time between input and result
‣ Throughput: rate of results (useful work) per unit time

Capacity-based metrics
❖ System has a certain amount of a resource
‣ Some used at the “current” level: useful work
‣ Some consumed at levels below: overhead
❖ Examples:
‣ In-Memory Hash-Table
‣ Multi-threaded HTTP Server

Latency vs. throughput

❖ 200 people want to go to Santa Barbara (~250 miles)
❖ How should we get there?
‣ Tesla Roadster 2.0 (Capacity 2, speed 250 MPH)
‣ Yellow school bus (Capacity 50, speed 50 MPH
Latency Tesla gets the rst 2 people there much faster: about 1 hour
Bandwidth School bus gets all people there much faster: about 40 hours

fi
Time-based metrics
❖ Latency: elapsed time for a single action
❖ Throughput: rate of actions per unit time
❖ Let’s study a modularized system!

‣ What’s the latency calculation?
‣ What’s the throughput calculation?

Pipelining and concurrency

❖ Pipelining: split operation into multiple individual stages
‣ Each stage operates independently
‣ Single request goes through multiple stages
‣ Di erent requests may go through di erent stages
❖ Latency: time for a single request to go through all stages
latencyA+B ≥ latencyA + latencyB
❖ Throughput: limited by throughput of slowest stage
throughputA+B ≤ min(throughputA, throughputB)

ff
ff
Sometimes the best choice isn’t obvious

❖ 1980s/1990s Silicon Chip databases
‣ Size of les / slowness of across-the-ocean
data transfers led to . . .
• Hand tape to engineer who boards a Boeing
747 to Japan
❖ Event Horizon Telescope
‣ Collected 5 petabytes around the world
‣ Shipped to central location for processing
on 800 computers
‣ “Nothing beats a 747 lled with hard disks.”

fi
fi
Sometimes the best choice isn’t obvious

❖ What’s the fastest way to move 100PB of data from Santa Cruz to
Seattle?
‣ Network (1 GB/s) is very fast, but is it fast enough?
‣ Is there a faster way?
Never underestimate the bandwidth of a station

wagon full of tapes hurtling down the highway.
—Andrew S. Tanenbaum
Amazon Snowmobile
https://aws.amazon.com/snowmobile/

Remember!
Latency (execution time) Time for one task
Throughput (bandwidth) Tasks per unit time

Designing for performance

❖ So how do we make a system run “better”?
❖ Build THEN Measure

‣ (Don’t prematurely optimize)
‣ (You will probably be incorrect about where the bottlenecks are)

Designing for performance

1. Measure the system to see if it is needs to be faster
2. Measure again to nd the bottleneck
3. Predict the impact of removing bottleneck
3.a.Removing the bottleneck doesn’t help performance: reconsider design?
3.b.Removing the bottleneck will help: nd ways to remove it
4. Implement new solution and repeat

fi
fi
Bottlenecks in Law and economics
By Lasse Havelund, based on content by Farcaster (Wikipedia)
❖ Eliminate NASA budget, all $18 billion of it

‣ How much can the government reduce spending?
❖ Reduce Defense by 20% (to 0.8× original)
‣ How much does this save?
Amdahl’s Law (Law of Diminishing Returns)

❖ How much will an optimization improve performance if it only
impacts one stage of the request?
‣ P = portion of total time that can be improved
1
‣ S = amount by which that portion is sped up (1 P) + P
S

Reducing latency: exploit workload properties

❖ Reducing latency is di cult: often limited by set rules
‣ Speed of light
‣ Algorithm run time
❖ Leverage non-uniformity: reduce latency for some requests
‣ Common requests
‣ “Easy” requests
❖ Create a fast path

ffi
Fast path example
Latencyfast Frequencyfast + Frequencyslow = 1 (proportion of fast vs. slow)

Latencyslow
Frequencyfast AverageLatency =
Latencyfast × Frequencyfast + Latencyslow × Frequencyslow
Frequencyslow
❖ One very common approach to fast-path: Caching

Concurrency
❖ Concurrency can reduce latency
‣ Run multiple stages (or parts of stages) in parallel
‣ Overall latency is reduced!
❖ Concurrency can increase throughput
‣ Each stage works on a di erent request
‣ Pipeline: assembly line
• Think: Factory line!

ff
Pipelining: hiding latency

❖ Pipelines increase throughput by keeping every stage busy
‣ Requires that each stage be handled by di erent resources
‣ Typically, thread/memory per stage which are likely virtualized
❖ Use a bounded bu er between each stage to keep stages busy

ff
ff
Queueing and overload

❖ What about bursts
❖ Pipelining works well if all stages are the same length
‣ Requests move in an orderly fashion
❖ What if some stages are di erent lengths?
‣ Or di erent lengths for di erent requests?
❖ Bounded bu ers queue requests between stages
❖ Problem: what if average throughput for stages is di erent?
‣ Bu ers will ll: more and more requests in each one
‣ Latency will increase: requests spend (idle) time in bu ers
❖ Incoming request rate > average throughput for one stage overload
‣ System can’t handle all of the incoming requests!
ff
ff
fi
ff
ff
ff
ff
ff
➡︎
Dealing with bottlenecks

❖ Batching: handle requests as a group to amortize xed overhead
❖ Dallying: delay a request

‣ Maybe it won’t be needed
‣ Maybe it can be batched
❖ Speculation: perform a request before it’s made

‣ Guess on what the request might be
‣ Increases work, but decreases latency
‣ Might be easier to do request at an earlier time
fi
Measuring performance
❖ Measuring computer system performance requires
‣ Computer system
‣ Workload
❖ Workload is important: di erent workloads perform di erently

‣ Systems are often optimized for a particular type of workload
‣ Measuring on other workloads is misleading (at best)
❖ How do we pick workloads to measure?

❖ Where do these workloads come from?
❖ How do we know the result is accurate?
ff
ff

07performance Cse130 Fa19 KV

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

07performance Cse130 Fa19 KV

Uploaded by

Copyright:

Available Formats

© 2019 Ethan L.

Computer systems performance

❖ How is performance measured?

❖ How can we design systems to improve performance?

CSE 130: Principles of Computer System Design 2

Refresher: SI pre xes

Pre x Pre x Power (base 2) Size (base 10)

Kilo (k) milli (m) 210 / 2-10 103 / 10-3

CSE 130: Principles of Computer System Design 3

❖ Capacity: amount of a resource that’s available (time, memory, etc.)

CSE 130: Principles of Computer System Design 4

CSE 130: Principles of Computer System Design 5

Latency vs. throughput

CSE 130: Principles of Computer System Design 6

❖ Let’s study a modularized system!

CSE 130: Principles of Computer System Design 7

Pipelining and concurrency

CSE 130: Principles of Computer System Design 8

Sometimes the best choice isn’t obvious

CSE 130: Principles of Computer System Design 9

Sometimes the best choice isn’t obvious

Never underestimate the bandwidth of a station

CSE 130: Principles of Computer System Design 10

Latency (execution time) Time for one task

Throughput (bandwidth) Tasks per unit time

CSE 130: Principles of Computer System Design 11

Designing for performance

❖ Build THEN Measure

CSE 130: Principles of Computer System Design 12

Designing for performance

CSE 130: Principles of Computer System Design 13

Bottlenecks in Law and economics

By Lasse Havelund, based on content by Farcaster (Wikipedia)

❖ Eliminate NASA budget, all $18 billion of it

Amdahl’s Law (Law of Diminishing Returns)

CSE 130: Principles of Computer System Design 15

Reducing latency: exploit workload properties

CSE 130: Principles of Computer System Design 16

Fast path example

Latencyfast Frequencyfast + Frequencyslow = 1 (proportion of fast vs. slow)

❖ One very common approach to fast-path: Caching

CSE 130: Principles of Computer System Design 17

CSE 130: Principles of Computer System Design 18

Pipelining: hiding latency

CSE 130: Principles of Computer System Design 19

Queueing and overload

Dealing with bottlenecks

❖ Dallying: delay a request

❖ Speculation: perform a request before it’s made

CSE 130: Principles of Computer System Design 21

❖ Workload is important: di erent workloads perform di erently

❖ How do we pick workloads to measure?

You might also like