Operating Systems

CPU scheduling
 CPU Scheduler
• A CPU scheduler, running in the dispatcher, is
responsible for selecting of the next running process.
– Based on a particular strategy
• When does CPU scheduling happen?
– Four cases:
• A process switches from the running state to waiting state
(e.g. I/O request)
• A process switches from the running state to the ready state.
• A process switches from waiting state to ready state
(completion of an I/O operation)
• A process terminates
 Scheduling Assumptions
• Many implicit assumptions for CPU scheduling:
– One program per user
– One thread per program
– Programs are independent
• Clearly, these are unrealistic but they simplify the problem
so it can be solved
– For instance: is “fair” about fairness among users or programs?
• If I run one compilation job and you run five, you get five times as much
CPU on many operating systems
• The high-level goal: Dole out CPU time to optimize some
desired parameters of system

USER1 USER2 USER3 USER1 USER2

Time
 Scheduling Policy Goals/Criteria
• Minimize Response Time
– Minimize elapsed time to do an operation (or job)
– Response time is what the user sees:
• Time to echo a keystroke in editor
• Time to compile a program
• Real-time Tasks: Must meet deadlines imposed by World
• Maximize Throughput
– Maximize operations (or jobs) per second
– Throughput related to response time, but not identical:
• Minimizing response time will lead to more context switching than if you
only maximized throughput
– Two parts to maximizing throughput
• Minimize overhead (for example, context-switching)
• Efficient use of resources (CPU, disk, memory, etc)
• Fairness
– Share CPU among users in some equitable way
– Fairness is not minimizing average response time:
• Better average response time by making system less fair
 Performance metrics for CPU
scheduling
• CPU utilization: percentage of the time that CPU
is busy.
• +Throughput: the number of processes
completed per unit time
• Turnaround time: the interval from the time of
submission of a process to the time of completion.
• Wait time: the sum of the periods spent waiting in
the ready queue
• Response time: the time of submission to the
time the first response is produced
 Assumption: CPU Bursts
Weighted toward small bursts

• Execution model: programs alternate between bursts of CPU and I/O

– Program typically uses the CPU for some period of time, then does I/O, then
uses CPU again
– Each scheduling decision is about which job to give to the CPU for use by its
next CPU burst
– With timeslicing, thread may be forced to give up CPU before finishing
current CPU burst
 Methods for evaluating CPU
scheduling algorithms
• Simulation:
– Get the workload from a system
– Simulate the scheduling algorithm
– Compute the performance metrics based on simulation
results

– This is practically the best evaluation method.

• Queueing models:
– Analytically model the queue behavior (under some
assumptions).
– A lot of math, but may not be very accurate because of
unrealistic assumptions.
 Deterministic modeling
example:
• Suppose we have processes A, B, and
C, submitted at time 0
• We want to know the response time,
waiting time, and turnaround time of
process A
turnaround time
wait time + +
response time = 0
A B C A B C A C A C Time

Gantt chart: visualize how processes execute.

Deterministic modeling example

• Suppose we have processes A, B, and

C, submitted at time 0
• We want to know the response time,
waiting time, and turnaround time of
process
turnaround timeB
wait time +
response time
A B C A B C A C A C Time
Deterministic modeling example

• Suppose we have processes A, B, and

C, submitted at time 0
• We want to know the response time,
waiting time, and turnaround time of
process
turnaround timeC
wait time + + +
response time
A B C A B C A C A C Time
 Preemptive versus
nonpreemptive scheduling
• Many CPU scheduling algorithms have both preemptive
and nonpreemptive versions:
– Preemptive: schedule a new process even when the current
process does not intend to give up the CPU
– Non-preemptive: only schedule a new process when the current
one does not want CPU any more.
• When do we perform non-preemptive scheduling?
• A process switches from the running state to waiting state
(e.g. I/O request)
• A process switches from the running state to the ready state.
• A process switches from waiting state to ready state
(completion of an I/O operation)
• A process terminates
 Scheduling Policies

• FIFO (first in, first out)

• Round robin
• SJF (shortest job first)
• Priority Scheduling
• Multilevel feedback queues

• This is obviously an incomplete list

 FIFO/FCFS

• FIFO: assigns the CPU based on the

order of requests
– Nonpreemptive: A process keeps running on
a CPU until it is blocked or terminated
– Also known as FCFS (first come, first served)
+ Simple
- Short jobs can get stuck behind long jobs
- Turnaround time is not ideal
 First-Come, First-Served (FCFS)
• First-Come, First-Served Scheduling
(FCFS)
– Also “First In, First Out” (FIFO) or “Run until done”
• In early systems, FCFS meant one program
scheduled until done (including I/O)
• Now, means keep CPU until thread blocks
• Example: Process Burst Time
P1 24
P2 3
P3 3
– Suppose processes arrive in the order: P1 , P2 , P3
The Gantt Chart for the schedule is:

P1 P2 P3

– Waiting time for P1 = 0; P2 = 24; P3 = 27

– Average0waiting time: (0 + 24 + 27)/324= 17 27 30
– Average Completion time: (24 + 27 + 30)/3 = 27
• Convoy effect: short process behind long process
 FCFS
• Example continued: Scheduling (Cont.)
– Suppose that processes arrive in order: P , P , P
2 3 1
Now, the Gantt chart for the schedule is:
P2 P3 P1

0 3 6 30
– Waiting time for P1 = 6; P2 = 0; P3 = 3
– Average waiting time: (6 + 0 + 3)/3 = 3
– Average Completion time: (3 + 6 + 30)/3 = 13
• In second case:
– average waiting time is much better (before it was 17)
– Average completion time is better (before it was 27)
• FIFO Pros and Cons:
– Simple (+)
– Short jobs get stuck behind long ones (-)
 Round Robin

• Round Robin (RR) periodically releases

the CPU from long-running jobs
– Based on timer interrupts so short jobs can get
a fair share of CPU time
– Preemptive: a process can be forced to leave
its running state and replaced by another
running process
– Time slice: interval between timer interrupts
 Round Robin (RR)
• FCFS Scheme: Potentially bad for short jobs!
– Depends on submit order
– If you are first in line at supermarket with milk, you don’t care who is
behind you, on the other hand…
• Round Robin Scheme
– Each process gets a small unit of CPU time
(time quantum), usually 10-100 milliseconds
– After quantum expires, the process is preempted
and added to the end of the ready queue.
– n processes in ready queue and time quantum is q 
• Each process gets 1/n of the CPU time
• In chunks of at most q time units
• No process waits more than (n-1)q time units
• Performance
– q large  FCFS
– q small  Interleaved (really small  hyperthreading?)
– q must be large with respect to context switch, otherwise overhead
is too high (all overhead)
 Example of RR with Time
• Example: Quantum = 20 Process Burst Time
P1 53
P2 8
P3 68
P4 24
– The Gantt chart is:

P1 P2 P3 P4 P1 P3 P4 P1 P3 P3

0 20 28 48 68 88 108 112 125 145 153

– Waiting time for P1=(68-20)+(112-88)=72
P2=(20-0)=20
P3=(28-0)+(88-48)+(125-108)=85
P4=(48-0)+(108-68)=88
– Average waiting time = (72+20+85+88)/4=66¼
– Average completion time = (125+28+153+112)/4 = 104½
• Thus, Round-Robin Pros and Cons:
– Better for short jobs, Fair (+)
– Context-switching time adds up for long jobs (-)
 Round-Robin Discussion
• How do you choose time slice?
– What if too big?
• Response time suffers
– What if infinite ()?
• Get back FIFO
– What if time slice too small?
• Throughput suffers!
• Actual choices of timeslice:
– Initially, UNIX timeslice one second:
• Worked ok when UNIX was used by one or two people.
• What if three compilations going on? 3 seconds to echo each
keystroke!
– In practice, need to balance short-job performance and long-job
throughput:
• Typical time slice today is between 10ms – 100ms
• Typical context-switching overhead is 0.1ms – 1ms
• Roughly 1% overhead due to context-switching
 Comparisons between
• FCFS
Assuming and
zero-cost Roundtime,
context-switching Robin
is RR always
better than FCFS?
• Simple example: 10 jobs, each take 100s of CPU time
RR scheduler quantum of 1s
All jobs start at the same time
• Completion Times: Job # FIFO RR
1 100 991
2 200 992
… … …
9 900 999
10 1000 1000
– Both RR and FCFS finish at the same time
– Average response time is much worse under RR!
• Bad when all jobs same length
• Also: Cache state must be shared between all jobs with RR
but can be devoted to each job with FIFO
– Total time for RR longer even for zero-cost switch!
 Earlier Example with Different
P Time
P Quantum
P 2 P 4 1 3
Best FCFS: [8] [24] [53] [68]
0 8 32 85 153
Quantum P1 P2 P3 P4 Average
Best FCFS 32 0 85 8 31¼
Q = 1 84 22 85 57 62
Q = 5 82 20 85 58 61¼
Wait
Q = 8 80 8 85 56 57¼
Time
Q = 10 82 10 85 68 61¼
Q = 20 72 20 85 88 66¼
Worst FCFS 68 145 0 121 83½
Best FCFS 85 8 153 32 69½
Q = 1 137 30 153 81 100½
Q = 5 135 28 153 82 99½
Completion
Q = 8 133 16 153 80 95½
Time
Q = 10 135 18 153 92 99½
Q = 20 125 28 153 112 104½
Worst FCFS 121 153 68 145 121¾
 More on Round Robin

• If time slice is too long

– Scheduling relegates to FIFO
• If time slice is too short
– Throughput suffers
– Context switching cost dominates
 FIFO vs. Round Robin
• Suppose we have three jobs of equal
length
turnaround time of C
turnaround time of B
turnaround time of A
A B C A B C A B C Time
Round Robin

turnaround time of C
turnaround time of B
turnaround time of A
A B C Time
FIFO
 FIFO vs. Round Robin

• Round Robin
+ Shorter response time
+ Fair sharing of CPU
- Not all jobs are preemptable
- Not good for jobs of the same length
- More precisely, not good in terms of the turnaround
time.
 What if we Knew the Future?
• Could we always mirror best FCFS?
• Shortest Job First (SJF):
– Run whatever job has the least amount of
computation to do
– Sometimes called “Shortest Time to
Completion First” (STCF)
• Shortest Remaining Time First (SRTF):
– Preemptive version of SJF: if job arrives and has a shorter time to
completion than the remaining time on the current job, immediately
preempt CPU
– Sometimes called “Shortest Remaining Time to Completion First”
(SRTCF)
• These can be applied either to a whole program or the
current CPU burst of each program
– Idea is to get short jobs out of the system
– Big effect on short jobs, only small effect on long ones
– Result is better average response time
 Discussion
• SJF/SRTF are the best you can do at minimizing
average response time
– Provably optimal (SJF among non-preemptive, SRTF
among preemptive)
– Since SRTF is always at least as good as SJF, focus on
SRTF
• Comparison of SRTF with FCFS and RR
– What if all jobs the same length?
• SRTF becomes the same as FCFS (i.e. FCFS is best can do if
all jobs the same length)
– What if jobs have varying length?
• SRTF (and RR): short jobs not stuck behind long ones
 Example to illustrate benefits of
A or B
SRTF C

C’s C’s C’s

I/O I/O I/O
• Three jobs:
– A,B: both CPU bound, run for week
C: I/O bound, loop 1ms CPU, 9ms disk I/O
– If only one at a time, C uses 90% of the disk, A or B could use
100% of the CPU
• With FIFO:
– Once A or B get in, keep CPU for two weeks
• What about RR or SRTF?
– Easier to see with a timeline
C
SRTF
A
Example continued:
B
Disk Utilization:
9/201
C ~ 4.5%

C’s RR 100ms time slice Disk

C’sUtilization:
I/O ~90%
I/O but lots
CABAB… C
of wakeups!

RR 1ms time slice

C’s C’s
I/O I/O
Disk Utilization:
C A A A 90%

SRTF
C’s C’s
I/O I/O
 Predicting the Length of the Next
CPU Burst
• Adaptive: Changing policy based on past behavior
– CPU scheduling, in virtual memory, in file systems, etc
– Works because programs have predictable behavior
• If program was I/O bound in past, likely in future
• If computer behavior were random, wouldn’t help
• Example: SRTF with estimated burst length
– Use an estimator function on previous bursts:
Let tn-1, tn-2, tn-3, etc. be previous CPU burst lengths. Estimate
next burst n = f(tn-1, tn-2, tn-3, …)
– Function f could be one of many different time series
estimation schemes (Kalman filters, etc)
– For instance,
exponential averaging
n = tn-1+(1-)n-1
with (0<1)
 SRTF Further discussion
• Starvation
– SRTF can lead to starvation if many small jobs!
– Large jobs never get to run
• Somehow need to predict future
– How can we do this?
– Some systems ask the user
• When you submit a job, have to say how long it will take
• To stop cheating, system kills job if takes too long
– But: Even non-malicious users have trouble predicting runtime of
their jobs
• Bottom line, can’t really know how long job will take
– However, can use SRTF as a yardstick
for measuring other policies
– Optimal, so can’t do any better
• SRTF Pros & Cons
– Optimal (average response time) (+)
– Hard to predict future (-)
– Unfair (-)
 Shortest Job First (SJF)
• SJF runs whatever job puts the least demand on
the CPU, also known as STCF (shortest time to
completion first)
+ Provably optimal in terms of turn-around time (anyone
can give an informal proof?).
+ Great for short jobs
+ Small degradation for long jobs
• Real life example: supermarket express
checkouts
 SJF Illustrated
turnaround time of C
turnaround time of B
turnaround time of A
wait time of C
wait time of B
wait time of A = 0
response time of C
response time of B
response time of A = 0

A B C Time
Shortest Job First
 SJF and SRTF vs. FIFO and
Round Robin
• If all jobs are the same length, SJF 
FIFO
– FIFO is the best you can do
• If jobs have varying length
– Short jobs do not get stuck behind long jobs
under SRTF
Drawbacks of Shortest Job First

- Starvation: constant arrivals of short jobs

can keep long ones from running
- There is no way to know the completion
time of jobs (most of the time)
– Some solutions
• Ask the user, who may not know any better
• If a user cheats, the job is killed
 Priority Scheduling (Multilevel
Queues)
• Priority scheduling: The process with the
highest priority runs first
• Priority 0: C

• Priority 1: A
• Priority 2: B
• Assume that low numbers represent high
priority C A B Time
Priority Scheduling
 Priority Scheduling

+ Generalization of SJF
– With SJF, priority = 1/requested_CPU_time
- Starvation
 Multilevel Feedback Queues

• Multilevel feedback queues use multiple

queues with different priorities
– Round robin at each priority level
– Run highest priority jobs first
– Once those finish, run next highest priority, etc
– Jobs start in the highest priority queue
– If time slice expires, drop the job by one level
– If time slice does not expire, push the job up by
one level
 Multilevel Feedback Queues
time = 0

• Priority 0 (time slice = 1): A B C

• Priority 1 (time slice = 2):

• Priority 2 (time slice = 4):

Time
 Multilevel Feedback Queues
time = 1

• Priority 0 (time slice = 1): B C

• Priority 1 (time slice = 2): A

• Priority 2 (time slice = 4):

A Time
 Multilevel Feedback Queues
time = 2

• Priority 0 (time slice = 1): C

• Priority 1 (time slice = 2): A B

• Priority 2 (time slice = 4):

A B Time
 Multilevel Feedback Queues
time = 3

• Priority 0 (time slice = 1):

• Priority 1 (time slice = 2): A B C

• Priority 2 (time slice = 4):

A B C Time
 Multilevel Feedback Queues
time = 3

• Priority 0 (time slice = 1):

• Priority 1 (time slice = 2): A B C

• Priority 2 (time slice = 4):

suppose process A is blocked on an I/O

A B C Time
 Multilevel Feedback Queues
time = 3

• Priority 0 (time slice = 1): A

• Priority 1 (time slice = 2): B C

• Priority 2 (time slice = 4):

suppose process A is blocked on an I/O

A B C Time
 Multilevel Feedback Queues
time = 5

• Priority 0 (time slice = 1): A

• Priority 1 (time slice = 2): C

• Priority 2 (time slice = 4):

suppose process A is returned from an I/O

A B C B Time
 Multilevel Feedback Queues
time = 6

• Priority 0 (time slice = 1):

• Priority 1 (time slice = 2): C

• Priority 2 (time slice = 4):

A B C B A Time
 Multilevel Feedback Queues
time = 8

• Priority 0 (time slice = 1):

• Priority 1 (time slice = 2):
• Priority 2 (time slice = 4): C

A B C B A C Time
 Multilevel Feedback Queues
time = 9

• Priority 0 (time slice = 1):

• Priority 1 (time slice = 2):
• Priority 2 (time slice = 4):

A B C B A C C Time
 Multilevel Feedback Queues

• Approximates SRTF
– A CPU-bound job drops down
– I/O-bound jobs stay near the top
– Still unfair for long running jobs
– Counter-measure: Aging
• Increase the priority of long running jobs if they are
not serviced for a period of time
• Tricky to tune aging
 Case study: Windows
• A priority-based, preemptive scheduling
• Highest priority thread will always run
• Also have multiple classes and priorities within classes
• Similar idea for user processes – Multilevel feedback
queue
• Lower priority when quantum runs out
• Increase priority after a wait event
• Some twists to improve “user perceived” performance:
• Boost priority and quantum for foreground process (the window that
is currently selected).
• Boost priority more for a wait on keyboard I/O (as compared to disk
I/O)
 Linux scheduling
• A priority-based, preemptive with global
round-robin scheduling
– Each process have a priority
– Processes with a larger priority also have a larger time slices
– Before the time slices is used up, processes are scheduled
based on priority.
– After the time slice of a process is used up, the process must
wait until all ready processes to use up their time slice (or be
blocked) – a round-robin approach.
• No starvation problem.
– For a user process, its priority may + or – 5 depending whether
the process is I/O- bound or CPU-bound.
• Giving I/O bound process higher priority.
 Summary for the case study
• Basic idea for schedule user processes is the
same for all systems:
– Lower priority for CPU bound process
– Increase priority for I/O bound process
• The scheduling in Solaris / Linux is more
concerned about fairness.
– More popular as the OSes for servers.
• The scheduling in Window is more concerned
about user perceived performance.
– More popular as the OS for personal computers.