COMPUTER SYSTEM PERFORMANCE

ANAYLYSIS
GENERAL FULL FACTORIAL
DESIGNS WITH k FACTORS

1
Analysis of General Design
Informal Methods
 Observation Method
 Ranking Method
 Range Method

2
Total Number of Experiments – (Levels)Factors

(3)4 = 81

3
TYPES OF INTERACTIONS
TWO FACTOR 4C
2 6

THREE FACTOR 4C
3 4

FOUR FACTOR 4C
4 1

4
ANOVA
Table
For
Paging
Study

5
Component Sum of Percentage Degrees of Mean
squares of variation freedom square

Y 730.01 81

Y 681.21 1

y-y.... 48.80 100 80

6
 MAIN EFFECTS

Component Sum of Percentage Degrees of Mean

squares of variation freedom square

(TOTAL) 45.80 93.85 8 5.7

Main Effects

A 1.30 2

D 6.10 2

P 12.30 2

M 26.20 2

7
 FIRST ORDER INTERACTIONS
Component Sum of Percentage of Degrees of Mean square
squares variation freedom
TOTAL 2.40 4.91 24 0.1
(FIRST ORDER
INTERACTIONS)
AD 0.07 4
AP 0.02 4
AM 0.03 4
DP 0.15 4
DM 1.96 4
PM 0.14 4

8
 SECOND ORDER INTERACTIONS
Component Sum of Percentage Degrees of Mean
squares of variation freedom square

TOTAL 0.48 0.98 32 0.015

(SECOND ORDER
INTERACTIONS)

ADP 0.05 8

ADM 0.13 8

APM 0.04 8

DPM 0.26 8

9
 THIRD ORDER INTERACTIONS

Component Sum of Percentage Degrees of Mean

squares of variation freedom square

ADPM 0.07 0.14 16 0.004

10
Simplified Model to represent the paging process

yijkl=µ+αi+βj+γk+δl+ξjl
µ Grand Mean

αi Main Effect of A

βj Main Effect of D

γk Main Effect of P

δl Main Effect of M

ξjl Interaction Between

Factors D & M

11
ERRORS IN THE PAGING STUDY

12
 Normal
Quatile-Quantile
Plot
Of Residuals

13
14
15
 INFORMAL METHODS

Analysis Techniques presented so far produce

 Main effects
 Interactions
 Confidence Intervals

Goal : “Find the best combination of factor levels that produces

best performance”
Too much simpler techniques
 Results can be easily explained to decision makers with out
statistical details.

 Preliminary Analysis before ANOVA

16
OBSERVATION METHOD
RANKING METHOD
RANGE METHOD

17
OBSERVATION METHOD

If the Response variable – HB(Higher the Better) or

LB(Lower the Better) Metric
and Goal – Find the combination of factor levels that
produces best response

Simple look at Mean response Column

Experiments corresponding to Highest Response or Lowest

Response is found – Its levels give Desired Combination

Other Responses – Close to the Best Performance

Factor levels common to all such responses – DESIRED ANSWER

18
Example: Scheduler Design Study

Goal: Find Best Factor Levels that

Maximize the Throughput

3 Classes of jobs:
 Word Processing
 Interactive Data Processing
 Background Data Processing

16 experiments – 25-1 design

19
SYMBOL FACTOR LEVEL -1 LEVEL 1

A Preemption NO YES

B Time Slice SMALL LARGE

C Queue Assignment 1 QUEUE 2 QUEUES

D Requeueing 2 QUEUES 5 QUEUES

E Fairness OFF ON

20
21
SYMBOL FACTOR LEVEL -1 LEVEL 1

A Preemption NO YES

B Time Slice SMALL LARGE

C Queue Assignment 1 QUEUE 2 QUEUES

D Requeueing 2 QUEUES 5 QUEUES

E Fairness OFF ON

22
23
SYMBOL FACTOR LEVEL -1 LEVEL 1

A Preemption NO YES

B Time Slice SMALL LARGE

C Queue Assignment 1 QUEUE 2 QUEUES

D Requeueing 2 QUEUES 5 QUEUES

E Fairness OFF ON

24
25
SYMBOL FACTOR LEVEL -1 LEVEL 1

A Preemption NO YES

B Time Slice SMALL LARGE

C Queue Assignment 1 QUEUE 2 QUEUES

D Requeueing 2 QUEUES 5 QUEUES

E Fairness OFF ON

26
 RANKING METHOD 
Similar to Observation method

Experiments are written in order of increasing or decreasing

responses –
 Experiment with BEST response FIRST
 Experiment with WORST response LAST

Observe Factor columns – Find levels that consistently

produce good or bad results

More Information than Observation Method

27
28
29
SYMBOL FACTOR LEVEL -1 LEVEL 1

A Preemption NO YES

B Time Slice SMALL LARGE

C Queue Assignment 1 QUEUE 2 QUEUES

D Requeueing 2 QUEUES 5 QUEUES

E Fairness OFF ON

30
No A B C D E Tw Ti Tb
9 -1 -1 -1 1 -1 66.1
5 -1 -1 1 -1 -1 63.9
2 1 -1 -1 -1 -1 41.0
14 1 -1 1 1 -1 38.0
7 -1 1 1 -1 1 36.3
3 -1 1 -1 -1 -1 36.0
15 -1 1 1 1 -1 35.0
11 -1 1 -1 1 1 34.6
13 -1 -1 1 1 1 26.0
1 -1 -1 -1 -1 1 25.0
8 1 1 1 -1 -1 23.0
12 1 1 -1 1 -1 23.0
16 1 1 1 1 1 22.0
4 1 1 -1 -1 1 15.7
6 1 -1 1 -1 1 13.2
10 1 -1 -1 1 1 9.2
31
SYMBOL FACTOR LEVEL -1 LEVEL 1

A Preemption NO YES

B Time Slice SMALL LARGE

C Queue Assignment 1 QUEUE 2 QUEUES

D Requeueing 2 QUEUES 5 QUEUES

E Fairness OFF ON

32
No A B C D E Tw Ti Tb
3 -1 1 -1 -1 -1 21.0
7 -1 1 1 -1 1 20.2
15 -1 1 1 1 -1 17.2
11 -1 1 -1 1 1 15.7
1 -1 -1 -1 -1 1 15.2
13 -1 -1 1 1 1 12.0
4 1 1 -1 -1 1 8.6
10 1 -1 -1 1 1 8.4
5 -1 -1 1 -1 -1 7.5
6 1 -1 1 -1 1 7.5
9 -1 -1 -1 1 -1 6.4
2 1 -1 -1 -1 -1 3.0
8 1 1 1 -1 -1 3.0
12 1 1 -1 -1 1 3.0
14 1 -1 1 1 -1 2.0
16 1 1 1 1 1 2.0
33
SYMBOL FACTOR LEVEL -1 LEVEL 1

A Preemption NO YES

B Time Slice SMALL LARGE

C Queue Assignment 1 QUEUE 2 QUEUES

D Requeueing 2 QUEUES 5 QUEUES

E Fairness OFF ON

34
RANGE METHOD

Importance of a factor in a model -> Find percentage of variation

explained by that factor

Informal alternative:
Find the average response
Find the difference between max & min -> RANGE

Factor with large range -> IMPORTANT

Example: Data of paging study

Calculate averages corresponding to three levels of each

of the four factors

35
 LEVEL
FACTOR
1 2 3

Replacement Algorithm 2056 2986 3781

Deck Arrangement 1584 2913 4326

Problem Program 592 2047 6185

Memory size 305 2006 6512

36
55523 27 = 2056
37
8233 27 = 305
38
 LEVEL Range of
FACTOR averages
1 2 3

Replacement Algorithm 2056 2986 3781 1725

Deck Arrangement 1584 2913 4326 2742
Problem Program 592 2047 6185 5593
Memory size 305 2006 6512 6207

****MEMORY SIZE – Most influential factor

***PROBLEM PROGRAM
**DECK ARRANGEMENT
*ALGORITHM

39
 The End
and
Thank You
40