Amdahl's Law

Amdahl's Law is a law governing the speedup of using parallel processors on a

problem, versus using only one serial processor.Amdahl's law, also known as
Amdahl's argument,[1] is named after computer architect Gene Amdahl, and is used
to find the maximum expected improvement to an overall system when only part of
the system is improved. It is often used in parallel computing to predict the
theoretical maximum speedup using multiple processors.

Amdahl's law is a model for the relationship between the expected speedup of
parallelized implementations of an algorithm relative to the serial algorithm, under
the assumption that the problem size remains the same when parallelized.

For example, if a program needs 20 hours using a single processor core, and a
particular portion of 1 hour cannot be parallelized, while the remaining promising
portion of 19 hours (95%) can be parallelized, then regardless of how many
processors we devote to a parallelized execution of this program, the minimal
execution time cannot be less than that critical 1 hour. Hence the speed up is
limited up to 20x, as the diagram illustrates.
The speedup of a program using multiple processors in parallel computing is limited
by the sequential fraction of the program. For example, if 95% of the program can
be parallelized, the theoretical maximum speedup using parallel computing would
be 20x as shown in the diagram, no matter how many processors are used.

The speedup of a program using multiple processors in parallel computing is limited

by the time needed for the sequential fraction of the program.

The speed of a program is the time it takes the program to excecute. This could be
measured in any increment of time. Speedup is defined as the time it takes a
program to execute in serial (with one processor) divided by the time it takes to
execute in parallel (with many processors). The formula for speedup is:
 T(1)

S = -------------

T(j)

Where T(j) is the time it takes to execute the program when using j processors.
Efficiency is the speedup, divided by the number of processors used.

For example:-

If there are N workers working on a project, we may assume that they would be
able to do a job in 1/N time of one worker working alone. Now, if we assume the
strictly serial part of the program is performed in B*T(1) time, then the strictly
parallel part is performed in ((1-B)*T(1)) / N time. With some substitution and
number manipulation, we get the formula for speedup as:

S = -----------------------

(B*N)+(1-B)

This formula is known as Amdahl's Law.

7.2 Handler's Classification

In 1977 Handler proposed an elaborate notation for expressing the pipelining and
parallelism of computers. Handler's taxonomy[2] addresses the computer at three
distinct levels: the processor control unit (PCU), the arithmetic logic unit (ALU), and
the bit-level circuit (BLC). The PCU corresponds to a processor or CPU, the ALU
corresponds to a functional unit or a processing element in an array processor, and
the BLC corresponds to a the logic needed to perform one-bit operations in the ALU.

Handler's taxonomy uses three pairs of integers to describe a computer:

 Computer = (k * k', d * d', w * w')

Where k = number of PCUs

Where k'= number of PCUs that can be pipelined

Where d = number of ALUs controlled by each PCU

Where d'= number of ALUs that can be pipelined

Where w = number of bits in ALU or processing element (PE) word

Where w'= number of pipeline segments on all ALUs or in a single PE

The following rules and operators are used to show the relationship between various
elements of the computer. The '*' operator is used to indicate that the units are
pipelined or macro-pipelined with a stream of data running through all the units.
The '+' operator is used to indicate that the units are not pipelined but work on
independent streams of data. The 'v' operator is used to indicate that the computer
hardware can work in one of several modes. The '~' symbol is used to indicate a
range of values for any one of the parameters. Peripheral processors are shown
before the main processor using another three pairs of integers. If the value of the
second element of any pair is 1, it may omitted for brevity. Handler's taxonomy is
best explained by showing how the rules and operators are used to classify several
machines.