Chapter One

Introduction and elementary data structures

contents
✓ Introduction
✓ What is algorithm?
✓ Why do we need algorithm analysis ?
✓ order notation
✓ Analysis of algorithm
✓ Review of elementary data structures heaps and Heap Sort
✓ Hashing
✓ sets representation
UNION, FIND operation.
 Introduction
• data structure is a way of storing data in a computer so that it
can be used efficiently.
• It is the specification of the elements of the structure, the
relationships between them, and the operation that may be
performed upon them.
• Consider as an example books in a library. It can be stored in a
shelf arbitrarily or using some defined orders such as sorted by
Title, Author and so on.
• It can be said that the study of data structures involves the
storage, retrieval and manipulation of information.
• In other words, the possible logical and physical arrangement
of data items to provide an efficient solution for a problem is
called Data Structure.
 Cont..
• A program is written in order to solve a problem.
• A solution to a problem actually consists of two things:
• A way to organize the data
• Sequence of steps to solve the problem
• The way data are organized in a computers memory is said to be Data
Structure and
• the sequence of computational steps to solve a problem is said to be an
algorithm. Therefore,
Program=Algorithm + Data structure
 What is algorithm?

It is a finite set of instruction that specifies a sequence of operation is to be

carried out in order to solve a specific problem or class of problems.
Or
It is a sequence of unambiguous instructions for solving a problem, i.e., for
obtaining a required output for any legitimate input in a finite amount of
time.
 Other definitions of an algorithm
oAn algorithm is a clearly specified set of simple instructions to be followed
to solve a problem
oAny well-defined computational procedure that takes some value (or set of
values) as an input and produces some value (or set of values) as an output
oA sequence of computational steps that transforms the input into the
output
o is a tool for solving a well - specified computational problem.
oA set of well-defined, finite rules used for problem solving
oA finite set of instructions that, if followed, accomplishes a particular task
oIt is a precise, systematic method for producing a specified result.
oan efficient method that can be expressed within finite amount of time
and space.
 Properties of an algorithm
Input:- An algorithm takes zero or more well defined inputs.
Output:- At least one output should be returned by the algorithm after the completion of
the specific task based on the given inputs.
Definiteness:- Every statement of the algorithm should be clear and unambiguous.
Finiteness:- The process should be terminated after a finite number of steps.
E.g: while(1<2){ number=number/2; }
Efficiency:- is defined as the ability to accomplish something with the least amount of
wasted time, money, and effort or competency in performance.
Effectiveness:- is defined as the degree to which something is successful in producing a
desired result; success
Feasibility:− Should be feasible with the available resources.
Independent:−An algorithm should have step-by-step directions, which should be
independent of any programming code.
What makes a “good” algorithm?

• A good algorithm should produce the correct outputs

for any set of legal inputs.
• A good algorithm should execute efficiently with the
fewest number of steps as possible.
• A good algorithm should be designed in such a way
that others will be able to understand it and
modify it to specify solutions to additional problems.
 Algorithm vs Program
• An algorithm consists of simply the steps (machine independent) needed to be
followed in some order to solve a problem.

❑ required at design phase

❑It needs to be analyzed
❑ done by any language
❑ Doesn’t depend on HW and OS used
❑Person should have domain knowledge i.e. done by domain knowledge experts
• A program is an instruction set for a specific type of machine to put an algorithm to
practice.
❑ required at Implementation phase
❑ it needs to be tested
❑ done through programming languages
❑ dependent on the HW and OS we used.
❑Person should be programmer i.e. done by programmer
Why do we need algorithm analysis ?
• writing a working program is not good enough
• If the program is run on a large data set, then the
running time becomes an issue.
• To choose better algorithm for solving a particular task
or problem.
• Provide an insight into reasonable directions of search
for efficient algorithms.
• Predict the behavior of an algorithm without
implementing it on machine.
 Algorithm Analysis Concepts
• Algorithm analysis is the study of effectiveness of algorithms with respect to time
taken for completion and space used up. This is done with the help of asymptotic
notations. There are three major asymptotic notations that is used to denote the time
and space complexities of algorithms. These are Big Oh, Omega and Theta notations.
• Complexity analysis is concerned with determining the efficiency of algorithms.
• What is Efficiency depends on?
• Execution speed (most important)
• Amount of memory used
• Efficiency differences may not be noticeable for small data, but become
very important for large amounts of data.
 Algorithm's Performance
• Two important ways to characterize the effectiveness of an algorithm are its space
complexity and time complexity.
• Time complexity determining the amount of time required to accomplish a given task.
• The space complexity determining the amount of memory space it needs from run to
completion.
• It will be difficult for us to determine the run time of a program exactly. Because the run
time may depend on:
• Processor speed
• Current processor load
• Input size of the given algorithm
• Available memory, etc.
So, it will be difficult for us to obtain exact running time of the program.
Con.t..
There are different types of time complexities which can be analysed
for an algorithm:
Best Case Time Complexity:- It is measure of minimum time that algorithm
will require for input of size “n‟.
Worst Case Time Complexity: It is measure of maximum time that
algorithm will require for input of size “n‟.
Average Case Time Complexity: the average number of operations
used to solve the problem for input size “n”.
 How do we estimate the complexity of algorithms?

1. Algorithm analysis – Find f(n)- helps to determine the

complexity of an algorithm
2. Order of Magnitude – g(n) belongs to O(f(n)) – helps to
determine the category of the complexity to which it
belongs.
Common time complexities/ Common Growth Rates

BETTER • O(1) constant time

• O(log n) log time
• O(n) linear time
• O(n log n) log linear time
• O(n2) quadratic time
• O(n3) cubic time
• O(2n) exponential time
WORSE
15
 Running Times for Small Inputs
Running time

Input size (x = n) 16
 Running Times for Large Inputs
Running time

Input size (x = n) 17
Asymptotic notation/ Growth of function
- It is a way to describe the characteristics of a function in the limit.
- It describes the rate of growth of functions.
- Focus on what’s important by abstracting away low-order terms and
constant factors.
- It is a way to compare “sizes” of functions
There are five notations used to describe a running time function. These are:
✓ Big-Oh Notation (O) >= asymptotic upper bound for f(n)
✓ Big-Omega Notation (Ω )<= asymptotic lower bound for f(n)
✓ Theta Notation (𝚹) = can be big-O or Big- Ω -> asymptotic tight
bound for f(n)
✓ Little-o Notation (o) >
✓ Little-Omega Notation () <
Cont…
Big Oh notation
A function f(n) is said to be O(g(n)) if there exist some constant
c0>0 and n0 ≥ 0 such that f(n) ≤ c0.g(n) for all n ≥ n0
Big Omega notation
A function f(n) is said to be Ω(g(n)) if there exist some constant
c0>0 and n0 ≥ 0 such that f(n)≥c0.g(n) for all n ≥ n0
Big Theta Notation
A function f(n) is said to be Ω(g(n)) if there exist some constant
C0 ,C1 >0 and n0 ≥ 0 such that c0.g(n) ≤ f(n)≤c1.g(n) for all n ≥ n0
Analysis Rule:
• 1. Assume an arbitrary time unit
• 2. Execution of one of the following operations takes time unit 1
– Assignment statement
– Single I/O statement;.
– Single Boolean statement.
– Single arithmetic.
– Function return.
• 3. selection statement
– Time for condition evaluation + the maximum time of from all cases
• 4. Loop statement
– Sum of(initialization time + time of condition checking+ time of action performed)+no of loop
iteration *(body of loop statement execution times)
• 5. For function call
– 1+ time(parameters) + body time
 Eg1.
• int sum (int n)
{
int partial_sum = 0;
for (int i = 1; i <= n; i++)
partial_sum = partial_sum +(i * i * i);
return partial_sum;
}
Time Units to Compute
-------------------------------------------------
1 for the assignment.
1 assignment, n+1 tests, and n increments.
n loops of 4 units for an assignment, an addition, and two multiplications.
1 for the return statement.
-------------------------------------------------------------------
T (n)= 1+(1+n+1+n)+4n+1 = 6n+4 = O(n)
 Eg2.
for (int i = 1; i <=n; i++) 1 + (n+1) + n
{
for (int j = 1; j <=n; j++) n*(1 + (n+1) +1)
{
cout<<“hello world”; 1
}}

T (n)= 1+(n+1)+n+n*(1 + (n+1) +1)= 2+2n+(n+n2+n+n)= n2 +5n+2

=O(n2)
Eg.3
int a = 0;
for (i = 0; i < N; i++) {
for (j = N; j > i; j--) {
a = a + i + j;
}
}
Explanation:
The above code runs total no of times
= N + (N – 1) + (N – 2) + … 1 + 0
= N * (N + 1) / 2
= 1/2 * N^2 + 1/2 * N
O(N^2) times.
 eg4
int i, j, k = 0;
for (i = n / 2; i <= n; i++) {
for (j = 2; j <= n; j = j * 2) {
k = k + n / 2;
}}
Explanation: If you notice, j keeps doubling till it is less than or equal to n.
Several times, we can double a number till it is less than n would be
log(n).
Let’s take the examples here.
for n = 16, j = 2, 4, 8, 16
for n = 32, j = 2, 4, 8, 16, 32
So, j would run for O(log n) steps.
i runs for n/2 steps.
So, total steps = O(n/ 2 * log (n)) = O(n*logn)
 Heaps and Heap Sort
• A Heap is a special Tree-based data structure in which the
tree is a complete binary tree. Generally, Heaps can be of
two types:
1.Max-Heap: In a Max-Heap the key present at the root node
must be greatest among the keys present at all of it’s
children. The same property must be recursively true for all
sub-trees in that Binary Tree.
2.Min-Heap: In a Min-Heap the key present at the root node
must be minimum among the keys present at all of it’s
children. The same property must be recursively true for all
sub-trees in that Binary Tree.
Application:
•Heap sort algorithm
•Graph algorithms like prism’s minimal
spanning algorithms and Dijkstra’s
shortest path algorithms
•A priority queue can be implemented
with a heap or variety of other methods.
Eg. Given Array
Build Heap Tree
 What is Hashing?
•Hashing is the process of mapping large amount of data item to smaller
table with the help of hashing function.
•Hashing is also known as Hashing Algorithm or Message Digest
Function.
•It is a technique to convert a range of key values into a range of indexes
of an array.
•It is used to facilitate the next level searching method when compared with
the linear or binary search.
•Hashing allows to update and retrieve any data entry in a constant time
O(1).
•Constant time O(1) means the operation does not depend on the size of
the data.
•Hashing is used with a database to enable items to be retrieved more
quickly.
•It is used in the encryption and decryption of digital signatures.
 What is Hash Function?
•A fixed process converts a key to a hash key is known as a Hash Function.
•This function takes a key and maps it to a value of a certain length which is
called a Hash value or Hash.
•Hash value represents the original string of characters, but it is normally smaller
than the original.
•It transfers the digital signature and then both hash value and signature are
sent to the receiver.
•Receiver uses the same hash function to generate the hash value and then
compares it to that received with the message.
•If the hash values are same, the message is transmitted without errors.
 What is Hash Table?
• Hash table or hash map is a data structure used to store key-
value pairs.
•It is a collection of items stored to make it easy to find them later.
• It uses a hash function to compute an index into an array of
buckets or slots from which the desired value can be found.
•It is an array of list where each list is known as bucket.
•It contains value based on the key.
•Hash table is used to implement the map interface and extends
Dictionary class.
•Hash table is synchronized and contains only unique elements.