Introduction

Outline of syllabus

• Introduction to data structures & algorithms –

Unit I
• Efficiency of Algorithms – Unit I
• Sorting & Searching – Unit III
• Lists – Arrays and linked lists– Unit I
• Stacks & Queues – Unit II
• Trees – Unit V
• Graphs – Unit IV
 Learning Resources
• Text Books
 Aaron M. Tenenbaum, Yedidyah Langsam and Moshe J.
Augenstein, “Data Structures Using C and C++”, PHI
 Horowitz and Sahani, “Fundamentals of Data Structures”,
Galgotia Publications Pvt Ltd Delhi India.
 Lipschutz, “Data Structures” Schaum’s Outline Series, Tata
McGraw-hill Education (India) Pvt. Ltd
 R. Kruse etal, “Data Structures and Program Design in C”,
Pearson Education.
 Learning Resources

• MOOCS and web resources

 https://www.coursera.org/specializations/dat
a-structures-algorithms
 https://www.coursera.org/learn/data-
structures#syllabus
 https://nptel.ac.in/courses/106102064/
 http://www.learnandteach.yolasite.com/lectu
res.php
 Contents

• Data Structures and their classification

• Abstract data type (ADT)
• Algorithms
• Efficiency of an algorithm
• Time and space complexity
• Asymptotic notations
 What is a data structure?
• A data structure is an arrangement of data either in
computer's memory or on the disk storage.
• Some common examples of data structures are arrays,
linked lists, queues, stacks, binary trees, graphs, and
hash tables.
• Algorithms, on the other hand, are used to
manipulate the data contained in these data structures
as in searching and sorting. Many algorithms apply
directly to a specific data structures.
 Classification of Data Structures

Data Structures

Primitive (Int,
Non-primitive
Float, Boolean,
etc

Linear (Arrays, Non- Linear

Lists, Stacks, (Trees, Graphs,
Queues, etc) etc)
 Primitive Data Structures
• These are the data structures which are directly
supported by the machine or programming language.
• The different primitive data types are
 Integer
 Float
 Double
 Character
 boolean
 Non-primitive Data Structures
• Non-primitive data structures are those data
structures which are created using primitive
data structures. These data structures do not
allow any specific instructions to be performed
on the data items directly
• Examples of such data structures include
linked lists, stacks, trees, and graphs.
• Non-primitive data structures can further be
classified into two categories: linear and non-
linear data structures.
 Linear and Non-linear Data
Structures
• If the elements of a data structure are stored in
a linear or sequential order, then it is a linear
data structure. Examples are arrays, linked
lists, stacks, and queues.
• If the elements of a data structure are not
stored in a sequential order, then it is a non-
linear data structure. Examples are trees and
graphs.
 array

Linked list

queue
tree stack
 Abstract Data Type

• An abstract data type (ADT) is a mathematical

model for data types where a data type is
defined by its behaviour (semantics) from the
point of view of a user of the data, specifically
in terms of possible values, possible operations
on data of this type, and the behaviour of these
operations.
 Abstract Data Type and Data
Structure
• Definition:-
 Abstract Data Types (ADTs) stores data and allow various operations
on the data to access and change it.
 A mathematical model, together with various operations defined on the
model
 An ADT is a collection of data and associated operations for
manipulating that data

• Data Structures
 Physical implementation of an ADT
 data structures used in implementations are provided in a language
(primitive or built-in) or are built from the language constructs (user-
defined)
 Each operation associated with the ADT is implemented by one
or more subroutines in the implementation
 Algorithm
• An algorithm is a
 well defined computational method that
 takes some value(s) as input and
 produces some value(s) as output.
• In other words, an algorithm is a sequence of
computational steps that transforms input(s) into
output(s).
• An algorithm is correct if for every input, it halts with
correct output.
• A correct algorithm solves the given problem, where
as an incorrect algorithm might not halt at all on some
input instance, or it might halt with other than
designed answer.
 Example of an algorithm

Algorithms are written in pseudo code that

resembles programming languages like C

Algorithm Factorial (n)

Step 1: FACT = 1
Step 2: for i = 1 to n do
Step 3: FACT = FACT * i
Step 4: return FACT
 Efficiency of an Algorithm

• In this course, we care most about asymptotic

performance
 How does the algorithm behave as the problem
size gets very large?
o Running time (Time Complexity)
o Memory/storage requirements (Space Complexity)
o Bandwidth/power requirements/logic gates/etc.

An asymptote of a curve is a line such that the distance

between the curve and the line approaches zero as one or
both of the x or y coordinates tends to infinity.
 Analysis of Algorithms
• Analysis is performed with respect to a computational
model
• We will usually use a generic uniprocessor random-
access machine (RAM)
 All memory equally expensive to access
 No concurrent operations
 All reasonable instructions take unit time
o Except, of course, function calls
 Constant word size
o Unless we are explicitly manipulating bits
 Input Size

• Time and space complexity

 This is generally a function of the input size
o E.g., sorting, multiplication
 How we characterize input size depends:
o Sorting: number of input items
o Multiplication: total number of bits
o Graph algorithms: number of nodes & edges
o Etc
 Running Time

• Number of primitive steps that are executed

 Except for time of executing a function call most
statements roughly require the same amount of
time
oy=m*x+b
o c = 5 / 9 * (t - 32 )
o z = f(x) + g(y)
• We can be more exact if need be
 Analysis

• Worst case
 Provides an upper bound on running time
 An absolute guarantee
• Average case
 Provides the expected running time
 Very useful, but treat with care: what is “average”?
o Random (equally likely) inputs
o Real-life inputs
 Time Complexity
• Execution time of an algorithm depends on numbers
of instruction executed.
• Consider the following algorithm fragment:
for i = 1 to n do
k= k + 1 ;
• The for loop executed n+1 times for i values 1,2,.......
n, n+1.
• Each instruction in the body of the loop is executed
once for each value of i = 1,2,......, n.
• So number of steps executed is 2n+1.
 Time Complexity

• Consider another algorithm fragment:

for i = 1 to n do
for j = 1 to n do
k = k +1
• From previous example, number of
instructions executed in the inner loop is 2n+1
which is the body of outer loop.
• Total number of instruction executed is
= n+1 + n(2n+1) = 2n2 + 2n + 1
 Calculating Algorithm Efficiency
Linear loops
for(i=0;i<100;i++)
statement block ;
f(n) = (n)
Logarithmic Loops
for(i=1;i<1000;i*=2) for(i=1000;i>=1;i/=2)
statement block; statement block;
f(n) = log n
Nested Loops
for(i=0;i<100;i++)
for (j=0;j<20;j++)
statement block ;
F(n) = n*m
Linear logarithmic
for(i=0;i<10;i++)
for(j=1; j<10;j*=2)
statement block;
f(n)= n log n
 Asymptotic Performance
• We compare algorithms based on large input
size
• Predict upper and lower bounds on the speed of
the algorithms rather than predicting their exact
speed.
• To compare two algorithms, we consider the
asymptotic behavior of the two functions for very
large input sizes.
• Asymptotic Notations
 Abstract away low order terms and constants
 Describe the running time of an algorithm as n grows
to .
 Asymptotic Upper Bound – O (Big-
Oh) Notation
• Definition
 O(g(n)) = { f(n):  positive
constants c and n0 such that
0  f(n)  c  g(n)  n  n0}
• We write f(n) = O(g(n))
instead of
f(n) ε O(g(n))
Intuitively: Set of all functions
whose rate of growth is the same as
or lower than that of g(n).
g(n) is an asymptotic upper bound for f(n).
 Example
• Consider the function f(n)=4n+32 - show that
f(n) = O(n2).
• This means that we need to find an integer n0
and a constant c>0 such that for all integers n 
n0, f(n)  cn2
• Suppose we choose c=1. Then
f(n)  cn2  4n + 32  n2
 0  n2 - 4n - 32
 0  (n -8) (n + 4)
For the above inequality to be satisfied (n0 – 8)  0
i.e. n0 = 8. This means that for c=1 and n0 = 8, f(n) 
cn2 for all n  n0. Hence, f(n) = O(n2).
• Of course there are other choices for c and n0
The important thing is that there should be
some choice for which it can be proven that
f(n)  cn2
• Can you tell me what will be the value of n0 if
c=4?
 Ans:- n0 = 6.
Show that f(n)=4n+32 = O(n)
Show that f(n)= 6n4 – 2 n3 +5 = O(n4)
 Loose & Tight Bounds
• f(n) = 3n2 – 100n + 6 = O (n2) because 3n2 – 100n + 6 < 3n2
• f(n) = 3n2 – 100n + 6 = O (n3) because 3n2 – 100n + 6 < 0.0001n3
• f(n) = 3n2 – 100n + 6  O (n) because 3n2 – 100n + 6 > cn
when n > c
• O(n2) is tight asymptotic upper bound whereas O(n3) is loose
asymptotic upper bounds. It is also correct to say that O(n4), O(n5)
…. are also loose asymptotic bounds for f(n).
 Asymptotic Lower Bound –  (Big-
Omega) Notation
• Definition
 (g(n)) = { f(n):  positive
constants c and n0 such that
f(n)  c  g(n) 0  n  n0}
• We write f(n) = (g(n))
instead of
f(n) ε (g(n))
Intuitively: Set of all functions
whose rate of growth is the same
as or higher than that of g(n).
g(n) is an asymptotic lower bound for f(n).
Show that f(n)= 5n2 – 64n +256 =  (n2)
 Asymptotic Tight Bound –  (Theta)
Notation
For function g(n), we define (g(n)),
big-Theta of n, as the set:
(g(n)) = {f(n) :
 positive constants c1, c2, and n0,
such that n  n0,
we have 0  c1g(n)  f(n)  c2g(n)
}
Technically, f(n)  (g(n)).
Usage, f(n) = (g(n)).

f(n) and g(n) are nonnegative, for large n.

Show that f(n)= (1/2)n2 – 3n =  (n2)
Prove that 6n3≠  (n2)
 Theorem

• f(n) =  (g(n)) if and only if f(n) = O(g(n)) and

f(n) =  (g(n))
 Other Asymptotic Notations

• A function f(n) is o(g(n)) if  positive

constants c and n0 such that
f(n) < c g(n)  n  n0
• A function f(n) is (g(n)) if  positive
constants c and n0 such that
c g(n) < f(n)  n  n0
• Intuitively,
 o() is like <  () is like >  () is like =
 O() is like   () is like 
 Categories of Algorithms
• Constant time algorithms have running time complexity given as
O(1)
• Linear time algorithms have running time complexity given as O(n)
• Logarithmic time algorithms have running time complexity given
as O(log n)
• Polynomial time algorithms have running time complexity given as
O(nk) where k>1
• Exponential time algorithms have running time complexity given as
O(2n)
n O(1) O(log n) O(n) O(n log n) O(n2) O(n3) O(2n)
1 1 1 1 1 1 1 2
2 1 1 2 2 4 8 4
4 1 2 4 8 16 64 16
8 1 3 8 24 64 512 256
16 1 4 16 64 256 4,096 65536
 Course Outcomes
After completing this course satisfactorily, a student will be able to:
• CO1: Describe various sorting and searching algorithms and
compute their efficiency.
• CO2: Explain the concept of abstract data types and basic data
organization schemes such as arrays and linked lists; and write
algorithms for their operations and applications.
• CO3: Describe operations and applications of stacks & queues
and implement them using array and linked list
• CO4: Describe properties of various types of trees and write
algorithms for their operations.
• CO5: Represent graphs using array and linked list and write
algorithms for graph traversal, minimum cost spanning trees and
shortest path