COMPILED BY: SULMAN

AHMED

Lecture # 3 & 4
ALGORITHMIC Toolkit - 1
INSTRUCTOR: Saeed
Ahmed
Algorithm
■ An algorithm is a sequence of instructions that one must perform in
order to solve a well-formulated problem.

■ A well-formulated problem is unambiguous and precise, leaving no

room for misinterpretation. The number of steps are always finite.

■ In other words, algorithm is the method of translating the inputs into

the outputs.
6 Characteristics of an Algorithm
Finiteness
■ Algorithms are finite in two aspects:
– Length of Algorithm
– Number of Computational Steps

■ If algorithm is not of finite length, then we do not have infinite

space/memory to save and implement it.
■ If the number of computational steps are not finite, then it means the
problem is not solved at all, as infinity is undefined.
Types of Algorithms
■ There are many types of algorithm but the most fundamental types of
algorithm are:

1. Recursive Algorithms 6. Dynamic Programming

2. Iterative Algorithms 7. Greedy Algorithms

3. Divide & Conquer Algorithms 8. Brute Force Algorithms

4. Decrease & Conquer 9. Backtracking Algorithms

5. Transform & Conquer 10. Randomized Algorithms

Problems vs Algorithms vs Programs
■ For each problem or class of problems, there may be many different
algorithms.
■ For each algorithm, there may be many different implementations
(programs).
Testing Correctness
■ How do we know whether an algorithm is actually
correct?
■ First, the logical analysis of the problem we
performed in order to design the algorithm should
give us confidence that we have identified a valid
procedure for finding a solution.
■ Second, we can test the algorithm by choosing
different sets of input values, carrying out the
algorithm, and checking to see if the resulting
solution does, in fact, work.
Testing Correctness

■ BUT… no matter how much testing

we do, unless there are only a finite
number of possible input values for the
algorithm to consider, testing can
never prove that the algorithm
produces correct results in all cases.
Testing Correctness
■ We can attempt to construct a formal,
mathematical proof that, if the algorithm is given
valid input values then the results obtained from
the algorithm must be a solution to the problem.
■ We should expect that such a proof be provided
for every algorithm.
■ In the absence of such a proof, we should view
the purported algorithm as nothing more than a
heuristic procedure, which may or may not yield
the expected results.
Example
■ Problem:
– Given the sequence of non-negative integers, find the maximum pairwise
product.
■ Input Format:
– The first line of the input contains total numbers, n, in the sequence.
– The second line contains n non-negative numbers.
■ Output:
– A single number which is the maximum pairwise product.
Sample
■ Input 1:
– 3
– 123
■ Output:
– 6
■ Input 2:
– 10
– 1 2 3 4 5 10 9 8 7 6
■ Output:
– 90
Algorithm #1
Problem: MaxPairwiseProduct(n, numbers[n])
Input: n non-negative integers as numbers
Output: result as maximum pairwise product
Start
result=0
for i=1 to n
for j=i+1 to n
if numbers[i] * numbers[j] > result
result = numbers[i] * numbers[j]
end if
end for j
end for i
End
 Implementation in C++
int MaxProd(vector<int>& numbers) { int main() {
int result=0; int n;
int n=numbers.size(); cin>>n;
for(int i=0;i<n;++i) { vector<int> numbers(n);
for(int j=i+1;j<n;++j) {
for(int i=0;i<n;++i) {

if(numbers[i]*numbers[j]>result) { cin>>numbers[i];
}
result=numbers[i]*numbers[j]; int result=MaxProd(numbers);
}
cout<<result;
}
return 0;
}
}
return result;
}
Testing Results
■ This program works for infinite input instances.

■ But, it does not work for many (may be infinite) of the instances, too.

■ It is because of wrong data structure (data type) being used.

■ There is an INTEGER OVERFLOW!!!

Integer Datatypes in C++
 Size of Datatypes in C++
#include <iostream>
using namespace std;

int main() {
cout << "Size of char : " << sizeof(char) << endl;
cout << "Size of int : " << sizeof(int) << endl;
cout << "Size of short int : " << sizeof(short) << endl;
cout << "Size of long int : " << sizeof(long) << endl;
cout << "Size of long long : " << sizeof(long long) << endl;
cout << "Size of float : " << sizeof(float) << endl;
cout << "Size of double : " << sizeof(double) << endl;
cout << "Size of wchar_t : " << sizeof(wchar_t) << endl;

return 0;
}
Output
Range of Datatypes in C++
#include<iostream>
#include<limits.h>
#include<float.h>
using namespace std;
int main() {
cout << "char ranges from : " << CHAR_MIN << " to " << CHAR_MAX;
cout << "\n\nshort char ranges from : " << SCHAR_MIN << " to " << SCHAR_MAX;
cout << "\n\nunsigned char ranges from : " << 0 << " to " << UCHAR_MAX;
cout << "\n\n\nshort int ranges from : " << SHRT_MIN << " to " << SHRT_MAX;
cout << "\n\nunsigned short int ranges from : " << 0 << " to " << USHRT_MAX;
cout << "\n\nint ranges from : " << INT_MIN << " to " << INT_MAX;
cout << "\n\nunsigned int ranges from : " << 0 << " to " << UINT_MAX;
cout << "\n\nlong int ranges from : " << LONG_MIN << " to " << LONG_MAX;
cout << "\n\nunsigned long int ranges from : " << 0 << " to " << ULONG_MAX;
cout << "\n\nlong long int ranges from : " << LLONG_MIN << " to " << LLONG_MAX;
cout << "\n\nunsigned long long int ranges from : " << 0 << " to " << ULLONG_MAX;
cout << "\n\n\nfloat ranges from : " << FLT_MIN << " to " << FLT_MAX;
cout << "\n\nnegative float ranges from : " << -FLT_MIN << " to " << -FLT_MAX;
cout << "\n\ndouble ranges from : " << DBL_MIN << " to " << DBL_MAX;
cout << "\n\nnegative double ranges from : " << -DBL_MIN << " to " << +DBL_MAX;
return 0;
}
Output
 Changing the Data Structure (Data Type)
long long MaxProd(vector<int>& numbers) { int main() {
long long result=0; int n;
int n=numbers.size(); cin>>n;
for(int i=0;i<n;++i) { vector<int> numbers(n);
for(int j=i+1;j<n;++j) {
for(int i=0;i<n;++i) {
if(((long long)numbers[i])*numbers[j]>
result) { cin>>numbers[i];

result=((long long)numbers[i])*numbers[j]; }
} long long result=MaxProd(numb
ers);
}
} cout<<result;

return result; return 0;

} }
Testing Results
■ This program works for all possible input instances on a machine.

■ Of course, machine has some limitations, but we will consider it.

■ But, testing is not just about the CORRECTNESS.

■ In fact, our algorithm is VERY SLOW!!!

Analysis of Algorithm #1
1

■ In algorithm 1, our approach is to find the

product of each possible pair. 2

3
■ We do not multiply a number with itself, but
we multiply each and every number with every
4
other number in the list and make comparison.

…
n
Analysis of Algorithm #1
1

■ We have n values in the list.

2

■ First, we multiply the 1st value with n-1 values. 3

■ So, there are n-1 comparisons and n-1 4

multiplications.

…
n
Analysis of Algorithm #1
1

■ In the next step, we multiply the 2nd value with

the next n-2 values. 2

3
■ So, there are n-2 comparisons and n-2
multiplications.
4

…
n
Analysis of Algorithm #1
1

■ At the end, we have a total of

2
(n-1) + (n-2) + (n-3) + … 1 = n(n-1)/2
multiplications and the same number of 3
comparisons.
4
■ So total cost = 2*n(n-1)/2 = n(n-1)

…
■ Do we actually need this number of operations,
or we can optimize that? n
Algorithm #2
Problem: MaxPairwiseProduct(n, numbers[n])
Input: n non-negative integers as numbers
Output: result as maximum pairwise product
Start
num1=FirstMaximum(numbers) //simple for loop with n-1 comparisons
num2=SecondMaximum(numbers) //simple for loop with n-2 comparisons
result=num1 * num2
End
Analysis of Algorithm #2
1

■ In algorithm 2, our approach is to find the two

maximum values in the list. 2

3
■ This can be done by simply scanning the
values of the list.
4

…
■ At the end, we just need 1 multiplication.

n
Analysis of Algorithm #2
1

■ The total cost of algorithm #2 is

2
(n-1)+(n-2) comparisons and 1 multiplication.
Note: At this stage, we are omitting other 3
operations
4

Hence, total cost = 2(n-1) operations

…
n
Comparison of Costs
size of n cost of algorithm #1 cost of algorithm #2

10 90 18

100 9900 198

1000 999000 1998

10000 99990000 19998

100000 9999900000 199998

1000000 999999000000 1999998

10000000 99999990000000 19999998

100000000 9999999900000000 199999998

Comparison of Costs
Chart Title
10000000000

9000000000

8000000000

7000000000

6000000000

5000000000

4000000000

3000000000

2000000000

1000000000

0
0 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000

cost of algorithm #1 cost of algorithm #2

Implementation in C++
long long MaxProd(vector<int>& numbers) {
int index1=0;
int index2=0;
int n = numbers.size();
for(int i=1;i<n;++i)
if(numbers[i]>numbers[index1])
index1=i;
if(index1==0)
index2=1;
for(int j=0;j<n;++j) {
if(j==index1)
continue;
if(numbers[j]>numbers[index2])
index2=j;
}
long long result=(((long long)(numbers[index1])*numbers[index2]));
return result;
}
Stress Testing
■ It is a program that generates random tests in an
infinite loop, and for each test, it launches our
solution on this test and an alternative correct
solution of the same test and compares the
results.

■ Then, we wait until we find a test on which our

solutions differ.

■ It helps to debug and correct the solution.

Stress Testing Implementation
int main() {
while(true) {
int n=rand() % 10000+200;
std::cout<<n<<"\n";
vector<int> numbers;
for(int i=0;i<n;++i) {
numbers.push_back(rand() % 10000000); }
for(int i=0;i<n;++i) {
std::cout<<numbers[i]<<" "; }
std::cout<<"\n";
long long result1=MaxPairWiseProduct(numbers);
long long result2=MPWP(numbers);
if(result1 != result2){
std::cout<<"Wrong Answer: "<<result1<<" "<<result2<<"\n";
break; }
else {
std::cout<<"OK"; } }
return 0; }
rand() function
Prototype: int rand (void);

Header File: <cstdlib>

Function: Returns a pseudo-random integral number.

v1 = rand() % 100; // v1 in the range 0 to 99

v2 = rand() % 100 + 1; // v2 in the range 1 to 100
v3 = rand() % 30 + 1985; // v3 in the range 1985-2014
Output
Altering the Functions
■ Let just make the following changes in
the function:
■ Comment the following lines of code.

// if(index1==0)
// index2=1;
Altering the Functions Again
■ Removing the comment from the second
line.

// if(index1==0)
index2=1;
Stress Testing
■ We have already run stress test on our
algorithms for a small number of n.

■ Now, Lets try a very large value of n and its

values.

int n=rand() % 10000+200;

&
numbers.push_back(rand() % 10000);
Another Implementation of Algorithm #2

■ Here is another implementation of Algorithm #2.

■ There is a small bug/error in this implementation.
Assignment #1
1. What is the function of following lines of code in the
first implementation of algorithm #2.
if(index1==0)
index2=1;
2. Perform stress test to indicate the error in second
implementation of algorithm #2 and then correct it.
Add proper snapshots where necessary.
Instructions for Assignment #1
■ Upload a .pdf file of your solution on LMS.

■ Assignment can be submitted in groups.

■ Maximum group size is 5.

■ Deadline: 27 August, before 23:55 HRS

 Actual Running Time
int main() { int main() {
int n=1000; int n=1000;
vector<int> numbers(n); vector<int> numbers(n);
for(int i=0;i<n;++i) { for(int i=0;i<n;++i) {
numbers.push_back(50000); numbers.push_back(50000);
} }
long long result=MPWP(numbers); long long result=MaxPairWiseProduct (numbers);
cout<<result; cout<<result;
return 0; return 0;
} }
Time Complexity of Algorithms
■ When we talk about time complexity of algorithms, what do we
actually mean?

■ It is not the actual running time. It is not the standard.

– It varies from machine to machine & compiler to compiler.
– It even varies on one machine due to availability of resources.
Time Complexity of Algorithms
■ We actually calculate the estimation, not actual running time.
■ Complexity can be viewed as the maximum number of primitive
operations that a program may execute.
■ Regular operations are:
– Addition
– Multiplication
– Assignment
– Accessing array element, etc.
■ We may leave some operations uncounted and concentrate on those
that are performed the largest number of times.
Time Complexity of Algorithms
■ We define complexity as a numerical function T(n) - time versus the
input size n.

■ We want to define time taken by an algorithm without depending on

the implementation details.

■ Talgorithm#1(n) = n(n-1) = n2 - n //quadratic time

■ Talgorithm#2(n) = 2(n-1) = 2n – 2 //linear time

 End of Lecture

THANK YOU