UNIT-V: Branch and Bound Method, O/I knapsack and traveling

salesperson problem, efficiency considerations. Techniques for algebraic

problems, some lower bounds on parallel computations.

BRANCH AND BOUND METHOD:

Branch-and-bound is a general technique for improving the searching process by

systematically enumerating all candidate solutions and disposing of obviously
impossible solutions.
Branch-and-bound usually applies to those problems that have finite solutions, in which
the solutions can be represented as a sequence of options. The first part of branch-and-
bound branching requires several choices to be made so that the choices will branch out
into the solution space. In these methods, the solution space is organized as a treelike
structure. Branching out to all possible choices guarantees that no potential solutions
will be left uncovered. But because the target problem is usually NP-complete or even
NP-hard, the solution space is often too vast to traverse.
An important advantage of branch-and-bound algorithms is that we can control the
quality of the solution to be expected, even if it is not yet found.

How to find bound for every node for 0/1 Knapsack?

The idea is to use the fact that the Greedy approach provides the best solution for
Fractional Knapsack problem.
To check if a particular node can give us a better solution or not, we compute the
optimal solution (through the node) using Greedy approach. If the solution computed
by Greedy approach itself is more than the best so far, then we can’t get a better
solution through the node.

Complete Algorithm:
1. Sort all items in decreasing order of ratio of value per unit weight so that an
upper bound can be computed using Greedy Approach.
2. Initialize maximum profit, maxProfit = 0
3. Create an empty queue, Q.
4. Create a dummy node of decision tree and enqueue it to Q. Profit and weight of
dummy node are 0.
5. Do following while Q is not empty.
o Extract an item from Q. Let the extracted item be u.
o Compute profit of next level node. If the profit is more than maxProfit,
then update maxProfit.
o Compute bound of next level node. If bound is more than maxProfit, then
add next level node to Q.
o Consider the case when next level node is not considered as part of
solution and add a node to queue with level as next, but weight and profit without
considering next level nodes.
Illustration:
Input:
// First thing in every pair is weight of item
// and second thing is value of item
Item arr[] = {{2, 40}, {3.14, 50}, {1.98, 100},
{5, 95}, {3, 30}};
Knapsack Capacity W = 10

Output:
The maximum possible profit = 235

Below diagram shows illustration. Items are

considered sorted by value/weight.
Note : The image doesn't strictly follow the
algorithm/code as there is no dummy node in the
image.
Following is C++ implementation of above idea.
// C++ program to solve knapsack problem using
// branch and bound
#include <bits/stdc++.h>
using namespace std;
  
// Stucture for Item which store weight and corresponding
// value of Item
struct Item
{
    float weight;
    int value;
};
  
// Node structure to store information of decision
// tree
struct Node
{
    // level  --> Level of node in decision tree (or index
    //             in arr[]
    // profit --> Profit of nodes on path from root to this
    //            node (including this node)
    // bound ---> Upper bound of maximum profit in subtree
    //            of this node/
    int level, profit, bound;
    float weight;
};
  
// Comparison function to sort Item according to
// val/weight ratio
bool cmp(Item a, Item b)
{
    double r1 = (double)a.value / a.weight;
    double r2 = (double)b.value / b.weight;
    return r1 > r2;
}
  
// Returns bound of profit in subtree rooted with u.
// This function mainly uses Greedy solution to find
// an upper bound on maximum profit.
int bound(Node u, int n, int W, Item arr[])
{
    // if weight overcomes the knapsack capacity, return
    // 0 as expected bound
    if (u.weight >= W)
        return 0;
  
    // initialize bound on profit by current profit
    int profit_bound = u.profit;
  
    // start including items from index 1 more to current
    // item index
    int j = u.level + 1;
    int totweight = u.weight;
  
    // checking index condition and knapsack capacity
    // condition
    while ((j < n) && (totweight + arr[j].weight <= W))
    {
        totweight    += arr[j].weight;
        profit_bound += arr[j].value;
        j++;
    }
  
    // If k is not n, include last item partially for
    // upper bound on profit
    if (j < n)
        profit_bound += (W - totweight) * arr[j].value /
                                         arr[j].weight;
  
    return profit_bound;
}
  
// Returns maximum profit we can get with capacity W
int knapsack(int W, Item arr[], int n)
{
    // sorting Item on basis of value per unit
    // weight.
    sort(arr, arr + n, cmp);
  
    // make a queue for traversing the node
    queue<Node> Q;
    Node u, v;
  
    // dummy node at starting
    u.level = -1;
    u.profit = u.weight = 0;
    Q.push(u);
  
    // One by one extract an item from decision tree
    // compute profit of all children of extracted item
    // and keep saving maxProfit
    int maxProfit = 0;
    while (!Q.empty())
    {
        // Dequeue a node
        u = Q.front();
        Q.pop();
  
        // If it is starting node, assign level 0
        if (u.level == -1)
            v.level = 0;
  
        // If there is nothing on next level
        if (u.level == n-1)
            continue;
  
        // Else if not last node, then increment level,
        // and compute profit of children nodes.
        v.level = u.level + 1;
  
        // Taking current level's item add current
        // level's weight and value to node u's
        // weight and value
        v.weight = u.weight + arr[v.level].weight;
        v.profit = u.profit + arr[v.level].value;
  
        // If cumulated weight is less than W and
        // profit is greater than previous profit,
        // update maxprofit
        if (v.weight <= W && v.profit > maxProfit)
            maxProfit = v.profit;
  
        // Get the upper bound on profit to decide
        // whether to add v to Q or not.
        v.bound = bound(v, n, W, arr);
  
        // If bound value is greater than profit,
        // then only push into queue for further
        // consideration
        if (v.bound > maxProfit)
            Q.push(v);
  
        // Do the same thing,  but Without taking
        // the item in knapsack
        v.weight = u.weight;
        v.profit = u.profit;
        v.bound = bound(v, n, W, arr);
        if (v.bound > maxProfit)
            Q.push(v);
    }
  
    return maxProfit;
}
  
// driver program to test above function
int main()
{
    int W = 10;   // Weight of knapsack
    Item arr[] = {{2, 40}, {3.14, 50}, {1.98, 100},
                  {5, 95}, {3, 30}};
    int n = sizeof(arr) / sizeof(arr[0]);
  
    cout << "Maximum possible profit = "
         << knapsack(W, arr, n);
  
    return 0;
}
Output :
Maximum possible profit = 235

lower bounds on parallel computations.

Analysis of an algorithm helps us determine whether the algorithm is useful or not.

Generally, an algorithm is analyzed based on its execution time (Time
Complexity) and the amount of space (Space Complexity) it requires.
Since we have sophisticated memory devices available at reasonable cost, storage
space is no longer an issue. Hence, space complexity is not given so much of
importance.
Parallel algorithms are designed to improve the computation speed of a computer. For
analyzing a Parallel Algorithm, we normally consider the following parameters −

 Time complexity (Execution Time),

 Total number of processors used, and
 Total cost.

Time Complexity
The main reason behind developing parallel algorithms was to reduce the computation
time of an algorithm. Thus, evaluating the execution time of an algorithm is extremely
important in analyzing its efficiency.
Execution time is measured on the basis of the time taken by the algorithm to solve a
problem. The total execution time is calculated from the moment when the algorithm
starts executing to the moment it stops. If all the processors do not start or end
execution at the same time, then the total execution time of the algorithm is the
moment when the first processor started its execution to the moment when the last
processor stops its execution.
Time complexity of an algorithm can be classified into three categories−
 Worst-case complexity − When the amount of time required by an algorithm for
a given input is maximum.
  Average-case complexity − When the amount of time required by an algorithm
for a given input is average.
 Best-case complexity − When the amount of time required by an algorithm for
a given input is minimum.

Asymptotic Analysis
The complexity or efficiency of an algorithm is the number of steps executed by the
algorithm to get the desired output. Asymptotic analysis is done to calculate the
complexity of an algorithm in its theoretical analysis. In asymptotic analysis, a large
length of input is used to calculate the complexity function of the algorithm.
Note − Asymptotic is a condition where a line tends to meet a curve, but they do not
intersect. Here the line and the curve is asymptotic to each other.
Asymptotic notation is the easiest way to describe the fastest and slowest possible
execution time for an algorithm using high bounds and low bounds on speed. For this,
we use the following notations −

 Big O notation
 Omega notation
 Theta notation

Big O notation

In mathematics, Big O notation is used to represent the asymptotic characteristics of

functions. It represents the behavior of a function for large inputs in a simple and
accurate method. It is a method of representing the upper bound of an algorithm’s
execution time. It represents the longest amount of time that the algorithm could take to
complete its execution. The function −
f(n) = O(g(n))
if there exists positive constants c and n0 such that f(n) ≤ c * g(n) for all n where n ≥ n0.

Omega notation

Omega notation is a method of representing the lower bound of an algorithm’s

execution time. The function −
f(n) = Ω (g(n))
iff there exists positive constants c and n0 such that f(n) ≥ c * g(n) for all n where n ≥
n 0.
Theta Notation

Theta notation is a method of representing both the lower bound and the upper bound
of an algorithm’s execution time. The function −
f(n) = θ(g(n))
iff there exists positive constants c1, c2, and n0 such that c1 * g(n) ≤ f(n) ≤ c2 * g(n) for
all n where n ≥ n0.

Speedup of an Algorithm
The performance of a parallel algorithm is determined by calculating its speedup.
Speedup is defined as the ratio of the worst-case execution time of the fastest known
sequential algorithm for a particular problem to the worst-case execution time of the
parallel algorithm.
speedup = 
Worst case execution time of the fastest known sequential for a particular problem/
Worst case execution time of the parallel algorithm

Number of Processors Used

The number of processors used is an important factor in analyzing the efficiency of a
parallel algorithm. The cost to buy, maintain, and run the computers are calculated.
Larger the number of processors used by an algorithm to solve a problem, more costly
becomes the obtained result.

Total Cost
Total cost of a parallel algorithm is the product of time complexity and the number of
processors used in that particular algorithm.
Total Cost = Time complexity × Number of processors used
Therefore, the efficiency of a parallel algorithm is −
Efficiency = 
Worst case execution time of sequential algorithm /
Worst case execution time of the parallel algorithm
The model of a parallel algorithm is developed by considering a strategy for dividing the data and
processing method and applying a suitable strategy to reduce interactions. In this chapter, we will
discuss the following Parallel Algorithm Models −

 Data parallel model

 Task graph model

 Work pool model

 Master slave model

 Producer consumer or pipeline model

 Hybrid model

Data Parallel

In data parallel model, tasks are assigned to processes and each task performs similar types of
operations on different data. Data parallelism is a consequence of single operations that is being
applied on multiple data items.

Data-parallel model can be applied on shared-address spaces and message-passing paradigms. In data-
parallel model, interaction overheads can be reduced by selecting a locality preserving decomposition,
by using optimized collective interaction routines, or by overlapping computation and interaction.

The primary characteristic of data-parallel model problems is that the intensity of data parallelism
increases with the size of the problem, which in turn makes it possible to use more processes to solve
larger problems.

Example − Dense matrix multiplication.

Task Graph Model

In the task graph model, parallelism is expressed by a task graph. A task graph can be either trivial or
nontrivial. In this model, the correlation among the tasks are utilized to promote locality or to minimize
interaction costs. This model is enforced to solve problems in which the quantity of data associated with
the tasks is huge compared to the number of computation associated with them. The tasks are assigned
to help improve the cost of data movement among the tasks.

Examples − Parallel quick sort, sparse matrix factorization, and parallel algorithms derived via divide-
and-conquer approach.
Here, problems are divided into atomic tasks and implemented as a graph. Each task is an independent
unit of job that has dependencies on one or more antecedent task. After the completion of a task, the
output of an antecedent task is passed to the dependent task. A task with antecedent task starts
execution only when its entire antecedent task is completed. The final output of the graph is received
when the last dependent task is completed (Task 6 in the above figure).

Work Pool Model

In work pool model, tasks are dynamically assigned to the processes for balancing the load. Therefore,
any process may potentially execute any task. This model is used when the quantity of data associated
with tasks is comparatively smaller than the computation associated with the tasks.

There is no desired pre-assigning of tasks onto the processes. Assigning of tasks is centralized or
decentralized. Pointers to the tasks are saved in a physically shared list, in a priority queue, or in a hash
table or tree, or they could be saved in a physically distributed data structure.

The task may be available in the beginning, or may be generated dynamically. If the task is generated
dynamically and a decentralized assigning of task is done, then a termination detection algorithm is
required so that all the processes can actually detect the completion of the entire program and stop
looking for more tasks.

Example − Parallel tree search

Master-Slave Model

In the master-slave model, one or more master processes generate task and allocate it to slave
processes. The tasks may be allocated beforehand if −

 the master can estimate the volume of the tasks, or

 a random assigning can do a satisfactory job of balancing load, or

 slaves are assigned smaller pieces of task at different times.

This model is generally equally suitable to shared-address-space or message-passing paradigms, since

the interaction is naturally two ways.

In some cases, a task may need to be completed in phases, and the task in each phase must be
completed before the task in the next phases can be generated. The master-slave model can be
generalized to hierarchical or multi-level master-slave model in which the top level master feeds the
large portion of tasks to the second-level master, who further subdivides the tasks among its own slaves
and may perform a part of the task itself.
Precautions in using the master-slave model

Care should be taken to assure that the master does not become a congestion point. It may happen if
the tasks are too small or the workers are comparatively fast.

The tasks should be selected in a way that the cost of performing a task dominates the cost of
communication and the cost of synchronization.

Asynchronous interaction may help overlap interaction and the computation associated with work
generation by the master.

Pipeline Model

It is also known as the producer-consumer model. Here a set of data is passed on through a series of
processes, each of which performs some task on it. Here, the arrival of new data generates the
execution of a new task by a process in the queue. The processes could form a queue in the shape of
linear or multidimensional arrays, trees, or general graphs with or without cycles.

This model is a chain of producers and consumers. Each process in the queue can be considered as a
consumer of a sequence of data items for the process preceding it in the queue and as a producer of
data for the process following it in the queue. The queue does not need to be a linear chain; it can be a
directed graph. The most common interaction minimization technique applicable to this model is
overlapping interaction with computation.

Example − Parallel LU factorization algorithm.

Hybrid Models

A hybrid algorithm model is required when more than one model may be needed to solve a problem.

A hybrid model may be composed of either multiple models applied hierarchically or multiple models
applied sequentially to different phases of a parallel algorithm.

Example − Parallel quick sort