DEPARTMENT OF

COMPUTER SCIENCE & ENGINEERING Research Paper

Numerical Methods and Optimization Using Python

“Optimization Techniques: A Foundational Approach for

Decision-Making”

Research Paper

Submitted by

ARNAV KUMAR (22BCS12122)

VISHAL KUMAR (22BCS12097)

in partial fulfilment for the award of the degree of

BACHELOR OF ENGINEERING

IN

COMPUTER SCIENCE

Optimization Techniques: A Foundational Approach for Decision-Making Page | 1

 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

ABSTRACT

Optimization methods have a crucial function in different scientific fields and

engineering uses. This paper discusses the basic ideas of optimization, including its
main rules and various ways to find optimal solutions. We discuss classical
mathematical programming methods such as linear programming (LP), non-linear
programming (NLP), and integer programming (IP). Second, we will examine
sophisticated optimization methods that involve dynamic programming,
evolutionary algorithms and stochastic programming. The article emphasizes the
significance of correctly framing problems by stating clear objective functions and
limits for successful optimization. We also talk about how optimization techniques
are applied to actual situations in various fields like resource allocation, design
improvement or decision making. Many areas of science and engineering, always
searching for the best solutions, depend on optimization techniques. This paper
reveals basic ideas in optimization, discussing its main principles and different
methods used to get the best results. We go into classical methods such as linear
programming, non-linear programming and integer programming that give readers
skills needed to handle problems with clear limits. Also, we are advancing towards
complex techniques like dynamic programming, evolutionary algorithms and
stochastic programming. We pay attention to their potential for dealing with
intricate and uncertain situations. The paper also underlines the significance of
optimization methods in practice, not just theoretically. We show how they change
different fields like resource division, design perfection and decision-making
methods across studies areas. This in-depth investigation provides readers with a
solid knowledge about optimization, giving them the ability to use it well in their
activities.

Optimization Techniques: A Foundational Approach for Decision-Making Page | 2

 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

Introduction

Optimization seeks to identify the best possible solution from a set of alternatives,
often maximizing desired outcomes or minimizing costs and resource
consumption. It underpins numerous disciplines, including engineering design,
machine learning, economics, and operations research. This paper provides a
comprehensive overview of optimization techniques, equipping readers with a
foundational understanding of this critical field. Imagine standing at a crossroads,
unsure of the best path forward. This scenario isn't limited to literal journeys; it
permeates every decision we make, from resource allocation in businesses to
design choices in engineering. In each instance, we strive to find the optimal
solution, the one that maximizes desired outcomes or minimizes costs and
inefficiencies. This is where optimization techniques emerge as powerful tools,
guiding us towards the most beneficial course of action.

This paper delves into the fascinating world of optimization techniques, equipping
you with the foundational knowledge to navigate complex decision-making
scenarios. We'll explore the core principles of optimization, classical and advanced
methodologies, and the crucial role of problem formulation. By understanding
these concepts, you'll gain the ability to leverage optimization techniques in
various domains, ensuring informed decisions and achieving the best possible
results.

Optimization Techniques: A Foundational Approach for Decision-Making Page | 3

 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

What is optimization?
 Optimization technique in the field of engineering is a powerful tool to
utilize the resources in an efficient way as well as to reduce the
environmental impact of a process. Application of optimization process
helps us achieve the most favourable operating conditions.
 The primary focus of using optimization techniques is to measure the
Maximum or Minimum value of a function depending on the circumstance.

 Any engineering or research discipline involving design, maintenance and

manufacturing requires certain technical decisions to be taken at different
stages. The outcome of taking these decisions is to maximize the profit with
minimum utilization of resources.
 Optimization search has a good significance in appearing a weakness of the
simulated model and guiding the software for best executing. Robust
Optimization Techniques: A Foundational Approach for Decision-Making Page | 4
 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

optimization could predict uncertainty parameters and complex confounding

(hidden) variables in interacted nonlinear systems.

 Knowledge of optimization theory as well its practical application is

essential for all engineers.

Optimization Techniques: A Foundational Approach for Decision-Making Page | 5

 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

Terms commonly used in optimization process:

Objectives to be minimized: cost, energy, loss, waste, processing time, raw
material consumption…. etc.
Objectives to be maximized: profit, conversion, yield, utility, efficiency,
capacity…etc.

+LP, linear programming problem: objective f(x) and constrains ci(x) are linear.
+NLP, nonlinear programming problem: f(x) is linear/nonlinear, ci(x):is
nonlinear/linear i.e, one of them is nonlinear.
+ quality constrains represented by numerical values.
+ Inequality constrains represented by related equation.

Static and dynamic problems:

Optimization Techniques: A Foundational Approach for Decision-Making Page | 6

 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

Most optimization problems are based on steady state models that could be
formulated from experimental data. Optimization problems involving dynamic
models are more suitable for "Optimal control".

Continuous and discrete variables:

Continuous variable is any process variable that has real value such as: pressure,
temperature, concentration…. etc, while discrete variable(design variable) is
integer value only, number of tubes in a heat exchanger and number of distillation
trays in a distillation column ….etc. Optimization problems without discrete
variables are far easier to solve. Reliability of optimization technique depends on
capturing of discrete variables.

Optimization process steps:

1. Selecting of objectives
2. Selecting of effective decision variables
3. Limiting of constraints
4. Formulating of reliable optimization model
5. Implementing of suitable optimization algorithm.

Example: Specifying the Optimization of a Steam Condenser

Variables, Objectives, Constraints

Figure below shows a steam condenser. The designer needs to design a condenser
that will cost a minimum amount and condense a specified amount of steam, m min.
Steam flow rate, ms, steam condition, x, water flow rate, mw, water temperature,
Tw, water pressure Pw, and materials are specified. Variables under the designer’s
Optimization Techniques: A Foundational Approach for Decision-Making Page | 7
 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

control include the outside diameter of the shell, D; tube wall thickness, t; length of
tubes, L; number of passes, N; number of tubes per pass, n; baffle spacing, B, tube
diameter, d. The model calculates the actual steam condensed, m cond, the corrosion
potential, CP, condenser pressure drops, ∆ P cond, cost and overall size, Vcond. The
overall size must be less than V max and the designer would like to limit overall
pressure drop to be less than ∆Pmax.

Figure: Schematic of steam condenser.

From this description it appears we have the following:

Design Variables: Process Variables:

Outside diameter of the shell, D Steam flow rate, ms
Tube wall thickness, t Steam condition, x Water
Length of tubes, L flow rate, mw Water
Number of passes, N temperature, Tw Water
Number of tubes per pass, n
Baffle spacing, B, pressure, Pw
Tube diameter, d. Material properties ,𝜌 ,CP

Constraints: Objective Functions:

Vcond ≤Vmax Minimize Cost

Optimization Techniques: A Foundational Approach for Decision-Making Page | 8

 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

mcond ≥ mmin Overall size, Vcond Steam

condensed, mcond
∆Pcond ≤∆Pmax
corrosion potential, CP
condenser pressure drop,∆Pcond

Optimization Methods:

1. Deterministic optimization: implemented for convex (differentiate)

equations, for LP, NLP, single and multi-variable functions:

-Lagrange multipliers
-Successive quadratic programming
-Newton’s method
-Qusi -Newton method
-Levenberg-Marquardt Algorithm
-Simplex Method
-Non-Simplex method
-Secant method

2. Statistical optimization: for correlation and design of experiment

-Least Square method

-Design of Experiment-Response Surface Methodology
-ANOVA Study

3. Stochastic Optimization: for complex non-convex (not differentiable)

equations.
Optimization Techniques: A Foundational Approach for Decision-Making Page | 9
 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

-Genetic Algorithm
- Pattern Search optimization
-Firefly Algorithm

Engineering Models in Optimization:

Engineering models play a key role in engineering optimization. In this section we
will discuss some further aspects of engineering models. We refer to engineering
models as analysis models.
In a very general sense, analysis models can be viewed as shown in Fig 1 below. A
model requires some inputs in order to make calculations. These inputs are called
analysis variables. Analysis variables include design variables (the variables we
can change) plus other quantities such as material properties, boundary conditions,
etc. which typically would not be design variables. When all values for all the
analysis variables have been set, the analysis model can be evaluated. The analysis
model computes outputs called analysis functions. These functions represent what
we need to determine the “goodness” of a design.
For example, analysis functions might be stresses, deflections, cost, efficiency,
heat transfer, pressure drop, etc. It is from the analysis functions that we will select
the design functions, i.e., the objectives and constraints.

Optimization Techniques: A Foundational Approach for Decision-Making Page | 10

 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

Fig. 1. The operation of analysis models

Models and Optimization by Trial-and-Error:

The analysis model is to compute the values of analysis functions. The designer
specifies values for analysis variables, and the model computes the corresponding
functions.
Note that the analysis software does not make any kind of “judgment” regarding
the goodness of the design. If an engineer is designing a reactor, for example, and
has software to predict conversion and yield, the analysis software merely reports
those values—it does not suggest how to change the reactor design to increase
conversion in a particular location. Determining how to improve the design is the
job of the designer.
To improve the design, the designer will often use the model in an iterative
fashion, as shown in Fig. 2 below. The designer specifies a set of inputs, evaluates
the model, and examines the outputs. Suppose, in some respect, the outputs are not
satisfactory. Using intuition and experience, the designer proposes a new set of
inputs which he or she feels will result in a better set of outputs. The model is
evaluated again. This process may be repeated many times.

Optimization Techniques: A Foundational Approach for Decision-Making Page | 11

 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

Fig. 2. Common “trial-and-error” iterative design process.

Optimization with Computer Algorithms:

Computer-based optimization is an attempt to bring some high-tech help to the
decision-making side of Fig. 2. With this approach, the designer is taken out of the
trial-and-error loop. The computer is now used to both evaluate the model and
search for a better operating condition. This process is illustrated in Fig.3.

Optimization Techniques: A Foundational Approach for Decision-Making Page | 12

 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

Fig..3. Moving the designer out of the trial-and-error loop with computer-based
optimization software.

The researcher now operates at a higher level. Instead of adjusting variables and
interpreting function values, the designer is specifying goals for the design
problem and interpreting optimization results. Usually, a better design can be
found in a shorter time.

Energy and Process Optimization:

Five Ways to Improve Energy Efficiency

The five ways in which improved energy efficiency can be achieved within plant
processes are highlighted below:
Optimization Techniques: A Foundational Approach for Decision-Making Page | 13
 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

 Minimizing wastes and losses

 Optimizing process operation
 Achieving better heat recovery
 Determing process changes
 Optimizing energy supply system

Multi-objective optimization methods:

The relevance and importance of multi-objective optimization (MOO) in chemical
engineering is increasing. The solutions of an MOO problem are known as the
pareto-optimal solutions. Several methods are deal with multi- objective problems
that are;

Optimization Techniques: A Foundational Approach for Decision-Making Page | 14

 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

A. Deterministic Methods:

1. Phase-plane method:
Objective space: plot f1(x) against f2(x)
2-Weighting method:
Convert MOO to SOO problem and using the same constraints
Y=w*f1(x) + (1-w)*f2(x)
Where w is a weight factor (0-1)
Even though the weighting method is conceptually straight forward, but choosing
of suitable (w) to find many pareto-optimal solutions is difficult.

3. €-constraints method:
The MOO problem is converted to a SOO problem by using one objective and
converting others into inequality constraints, for example;

Minimize f1(x) (1)

Maximize f2(x) (2)

Subject to X1 < x<Xu (3)

h(x) =0 (4)
g(x) <0 (5)
It is desired to maximize f2(x);
With respect to x subject to the bounds and constraints in equations (3 to 5) as well
as an additional constraints will be used:

Optimization Techniques: A Foundational Approach for Decision-Making Page | 15

 DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING Research Paper

f1(x) <€ (6)

Obviously, the user will have to select which objective to be retained and the value
of €. The difficulties in this method are the selection of € value and solving the
optimization problem.

Optimization Techniques: A Foundational Approach for Decision-Making Page | 16