Pengantar Search-Based

Software Engineering
Lecture 9

Adopted from Shin Yoo, Mark Harman

Prepared by I Made Murwantara, Universitas Pelita Harapan
©IMM 2021
 Overview
• Software Engineering Crisis
• What is SBSE?
• Stand Point
• View Point

2021 2
 Software Engineering
• "the systematic application of scientific and technological
knowledge, methods, and experience to the design,
implementation, testing, and documentation of software"—
The Bureau of Labor Statistics—IEEE Systems and software
engineering – Vocabulary[18]
• "The application of a systematic, disciplined, quantifiable
approach to the development, operation, and maintenance of
software"—IEEE Standard Glossary of Software Engineering
Terminology[19]
• "'software engineering' encompasses not just the act of writing
code, but all of the tools and processes an organization uses to
build and maintain that code over time. [...] Software
engineering can be thought of as 'programming integrated over
time.'"—Software Engineering at Google[23]
2021 3
 Software Crisis
As long as there were no machines, programming was no
problem at all; when we had a few weak computers,
programming became a mild problem, and now we have gigantic
computers, programming has become an equally gigantic
problem.
-- Edsger Dijkstra, The Humble Programmer, Communications of
the ACM, 1972

[NATO Conf 1968]…concluded that software engineering should

use the philosophies and paradigms of established engineering
disciplines, to solve the problem of software crisis.

2021 4
 How To Improve Quality
• Theory
• Simulation
• Optimization

2021 5
 An Example
We want to build a bridge

Theory,
𝐹𝑜𝑟𝑐𝑒
• Stress =
𝐴𝑟𝑒𝑎
• The maximum stress material can resists

2021 6
 An Example
• Simulation, Given the physical law as the
foundation

https://www.midasoft.com/bridge-library/civil/products/midascivil
2021 7
 An Example
• Optimisation
• Simulation make it better for optimization
• It is, simply, trial and error, which is only
possible because we have the simulation
environment.

2021 8
 Software Engineering
Theory Simulation Optimisation Product
Building a Abstract Computation Computation Real-world
Bridge
Software Best Practice ? ? Computation
Engineering

• Theory of software engineering is (somewhat) lacking.

• Simulation is done via Modeling
• Optimisation, how to do it in software development?
• SWE is the only engineering principle, in which the materials
for product, simulation, and optimisation are the same.

2021 9
 Search-Based Software Engineering
• A large movement(?) that seeks to apply
various optimisation techniques to software
engineering problems (NOT search engines or
code search)

2021 10
 Why Optimisation
• Automate SE tasks (either fully, or at least
until human engineers can attend to the issue)
• Gain insights into complicated problem
domain that are either too large or too
complicated for humans to understand
• Unbiased decision support that is data-driven

2021 11
 Tools
• Classical (exact) optimisation would be desirable
but often cannot cope with the scale
• A heavy focus on stochastic optimisation, with a
heavy emphasis on evolutionary computation and
other nature-inspired algorithms (mostly due to
historical reasons).
• Open challenges to applications of any machine
intelligence: neural nets and reinforcement
learning are gaining much attention these days
naturally.
2021 12
 Our Stand Point
• SBSE stand at the intersection of
computational intelligence and software
engineering.
• Pragmatic application has stimulated
theoretical results in computational
intelligence, and vice versa.

2021 13
 Our Stand Point

https://coinse.kaist.ac.kr/research/
2021 14
 Meta Heuristic
Heuristic [mirriam-webster]
• : involving or serving as an aid to learning,
discovery, or problem-solving by experimental
and especially trial-and-error methods
Meta
• showing or suggesting an explicit awareness of
itself or oneself as a member of its category :
cleverly self-referential
– occurring later than or in succession to : after
– : later or more highly organized or specialized form of

2021 15
 Meta Heuristic
• Strategies that guide the search of the
acceptable solution
• Approximate and usually non-deterministic
• Not problem specific
• Smart trial and error

2021 16
 Trial & Error
• Abundance of computational resources means
many domains are adopting (knowingly or not)
a similar approach.
• Corpus-based NLP
• Go (the only competitive AI players are based
on Monte-Carlo Method)
• Many application of machine learning

2021 17
 Key & Ingredients
• What are we going to try this time?
(representation)
• How is it different from what we tried before?
(operators)
• How well did we do this time?
(fitness/objective function)
• Minor (but critical) ingredients: constraints

2021 18
 Viewpoints
• There are many algorithms in computational
intelligence; you do need to learn individual
algorithms in detail.
• However, I also want to communicate a frame
of thinking, not only individual algorithms.
• The tuple of (representation, operators, fitness
function) can be a universal platform to
understand different classes of algorithms.

2021 19
 Exploitation VS Exploration
• Exploitation: if a candidate solution looks
promising, optimisation should focus on that
particular direction. However,
• Exploration: unexplored solution space may
contain something *much better*.
• How to balance these two is critical to all
learning/optimisation algorithms.

2021 20
 Machines are Dumb & Lazy
• Like human, they will do the minimum work
that passes your criteria, i.e. design of the
optimization problem.
• Not because of their work ethic, but because of
the fact that, usually, minimum work is the
easiest to find solution.

2021 21
 Case: GenProg
• GenProg uses stochastic optimisation to
modify existing faulty software code, until it
passes all tests.
• We can only tell it to try until it passes all tests,
not until the program is correct.

2021 22
 How to do better
• Try to learn from human engineers more.
• Eventually, solutions from SBSE should be
adopted because humans accept it, not because
the fitness value is above an arbitrary cut-off
point X.
• Turing-test as fitness function!
– a way of dealing with the question whether
machines can think.
– https://plato.stanford.edu/entries/turing-test/

2021 23
 SBSE
In SBSE we apply search techniques to
search large search spaces, guided by a
fitness function that captures properties of
the acceptable software artefacts we seek.

Genetic Algorithms, Hill climbing, Simulated

Annealing, Random, Tabu Search,
Estimation of Distribution Algorithms,
Particle Swarm Optimization

2021 24
 SBSE: Generic
Testing
Fitness function: coverage, time, …
Representation: input vector

Restructuring
Fitness function: cohesion and coupling
Representation: mapping nodes to clusters

2021 25
 SBSE: Generic
Testing is a search problem
Fitness function: coverage, time, …
Representation: input vector

Requirements is a search problem

Fitness function: cost, value, dependence constraints
Representation: bitset

2021 26
Requirement Selection & Regression
Testing

2021 27
Requirement Selection & Regression
Testing

2021 28
Hill Climbing
Hill Climbing seeks to improve a
single solution, initially selected at
random, by iteratively exploring
its neighbourhood

2021 29
 Some Dangers in SBSE
• Simply combining two
successful areas (optimisation
and SE) does not equal
success. You need to optimise
the ideas themselves and form
a new, cohesive whole
• Don’t use or reinvent the same
flawed wheel when it looks
like this.
• (multi objective optimisation,
constraint handling, scalable
search)
2021 30
 Some Dangers in SBSE

• Beware! When you’re holding a

hammer, everything looks like a
nail.

• For example, if you can

calculate the answer directly or
other methods solve the problem
better (e.g. best prioritised list of
requirements), search is the
wrong tool for the job.
2021 31