Categories of Defects

Usual Defects in Algorithms

• incorrect logical conditions
– defect
• logical conditions poorly formulated for if-then-
else instructions
– testing strategies
• equivalence classes and class border testing
• each variable used in a logical condition must be
considered as input

2
 Example
• specification - landing alarm
– the landing gear must be deployed 2 minutes
before landing or after takeoff or when altitude
drops below 2000 feet
– if visibility is below 1000 feet, the landing gear
must be deployed 3 minutes before landing or
when altitude drops below 2500 feet
• false alarm - just as bad as missing alarm

3
 Example (continued)
if(!landingGearDeployed &&
(min(now-takeoffTime,estLandTime-now))<
(visibility < 1000 ? 180 : 120) ||
relativeAltitude<
(visibility < 1000 ? 2500 : 2000))
throw new LandingGearException();

• where is the defect in the implementation

above?
4
 Usual Defects in Algorithms
• performing an operation on the wrong
branch of a control structure
– defect
• the program performs an action when it shouldn't or
does not perform an action when it should
– testing strategies
• tests going through each program loop: zero times;
exactly once; more than once
• each possible event in the loop should occur in: the
first iteration; an intermediate iteration; the last
iteration
5
 Example 1
while(j<maximum) {
k=someOperation(j);
j++;
}
if(k==-1) signalAnError();
• the last code line must be relocated inside
the loop

6
 Example 2
if (j<maximum)
doSomething();
if (debug) printDebugMessage();
else doSomethingElse();
• outside the debugging phase
– doSomethingElse() is always called
– although that is not what was intended

7
 Usual Defects in Algorithms
• unending loop or recursion
– defect
• a loop or recursion has an infinite (neverending)
execution path
– testing strategies
• analysis of end conditions
• test cases which we expect not to be handled
properly

8
 Usual Defects in Algorithms
• preconditions of an algorithm are not
observed
– defect
• the program carries out its activity even when
preconditions are not satisfied
– testing strategies
• test cases where each precondition is not satisfied

9
 Example
• program that computes the total number of
atoms in a molecule
• unending loop/recursion
– the molecule may contain circular structures
• possible case of precondition not satisfied
– counting the atoms in a single molecule
– test - two disjoint molecules at the same time

10
 Usual Defects in Algorithms
• ignoring zero conditions
– defect
• normally, the process contains one or more items of
a certain type, but sometimes there is none
• the program may exhibit abnormal behavior in such
a case, provided it does not handle separately the
zero condition
– testing strategies
• brainstorming - finding the unusual conditions

11
 Example
• computing the average salary for each
division in a company
• a division has no members at the current
time
– all members left the division
– the division has just been created
• possible defect - division by zero

12
 Usual Defects in Algorithms
• ignoring singleton conditions
– defect
• normally, the process contains multiple items of a
certain type, but sometimes there is only one
• or the other way around
• the program may exhibit abnormal behavior in such
a case, if it does not handle separately the singleton
condition
– testing strategies
• just as before

13
 Example
• randomly grouping the members of a group
into pairs
– e.g., for sporting activities
• odd number of members
• what about the person that remains single?

14
 Usual Defects in Algorithms
• "off-by-one" errors
– defect
• wrong increment/decrement of a variable
• the loop is iterated one time more (or less) than it
should be
– testing strategies
• check if the correct numerical results were obtained
• check if the number of iterations during loop
execution is correct

15
 Example
for(i=1;i<arrayname.length;i++) {
/* do something */
}
• index starts at 0 - the first element is omitted
• safer approach - use iterators
while(iterator.hasNext()) {
anOperation(++val);
}
• variable val is incremented too soon
16
 Usual Defects in Algorithms
• operator precedence
– defect
• the programmer either does not use parentheses
where he should, or misplaces them
• such errors may be masked under certain
circumstances
– testing strategies
• tests that avoid masking defects
• example: x*y+z instead of x*(y+z)
– the defect is masked if z = 0 or x = 1
17
 Usual Defects in Algorithms
• using improper standard algorithms
– defect
• inefficient algorithm (there are better alternatives)
– either in the general case or for the specific problem
• algorithm with well-known undesired properties
– same as above
– testing strategies
• the properties of the algorithms must be known

18
 Examples
• inefficient sort algorithm
– most widely used - bubble sort
• inefficient search algorithm
– search time may grow along with the length of
the list
– the position of the element to be found may
have an influence on the search time
• search algorithm that is case sensitive (or
case insensitive) when it shouldn't be
19
Defects in Numerical Algorithms
• using weak data types
– defect
• the variables have too few bits/digits and thus
cannot represent the maximum values in the range
allowed by the application
• an unhandled exception may occur or data may be
stored with wrong values
– testing strategies
• tests with very big numbers

20
Defects in Numerical Algorithms
• not enough digits before/after the decimal point
– defects
• a floating-point value cannot hold enough significant digits
(before/after the decimal point)
• a fixed-point value cannot hold enough significant digits after
the decimal point
– testing strategies
• tests with values of various magnitudes and many digits
• check the correctness of results
– example: stock exchange - four significant digits
($135.5); cannot represent $0.0344
21
Defects in Numerical Algorithms
• wrong ordering of operations
– defect
• very big numbers are represented with insufficient
precision (floating-point)
– testing strategies
• tests numbers whose exponents have big (positive
and negative) values
• tests with numbers of various magnitudes
• check the correctness of results

22
 Example
public class OrderingOperations {
public static void main(String args[]) {
float f = Float.MAX_VALUE;
System.out.println("f = " + f);
int i;
for (i = 0; i <= 1000000; i++)
f -= i;
System.out.println("f = " + f);
}
}
• displayed results: 3.4028235E38 in both cases
23
Defects in Numerical Algorithms
• assuming that two floating-point values are
equal
– defect
• floating-point computing is affected by
approximations
• equality tests should be expressed by small intervals
around the exact values
– testing strategies
• border testing between equivalence classes

24
 Example
for(double d=0.18;d!=18.0;d+=0.02) {
System.out.println("d = " + d); }

• improvement
for(double d=0.18;d<=18.0;d+=0.02)
25
 Coordination Defects
• deadlock and livelock
– defects
• deadlock: two (or more) threads wait for each other
– none can move further until the other one performs a
certain action
• livelock: the system can perform certain actions, but
cannot get out from a certain state

26
 Coordination Defects
• deadlock and livelock (continued)
– testing strategies
• these defects originate in combinations of conditions
which are hard to anticipate or reproduce
• inspection is usually more efficient than testing
• what testing can do
– enforcing different values of time consumption for the
threads
– concurrent running of a large number of threads
– denying access to resources for some threads

27
 Coordination Defects
• race conditions
– defect
• a thread cannot execute normally due to interference
with another thread
– testing strategies
• black-box testing is not successful
• delay one of the threads
• again, inspection can provide better results

28
 Example

A:Thread Data: B:Thread A:Thread Data: B:Thread

set

get get

set

a) Normal b) Abnormal due to delay in thread A

29
Semaphores and Synchronization
• race conditions can be prevented through
mutual exclusion on accessing the shared
resources
– the simplest mechanism - the semaphore
– or other similar variants
• e.g., synchronized methods
• no other thread can access a certain resource until
the synchronized method is finished

30
 Java - Synchronized Methods
A:Thread Data: B:Thread A:Thread Data: B:Thread

get get
waiting for A:
get to complete its
calc calc synchronized
put put operation
calc get
put

calc
put

a) Abnormal: The value put by b) The problem has been solved

thread A is immediately by accessing the data using
overwritten by the value put synchronized methods
by thread B.

31
Defects in Handling Stress Situations
• incorrect or slow response of the system on
minimal configurations
– defect
• on a minimal configuration, the response or the
response time of the system do not meet the
requirements
– testing strategies
• tests on minimal platforms
• tests on platforms with configurations lower than
minimal - must signal they cannot work correctly

32
Defects in Handling Stress Situations
• incompatibility with certain hardware or software
configurations
– defect
• the system fails when running on certain hardware
configurations/operating systems/external libraries
– testing strategies
• run on all configurations available to users
• examples of failures - when using: a different
graphic card; missing fonts; a different version of
the web browser
33
Defects in Handling Stress Situations
• handling extreme load or insufficient resources
– defects
• the system responds poorly when the available resources are
reduced
– memory, disk space, bandwidth, permissions
• the program does not report the problem in a way that is
intelligible to the user
– testing strategies
• ban the access to some resources
• run a large number of instances of the program at the same
time

34
Defects in Handling Stress Situations
• poor resource management
– defect
• the program uses certain resources, but does not
release them when they are no longer needed
– testing strategies
• run the program such that it has to repeatedly release
and reuse the resources
• example: the program opens a series of
files, but does not close them
35
Defects in Handling Stress Situations
• recovery from a severe failure
– defects
• after a hardware error or power loss, the system remains in an
unstable state and cannot fully recover
• the system cannot handle properly the failures of other systems
it is cooperating with
– testing strategies
• force termination of the program at various moments of its
execution
• cut the power during execution
– problem - operating systems do not respond well to such
situations

36
 Documentation Defects
• defect
– user's manual, reference manual or on-line help system
• provide incorrect information
• do not provide relevant information for the problems addressed
• testing strategies
– examine the end-user documentation
– check if all use-cases are well explained to the user

37
Code Inspection
 Inspection
• examine the source code and documentation
• how NOT to do it
– ad-hoc (without planning)
– individually
• by the programmer who wrote the code
• by others
• how to do it
– sistematically
– an inspection team is required
39
 What Is Inspection Good for
• what we can detect
– by testing - failures
– by inspection - defects
• some types of defects are hard to detect by
testing
– synchronization defects
– masked defects
• the two techniques are complementary
40
 The Inspection Team
• the moderator
• the code author
• the secretary
• programmers, testers, etc.

• it is recommended to have no more than 4-5

members

41
 Actions During the Inspection
• the author describes the program's logic
– instruction by instruction
– the participants ask questions
• program analysis
– based on a checklist
– looking for common programming errors

42
 Principles of Inspection (1)
• the expertise of the members must cover
– programming
– software engineering
• documents must be studied before the
meeting
• must avoid discussions about
– the writing style of the program
– fixing the errors - detection is the only goal
43
 Principles of Inspection (2)
• recommended inspection speed
– 200 code lines per hour
– 10 text pages per hour
• code inspection also includes the comments
• logs of the discussions must be kept
• inspection must be carried out again when
changes are made
– especially if changes exceed 20%
44
 The Checklist
• must concentrate on programming errors,
not other aspects
– e.g., writing style of the code
• clearly express the types of errors the team
has to look for
– example: "does the code meet the design
requirements?"
• the question above is useless (too vague)
• in most part - language independent
45
 Data Referencing Errors
• variables read before being initialized
• index outside the array limits
• array index with non-integer values
• pointers used without being allocated
• incorrect read operations for structures (e.g.,
from files)
• structures with different definitions in the
functions that access them
46
 Data Declaration Errors
• variables that have not been explicitly
declared
– for interpreted languages
• variables with similar names
– not exactly an error, but also not recommended
• parameters with implicit values which are
used incorrectly
– e.g., in constructors
47
 Computing Errors (1)
• arithmetic operations performed on
variables of non-arithmetic types
• operations with variables of different types
– these are not errors, but must be carefully
verified
• operations with variables of similar types,
but different sizes
– any kinds of arrays/vectors
• upper/lower overflow
48
 Computing Errors (2)
• division by 0
• variables that can get values outside the
range allowed by the application logic
– e.g., probabilities are floating-point variables
• but may not exceed 1.0
• operator precedence errors
• incorrect use of integer arithmetic

49
 Comparison Errors
• between variables of different types
• between variables of different sizes
• Boolean operators with operands of other
types
• misused logical conditions (and, or, not) in
comparisons
• comparisons affected by the way that the
compiler evaluates Boolean expressions

50
 Flow Control Errors (1)
• infinite loops
– at least for some execution paths
• loops that are never executed
– depending on the input values
– not necessarily errors, but must be carefully
analyzed
• loops controlled by iterators and Boolean
conditions
– again, not necessarily errors
51
 Flow Control Errors (2)
• "off-by-one" errors
• incorrect loop nesting
• instruction blocks incorrectly delimited
– e.g., misplaced open/closed braces
• tests that do not cover all possible values
– e.g., input variable with values between 1 and 5
• if the input value is not between 1 and 4, then it is 5
by default

52
 Interface Errors
• parameters sent in an incorrect order
• writing a parameter that was supposed to be
only read
• global variables with different declarations
within the modules that use them

53
 I/O Errors
• opening files in wrong modes
• not enough memory to store the data read
from the file
• files used prior to being opened
• files not closed after using them
• end-of-file conditions handled incorrectly

54
 Peer-review
• an alternative to inspection
• the team is not gathered
• the team members write reviews to the code
written by others
– the reviewer is unknown to the code author
– allows expressing the suggestions more directly
• the goal is not to find a "responsible" person
– managers are not involved
55
Begin with Testing or Inspection?
• recommended - begin with inspection
– before any kind of testing
– even the one carried out by the developer
• rationale
– allows the quick elimination of many defects
– and thus testing can be focused on hard defects
– if we begin with testing, and then inspection
recommends redesigning the program, the
testing effort was useless
56