Chapter 13 & 14

Software Testing Strategies and Techniques

Software Engineering: A Practitioners Approach, 6th edition by Roger S. Pressman
1

Software Testing
Testing is the process of exercising a program with the specific intent of finding errors prior to delivery to the end user.

What Testing Shows

errors requirements conformance performance an indication of quality

Who Tests the Software?

developer
Understands the system but, will test "gently" and, is driven by "delivery"

independent tester
Must learn about the system, but, will attempt to break it and, is driven by quality
4

Validation vs Verification
 Verification Are we building the product right?
Is the code correct with respect to its specification?

 Validation Are we building the right product?

Does the specification reflect what it should?

Testing Strategy
unit test integration test

system test

validation test
6

Testing Strategy
 Begin with unit testing and work your way up to system testing.  Unit testing test individual components (modules in procedural languages; classes in OO languages)  Integration testing test collections of components that must work together  Validation testing test the application as a whole against user requirements  System testing test the application in the context of an entire system

Unit Testing
module to be tested
results software engineer test cases

Unit Testing
module to be tested interface local data structures boundary conditions independent paths error handling paths

test cases

Unit Test Environment

driver
interface local data structures

Module

boundary conditions independent paths error handling paths

stub

stub

test cases RESULTS

10

Integration Testing Strategies

Options: the big bang approach an incremental construction strategy

11

Top Down Integration

A top module is tested with stubs G

stubs are replaced one at a time, "depth first" as new modules are integrated, some subset of tests is re-run re-

12

Bottom-Up Integration
A

drivers are replaced one at a time, "depth first" worker modules are grouped into builds and integrated

cluster

13

Regression Testing
 The selective retesting of a modified system to help ensure that no bugs have been introduced during modification.
Fixing one part of the code can break another

14

High Order Testing

       Validation testing
Focus is on software requirements

System testing
Focus is on system integration

Alpha/Beta testing
Focus is on customer usage

Recovery testing
forces the software to fail in a variety of ways and verifies that recovery is properly performed

Security testing
verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration

Stress testing
executes a system in a manner that demands resources in abnormal quantity, frequency, or volume

Performance Testing
test the run-time performance of software within the context of an integrated system

15

What is a Good Test?

 A good test is one that has a high probability of finding an error.

16

Test Case Design

"Bugs lurk in corners and congregate at boundaries ..." Boris Beizer OBJECTIVE CRITERIA CONSTRAINT to uncover errors in a complete manner with a minimum of effort and time
17

Exhaustive Testing

loop < 20 X

There are 10 possible paths! If we execute one test per millisecond, it would take 3,170 years to test this program!!

14

18

Selective Testing
Selected path

loop < 20 X

19

Software Testing
white-box methods black-box methods

Methods Strategies

20

White-Box Testing

... our goal is to ensure that all statements and conditions have been executed at least once ...
21

Why Cover?
logic errors and incorrect assumptions are inversely proportional to a path's execution probability we often believe that a path is not likely to be executed; in fact, reality is often counter intuitive typographical errors are random; it's likely that untested paths will contain some
22

Basis Path Testing

First, we compute the cyclomatic complexity: number of simple decisions + 1 or number of enclosed areas + 1 In this case, V(G) = 4
23

Cyclomatic Complexity
A number of industry studies have indicated that the higher V(G), the higher the probability or errors.

modules

V(G) modules in this range are more error prone

24

Basis Path Testing

Next, we derive the independent paths:
1

2 3 4 5 6

Since V(G) = 4, there are four paths Path 1: Path 2: Path 3: Path 4: 1,2,3,6,7,8 1,2,3,5,7,8 1,2,4,7,8 1,2,4,7,2,4,...7,8

Finally, we derive test cases to exercise these paths.

25

Basis Path Testing Notes

you don't need a flow chart, but the picture will help when you trace program paths count each simple logical test, compound tests count as 2 or more basis path testing should be applied to critical modules
26

Black-Box Testing
requirements

output

input

events
27

Equivalence Partitioning
user queries output formats prompts FK input data

mouse picks

28

Sample Equivalence Classes

Valid data user supplied commands responses to system prompts file names computational data physical parameters bounding values initiation values output data formatting responses to error messages graphical data (e.g., mouse picks) Invalid data data outside bounds of the program physically impossible data proper value supplied in wrong place
29

Boundary Value Analysis

user queries output formats prompts FK input data

mouse picks

input domain

output domain
30

OOT Methods: Behavior Testing

The tests to be designed should achieve all state coverage [KIR94]. That is, the operation sequences should cause the Account class to make transition through all allowable states
open e mp t y ac c t s etup Ac c nt s et up ac c t

de p o s it (in it ia l) d e p o s it w o rk in g ac c t wit h d ra w

b a lan c e c re d it ac c ntInfo

wit h draw a l (f ina l)

dead ac c t

c lo s e

n o nw o rk in g ac c t

Figure 1 4 .3 St at e diagram f or A ccount class ( adapt ed f rom [ KIR9 4 ] )

31