Integration-And System Testing: O O S C

Chapter 11: Chapter 16: Testingand (2/2) IntegrationSystem Testing
Object-Oriented SoftwareConstruction
Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit)
Roadmap for lecture
4.7.2006 (Lecture Testing and Agile Software Development) 6.7.2006 (Practice Talk T-Mobile) 11.7.2006 (Exercise, on demand) 18.7.2006 (Written Exam), 18:00
Object-Oriented Software Construction
Software Lifecycle Activities ...and their models

Requirements Elicitation System Design Object Design Implementation
Analysis
Testing
Expressed in Terms of
Structured by
Realized by
Implemented by
Verified by
class... class... class...
?
class.... ?
Use Case Model
Application Domain Objects
Subsystems
Solution Domain Objects
Source Code
Test Cases
3
Test Team Organizational issues

Professional Tester
Programmer
too familiar with code
Analyst
User
Test Team
System Designer
Configuration Management Specialist
Types of Testing
Unit Testing (last lecture):

- Individual subsystem - Carried out by developers (of components) - Goal: Confirm that subsystems is correctly coded and carries out the intended functionality
Integration Testing (mainly this lecture):

- Groups of subsystems (collection of classes) and eventually the entire system - Carried out by developers - Goal: Test the interface and the interplay among the subsystem
Types of Testing
System Testing:
The entire system Carried out by developers (testers!) Goal: Determine if the system meets the requirements (functional and global) Functional Testing: Test of functional requirements Performance Testing: Test of non-functional requirements
Acceptance and Installation Testing:

Evaluates the system delivered by developers Carried out by the client. Goal: Demonstrate that the system meets customer requirements and is ready to use
Integration Testing Strategy

Test (sub-)systems (cluster) for problems that arise from subsystem interactions Assumption:
The entire system is viewed as a collection of subsystems determined during the system and object design. System Decomposition is hierarchical
The order in which the subsystems are selected for testing and integration determines the testing strategy
Big bang integration (Nonincremental) Bottom up integration Top down integration Sandwich testing Variations of the above
For the selection use the system decomposition from the System Design
Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 7
Example: Three Layer Call Hierarchy

User Interface
Layer III
Billing System
Event Service
Learning
Layer II
Database
Network Access
Neural Network
Layer I
Integration Testing: Big-Bang Approach

Unit Test A Unit Test B Unit Test C Unit Test D

Dont try this!
System Test
All components (units) are first tested individually and then together as a single and entire system: Pros:
No test stubs (mocks) and drivers are needed
Cons:
Difficult to pinpoint the specific component responsible for the failure

Object-Oriented Software Construction 9
Results in Strategies that integrate only a few components at the time

Bottom-up Testing Strategy
The subsystem in the lowest layer of the call hierarchy are tested individually
Infrastructure is tested first
Then the next subsystems are integrated and tested from the next layer up that call the previously tested subsystems
Increment one subsystem at a time Order of integration depends on importance of subsystem etc.
This is done repeatedly until all subsystems are included in the testing
Regression Tests: Rerun previous tests
Only Test Drivers are used to simulate the components of higher layers No Test Stubs!
Bottom-up Integration
A Layer III
Test E Test B, E, F Test F Test C

E
Layer I
Layer I
Test A, B, C, D, E, F, G
Test D,G Test G
11
Pros and Cons of bottom up integration testing
Pros:
Interface faults can be more easily found (the usage of test drivers accomplishes a clear intention of the underlying interfaces of the lower layer) No Stubs are necessary
Cons:
Components of the User Interface are tested last Test cases often hard to derive Faults found in the top layer may lead to changes in the subsystems of lower layers, invalidating previous tests.
12
Top-down Testing Strategy

Test the top layer of the controlling subsystem first

The skeleton of the program is tested
Then combine all the subsystems that are called by the tested subsystems and test the resulting collection of subsystems
Increment one subsystem at a time
Do this until all subsystems are incorporated into the test Test Stubs are used to simulate the components of lower layers that have not yet been integrated. No drivers are needed
13
Top-down Integration Testing

A Layer III
Layer I
Layer I
Test A
Test A, B, C, D
Test A, B, C, D, E, F, G
Layer III Layer III + II All Layers

Pros and Cons of top-down integration testing
Pros:
Test cases can be defined in terms of the functionality of the system (functional requirements) More effective for finding faults that are visible to the user
Cons:
Writing stubs can be difficult: Stubs must allow all possible conditions to be tested. Possibly a very large number of stubs may be required
15
Sandwich Testing Strategy

Combines top-down strategy with bottom-up strategy (parallel testing is possible) The system is view as having three layers
A target layer in the middle A layer above the target (top layer) A layer below the target (bottom layer) Testing converges towards the target layer
No Test Stubs and Drivers are necessary for bottom and top layer
16
Sandwich Testing Strategy

A Layer III
Layer I
Test E
E F G Layer I
Bottom Layer Tests
Test B, E, F Test F Test A, B, C, D, E, F, G
Test D,G Test G Test A,B,C, D Test A

Top Layer Tests
17
Pros and Cons of Sandwich Testing
Pros:
Top and Bottom Layer Tests can be done in parallel No Stubs and Drivers (saves development time)
Cons:
System implementation needs to be finished Does not test the individual subsystems on the target layer thoroughly before integration (C in the example)
Solution: Modified sandwich testing strategy
18
Modified Sandwich Testing Strategy
Tests the three layers individually before combining them in incremental tests with one another The individual layer tests consists of three tests:
Target layer test with drivers and stubs Top layer test with stubs Bottom layer test with drivers
The combined Layer Tests consist of two tests:

Top layer accessing target layer (top layer replaces drivers) Bottom accessed by target layer (bottom layer replaces stubs)
19
Modified Sandwich Testing Strategy

A Layer III
Test A
Test A,B Test A,C

E
B F
Layer II
Layer I
Test B Test C Test D
Test A,D
Test A, B, C, D
Test E Test F Test G
Test D,G Test A, B, C, D, E, F, G

Test B, E, F
Using the Bridge Pattern to enable early Integration Testing

Usage of Design Patterns supports testing Use the bridge pattern to provide multiple implementations under the same interface.
Client
DBInterface
DBImplementation
Stub Code
Oracle
MySQL
21
Summary
Testing Testing is is still still a a black black art, art, but but many many rules rules and and heuristics heuristics
are are available available unit Testing Testing consists consists of of component-testing component-testing (( unit testing, testing, integration integration testing) testing) and and system system testing testing Design Design Patterns Patterns can can be be used used for for integration integration testing testing Testing Testing has has its its own own lifecycle lifecycle
22
Excurse: Agile Development

Agile Development Extreme Programming (XP)
23
Agile Software Development:
Different Methods and Approaches
Overall problem in software production:

Rapidly changing requirements Complexity (software, models, notations)
Dissatisfaction with the overheads involved in design methods led to the creation of agile methods. Core issues:
Focus on the code rather than the design; Are based on an iterative approach to software development; Are intended to deliver working software quickly and evolve this quickly to meet changing requirements.
24
Agile Software Development:

The Agile Software Development Manifesto

Permanent Customer Involvement
Role: Provide and prioriterize new system requirements
Incremental delivery
Software is developed in increments
People not process

Skills of team (members) should be appreciated Should be left to work with their own methods, tools etc.
Embrace change
Expect system requirements to change, so design the system to accommodate these
Maintain simplicity
Focus on simplicity in both software and development process Work together in eliminate complexity
http://agilemanifesto.org/
Problems with agile methods

z
It can be difficult to keep the interest of customers who are involved in the process. Team members may be unsuited to the intense involvement that characterizes agile methods.
-
In particular shy and reserved people (nerds?)
Prioritizing requirements can be difficult when there are multiple stakeholders. Maintaining simplicity requires extra work.
26
Extreme Programming (Beck, 2000)

Perhaps the best-known and most widely used agile method. Extreme Programming (XP) takes an extreme approach to iterative development.
New versions may be built several times per day; Increments are delivered to customers every 2 weeks; All tests must be run for every build and the build is only accepted if tests run successfully.
Programming in pairs Continuously re-prioritizing of requirements (client, customer, users) Client is part of development team and in charge
Life Cycle of XP
In XP, user requirements are expressed as scenarios or user stories. These are written on cards and the development team break them down into implementation tasks. These tasks are the basis of schedule and cost estimates. The customer chooses the stories for inclusion in the next release based on their priorities and the schedule estimates.
How does XP work? (1)
The planning game

long-term plans only diffuse Details in short-terms (some days) customer is involved in every development cycle (some weeks) tasks are handed out according to skills
Short development cycles

enforce decomposition into small tasks minimize risk better integration of customer
Simple Design
Design takes into account only short-term goals complexity is low
Sustainable amount of time

Large amounts of overtime are not considered acceptable (reduce
quality)
29
Tests
Unit tests: For implementation details, are developed by the developers themselves, are designed before development Test-first development enhances trust in system
Refactoring
allow for continuous changes in design Keep code simple and maintainable
Programming in pairs
always pair-programming changing pairs (daily) distributes knowledge within the team
30
Collective ownership: Source code belongs to the team

everybody can make changes (consultation) no fixed responsibilities pairs work on all areas, no islands of expertise
Continuous integration
integration several times a day system is error-free every evening
Standards (Coding)
is commonly accepted
Customer agent is part of the team

available (in case of questions) works out feature tests
31
Conclusions (XP)
Advantages
Flexible Quick Customer-centered Team-oriented
Disadvantages
little documentation (only tests und code) Scalability problems (only small and medium projects) Depends strongly on team members
32

Integration-And System Testing: O O S C

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Integration-And System Testing: O O S C

Uploaded by

Copyright:

Available Formats

Chapter 11: Chapter 16: Testingand (2/2) IntegrationSystem Testing

Roadmap for lecture

Object-Oriented Software Construction

Software Lifecycle Activities ...and their models

class... class... class...

Use Case Model

Application Domain Objects

Solution Domain Objects

Object-Oriented Software Construction

Test Team Organizational issues

too familiar with code

Configuration Management Specialist

Object-Oriented Software Construction

Unit Testing (last lecture):

Integration Testing (mainly this lecture):

Object-Oriented Software Construction

Acceptance and Installation Testing:

Object-Oriented Software Construction

Integration Testing Strategy

Example: Three Layer Call Hierarchy

Object-Oriented Software Construction

Integration Testing: Big-Bang Approach

Dont try this!

Difficult to pinpoint the specific component responsible for the failure

Results in Strategies that integrate only a few components at the time

Bottom-up Testing Strategy

Test E Test B, E, F Test F Test C

Test D,G Test G

Object-Oriented Software Construction

Pros and Cons of bottom up integration testing

Object-Oriented Software Construction

Top-down Testing Strategy

Test the top layer of the controlling subsystem first

Object-Oriented Software Construction

Top-down Integration Testing

Layer III Layer III + II All Layers

Pros and Cons of top-down integration testing

Object-Oriented Software Construction

Sandwich Testing Strategy

Object-Oriented Software Construction

Sandwich Testing Strategy

Bottom Layer Tests

Test B, E, F Test F Test A, B, C, D, E, F, G

Test D,G Test G Test A,B,C, D Test A

Top Layer Tests

Pros and Cons of Sandwich Testing

Solution: Modified sandwich testing strategy

Object-Oriented Software Construction

Modified Sandwich Testing Strategy

The combined Layer Tests consist of two tests:

Object-Oriented Software Construction

Modified Sandwich Testing Strategy

Test A,B Test A,C

Test B Test C Test D

Test E Test F Test G

Test D,G Test A, B, C, D, E, F, G

Using the Bridge Pattern to enable early Integration Testing

Object-Oriented Software Construction

Object-Oriented Software Construction

Excurse: Agile Development

Agile Development Extreme Programming (XP)

Object-Oriented Software Construction