Unit Testing with

JUnit and CppUnit

Software Testing
Fundamentals (1)
z What is software testing?
z The process of operating a system or component under
specified conditions, observing or recording the results,
and making an evaluation of some aspect of system or
component. (IEEE Definition)
z Why do we test?
z To ensure the quality and satisfaction of the product
z To gain the confidence in the correctness of the product
z Testing vs. debugging
z Testing is to show that a program has bugs
z Debugging is to locate and correct the error or
misconception that cause the program failures
2
Software Testing
Fundamentals (2)
z Software Testing Techniques
z White-box testing
z The tester has access to the details of the
program under test and performs the testing
according to such details.
z Examples: path testing, branch testing
z Black-box testing
z Black-box testing treats the program under test as
a “black box.” No knowledge about the
implementation is assumed.
z Examples: equivalent partitioning, boundary value
analysis
3

Software Testing
Fundamentals (3)
z Testing should begin “in the small” and progress
toward testing “in the large”
z Unit testing
z Concentrates on each unit (i.e., component) of the software
z Integrated testing
z Building a system from its components and testing the
resultant system for detecting component interaction
problems
z System testing
z Concern with testing an increment to be delivered or entire
system
z Acceptance testing
z Performed by the clients/users to confirm that the product
meets the business requirements
4
Unit Testing
z Unit Testing
z Is normally considered as an adjunct to the coding step
z Focuses verification effort on the smallest unit of software
design – the software component or module
z Using the component-level design description as a guide
z Provide a release criterion for a programming task
z Unit Testing in the OO Context
z Smallest testable unit is the encapsulated class
z Conducting class testing (i.e., unit testing)
z Methods within the class are tested
z The state behavior of the class is examined

Unit Testing Example:

Statement coverage (1)
class X { Function under test
. . .
void f() {
. . .
. . . Test case(s)
} class XTest {
}; . . .
void testf() {
X x;
x.f();
Statement coverage: CPPUNIT_ASSERT(…);
are all statements }
executed? . . .
};
6
Unit Testing Example:
Statement coverage (2)
class X {
. . .
void f(bool x) {
. . .
if (x) {
. . . class XTest {
} . . .
void testf() {
. . .
X x;
} x.f(false);
}; CPPUNIT_ASSERT(…);
There are unexecuted
}
statements
. . .
};
7

Unit Testing Example:

Statement coverage (3)
class X {
. . .
void f(bool x) {
. . .
if (x) {
. . . class XTest {
} . . .
void testf() {
. . .
X x;
} x.f(true);
}; CPPUNIT_ASSERT(…);
All statements
}
are executed
. . .
};
8
 Unit Testing Example: Branch
coverage (1)
class X {
. . .
void f(bool x) {
. . .
if (x) {
Unexercised . . . class XTest {
} . . . All statements
void testf() { are executed
. . .
X x;
} x.f(true);
}; CPPUNIT_ASSERT(…);
}
Brach coverage:
. . .
are all control
};
transfer exercised?
9

Unit Testing Example: Branch

coverage (2)
class X {
. . .
void f(bool x) { Two test cases
. . .
if (x) { class XTest {
. . . . . . Is reusing x
} void testf() { a good idea?
X x;
. . .
x.f(true);
}
CPPUNIT_ASSERT(…);
}; . . .
All control
x.f(false);
transfers are
CPPUNIT_ASSERT(…);
exercised
}
. . .
10
};
Unit Testing Example: Branch
coverage (3)
class X {
. . .
void f(int x) {
. . .
do {
x--;
. . . class XTest {
} . . .
while (x>0); void testf() {
. . .
X x;
x.f(0);
}
CPPUNIT_ASSERT(…);
Statement
}; coverage? Yes }
. . .
Branch coverage? No }; 11

Advantages of Unit Testing

z Write test = understand better what has to be done
z Silly bugs appear immediately = less time spent on
debugging
z Provide a working specification of your functional
code
z Speed-up the development – write test once, use
test to find errors many times
z Gain confidence in your code
z Repeatable and deterministic
z Regression tests – incorrect changes discovered
immediately; unautomated refactoring enabled
Encouraged by eXtreme Programming community
12
Old-fashioned Low-level
Testing
z Stepping through a debugger
z drawbacks:
1. not automatic
2. time-consuming

z Littering code with stream output calls

z drawbacks:
1. makes code ugly
(ok, nowadays you could use aspects to avoid it;-))

2. generates to much information

13

What is xUnit?
z An automated unit test framework
z Provides the Driver for unit(s)
z Provides automatic test runs
z Provides automatic result checks
z Available for multiple languages:
z JUnit (from Kent Beck (XP) and Erich Gamma(Gang of Four))
z cppUnit
z httpUnit
z NUnit
z …
14
The Goals of JUnit
z To write a framework within which we have some
glimmer of hope that developers will actually write
tests.
z The framework has to use familiar tools, so there is little
new to learn.
z The second goal of testing is creating tests that
retain their value over time.
z Someone other than the original author has to be able to
execute the tests and interpret the results.
z Creating a setup or fixture is expensive
z A framework has to enable reusing fixtures to run different
tests.
15

The Design of JUnit

z Patterns Generate Architectures
z Used to present the design of JUnit.
z The idea is to explain the design of a system by starting
with nothing and applying patterns, one after another, until
you have the architecture of the system.
z Presentation flow
(1) Present the architectural problem to be solved
(2) Summarize the pattern that solves it
(3) Show how the pattern was applied to JUnit.
z The details can refer to JUnit A Cook's Tour
http://junit.sourceforge.net/doc/cookstour/cookstour.htm

16
The Patterns Used in JUnit

17

How to Use JUnit (1)

z Write a test case (Fixture)
z Create your own test case as a subclass of JUnit TestCase
z Add an instance variable for each known object in the fixture
z Override the setUp() method to initialize object(s) under test
z Override the tearDown() method to release object(s) under test
z Run the test
z Define a public test???() method for exercising the object(s)
under test
z Verify the result
z Assert expected result of the test case using assertEquals(),
assertTrue, …
z Clean up the fixture
z through the tearDown() method

18
How to Use JUnit (2)
z Suite management
z A test suite is a collection of test cases that are intended to be
used to show that a program under test has some specified set of
behaviors
z Write a static suite() containing all the test???() in the fixture
class
z Define a main() method that runs the TestCase in batch mode
z Error vs. Failures
z Error: unanticipated problem like an
ArrayIndexOutOfBoundsException
z Failure: is anticipated and can be checked with assertions

19

JUnit FAQ: Best Practices

z Test-first programming
z Tests should be written before the code.
z Test-first programming is practiced by only writing new
code when an automated test is failing.
z When all the tests pass, you know you're done!
z When a bug is reported, first write unit test(s) to expose the
bug(s), then fix them.
z This makes it almost impossible for that particular bug to
resurface later.
z Good tests tell you how to best design the system for its
intended use.
z Test-driven development is a lot more fun than writing tests
after the code seems to be working.
20
JUnit FAQ: Best Practices
z Do I have to write a test for everything?
z No, just test everything that could reasonably break.
z Investments in testing are equal investments in design.
z If defects aren't being reported, and your design responds
well to change, then you're probably testing enough.
z If you're spending a lot of time fixing defects and your
design is difficult to grow, you should write more tests.
z If something is difficult to test, it's usually an opportunity for
a design improvement.

21

JUnit FAQ: Best Practices

z How simple is “too simple to break”?
z If it can't break on its own, it's too simple to break.
z Example
z getX() method cannot break unless the compiler
is also broken. Therefore, don't test getX().
z setX() method is also too simple to break.
However, if it does any parameter validation, you
likely need to test it.

22
JUnit FAQ: Best Practices
z How often should I run my tests?
z Run all your unit tests as often as possible
z Ideally every time the code is changed.
z Make sure all your unit tests always run at 100%.
z Frequent testing gives you confidence that your changes
didn't break anything.
z For larger systems, you may just run specific test suites
that are relevant to the code you're working on.
z Run all the tests of the a system at least once per day (or
night).

23

JUnit FAQ: Best Practices

z What do I do when a defect is reported?
z Write a failing test that exposes the defect
z When the test passes, you know the defect is fixed!

z This is a learning opportunity

z Perhaps the defect could have been prevented by being more
aggressive about testing everything that could reasonably break.
z Why not just use print?
z It requires that output be scanned manually every time the
program is run to ensure that the code is doing what's expected.
z Tests should retain its value over time.
z Why not just use a debugger?
z The same as that of using print.
24
JUnit FAQ: Best Practices
z Testing Idioms
z Code a little, test a little, code a little, test a little...
z Begin by writing tests for the areas of code that you're most
worried about breaking.
z Write tests that have the highest possible return on your
testing investment.
z When you need to add new functionality to the system,
write the tests first.
z If you find yourself debugging using System.out.println(),
write a test case instead.
z The next time someone asks you for help debugging, help
them write a test.
z Don't deliver software that doesn't pass all of its tests.
25

CppUnit Cookbook (1)

z Simple Test Case (Ordinarily, you will not use such simple test)
z Subclass the TestCase class.
z Override the method runTest().
z When you want to check a value, call CPPUNIT_ASSERT(bool)
and pass in an expression that is true if the test succeeds

class ComplexNumberTest : public CppUnit::TestCase {

public:
ComplexNumberTest( std::string name ) :
CppUnit::TestCase( name ) {}

void runTest() {
CPPUNIT_ASSERT( Complex (10, 1) == Complex (10, 1) );
CPPUNIT_ASSERT( !(Complex (1, 1) == Complex (2, 2)) );
}
};
26
CppUnit Cookbook (2)
z Fixture - a known set of objects served as a base for a set of test cases
z To add new tests
z Add member variables for each part of the fixture
z Override setUp() to initialize the variables
z Override tearDown() to release any resources allocated in setUp()

class ComplexNumberTest : public CppUnit::TestFixture {

private:
Complex *m_10_1, *m_1_1, *m_11_2;
protected:
void setUp()
{
m_10_1 = new Complex( 10, 1 );
m_1_1 = new Complex( 1, 1 );
m_11_2 = new Complex( 11, 2 );
}
void tearDown()
{
delete m_10_1;
delete m_1_1;
delete m_11_2;
} 27
};

CppUnit Cookbook (3)

z How do you write and invoke individual tests using a fixture?
z Write the test case as a method in the fixture class
z Create a TestCaller which runs that particular method
class ComplexNumberTest : public CppUnit::TestFixture {
private:
Complex *m_10_1, *m_1_1, *m_11_2;
protected:
void setUp() {...}
void tearDown() {...}
void testEquality()
{
CPPUNIT_ASSERT( *m_10_1 == *m_10_1 );
CPPUNIT_ASSERT( !(*m_10_1 == *m_11_2) );
}
void testAddition()
{
CPPUNIT_ASSERT( *m_10_1 + *m_1_1 == *m_11_2 );
}
};
CppUnit::TestCaller<ComplexNumberTest> test("testEquality",
&ComplexNumberTest::testEquality );
CppUnit::TestResult result;
test.run( &result ); 28
CppUnit Cookbook (4)
z Use TestSuite class that runs any number of TestCases together
CppUnit::TestSuite suite;
CppUnit::TestResult result;
suite.addTest( new CppUnit::TestCaller<ComplexNumberTest>(
"testEquality",
&ComplexNumberTest::testEquality ) );
suite.addTest( new CppUnit::TestCaller<ComplexNumberTest>(
"testAddition",
&ComplexNumberTest::testAddition ) );
suite.run( &result );

z TestSuite can contain any object implementing the Test interface

CppUnit::TestSuite suite;
CppUnit::TestResult result;
suite.addTest( ComplexNumberTest::suite() );
suite.addTest( SurrealNumberTest::suite() );
suite.run( &result );
29

CppUnit Cookbook (5)

z Run TestSuite using TestRunner
z Using Helper Macros
z Defined in cppunit/extensions/HelperMacros.h which must be
integrated for initiating and finishing a test suite
(CPPUNIT_TEST_SUITE and CPPUNIT_TEST_SUITE_END)
z Rewrite the fixture to include the HelperMacros.h
#include <cppunit/extensions/HelperMacros.h>
class ComplexNumberTest : public CppUnit::TestFixture
{...}

z TestSuite can be created with the HelperMacros

CPPUNIT_TEST_SUITE( ComplexNumberTest );
CPPUNIT_TEST( testEquality );
CPPUNIT_TEST( testAddition );
CPPUNIT_TEST_SUITE_END();
30
CppUnit Cookbook (6)
z Run TestSuite using TestRunner
#include <cppunit/ui/text/TestRunner.h>
#include "ComplexNumberTest.h"
int main( int argc, char **argv)
{
CppUnit::TextUi::TestRunner runner;
CPPUNIT_TEST_SUITE( ComplexNumberTest );
CPPUNIT_TEST( testEquality ); ...
CPPUNIT_TEST_SUITE_END();
bool wasSuccessful = runner.run();
return wasSuccessful ? 0 : 1;
}

z If all the tests pass, you'll get an informative message.

z If any fail, you'll get the following information:
z The name of the test case that failed
z The name of the source file that contains the test
z The line number where the failure occurred
z All of the text inside the call to CPPUNIT_ASSERT() which detected the 31
failure

CppUnit Cookbook (7)

z The classes of the CppUnit-framework are placed in a
namespace which is defined by the macro CPPUNIT_NS
z CppUnit macros
z CPPUNIT_ASSERT
z Check whether the passed expression returns the value True
z CPPUNIT_ASSERT_EQUAL
z Check whether the first parameter is like the second one
z CPPUNIT_ASSERT_THROW
z Check whether the passed expression throws an exception of the
passed type
z CppUnit Examples
z See LinkedListTest

32
Unit testing and
Test-driven Development (TDD)
z How to write JUnit/CPPUnit tests? Do it TDD!
z Workflow:
1. Think about functionality to be implemented &
scenarios in which the unit to be tested will play a role.
2. Create a stub of the unit you want to implement.
3. Write a test for each scenario and/or use of the unit.
4. Make the unit fail the tests (nothing is implemented
yet!).
5. Develop the unit until it passes every test.
z Encountered new scenario/use?
z make a test before implementing it!

33

Test-driven Development (TDD)

z Test-driven development (TDD) is an evolutionary
approach to development which combines
z Test-first development
z Write a test before you write just enough production code to
fulfill that test
z Refactoring
z The goal of TDD
z Think through your design before your write your functional
code
z To write clean code that works
z TDD is primarily a design technique with a side effect of
ensuring that your source code is thoroughly unit tested
34
References
z W.-K Chen, “Object-Oriented Programming -
Software testing”
z JUnit A Cook's Tour.
http://junit.sourceforge.net/doc/cookstour/cookstour.
htm
z CppUnit Cookbook.
http://cppunit.sourceforge.net/doc/1.8.0/cppunit_coo
kbook.html
z Krzysztof Pietroszek, “Unit testing with JUnit and
CPPUnit”.
http://swen.uwaterloo.ca/~se2/project/project-
junit.pdf

35