Unit testing and code

coverage for C++

1
cppUnit

Coding an individual test

Use assertion macros defined in unit test library
Elements for building tests
TestFixture, TestSuite, TestRunner

2
 Code in an individual test
Use assertion macros defined in cppunit
If assertion fails, an exception is thrown
 CPPUNIT_ASSERT(condition):
 Checks the condition and throws an exception if it's false.
 CPPUNIT_ASSERT_MESSAGE(message, condition):
 Checks the condition and throws an exception and showing specified message if
it is false.
 CPPUNIT_ASSERT_EQUAL(expected,current):
 Checks whether the expected condition is the same as current, and raises an
exception showing the expected and current values.
 CPPUNIT_ASSERT_EQUAL_MESSAGE(message,expected,c
urrent)
 Checks whether the expected is the same as the actual, and raises an exception
showing the expected and current values, and specified message.
 CPPUNIT_ASSERT_DOUBLES_EQUAL(expected,current,delta):
 Checks whether the expected and current difference is smaller than delta. If it 3
fails, the expected and current values are shown.
TestFixture
 Each test should be derived from the abstract
class TestFixture
 Override two functions:
setUp
 operations that would be invoked prior to each individual test,
e.g. create new clean instances of class ready for next test
tearDown
 operations that would be invoked after to each individual test,
e.g. deletes instances

4
TestSuite, TestRunner

TestSuite
Groups a number of tests (using TestFixture) so
that they can all be run
TestRunner
Driver program

5
Test driven development
Test twice, code once
“The style here is to write a few lines of
code, then a test that should run, or even
better, to write a test that won't run, then
write the code that will make it run.”

6
Test driven development – step 0
 Define class Bitmap
Owns an array of unsigned long integers
 Methods
boolean Identity(const Bitmap& other)
 Are we the same Bitmap
 Compare our address with other’s address!
boolean Equals(const Bitmap& other)
 Are we equal?
 What do you mean equal?
• This is the part we test twice – code once!
void Zero()
 Set my bits all to zero.

7
#ifndef __MYBITSCLASS__
#define __MYBITSCLASS__
Class Bitmap
// Code assumes 32-bit unsigned long integers
#define MAXBITS 512
#define NUMWORDS 16
typedef unsigned long Bits;

class Bitmap {
public:
Bitmap();
void Zero(void);
bool Equals(const Bitmap &other) const;
bool Identity(const Bitmap &other) const;
private:
Bits fBits[NUMWORDS];
};
#endif

8
#include "Bitmap.h"
Bitmap::Bitmap() Class Bitmap
{
Zero();
}
void Bitmap::Zero(void)
{
for(int i=0;i<NUMWORDS; i++)
fBits[i] = 0;
}
bool Bitmap::Identity(const Bitmap& other) const
{
return (this == &other);
}
bool Bitmap::Equals(const Bitmap& other) const
{
return Identity(other);
// Incorrect implementation
}

9
Now write a test for Bitmap -- TestFixture
#ifndef BITMAPTEST_H
#define BITMAPTEST_H

#include <cppunit/TestCase.h>
#include <cppunit/extensions/HelperMacros.h>
#include "Bitmap.h"

class BitmapTest : public CppUnit::TestFixture

{
CPPUNIT_TEST_SUITE (BitmapTest);
CPPUNIT_TEST (testEquals);
CPPUNIT_TEST_SUITE_END ();
private:
Bitmap bitmap1;
Bitmap bitmap2;
public:
void setUp();
void tearDown();
protected:
void testEquals();
};

#endif 10
#include "BitmapTest.h"
#include "Bitmap.h"

CPPUNIT_TEST_SUITE_REGISTRATION (BitmapTest);

void BitmapTest::setUp()
{
// Put the bitmaps back in standard state
// (Need at least two as doing things like
// checking equality)
bitmap1.Zero();
bitmap2.Zero();
}

void BitmapTest::tearDown()
{
// Nothing to do
}

11
void BitmapTest::testEquals()
{
// Identity should work
CPPUNIT_ASSERT(bitmap1.Identity(bitmap1));
CPPUNIT_ASSERT(bitmap2.Identity(bitmap2));
CPPUNIT_ASSERT(!bitmap1.Identity(bitmap2));

// At this stage - equality incorrectly defined

// It is identity
// (It shouldn't be!)

// This test should succeed - I'm equal to me

CPPUNIT_ASSERT(bitmap1.Equals(bitmap1));

// so should this
CPPUNIT_ASSERT(bitmap2.Equals(bitmap2));

// But this should fail

CPPUNIT_ASSERT(bitmap1.Equals(bitmap2));
}

12
The Driver -- TestRunner

int main (int argc, char* argv[])

{
// Result collection
CPPUNIT_NS :: TestResult testresult;
CPPUNIT_NS :: TestResultCollector collectedresults;
testresult.addListener (&collectedresults);
CPPUNIT_NS :: TextTestProgressListener tracker;
testresult.addListener(&tracker);

// Setting up runner and running test

CPPUNIT_NS :: TestRunner testrunner;
testrunner.addTest (CPPUNIT_NS :: TestFactoryRegistry :: getRegistry ().makeTest
());
testrunner.run (testresult);

// Reporting
CPPUNIT_NS :: CompilerOutputter compileroutputter (&collectedresults, std::cerr);
compileroutputter.write ();

return collectedresults.wasSuccessful () ? 0 : 1; 13
}
Build and run

!!!FAILURES!!!
Test Results:
Run: 1 Failures: 1 Errors: 0

1) test: BitmapTest::testEquals(F) line: 34

BitmapTest.cc
assertion failed
Expression: bitmap1.Equals(bitmap2)

14
“Refactor”

bool Bitmap::Equals(const Bitmap& other) const

{
for(int i = 0; i < NUMWORDS; i++)
if(this->fBits[i] != other.fBits[i]) return false;
return true;
}

15
Next increment – more functionality in
Bitmap

class Bitmap {
public:
Bitmap();
void Zero(void);
void SetBit(int bitnum);
int Count(void) const;

bool Equals(const Bitmap &other) const;

bool Identity(const Bitmap &other) const;
private:
Bits fBits[NUMWORDS];
};

16
BitmapTest
class BitmapTest : public CppUnit::TestFixture
{
CPPUNIT_TEST_SUITE (BitmapTest);
CPPUNIT_TEST (testEquals);
CPPUNIT_TEST (testSetBit);
CPPUNIT_TEST (testSetBitAndCount);
CPPUNIT_TEST_SUITE_END ();
private:
Bitmap bitmap1;
Bitmap bitmap2;
public:
void setUp();
void tearDown();
protected:
void testEquals();
void testSetBit();
void testSetBitAndCount();
};

17
BitmapTest

Extended equality test

Can now make Bitmaps that are not equal (by
using set bit operation)
Add two more extensive tests on new test
and count operations

18
void BitmapTest::testEquals()
{ BitmapTest
// Identity should work
CPPUNIT_ASSERT(bitmap1.Identity(bitmap1));
CPPUNIT_ASSERT(bitmap2.Identity(bitmap2));
CPPUNIT_ASSERT(!bitmap1.Identity(bitmap2));
CPPUNIT_ASSERT(bitmap1.Equals(bitmap1));
CPPUNIT_ASSERT(bitmap2.Equals(bitmap2));
CPPUNIT_ASSERT(bitmap1.Equals(bitmap2));

// Have some functionality to change the bits

// See if equals works after that
// Set a different bit
bitmap1.SetBit(22);
bitmap2.SetBit(123);
// Shouldn't be equals any more!
CPPUNIT_ASSERT(!bitmap1.Equals(bitmap2));
}
19
void BitmapTest::testSetBit()
{ BitmapTest
// If set bit 0 in one, they should not be equal
bitmap1.SetBit(0);
CPPUNIT_ASSERT(!bitmap1.Equals(bitmap2));
// Just being pedantic - reverse roles in test
CPPUNIT_ASSERT(!bitmap2.Equals(bitmap1));
bitmap1.Zero();
// But should be equal after it got cleared
CPPUNIT_ASSERT(bitmap1.Equals(bitmap2));
CPPUNIT_ASSERT(bitmap2.Equals(bitmap1));
// If set a non-existent bit it isn't supposed
// to change anything
bitmap1.SetBit(-1);
bitmap2.SetBit(513);
CPPUNIT_ASSERT(bitmap2.Equals(bitmap1));
}

20
void BitmapTest::testSetBitAndCount()
{ BitmapTest
// Set some bits and check that count equals number of bits set
int bits1[] = { 0, 3, 17, 21, 33, 54, 68, 77, 91, 103, 211, 304 };
int bits2[] = { 0, 31, 32, 33, 63, 64, 65, 300, 400, 500, 511 };
int count1 = sizeof(bits1)/sizeof(int);
int count2 = sizeof(bits2)/sizeof(int);
for(int i=0;i<count1;i++)
bitmap1.SetBit(bits1[i]);
for(int i=0;i<count2;i++)
bitmap2.SetBit(bits2[i]);
CPPUNIT_ASSERT_EQUAL(count1, bitmap1.Count());
// Deliberate error to get a failed test
CPPUNIT_ASSERT_EQUAL(count1, bitmap2.Count());
}

21
Rebuild, rerun

!!!FAILURES!!! Text
Test Results:
Run: 3 Failures: 1 Errors: 0

1) test: BitmapTest::testSetBitAndCount (F) line: 77

BitmapTest.cc
equality assertion failed
- Expected: 12
- Actual : 11
22
cppunit

Available at
http://sourceforge.net/projects/cppunit

Is installed on Linux machines in lab used

for CSCI222

23
Tutorials …

http://www.comp.nus.edu.sg/~cs3214s/too
ls/cppunitSol.html
http://www.evocomp.de/tutorials/tutorium_
cppunit/howto_tutorial_cppunit_en.html
Documentation with cppUnit
Docs folder in /share/cs-pub/222/testing
“Money” tutorial

24
Code coverage
g++ compiler and
gcov analysis tool

25
Compilation for code coverage

Set extra compile flags

In NetBeans setup, it is simplest just to specify
--coverage
in the "command line" options for both the compilation
and linker steps

26
Run, then use gcov to analyze
 NetBeans can be used to run the application
under test
Best to run several times with different input data so as
to try to test all options
 gcov analysis should be done in terminal
session
cd to directory with C++ source
Identify directory where gcov files (.gcno, .gcda) were
created – should be ./build/Debug/GNU-Linux-x86
Run gcov specifying the data directory and file for
analysis

27
Gcov analysis : function summaries
gcov –o ./build/Debug/GNU-Linux-x86 -f Bitmap.cc
100.00% of 3 source lines executed in function void Bitmap::Zero()
100.00% of 7 source lines executed in function void Bitmap::SetBit(int)
100.00% of 8 source lines executed in function void Bitmap::ClearBit(int)
0.00% of 4 source lines executed in function void Bitmap::SetAs(int, int)
85.71% of 7 source lines executed in function int Bitmap::TestBit(int) const
100.00% of 7 source lines executed in function void Bitmap::FlipBit(int)
100.00% of 2 source lines executed in function void
Bitmap::ReadFrom(std::fstream&)
…
100.00% of 4 source lines executed in function int Bitmap::Equals(const
Bitmap&) const
94.32% of 88 source lines executed in file Bitmap.cc

28
Gcov analysis: branch counts
$ gcov –o ./build/Debug/GNU-Linux-x86 -c Bitmap.cc
…
94.32% of 88 source lines executed in file Bitmap.cc
Creating Bitmap.cc.gcov.
$ cat Bitmap.cc.gcov
#include "Bitmap.h"
Bitmap::Bitmap()
14 {
14 Zero();
}
void Bitmap::Zero(void)
15 {
255 for(int i=0;i<NUMWORDS; i++)
240 fBits[i] = 0;
}
void Bitmap::SetBit(int bitnum)
113 {
113 if((bitnum < 0) || (bitnum >= MAXBITS))
113 return;
113 int word = bitnum / 32;
…

29
Inspect profiles

Profiles and function summaries

immediately reveal untested code.

30