CS 308: Large Applications Practicum

Week 5

Varun Dutt
School of Computing and Electrical Engineering
School of Humanities and Social Sciences
Indian Institute of Technology Mandi, India

Scaling the Heights

1
 Software Testing

Motivation: “Coders love to code but hate to test!” –

Software testing is considered a waste of time, energy and
resources but that is so NOT true…

Software testing is the process of verifying and validating

that a software program/application/product:
ü meets the business and technical requirements that guided
its design and development
ü works as expected
It is an investigation conducted to provide stakeholders with
information about quality of product or service under test.
 Verification and Validation (CMMI-
SW v1.1)
Verification is the process of evaluating a system or
component to determine whether the products of a given
development phase satisfy the conditions imposed at the
start of that phase
Verification is to ask “Have we built the software right?”

Validation is the process of evaluating a system or

component during or at the end of the development
process to determine whether it satisfies specified
requirements, i.e., is it what the customer wants
Validation is to ask “Have we built the right software?”
 Software Testing
CAUTION!
ü Testing can never completely identify all the defects within
software.
ü Instead, it furnishes a criticism or comparison that
compares the state and behavior of the product against
oracles like specifications, contracts, comparable products,
past versions of the same product, inferences about
intended or expected purpose, user or customer
expectations, relevant standards, applicable laws etc.
 Finding faults early has value
Earlier a defect is found, cheaper it is to fix it!
The illustration below states that if a problem in the
requirements is found only post-release, then it would cost
10–100 times more to fix than if it had already been found
by the requirements review.
 Functional vs. Non-Functional Testing

Functional testing refers to activities that verify a specific

action or function of the code
ü usually found in the code requirements documentation
ü tends to answer the question of “can the user do this” or
“does this particular feature work”

Non-functional testing refers to aspects of the software that

may not be related to a specific function or user action,
such as scalability or security
ü tends to answer such questions as "how many people can
log in at once"
 Static vs. Dynamic Testing
Reviews, walkthroughs, or inspections are considered as
static testing
ü Investigation without operation
ü Pencil and paper kind of reviews
ü Cost effective
ü What some of you did in the software engineering class…
Actually executing programmed code with a given set of
test cases is referred to as dynamic testing
ü Takes longer time
ü Might catch bugs that might be skipped in static testing
ü What all of you will be doing in this class…
 Testing Methods: The Box Approach!

Testing methods are divided into three categories based on

point of view that a test engineer takes when designing:

ü White box testing: when the tester has access to the

internal data structures and algorithms including the code
that implement these e.g. unit testing, code coverage
testing, mutation testing methods, static testing etc. (Focus
of this class)

.
 Testing Methods: The Box Approach!

ü Black Box Testing: Black box testing treats the software

as a "black box"—without any knowledge of internal
implementation, e.g., boundary value analysis,
specification-based testing

ü Grey Box Testing: Involves having knowledge of internal

data structures and algorithms for purposes of designing
the test cases; but, still testing at the black-box level, e.g.,
reverse engineering to determine, for instance, boundary
values or error messages
 Testing Levels
Unit testing
ü Tests that verify the functionality of a specific section of code, usually at
the function level or at class level in an object-oriented environment
ü Ensures that the specific individual function or module is working as
expected in the code, does not verify integrity of the whole software

Integration testing
ü Testing that verifies the interfaces between components against a
software design
ü Tries to expose defects in the interfaces and interaction between
integrated components (modules)

Regression testing
ü Focuses on finding defects after a major code change has occurred
ü Uncover software regressions, or old bugs that have come back
 Testing Process
Traditional Waterfall development model

ü Testing is performed by an independent group of testers after the

functionality is developed, before it is shipped to the customer

Agile or (Extreme Programming) development model

ü Adheres to a "test-driven software development" model
ü Unit tests are written first, which are expected to fail initially but
as code is written it passes incrementally larger portions of the
test suites
ü The test suites are continuously updated as new failure
conditions and corner cases are discovered, and they are
integrated with any regression tests that are developed
Basic Software Testing Architecture

UNIT TESTING
CODE (.Java TOOL (e.g.,
file) JUnit)

COVERAGE MUTATION
TOOL (e.g., TESTING TOOL
Emma) (e.g., Jester)
JUnit: A popular unit testing tool for Java

- JUnit is a simple framework for writing and

running automated tests for Java.
- It has been extremely important in the
development of test-driven development.
- JUnit uses annotations and methods to
write test cases. Typically these test
methods are contained in a class which is
only used for testing. The class is typically
called a Test class.
13
Installation and Running JUnit

• Eclipse (a popular Java IDE ~ aka Java

editor) allows you to use JUnit as an
integrated component in Eclipse.
• In fact, eclipse supports the creation of
JUnit tests via wizards.

14
 Writing and Running JUnit Tests

Create a new project. Add a class file to it.

Create a new project in Eclipse. Add a .java file to it.
Let us consider the scenario in which- this is the class, we want to test.
Let us consider the scenario in which- this is the class, we want to test.

15
 Writing and Running JUnit Tests

Install JUnit in Eclipse:

- First right click the project folder name

- Now, choose Properties
- In Properties, choose Java Build Path
- Click the Libraries tab
- Click the Add External JARs
- And choose, Junit.jar from
\plugins\org.junit_4.11.0.v201303080030 folder
- Hit ok are you are all set to use JUnit
Adding JUnitTestCase

To an existing Project, add a JUnit TestMultiply.java suite file via Package Explorer.
Write the code in the file as mentioned below and then run the test case by right
clicking the TestMultiply.java file and doing Run As and JUnit Test.
Name of the method must begin with "test"

17
Output of the Test

18
Other Annotations

19
Assert Statements

• JUnit provides static methods in the Assert

class to test for certain conditions.
• These assertion methods typically start with
assert and allow you to specify the error
message, the expected and the actual
result.
• An assertion method compares the actual
value returned by a test to the expected
value, and throws an AssertionException if
the comparison test fails. 20
Assert Test Methods

21
Example

JAVA CLASS

JUNIT CLASS

JUNIT OUTPUT 22
Code Coverage: Emma

• In addition to writing test cases, it is

important to know the code coverage
provided by a test case
• Code coverage refers to the number of
lines of code that is executed by the test
case
• A popular tool to check coverage is Emma,
which can be installed in Eclipse very
quickly (the version for Eclipse is called
EclEmma) 23
Installing EclEmma

24
Installing EclEmma

25
Installing EclEmma: Coverage Decorators

26
Installing EclEmma: Coverage Mode

27
Installing EclEmma: Source Code
Annotation

28
Installing EclEmma: Coverage View

29
What Next?

Mutation Testing on Code

30
References: Please read them!

• Introduction to Software Engineering/Testing

(http://en.wikibooks.org/wiki/Introduction_to_Software_Engineering/Testing
)
• Unit Testing with JUnit http://www.vogella.com/tutorials/JUnit/article.html
• Unit Testing in Eclipse Using Junit
http://realsearchgroup.org/SEMaterials/tutorials/junit/junit_tutorial_3.1.html
• A JUnit Tutorial for Beginners http://java.dzone.com/articles/junit-tutorial-
beginners
• Using JUnit in Eclipse
• http://courses.cs.washington.edu/courses/cse143/11wi/eclipse-
tutorial/junit.shtml
• EclEmma 2.3.0 http://www.eclemma.org/installation.html
• JUnit: https://github.com/junit-team/junit
Acknowledgements: Saumya Singh and Antra Grover for help with slides.
31