An Introduction to

Agile Software Testing

1

Course Objectives
After this seminar, you will:
Understand what an Agile project
Know the differences between Kanban and Scrum
Understand what and how a tester performs the

testing in Scrum process

Understand what testing techniques required for a

tester in Agile

Know what challenges of Agile Software Testing

Course Content
What is an Agile Project?
What are Scrum and Kanban?
What are Scrum & Kanban Processes?

What are testing Activities in Scrum?

What are Agile Testing Techniques?

What is an Agile Project?

All projects are agile, but some agile projects
are more agile than others

How to agile a project

An Agile purist would point to agile manifesto and
its principles
Individuals and interactions over process and tools
Working SW over comprehensive documents

In practice, Agile labeled

to projects that its delivery of

each increment taking no
longer 4 weeks
Is it met level of agility?

Customer collaboration over contract negotiation

Responding to change over following a plan

Agile Project
An
introduction
of Scrum and
Kanban

What is Scrum?
Scrum in 100 words
Scrum is an agile process that allows us to focus on

delivering the highest business value in the shortest

time.
It allows us to rapidly and repeatedly inspect actual
working software (every two weeks to one month).
The business sets the priorities. Teams self-organize to
determine the best way to deliver the highest priority
features.
Every two weeks to a month anyone can see real
working software and decide to release it as is or
continue to enhance it for another sprint.
6

What is Scrum? - Principles

9 Scrum principles
Scrum contains sets of principles (some of them
required while others are optional) and
predefined roles
(1)Demo
(2)Sit together

(3)Daily Scrum
(4)Retrospective
(5)Sprint backlog

(6)Product Backlog
(7)Sprint planning meetings
(8)Time-boxed iterations
(9)Definition of Done

What is Kanban?
Kanban in 100 words
Visualize the workflow
Split the work into pieces, write each item on a card and put on
the wall.
Use named columns to illustrate where each item is in the
workflow..
Limit Work In Progress (WIP) assign explicit limits to how many
items may be in progress at each workflow state
Measure the lead time (average time to complete one item,
sometimes called cycle time), optimize the process to make
lead time as small and predictable as possible
8

Kanban vs Scrum Board

Kanban and Scrum Board

V.S.
Scrum and Kanban over time

V.S.
Scrum resists change within
an Iteration

V.S.

Kanban limits number of items

in workflow state
9

An
introduction
of Scrum
Process

10

What is Scrum process?

Scrum Roles

Product Owner

Key Artifacts
Product Backlog
Requirements user stories
Desired work
Prioritized by Product Owner
Anybody can add to it
Sprint Goal
Summary of focused work in
sprint
Declared by Product Owner
Accepted by team

Scrum Master

Scrum Team
(max 9)

Sprint Backlog
Team signs up for work of
their own work never
assigned
Owned/managed by the team
Estimated work remaining is
updated daily
Blocks List
List of blocks & unmade
decisions
Owned by Scrum Master
Updated daily
Burndown chart
Effort spent over period
Stories/ features completed

Ceremonies
Sprint planning
Hosted by Scrum Master
Pick highest priority items in
Product backlog and the team turns
the items into Sprint Backlog
Estimate sprint backlog in hours
Work breakdown
Declare Sprint Goal

Daily Scrum
Hosted by Scrum Master
15 minutes - same time every day
Not for problem solving
3 questions in meeting: (1) What
did you do? (2) What will you do?
(3) Whats in your way?
Team updates sprint backlog
Sprint Review
Hosted by Scrum Master 2-4 hours
Accomplishments
Whole team participants
Take form of demo for features
Sprint Retrospective
Hosted by SM 15-30 minutes
Discuss on Start doing, Continue
doing and Stop doing

Process

Product
Backlog
Sprint 2-4 weeks
Sprint Planning

Sprint Goal

Daily Scrum

Sprint
Backlog
Blocks

Daily Work

Product

Sprint Review/
Retrospective

Product
Backlog
11

What are roles in Scrum?

Product Owner (PO) is a
clients representation

Scrum Master represents

Management

Define features of product

Decide Release Date and
content
Prioritize features
according to market value
Be responsible for the
profitability of product
Accept or reject work
item result

Enacting Scrum values

Ensure teams productivity
Prioritize features according
to market value
Corporate across all roles
and functions
Shield team from external
interferences

The Team
5-9 members
Including: developers,
testers, designers,
Full time
Work Self-organizing
Membership should be
changed each sprint

A practice for prioritizing features (or User Story)

Priority = Max (5 (or 6) x priority that assigned by PO, sum
(priority that assigned by team members)

12

What are roles of tester in Scrum?

There is NO tester in formal Scrum Process
Testing is carried out by developer with Unit Test
Test Driven Development means testing
executed before development and refactoring
performed to optimize development
Testing coverage

Testing is carried out by Product Owner/ Client

Frequently testing by Product Owner each
sprint
Testing Acceptance Criteria

Isnt it really necessary tester position in Scrum?

What happens without tester?
No future for tester?

13

What are roles of tester in Scrum ? (cont.)

Are testers obsolete?
Questions:

Will developers do acceptance

testing well?
Will developers test their work
product well?
Is that enough with only one
kind of testing Unit Test?
How to test non-functional
requirements?

Benefits when have

dedicated test team:

Focus on customer usage over

technical implementation
Focus on uncovering flaws over
confirming completeness
14

What are roles of tester in Scrum? (cont.)

Participant

in

Release/

Sprint

Planning

Support developers in Unit Testing

say them about testing view instead
of development view

Test

User Story when completed,

last
gate
to
confirm
testing
completeness

Collaborate

with customer and

Product Owner to define acceptance
criteria

Provide positive feedback

customer
Develop automation testing

to

15

Testing Activity
in Scrum &
Kaban Process

16

What are Testing Activities in Scrum Process?

Process
Product
Backlog

Sprint Goal

Daily Scrum

Sprint
Backlog
Blocks

Daily Work

Product

Three
cadence

Sprint Planning

event
Driven

Sprint 2-4 weeks

single
cadence

Tester must know what model of cadence used

In Sprint Planning:
Sprint Review/
Retrospective

Product
Backlog

As a tester, he/she says what should be picked from

Product Backlog for this Sprint to bring value to customer
& to create high performance

As a tester, he/she says how many hours should have

to complete testing for each of selected User Stories
As a tester, he/she must understand what sprint goals
As a tester, involves into the prioritizing process
17

What are Testing Activities ? estimate effort

Effort Estimate techniques:

Heuristic (expert judgment based)

Parametric (calculation based)

Comparison to similar systems

Expert Judgment
Activity Based (top down)
Task Based (bottom up)
Three-point estimate
50/90 estimate

Function Points
Use Case Points
User Story Points

In Agile, testers are empowered to make decision. It means they

are given authority to estimate effort for each work item that
stated in Story Card for Product Owner reviews
As a result, testers are responsible for all testing activities that
they estimated.

18

What are Testing Activities ? 50/90 estimate

50/90 Estimation

Worst case: number of hours (cost) in the worse-case scenario

Most -likely: number of hours (cost) in the Most -likely scenario
Total Hours = Most Likely
Task variance = (Worst Case (W) Most Likely (A))
Deviation = sqrt (sum((Wi-Ai)^2))
19

What are Testing Activities ? 3-point estimate

3 Points Estimate (PERT)

Opt: indicates number of hours (cost) in the best-case scenario

Pess: indicates number of hours (cost) in the worse-case scenario
Most -likely: indicates number of hours (cost) in the Most -likely scenario
PERT = (Opt + 4 x Most-Likely + Pess) / 6
Variance = (Pess Opt) / 6

20

What are Testing Activities ? User Story Points

User Story points (USP)

Story points are a unit of measure for expressing the overall size of a
user story, feature, or other piece of work
The raw values assigned are unimportant
Image a story with 2 USP(s) will 2 times bigger than 1-USP one

Two estimation scales

or

or

Velocity is a measure of a teams rate of progress Ideal Time

The beauty of this is that estimating in story points completely separates

the estimation of effort from the estimation of duration
If we find that a particular work unit is twice the size we thought.
The estimate remains the same, but rate of progress through it
will be slower
21

What are Testing Activities in Scrum Process?

Process
Product
Backlog
Sprint 2-4 weeks

Sprint Planning

Sprint Goal

Daily Scrum

Sprint
Backlog

Daily Work

In Daily Scrum:

Blocks

As a tester, he/she joins all daily meeting (standup)

Product

to speak up
1. what was tested and did yesterday/ progress;

Sprint Review/
Retrospective

2. What will test and do today?

3. Any in testing progress

As a tester, he/she can bring any backlog item that

Product
Backlog

cannot be completed in current sprint and put to next

sprint EX: I dont have enough time to execute
testing for item #4. it will be tested in next sprint.
22

What are Testing Activities in Scrum Process?

Process
Product
Backlog
Sprint 2-4 weeks

Sprint Planning

Sprint Goal

Daily Scrum

Sprint
Backlog
Blocks

Daily Work

Sprint Review/
Retrospective

Product
Backlog

Product

In Daily Work:

As a tester, he/she is responsible for develop

automation scripts (for automation testing
As a tester, he/she schedules automation testing
with Continuous Integration (CI) System.
As a tester, he/she reviews & reports test results to
stakeholders via CI System or by himself/herself.
As a tester, he/she executes non-functional testing
with approved user stories
Some organizations, Product Owner does not have
enough energy to do acceptance testing each sprint.
Tester is hired for this case

As a tester, in end of Sprint, he/she executes UAT

and confirms testing completeness for current sprint.

23

What are Testing Activities in Scrum Process?

In Scrum, Software is integrated daily, build and test
each day. Therefore, CI System is deployed to most
of Agile Project. A CI System & test components can
be identified as below

Hudson / CruiseControl CI Framework

Junit (for Java)/ Nunit (.Net) Unit Test
Cobertura (Java)/ NCoverage Testing
Coverage
Ant (for Java)/ Nant (.Net) Automation Build
SVN Version Control
Selenium integrated with Hudson - Functional
testing

In Daily Work, tester will responsible for develop

automation scripts, integrate testing automation
testing tools/scripts with CI for regression test and
UAT
As a part of duty, tester is playing role supporter in
helping developers create Unit Tests effectively and
efficient
24

What are Testing Activities in Scrum Process?

In Daily Work:
As a practice, Done of sprint can be considered when
Code checked in.

Builds ready
Unit Test complete successfully
80% code branch coverage on UT

Testers
supports

Non-functional testing completed

(confirmed by testers)
Finish deploying to Production

25

What are Testing Activities in Scrum Process?

Process
Product
Backlog
Sprint 2-4 weeks

Sprint Planning

Sprint Goal

Daily Scrum

Sprint
Backlog
Blocks

Daily Work

Product

In Review & Retrospective:

As a tester, he/she identifies what went right and

what went wrong in current sprint
As a tester, he/she identifies lesson learned and
best practices.
As a tester, he/she is encouraged to write user
stories that support testing.

Sprint Review/
Retrospective

EX: As a tester, I need interface with inputs A, B, C

to execute testing of feature X.

As a tester, he/she is encouraged to write user

stories that bring value to customer.

Product
Backlog

EX: As a user, I need to be able to sort items.

All these user stories are put to Product Backlog
for review and Approved
26

What are Testing Activities in Scrum Process?

In Review & Retrospective:

Any items remaining will be

Review why tester could

NOT complete in

current sprint

Problem may be identified by lacking of

experience in estimate or any roadblock
prevented it completed. Whatever reason
identified will be noted as lesson learned

Put to next sprint for continuing testing

Any roadblock comes from lacking of support of

Product Owner, should be pointed out for Scrum
Masters follow-up
To bring effectiveness and efficiency to review sessions,
Scrum Master should request participants to prepare list of
items/ issues at least 1 day prior to meeting date
27

What are Testing Activities in Kanban Process?

Kanban is less prescriptive than Scrum
All testing activities are similar with Scrum

No required time-boxed iteration

Testing is not limited by time duration
Accept changes at anytime Scrum protects changing backlog items
in sprint while Kanban still accepts removing and adding new items
Testing item in workflow state can be added or removed to get the
most effectiveness and efficiency.

Kanban limits number of items for each workflow state

Testing items should be picked logically to put to workflow stated

28

What are Testing Activities in Kanban Process?

An Example of Managing Tasks/ Work Item
1. Backlog: state all work items/ user stories translated into
tasks - As a characteristic of Backlog may be limited
number of items
2. Ready: List of work items that ready for executing
3. Specifics: Identify what criteria to confirm work item
completeness
4. Execute: Work Item in executing

Tuan completed work item testing for Sorting Items. He

reviews and confirms that it
satisfied criteria state. He moves
this item to Done

Next, he moves another work

item testing for Filter Items
from ready to execute to
start new work

He also moves another work

item from Backlog as a
pipeline item.
29

What are Testing Activities in Product Development?

Testing against
user story

Unit Testing
Integration Test

Testing in live
environment - UAT

Non-functional
testing/ confirms
story is met

In Agile, testing is focused on

1. Unit Test that developers need support of testers to
create effective set of tests and
2. Acceptance Test that the responsibility is belonged
to tester
Integration & regression test are executed daily by
automation.

Tester is also responsible for

testing user story that cannot be
verified by UT to confirm testing
completed before demo and
release to production.
30

What are Testing Activities in Product Development?

Testing against
user story

Unit Testing
Integration Test

Story Exploration
Estimation
Story Planning
Story Progression
Story Acceptance

Testing in live
environment - UAT

Non-functional
testing/ confirms
story is met

Communication within an agile team to explore story

understanding
Consider flows of between Product Owner and team on story
Testers also speak up with questions to clarify any vagueness
in the story

What are Testing Activities in Product Development?

EX: For User story: As a user, I can
reset my account password if I forgot
my password
Test may have questions relate to this
story as follows

1. What does it mean for an email

address to be unknown?
2. What does it mean to require
confirmation of the password?
And then s/he temporarily states their
understanding as next picture

Story Exploration
Estimation

Estimation is a part of iteration where tester is using appropriate

techniques

Story Planning

Story Planning occurs after a story has been estimated and be

placed in iteration. Details of test tasks defined as

Story Progression

1. Prepare test data

2. Extend acceptance test

Story Acceptance

3. Execute manual

4. Automate CI tests

What are Testing Activities in Product Development?

If acceptance criteria are not in
place, agile teams may collaborate
on writing acceptance tests during
story planning
Tester can select acceptance
tests

for

automation.

Not

all

acceptance tests will be automated

Ex: for Forgot Password story, team may
have scenario as next picture for acceptance
test
Story Exploration
Estimation

Story Planning
Story Progression
Story Acceptance

Story Progression: uncover additional tests that reply on

exploratory testing and communication. Conducting testing while
monitor results from CI system
Story acceptance: done in test may be understood as all tests
pass or level of test automation is met

What are documents of testing in Agile Project?

Agile in common, or Scrum/ Kanban in
particular, is focused on how to deliver
value to customer by involving customer
(that presented by Product Owner) in
whole process; transferring product to
live (production) environment to bring
benefits to business soon as possible
and expect changes to maximize
customer business value. While Agile
reduces documentation overhead by
simplifying process and number of
artifacts created

As a part of process, testing in Agile has

lightweight documentation with:

Use reusable checklists to

suggest tests

Focus on essential of test

rather than the incidental details

Capture test ideas for

Exploratory Testing

In case test-cases required, they should be

created at high level
If any, leverage one artifact for Manual and
Automation Tests
34

How to track bugs in Agile Process?

Do or Do Not need Bug Tracking.
No bug Tracking is still working well when testers perform their tests
in Dev environment

However, in Test environment, testers need to track bugs because they

dont see fixes for a while
With Bug Tracking, any bugs which are identified (opened) by
developers also will be regress tested by testers. Then test scripts will
be updated to cover the case
Ultimately, bugs can be prioritized with stories
Keep in mind, in Agile, testing is not concrete phase. It is a part of
development

Development
Testing

Development &
Testing
35

Agile Testing
Techniques

36

What are Agile Testing Techniques? Quadrants

Business Facing

Automation &
Manual Test

Exploratory Testing

Story Test
Prototypes Test

Scenario Test
Usability Test
User Acceptance Test

Unit Testing

Performance & Load Testing

Unit tests combined with

Test coverage

Performance Test
Load Test
Security Test

Technology Facing

Critique Product

Support the team

Functional Testing

Automation
Test

Manual
Test

Special Tool &

Automation
Test
37

What are Agile Testing Techniques?

As typical testing, black box testing is used.
Agile testing is focus on using
Automation testing techniques
Test driven development, auto builds and continuous regression test
daily all reply on automation and will not work without it. And
acceptance test is carried out each Sprint that automation also has
benefits
Exploratory testing technique
This technique simultaneous learning, test design and test execution.
While the software is being tested, the tester learns things that
together with experience and creativity generates new good tests to
run.
Scenario testing technique
Use real-world scenarios that exercise multiple stories
Dont think of Exploratory is Agile Testing it is only one of Agile
testing approaches
38

What are Agile Testing Techniques? Exploratory

An example of Exploratory testing

technique.
User Story
As an online shopper, I want to be able to delete
items from my shopping cart
Exploratory is kind of creativity techniques
We start with thinking of some personas or
characters and devise exploratory scenarios which
role may get into. Ex:
1. Senior citizen who never shopped on web
2. Hacker looking for troubles
3. Internet-savvy teen
For each role, behaviors will be different to have
cases of testing generated and tested

39

References
Scrum and Kaban at InfoQ
http://www.infoq.com/
Testing Experience
http://www.testingexperience.com/
Mountain Goat Software
http://www.mountaingoatsoftware.com/
Agile Estimating and Planning Mike Cohn
User Stories Applied: For Agile Software Development

40

Recommended useful tool for Agile Team

- the ultimate screenshot capture software designed specifically for
testers that:
Snap a single screenshot or record a sequence of screens
-

Automatically generate intelligent step-by-step narration (defect report)

Auto submit tickets to defect tracking and ALM systems

- calling for BETA user - Full featured Test Management System to

give testers a simple yet productive working environment. Some of the qTest
features and modules include:
Test Planning
-

Requirement Management
Test Case Management
Test Execution
Defect Tracking
Dashboards and Reports
Built in workflows and collaboration

Get the tools as free download at http://www.qasymphony.com/

41

Thank You
Welcome your questions
Agile
Project
Scrum &
Kanban
Agile

1
2

Testing
Others

42