Effective utilization of Agile Methods in QA for defect free software

Authors: Vineet Kalra (vineet_kalra@adp.com) ADP Small Business Services, 71 Hanover Road, Florham Park, NJ - 07932, USA. Srinivasa Padakanti (srinivasa_padakanti@adp.com) ADP Private Ltd. 6-3-1091/C/1, Fortune 9 Raj Bhavan Road, Hyderabad, AP India - 500082 Aruna Bala Kumari Dwibhashyam ArunaBala_Dwibhashyam@adp.com ADP Private Ltd. 6-3-1091/C/1, Fortune 9 Raj Bhavan Road, Hyderabad, AP India - 500082

1. Abstract
This paper discusses regarding the agile Processes where testing is integrated throughout the lifecycle that Reduces development risk with phased Integration and continuous user feedback which made the defect ratio as 12:1(approx) and kept a final stop to Last Minute Surprises. Apart from being geared towards better quality software, this also supports the principle of small iterative incremental releases which helps us to achieve faster ROI through rapid release of business functionality. This Paper also gives enough focus on the issues faced by the QA team while moving to agile methodology from the traditional approach and the processes we followed to overcome them and ensue ahead successfully. In fact this test driven development has given the benefits above expectations which reflected in the decrease in defect rate as well as achieving the unanimous target of Software Industry High Quality Software.

2. Introduction
The main idea behind developing High quality software depends on the methods followed from the initial phase; the approach of agile comes into picture where the QA activities become a part and parcel from the kickoff of the development activities itself. In this paper we will describe our experiences and discuss about the processes developed from the gist of issues faced while migration. For this paper, we took Full Level Tax (FLT) RUN application and analyzed the entire agile process where the QA people were involved along with the development team sprint wise listing the issues in the QA methodology. We listed the main differences. We than applied these process improvements to the later Sprints of FLT and compared the performance of the releases with the previous ones. We also highlight the current methodologies that are being implemented in ADP Small Business Services (ADP-SBS) Division making the Agile approach grand success.

3. Initial Scenario of the FLT project

ADP Small business Services (ADP SBS) started its offshore operations in mid 2004. A team of four members was formed to test the FLT SBS applications. The application was initially tested in the Basic model of testing where in which the release comes for every 3 months and a thorough QA for about 4 weeks and then logging the defects and then proceeding with those bug fixes in the later releases. This process can be depicted as shown below Figure 1 represents the formal approach we followed in FORMAL Release and Figure 2 depicts the AGILE approach
R q ire e ts R ) e u m n (S S D v lo m n ee p et Ts g e tin

P n in la n g

R q ire e ts A a s s e u m n n ly y Aa s n ly is

Aa s n ly

is

T s P n in e t la n g

Ds n e ig

Ts Ds n e t e ig

C d g&D v lo m n o in ee p et

T s D v lo m n et ee p e t

U it T s g n e tin

R le s toT s g e ae e tin

T s Ee u n e t x c tio

T s / B gR p rts et u eo

D liv ry e e

Figure-1

Figure -2 For the FLT application once the code is released to QA we used to log around 120 defects (approx) and the fixes would be followed in the next releases and so on. Once the agile methodology was implemented the defects found in every sprint once summarized the figures are fascinating, around 90 defects were identified in the unit testing for sprints and once the process in QA we could identify only 10 defects which can be considered negligible when compared to the initial process . The figure below depicts the figures mentioned above
140 120 100 80 60 40 20 0 Unit Testing QA RATE Formal Approach Agile Methodology

The Bars above reveal the picture, in normal integration it was reported that only 30 defects were identified, but when the code is in QA we identified around 120 defects and some areas we couldnt focus during QA which came up in RATE (Regional Acceptance Testing Environment) around 15 of them which was very high but later it reduced to 2 defects which was one of the unexpected benefits we grabbed through Agile process adoption

4. Challenges Encountered
The above figures were so fascinating but these were not achieved overnight as well as adoption of Agile was not a cake walk too. One of the issues the Initial resistance from the team was very high as the team considered the daily Scrum calls as micromanagement and couldnt perform as expected. The challenges can be summarized as below

Daily Scrum Calls:

As the meeting scheduled to just update the status of the observations it was treated as an overhead by the team initially and couldnt raise the involvement of the team which didnt give the performance as expected.

Lack of in depth Information providing Documents:

As the information received are just confined to the respective sprint and the team got habituated to learn all about the product, created a lot of ambiguity regarding the future releases of the product working on

Repetitive testing:
As the sprints needs to be integrated and tested for all the sprints the regression testing was so repetitive that some times created fatigue to the QA team

5. Methods followed to mitigate the impact of the Challenges faced: Scrum Calls made more interactive:
The scrum Calls were made a sort of more interactions between the people and was made clear that they were not daily update calls instead providing a platform to know more about.

All the Required Information is provided:

The team was given with all the information about the sprint they are working on on the day1 itself making them more comfortable of testing the sprint thoroughly, though hardly any information regarding the future releases its need is not felt by the team while testing respective sprint

Repetitive testing was done through Automation:

Automation came into picture, which avoided the need of manual repetitive testing and made the task of repetitive testing as easy as expected. Along with the above the following process also produced a better and effective utilization of the agile methodology

6. Processes that make the agile more beneficiary:

1. The whole team was made responsible for QA: The responsibility for quality was shifted to the whole team. Each of the different roles is responsible for doing some form of testing. Programmers, architects, analysts, and even managers are all intimately involved with testing activities and work closely together to achieve quality goals. The whole team was motivated to produce defect free software by the Sprints release. When this is done, testing becomes everyone's job everyday. This doesn't mean that there is no need for dedicated QA personnel on an agile team. The difference is that they must learn to work alongside the team, and expand their job description beyond testing only when development is finished. 2. Keep the system running all the time: Because the only practical way to test software (formal proofs aside) is by running it, the system Has to be up and running every day. This sounds obvious, but there are a surprising number of Projects that go for hours or days without a functioning build. Without running the software you cant test it, and without testing it you don't know where you are. Keeping the system stable and Run able is crucial to keep the quality problem from getting out of hand.

3. Automate as much as possible As the system grows, there are more and more tests that must be run to ensure that it is still working. With only a handful of features, manual testing might be practical. But as features existing features continue to function as before. The only way to accomplish this within the short time boxes on an agile project is to automate all of the testing you can. There is a lot of power and freedom in being able to literally push a button and have multiple types of regression tests run against your system. Because automated tests are easy to run, they will get run a lotespecially when the team is under pressure. Manual testing is just the opposite. The more pressure you feel, the less likely you are to take sufficient time to test carefully and the more potential you have for injecting undiscovered defects. Automation is essential in the continuous testing approach. These Processes guide QA practices that minimize project risk through short development cycles, enhance communication, and rapidly adapt to changing business needs. The end result is a value-driven software development approach fundamentally different from any formal methodology of testing widely used today. One of the (often unexpected) benefits of adopting an agile software development approach is a significant increase in software quality. While teams rarely decide to try agile development for quality reasons alone, dramatically lower defect rates are almost always one of the first things that they notice. The projects which had implemented agile methodology have succeeded in reducing defects to the point where there are hardly any major defects after the release of the entire build to QA. The answer lies in the fundamentally different approach taken by agile methodologies to both building and testing software. Agile development rejects the Notion of distinct, activity-specific phases as the best way to scale software Project. Rather than doing requirements, then analysis, then design, build, and test as Separate, Sequential phases (often with different groups of people), agile teams slice the problem differently. It works by building a small but complete piece of a Larger System and then growing it incrementally.

Defect coverage Process in FLT:

The project started with a kick off meeting on August 2nd 2007, explaining the principles and the discipline of agile. The QA team was given the data of the sprints dealing with and the developers working on. Once the first sprint was out to the QA the fixes were identified and initially communicated through the mail got them fixed later they are properly handled through QC This early identification of defects reduced the turnaround time of the fixes as well as made the QA people more comfortable reducing the last minute surprises. In December we got a major change in the business requirement which was achieved very simple through this approach other than which would have created a lot of chaos in traditional process. We got through that final sprint testing too and then the code was distributed to QA environment As worked for more than 6 months along with the development, we could hardly find any major defect in QA; we could find 10 of them which were missed due to the dependency levels. The most important of all is as most of the defects were identified at the initial stage of the code the cost reduction also was high enough to implement this process to the projects . The defects can be tabulated as below: Defects uncovered 165 102 In Unit Testing 30 90 In QA environment 120 10 In RATE environment 15 2

Product FLT FLT-RUN

Process Development Agile Development

The Agile methodology can be summarized as below Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan

8. Current Improvements in SBS

As the implemented methods delivered the product with high quality and the team is also much aware and organized towards agile practices, Agile methodology was taken up by another project too AOS, which would be soon into QA environment. Adhering to the practice of continuous SQA improvement, we are always on lookout for the various standards and best practices to implement in the existing process. Currently, we are working together with the developers end to end of the application to formulate better functionality test cases and to make the Rate defects to Zero as well as negligible amount in QA environment too. .. We are also working on to build better and effective Test Automation suites for the AOS Applications too . This will help us to do better and faster regression testing for the upcoming module .

Authors Biography:
Vineet Kalra is currently Project Manager at ADP Inc. NJ, USA. He did his masters from Stevens Institute of Technology. His experience covers development and IT project management, including offshore project management. He manages several large-scale Quality Assurance projects for the Small Business Services Division of ADP Inc, NJ. He is also managing the entire offshore operations for the SBS Quality Assurance Division. Srinivasa Padakanti is currently Senior Project Leader at ADP Private Ltd. Hyderabad, India. He has more than 8 years of experience in IT field. From the beginning of his career, he has handled various development and QA Projects. His specialties are IT Project Management, Development and Quality Assurance. He manages several large-scale Quality Assurance projects at ADP Hyderabad. Aruna Bala Kumari is working as a Senior Software Engineer at ADP India Limited. She has an overall experience of 3 years in Software industry that includes 3 years in Automation testing. She has obtained her Masters In Computer Applications from Indira Gandhi University (IGNOU). Her areas of interest are test automation and Agile testing..

Acknowledgements:
I would like to thank Akkiraju Bhattiprolu for the amount of time and effort he spent in reviewing the paper and giving his valuable suggestions.

Appendix:
ADP SBS FLT Dev QA RATE QC or Quality Center Automatic Data Processing, a financial services company. Small Business Services, a Division of ADP Inc Full Level Tax Development Quality Assurance Regional Acceptance Testing Environment Test Management Tool by Mercury Cooperation

(Number of defects found in RATE) Defect Leakage Percentage in RATE: = ______________________________________ X 100 (Total number of defects found in the QA and RATE) Defect Severity levels 1 & 2 severity defects 1 Immediate Response 2 Fix required for Certain Other severity defects 3 Workaround available 4 Cosmetic 5 Issue not in requirements 6 Non SBS Issue 7 To Be Determined