Software Testing Guide Book Part 1

Software Testing Guide Book
Part I: Fundamentals of Software Testing
Ajitha, Amrish Shah, Ashna Datye, Bharathy J, Deepa M G, James M, Jayapradeep J, Jeffin Jacob M, Kapil Mohan Sharma, Leena Warrier, Mahesh, Michael Fran , M!hammad Kashif
Jamil "arendra ", "a#eed M, $haneendra %, $rathima ", &a#i Kiran ", &ajee# D, Sarah
Salah!ddin, Si#a $rasad B, Shalini &, Shilpa D, S!bramanian D &amprasad, S!nitha ' ", S!nil K!mar M K, (sha $admini K, Winston Geor)e and *arinath $ +
Copyright (c) SofTReL 2004. Permission is granted to copy, distrib te and!or modify this doc ment nder the terms of the "#$ %ree &oc mentation License, 'ersion (.2 or any )ater *ersion p b)ished by the %ree Soft+are %o ndation, +ith no -n*ariant Sections, no %ront.Co*er Te/ts, and no 0ac1.Co*er Te/ts. 2 copy of the )icense is inc) ded in the section entit)ed 3"#$ %ree &oc mentation License3.
Software Testing Research Lab http,--.../Sof0&eL/or)
Soft.are 0estin) G!ide Boo
$art 1, F!ndamentals of Soft.are 0estin)
Revision "er# $o# 2/3 2/7 2/8 %ate 245Apr526 235May526 285J!ly526
istor! 'uthor *arinath, on behalf of S0GB 0eam/ *arinath, on behalf of S0GB 0eam/ *arinath, on behalf of S0GB 0eam
%escri&tion 1nitial doc!ment creation 1ncorporated &e#ie. 'omments Draft &elease
http,--.../Sof0&eL/or)
7 of 367
Table of (ontents
Software Testing Guide Book################################################################################) )# The Software Testing Guide Book######################################################################*
+# ,hat is Software Testing and ,h! is it Im&ortant-#########################################). /# T!&es of %evelo&ment S!stems#######################################################################)+
For.ard/////////////////////////////////////////////////////////////////////////////////////////////////////////4 Abo!t Sof0&eL///////////////////////////////////////////////////////////////////////////////////////////////9 $!rpose of this Doc!ment/////////////////////////////////////////////////////////////////////////////9 A!thors/////////////////////////////////////////////////////////////////////////////////////////////////////////: 1ntended A!dience/////////////////////////////////////////////////////////////////////////////////////////; *o. to !se this Doc!ment/////////////////////////////////////////////////////////////////////////////; What this G!ide Boo is not//////////////////////////////////////////////////////////////////////////; *o. to 'ontrib!te/////////////////////////////////////////////////////////////////////////////////////////; F!t!re <nhancements///////////////////////////////////////////////////////////////////////////////////; 'opyri)hts/////////////////////////////////////////////////////////////////////////////////////////////////////;
0# T!&es of Software S!stems##############################################################################)/
8/3 8/7 8/8 8/6
0raditional De#elopment Systems//////////////////////////////////////////////////////////37 1terati#e De#elopment////////////////////////////////////////////////////////////////////////////37 Maintenance System//////////////////////////////////////////////////////////////////////////////37 $!rchased-'ontracted Soft.are////////////////////////////////////////////////////////////38
6/3 Batch Systems///////////////////////////////////////////////////////////////////////////////////////38 6/7 <#ent 'ontrol Systems//////////////////////////////////////////////////////////////////////////38 6/8 $rocess 'ontrol Systems////////////////////////////////////////////////////////////////////////38 6/6 $roced!re 'ontrol Systems////////////////////////////////////////////////////////////////////36 6/= Ad#anced Mathematical Models/////////////////////////////////////////////////////////////36 6/4 Messa)e $rocessin) Systems/////////////////////////////////////////////////////////////////36 6/9 Dia)nostic Soft.are Systems/////////////////////////////////////////////////////////////////36 6/: Sensor and Si)nal $rocessin) Systems//////////////////////////////////////////////////36 6/; Sim!lation Systems///////////////////////////////////////////////////////////////////////////////3= 6/32 Database Mana)ement Systems///////////////////////////////////////////////////////////3; 6/33 Data Ac>!isition /////////////////////////////////////////////////////////////////////////////////3; 6/37 Data $resentation ///////////////////////////////////////////////////////////////////////////////3; 6/38 Decision and $lannin) Systems///////////////////////////////////////////////////////////3; 6/36 $attern and 1ma)e $rocessin) Systems////////////////////////////////////////////////3; 6/3= 'omp!ter System Soft.are Systems////////////////////////////////////////////////////72 6/34 Soft.are De#elopment 0ools////////////////////////////////////////////////////////////////72
1# euristics of Software Testing########################################################################+. *# ,hen Testing should occur-###########################################################################+0 2# The Test %evelo&ment Life (!cle 3T%L(4#########################################################+5 5# ,hen should Testing sto&-#############################################################################/. 6# "erification Strategies####################################################################################/.
;/3 &e#ie.///////////////////////////////////////////////////////////////////////////////////////////////////82 ;/7 Wal thro!)h//////////////////////////////////////////////////////////////////////////////////////////88

8 of 367
;/8 1nspection//////////////////////////////////////////////////////////////////////////////////////////////8=
).# Testing T!&es and Techni7ues######################################################################/*
32/3 White Bo? 0estin)////////////////////////////////////////////////////////////////////////////////8:
10.1.1 Basis Path Testing...........................................................................................42 10.1.2 Flow Graph Notation......................................................................................42 10.1.3 Cyclomatic Complexity..................................................................................42 10.1.4 Graph atrices................................................................................................42 10.1.! Control "tr#ct#re Testing................................................................................43 10.1.$ %oop Testing...................................................................................................43
32/7 Blac Bo? 0estin) ///////////////////////////////////////////////////////////////////////////////66
10.2.1 Graph Base& Testing etho&s........................................................................4! 10.2.2 'rror G#essing................................................................................................4! 10.2.3 Bo#n&ary (al#e )nalysis................................................................................4! 10.2.4 '*#i+alence Partitioning.................................................................................4$ 10.2.! Comparison Testing........................................................................................4, 10.2.$ -rthogonal )rray Testing................................................................................4,
))# %esigning Test (ases####################################################################################02 )+# "alidation Phase###########################################################################################05
37/3 (nit 0estin)/////////////////////////////////////////////////////////////////////////////////////////6: 37/7 1nte)ration 0estin)///////////////////////////////////////////////////////////////////////////////=8
12.2.1 Top./own 0ntegration.....................................................................................!3 12.2.2 Bottom.1p 0ntegration....................................................................................!3

37/8 System 0estin)////////////////////////////////////////////////////////////////////////////////////=6
12.3.1 Compati2ility Testing......................................................................................!4 12.3.2 3eco+ery Testing.............................................................................................!! 12.3.3 1sa2ility Testing.............................................................................................!! 12.3.4 "ec#rity Testing...............................................................................................!4 12.3.! "tress Testing..................................................................................................!4 12.3.$ Per5ormance Testing ......................................................................................!4 12.3., Content anagement Testing.........................................................................$4 12.3.4 3egression Testing .........................................................................................$4
37/6 37/= 37/4 37/9 Alpha 0estin)///////////////////////////////////////////////////////////////////////////////////////93 (ser Acceptance 0estin)//////////////////////////////////////////////////////////////////////97 1nstallation 0estin)//////////////////////////////////////////////////////////////////////////////97 Beta 0estin) ///////////////////////////////////////////////////////////////////////////////////////98
)/# 8nderstanding 9:&lorator! Testing###############################################################2/ )0# 8nderstanding Scenario Based Testing##########################################################6. )1# 8nderstanding 'gile Testing#########################################################################6) )*# 'PI Testing###################################################################################################6* )2# 8nderstanding Ra&id Testing######################################################################)./ )5# Test ,are %evelo&ment ############################################################################## ).1
)6# %efect ;anagement####################################################################################)+.

6 of 367
3:/3 0est Strate)y//////////////////////////////////////////////////////////////////////////////////////32= 3:/7 0est $lan///////////////////////////////////////////////////////////////////////////////////////////32: 3:/8 0est 'ase Doc!ments////////////////////////////////////////////////////////////////////////336
3;/3 What is a Defect@///////////////////////////////////////////////////////////////////////////////372 3;/7 Defect 0a?onomies/////////////////////////////////////////////////////////////////////////////372 3;/8 Life 'ycle of a Defect//////////////////////////////////////////////////////////////////////////373
+.# ;etrics for Testing######################################################################################)++ References########################################################################################## ##############)/2 G$8 Free %ocumentation License#####################################################################)/5
= of 367
)# The Software Testing Guide Book

Forward
Soft.are 0estin) has )ained a phenomenal importance in the recent years in the System De#elopment Life 'ycle/ Many learned people ha#e .or ed on the topic and pro#ided #ario!s techni>!es and methodolo)ies for effecti#e and efficient testin)/ 0oday, e#en tho!)h .e ha#e many boo s and articles on Soft.are 0est <n)ineerin), many people are fallacio!s in !nderstandin) the !nderlyin) concepts of the s!bject/ Soft.are 0estin) G!ide Boo !nderstandin)/ 0his )!ide boo has been a!thored by professionals .ho ha#e been e?posed to 0estin) no.led)e ban .here 0estin) #ario!s applications/ We .anted to brin) o!t a base this boo has come o!t/ 0his )!ide boo does not pro#ide any specific methodolo)ies to be follo.ed for 0estin), instead pro#ides a concept!al !nderstandin) of the same/ Note to the Reader4 1t is not o!r intention to tell yo! that this is a one5stop place for learnin) 0estin)/ 0his is j!st a )!ide/ Many eminent scientists ha#e researched e#ery topic yo! find in this boo / We ha#e j!st compiled e#erythin) in one place and made s!re .e e?plained each topic relatin) it to the practical .orld as .e e?perienced it/ 1f yo! find any s!bject matter that mi)ht loo li e .e ha#e copied from any e?istin) boo , .e re>!est yo! to let !s no./ 1t is not o!r intention to copy any material, and .e bro!)ht o!t this boo j!st to help 0estin) aspirants to ha#e a basic !nderstandin) of the s!bject and )!ide them to be )ood at their job/ All the material in this doc!ment is .ritten in plain <n)lish, as .e !nderstand testin)/ $lease send in yo!r comments, s!))estions or a .ord of enco!ra)ement to the team/ &e)ards, 0he Sof0&eL 0eam AS0GBB is an open so!rce project aimed at brin)in) the technicalities of Soft.are 0estin) into one place and arri#in) at a common
enth!siasts can start to learn the science and art of Soft.are 0estin), and this is ho.
4 of 367
'bout SofTReL
0he Soft.are 0estin) &esearch Lab ASof0&eLB is a non5profit or)aniCation dedicated for &esearch and Ad#ancements of Soft.are 0estin)/ 0he concept of ha#in) a common place for Soft.are 0estin) &esearch .as form!lated in 7223/ 1nitially .e named it DSoft.are E!ality and <n)ineerin)F/ &ecently in March 7226, .e renamed it to GSoft.are 0estin) &esearch LabH I Sof0&eL/ $rofessionals .ho are c!rrently .or in) .ith the ind!stry and possess rich e?perience in testin) form the members of the Lab/ +isit http,--.../softrel/or) for more information/
Pur&ose of this %ocument

0his doc!ment does not pro#ide the reader .ith short c!tFs to perform testin) in daily life, b!t instead e?plains the #ario!s methodolo)ies and techni>!es .hich ha#e been proposed by eminent scientists in an easy and !nderstandable .ay/ 0his )!ide boo is di#ided into three parts, Part I Fundamentals of Software Testing 0his section addresses the f!ndamentals of Soft.are 0estin) and their practical application in real life/ Part II Software Testing for various Architectures 0his section .o!ld concentrate in e?plainin) testin) applications !nder #ario!s architect!res li e 'lient-Ser#er, Web, $oc et $', Mobile and <mbedded/ Part III Platform Specific Testing 0his section addresses testin) 'JJ and Ja#a applications !sin) .hite bo? testin) methodolo)ies/ This is Part -. 2)) pdates on the pro5ect are a*ai)ab)e at http4!!+++.SofTReL.org!stgb.htm).
9 of 367
'uthors
0he )!ide boo has been a!thored by professionals .ho D0estF e#eryday/ Ajitha 5 GrayLo)ic 'orporation, "e. Jersey, (SA Amrish Shah 5 MAESoft.are, M!mbai Ashna Datye 5 &S 0ech 1nc, 'anada Bharathy Jayaraman 5 1#esia Sol!tions A1B $#t Limited, 'hennai Deepa M G 5 Kc.en 0echnolo)y Lchan)e, Ban)alore James M 5 'SS, 'hennai Jayapradeep Jiothis 5 Satyam 'omp!ter Ser#ices, *yderabad Jeffin Jacob Mathe. 5 1'FA1 B!siness School, *yderabad Kapil Mohan Sharma 5 $i?tel 'omm!nitations, "e. Delhi Leena Warrier I Wipro 0echnolo)ies, Ban)alore Mahesh, i$ointSoft, *yderabad Michael Fran 5 (SA M!hammad Kashif Jamil, A#anCa Sol!tions, Karachi, $a istan "arendra "a)aram I Satyam 'omp!ter Ser#ices, *yderabad "a#eed Mohammad I #Mo sha, Ban)alore $haneendra % 5 Wipro 0echnolo)ies, Ban)alore $rathima "a)apra ash I Wipro 0echnolo)ies, Ban)alore &a#i Kiran " 5 Andale, Ban)alore &ajee# Daithan ar 5 $ersistent Systems $#t/ Ltd/, $!ne Sarah Salah!ddin 5 Arc Sol!tions, $a istan Si#a $rasad Badimi 5 Danla. 0echnolo)ies, *yderabad Shalini &a#i !mar 5 (SA Shilpa Dodla 5 Decatrend 0echnolo)ies, 'hennai S!bramanian Dattaramprasad 5 Mind0ec , Ban)alore S!nitha ' " 5 1nfosys 0echnolo)ies, Mysore S!nil K!mar M K I %ahooM 1ndia, Ban)alore (sha $admini Kandala 5 +irt!sa 'orp, Massach!setts Winston Geor)e I +biNap Soft Sol!tions A$B Ltd/, 'hennai *arinath I Sof0&eL, Ban)alore
: of 367
Intended 'udience
0his )!ide boo is aimed at all 0estin) $rofessionals I from a be)inner to an ad#anced
!ser/ 0his boo .o!ld pro#ide a baseline !nderstandin) of the concept!al theory/
ow to use this %ocument

0his boo can be !sed as a )!ide for performin) the 0estin) acti#ities/ A D)!ideF here, .e mean that this can pro#ide yo! a road map as to ho. to approach a specific problem .ith respect to 0estin)/
,hat this Guide Book is not

0his )!ide boo is definitely not a sil#er-)old-diamond b!llet .hich can help yo! to
test any application/ 1nstead this boo .o!ld be reference help to perform 0estin)/
ow to (ontribute
0his is an open so!rce project/ 1f yo! are interested in contrib!tin) to the boo or to the Lab, please do .rite in to st)b at SoF0&eL dot or)/ We need yo!r e?pertise in the research acti#ities/
Future 9nhancements
0his is the first part of the three5part Soft.are 0estin) G!ide Boo AS0GBB series/ %o!
can #isit http,--.../softrel/or)-st)b/html for !pdates on the $roject/
(o&!rights
Sof0&eL is not proposin) the 0estin) methodolo)ies, types and #ario!s other concepts/ We tried presentin) each and e#ery theoretical concept of Soft.are 0estin) .ith a li#e e?ample for easier !nderstandin) of the s!bject and arri#in) at a common !nderstandin) of Soft.are 0est <n)ineerin)/ *o.e#er, .e did p!t in fe. of o!r proposed .ays to achie#e specific tas s and these are )o#erned by 0he G"( Free Doc!mentation License AG"(5FDLB/ $lease #isit http,--.../)n!/or)-doc-doc/html for complete )!idelines of the license or alternati#ely yo! can find the license to.ards the end of this doc!ment
; of 367
+# ,hat is Software Testing and ,h! is it Im&ortant2 brief history of Soft+are engineering and the S&LC. 0he soft.are ind!stry has e#ol#ed thro!)h 4 eras, =2Fs I42Fs, mid 42Fs Ilate 92Fs, mid 92Fs5 mid :2Fs, and mid :2Fs5present/ <ach era has its o.n distincti#e characteristics, b!t o#er the years the soft.areFs ha#e increased in siCe and comple?ity/ Se#eral problems are common to almost all of the eras and are disc!ssed belo./ The Soft+are Crisis dates bac to the 3;42Fs .hen the primary reasons for this sit!ation .ere less than acceptable soft.are en)ineerin) practices/ 1n the early sta)es of soft.are there .as a lot of interest in comp!ters, a lot of code .ritten b!t no established standards/ 0hen in early 92Fs a lot of comp!ter pro)rams started failin) and people lost confidence and th!s an ind!stry crisis .as declared/ +ario!s reasons leadin) to the crisis incl!ded,

*ard.are ad#ances o!tpacin) the ability to b!ild soft.are for this hard.are/ 0he ability to b!ild in pace .ith the demands/ 1ncreasin) dependency on soft.areFs Str!))le to b!ild reliable and hi)h >!ality soft.are $oor desi)n and inade>!ate reso!rces/
0his crisis tho!)h identified in the early years, e?ists to date and .e ha#e e?amples of soft.are fail!res aro!nd the .orld/ Soft.are is basically considered a fail!re if the project is terminated beca!se of costs or o#err!n sched!les, if the project has e?perienced o#err!ns in e?cess of =2O of the ori)inal or if the soft.are res!lts in client la.s!its/ Some e/amp)es of fai) res incl!de fail!re of Air traffic control systems, fail!re of medical soft.are, and fail!re in telecomm!nication soft.are/ 0he primary reason for these fail!res other than those mentioned abo#e is d!e to bad soft.are en)ineerin) practices adopted/ Some of the .orst soft.are practices incl!de,

"o historical soft.are5meas!rement data/ &ejection of acc!rate cost estimates/ Fail!re to !se a!tomated estimatin) and plannin) tools/ <?cessi#e, irrational sched!le press!re and creep in !ser re>!irements/ Fail!re to monitor pro)ress and to perform ris mana)ement/ Fail!re to !se desi)n re#ie.s and code inspections/
0o a#oid these fail!res and th!s impro#e the record, .hat is needed is a better !nderstandin) of the process, better estimation techni>!es for cost time and >!ality meas!res/ B!t the >!estion is, .hat is a process@ $rocess transform inp!ts to o!tp!ts
32 of 367
i/e/ a prod!ct/ A soft.are process is a set of acti#ities, methods and practices in#ol#in) transformation that people !se to de#elop and maintain soft.are/ At present a lar)e n!mber of problems e?ist d!e to a chaotic soft.are process and the occasional s!ccess depends on indi#id!al efforts/ 0herefore to be able to deli#er s!ccessf!l soft.are projects, a foc!s on the process is essential since a foc!s on the prod!ct alone is li ely to miss the scalability iss!es, and impro#ements in the e?istin) system/ 0his foc!s .o!ld help in the predictability of o!tcomes, project trends, and project characteristics/ 0he process that has been defined and adopted needs to be mana)ed .ell and th!s process mana)ement comes into play/ Process management is concerned .ith the no.led)e and mana)ement of the soft.are process, its technical aspects and also ens!res that the processes are bein) follo.ed as e?pected and impro#ements are sho.n/ From this .e concl!de that a set of defined processes can possibly sa#e !s from soft.are project fail!res/ B!t it is nonetheless important to note that the process alone cannot help !s a#oid all the problems, beca!se .ith #aryin) circ!mstances the need #aries and the process has to be adapti#e to these #aryin) needs/ 1mportance needs to be )i#en to the h!man aspect of soft.are de#elopment since that alone can ha#e a lot of impact on the res!lts, and effecti#e cost and time estimations may )o totally .aste if the h!man reso!rces are not planned and mana)ed effecti#ely/ Secondly, the reasons mentioned related to the soft.are en)ineerin) principles may be resol#ed .hen the needs are correctly identified/ 'orrect identification .o!ld then ma e it easier to identify the best practices that can be applied beca!se one process that mi)ht be s!itable for one or)aniCation may not be most s!itable for another/ 0herefore to ma e a s!ccessf!l prod!ct a combination of $rocess and 0echnicalities .ill be re>!ired !nder the !mbrella of a .ell5defined process/ *a#in) tal ed abo!t the Soft.are process o#erall, it is important to identify and relate the role soft.are testin) plays not only in prod!cin) >!ality soft.are b!t also mane!#erin) the o#erall process/ 0he comp!ter society defines testin) as follo.s, GTesting 55 A #erification method that applies a controlled set of conditions and stim!li for the p!rpose of findin) errors/ 0his is the most desirable method of #erifyin) the f!nctional and performance re>!irements/ 0est res!lts are doc!mented proof that re>!irements .ere met and can be repeated/ 0he res!ltin) data can be re#ie.ed by all concerned for confirmation of capabilities/H
33 of 367
0here may be many definitions of soft.are testin) and many .hich appeal to !s from time to time, b!t its best to start by definin) testin) and then mo#e on dependin) on the re>!irements or needs/
/# T!&es of %evelo&ment S!stems

0he type of de#elopment project refers to the en#ironment-methodolo)y in .hich the soft.are .ill be de#eloped/ Different testin) approaches need to be !sed for different types of projects, j!st as different de#elopment approaches/
/#) Traditional %evelo&ment S!stems

0he 0raditional De#elopment System has the follo.in) characteristics, 0he traditional de#elopment system !ses a system de#elopment methodolo)y/ 0he !ser c!stomerB/ 0he de#elopment system determines the str!ct!re of the application/ no.s .hat the c!stomer re>!ires A&e>!irements are clear from the
What do yo! do .hile testin), 0estin) happens at the end of each phase of de#elopment/ 0estin) sho!ld concentrate if the re>!irements match the de#elopment/ F!nctional testin) is re>!ired/
/#+ Iterative %evelo&ment

D!rin) the 1terati#e De#elopment, 0he re>!irements are not clear from the !ser Ac!stomerB/ 0he str!ct!re of the soft.are is pre5determined/
0estin) of 1terati#e De#elopment projects sho!ld concentrate only if the 'AS< A'omp!ter Aided Soft.are <n)ineerin)B tools are properly !tiliCed and the f!nctionality is thoro!)hly tested/
/#/ ;aintenance S!stem

0he Maintenance System is .here the str!ct!re of the pro)ram !nder)oes chan)es/ 0he system is de#eloped and bein) !sed, b!t it demands chan)es in the f!nctional aspects of the system d!e to #ario!s reasons/ 0estin) Maintenance Systems re>!ires str!ct!ral testin)/ 0op priority sho!ld be p!t into &e)ression 0estin)/
37 of 367
/#0 Purchased<(ontracted Software

At times it may be re>!ired that yo! p!rchase soft.are to inte)rate .ith yo!r prod!ct or o!tso!rce the de#elopment of certain components of yo!r prod!ct/ 0his is $!rchased or 'ontracted Soft.are/ When yo! need to inte)rate third party soft.are to yo!r e?istin) soft.are, this demands the testin) of the p!rchased soft.are .ith yo!r re>!irements/ Since the t.o systems are desi)ned and de#eloped differently, the inte)ration ta es the top priority d!rin) testin)/ Also, &e)ression 0estin) of the inte)rated soft.are is a m!st to cross chec the t.o soft.areFs are .or in) as per the re>!irements/ if
0# T!&es of Software S!stems

0he type of soft.are system refers to the processin) that .ill be performed by that system/ 0his contains the follo.in) soft.are system types/
0#) Batch S!stems

0he Batch Systems are a set of pro)rams that perform certain acti#ities .hich do not re>!ire any inp!t from the !ser/ A practical e?ample is that .hen yo! are typin) somethin) on a .ord doc!ment, yo! press the ey yo! re>!ire and the same is printed on the monitor/ B!t processin) Acon#ertin)B the !ser inp!t of the ey to the machine !nderstandable lan)!a)e, ma in) the system !nderstand .hat to be displayed and in ret!rn the .ord doc!ment displayin) .hat yo! ha#e typed is performed by the batch systems/ 0hese batch systems contain one or more Application $ro)rammin) 1nterface AA$1B .hich perform #ario!s tas s/
0#+ 9vent (ontrol S!stems

<#ent 'ontrol Systems process real time data to pro#ide the !ser .ith res!lts for .hat command AsB he is )i#en/ For e?ample, .hen yo! type on the .ord doc!ment and press 'trl J S, this tells the comp!ter to sa#e the doc!ment/ *o. this is performed instantaneo!sly@ 0hese real time command comm!nications to the comp!ter are pro#ided by the <#ent 'ontrols that are pre5defined in the system/
0#/ Process (ontrol S!stems

0here are t.o or more different systems that comm!nicate to pro#ide the end !ser a specific !tility/ When t.o systems comm!nicate, the co5ordination or data transfer becomes #ital/ $rocess 'ontrol Systems are the oneFs .hich recei#e data from a different
38 of 367
system and instr!cts the system .hich sent the data to perform specific tas s based on the reply sent by the system .hich recei#ed the data/
0#0 Procedure (ontrol S!stems

$roced!re 'ontrol Systems are the oneFs .hich control the f!nctions of another system/
0#1 'dvanced ;athematical ;odels

Systems, .hich ma e !se of hea#y mathematics, fall into the cate)ory of Mathematical Models/ (s!ally all the comp!ter soft.are ma e !se of mathematics in some .ay or the other/ B!t, Ad#ance Mathematical Models can be classified .hen there is hea#y !tiliCation of mathematics for performin) certain actions/ An e?ample of Ad#anced Mathematical Model can be sim!lation systems .hich !ses )raphics and controls the positionin) of soft.are on the monitor or Decision and Strate)y ma in) soft.areFs/
0#* ;essage Processing S!stems

A simple e?ample is the SMS mana)ement soft.are !sed by Mobile operatorFs .hich handle incomin) and o!t)oin) messa)es/ Another system, .hich is note.orthy is the system !sed by pa)in) companies/
0#2 %iagnostic Software S!stems

0he Dia)nostic Soft.are System is one that helps in dia)nosin) the comp!ter hard.are components/ When yo! pl!) in a ne. de#ice to yo!r comp!ter and start it, yo! can see the dia)nostic soft.are system doin) some .or / 0he G"e. *ard.are Fo!ndH dialo)!e can be seen as a res!lt of this system/ 0oday, almost all the Kperatin) SystemFs come pac ed .ith Dia)nostic Soft.are Systems/
0#5 Sensor and Signal Processing S!stems

0he messa)e processin) systems help in sendin) and recei#in) messa)es/ 0he Sensor and Si)nal $rocessin) Systems are more comple? beca!se these systems ma e !se of mathematics for si)nal processin)/ 1n a si)nal processin) system the comp!ter recei#es inp!t in the form of si)nals and then transforms the si)nals to a !ser !nderstandable o!tp!t/
36 of 367
0#6 Simulation S!stems

A sim!lation system is a soft.are application, some times !sed in combination .ith specialiCed hard.are, .hich re5creates or sim!lates the comple? beha#ior of a system in its real en#ironment/ 1t can be defined in many .ays, P0he process of desi)nin) a model of a real system and cond!ctin) e?periments .ith this model for the p!rpose of !nderstandin) the beha#ior of the system and-or e#al!atin) #ario!s strate)ies for the operation of the systemP55 1ntrod!ction to Sim!lation (sin) S1MA", by '/ D/ $e)den, &/ </ Shannon and &/ $/ Sado.s i, McGra.5 *ill, 3;;2/ GA sim!lation is a soft.are pac a)e Asometimes b!ndled .ith special hard.are inp!t de#icesB that re5creates or sim!lates, albeit in a simplified manner, a comple? phenomena, en#ironment, or e?perience, pro#idin) the !ser .ith the opport!nity for some ne. le#el of !nderstandin)/ 1t is interacti#e, and !s!ally )ro!nded in some objecti#e reality/ A sim!lation is based on some !nderlyin) comp!tational model of the phenomena, en#ironment, or e?perience that it is sim!latin)/ A1n fact, some a!thors !se model and modelin) as synonyms of sim!lation/BP 55K!rt Sch!ma er, A 0a?onomy of Sim!lation Soft.are/P Learnin) 0echnolo)y &e#ie./ 1n simple .ords sim!lation is nothin) b!t a representation of a real system/ 1n a pro)rammable en#ironment, sim!lations are !sed to st!dy system beha#ior or test the system in an artificial en#ironment that pro#ides a limited representation of the real en#ironment/ ,h! Simulation S!stems Sim!lation systems are easier, cheaper, and safer to !se than real systems, and often the only .ay to b!ild the real systems/ For e?ample, learnin) to fly a fi)hter plane !sin) a sim!lator is m!ch safer and less e?pensi#e than learnin) on a real fi)hter plane/ System sim!lation mimics the operation of a real system s!ch as the operation in a ban , or the r!nnin) of the assembly line in a factory etc/ Sim!lation in the early sta)e of desi)n cycle is important beca!se the cost of mista es increases dramatically later in the prod!ct life cycle/ Also, sim!lation soft.are can analyCe the operation of a real system .itho!t the in#ol#ement of an e?pert, i/e/ it can also be analyCed .ith a non5e?pert li e a mana)er/ ow to Build Simulation S!stems 1n order to create a sim!lation system .e need a realistic model of the system beha#ior/ Kne .ay of sim!lation is to create smaller #ersions of the real system/
3= of 367
0he sim!lation system may !se only soft.are or a combination of soft.are and hard.are to model the real system/ 0he sim!lation soft.are often in#ol#es the inte)ration of artificial intelli)ence and other modelin) techni>!es/ ,hat a&&lications fall under this categor!Sim!lation is .idely !sed in many fields/ Some of the applications are, Models of planes and cars that are tested in .ind t!nnels to determine the aerodynamic properties/ (sed in comp!ter Games A</)/ Sim'ity, car )ames etcB/ 0his sim!lates the .or in) in a city, the roads, people tal in), playin) )ames etc/ War tactics that are sim!lated !sin) sim!lated battlefields/ Most <mbedded Systems are de#eloped by sim!lation soft.are before they e#er ma e it to the chip fabrication labs/ Stochastic sim!lation models are often !sed to model applications s!ch as .eather forecastin) systems/ Social sim!lation is !sed to model socio5economic sit!ations/ 1t is e?tensi#ely !sed in the field of operations research/
,hat are the (haracteristics of Simulation S!stemsSim!lation Systems can be characteriCed in n!mero!s .ays dependin) on the characteriCation criteria applied/ Some of them are listed belo./ %eterministic Simulation S!stems Deterministic Sim!lation Systems ha#e completely predictable o!tcomes/ 0hat is, )i#en a certain inp!t .e can predict the e?act o!tcome/ Another feat!re of these systems is idempotency, .hich means that the res!lts for any )i#en inp!t are al.ays the same/ <?amples incl!de pop!lation prediction models, atmospheric science etc/ Stochastic Simulation S!stems Stochastic Sim!lation systems ha#e models .ith random #ariables/ 0his means that the e?act o!tcome is not predictable for any )i#en inp!t, res!ltin) in potentially #ery different o!tcomes for the same inp!t/ Static Simulation S!stems Static Sim!lation systems !se statistical models in .hich time does not play any role/ 0hese models incl!de #ario!s probabilistic scenarios .hich are !sed to calc!late the res!lts of any )i#en inp!t/ <?amples of s!ch systems incl!de financial portfolio #al!ation models/ 0he most common sim!lation techni>!e !sed in these models is the Monte 'arlo Sim!lation/
34 of 367
%!namic Simulation S!stems A dynamic sim!lation system has a model that accommodates for chan)es in data o#er time/ 0his means that the inp!t data affectin) the res!lts .ill be entered into the sim!lation d!rin) its entire lifetime than j!st at the be)innin)/ A sim!lation system !sed to predict the )ro.th of the economy may need to incorporate chan)es in economic data, is a )ood e?ample of a dynamic sim!lation system/ %iscrete Simulation S!stems Discrete Sim!lation Systems !se models that ha#e discrete entities .ith m!ltiple attrib!tes/ <ach of these entities can be in any state, at any )i#en time, represented by the #al!es of its attrib!tes/ / 0he state of the system is a set of all the states of all its entities/ 0his state chan)es one discrete step at a time as e#ents happens in the system/ 0herefore, the act!al desi)nin) of the sim!lation in#ol#es ma in) choices abo!t .hich entities to model, .hat attrib!tes represent the <ntity State, .hat e#ents to model, ho. these e#ents impact the entity attrib!tes, and the se>!ence of the e#ents/ <?amples of these systems are sim!lated battlefield scenarios, hi)h.ay traffic control systems, m!ltiteller systems, comp!ter net.or s etc/ (ontinuous Simulation S!stems 1f instead of !sin) a model .ith discrete entities .e !se data .ith contin!o!s #al!es, .e .ill end !p .ith contin!o!s sim!lation/ For e?ample instead of tryin) to sim!late battlefield scenarios by !sin) discrete entities s!ch as soldiers and tan s, .e can try to model beha#ior and mo#ements of troops by !sin) differential e>!ations/ Social Simulation S!stems Social sim!lation is not a techni>!e by itself b!t !ses the #ario!s types of sim!lation described abo#e/ *o.e#er, beca!se of the specialiCed application of those techni>!es for social sim!lation it deser#es a special mention of its o.n/ 0he field of social sim!lation in#ol#es !sin) sim!lation to learn abo!t and predict #ario!s social phenomenon s!ch as #otin) patterns, mi)ration patterns, economic decisions made by the )eneral pop!lation, etc/ Kne interestin) application of social sim!lation is in a field called artificial life .hich is !sed to obtain !sef!l insi)hts into the formation and e#ol!tion of life/ ,hat can be the &ossible test a&&roachA sim!lation systemFs primary responsibility is to replicate the beha#ior of the real system as acc!rately as possible/ 0herefore, a )ood place to start creatin) a test plan .o!ld be to !nderstand the beha#ior of the real system/
39 of 367
Sub=ective Testing S!bjecti#e testin) mainly depends on an e?pertQs opinion/ An e?pert is a person .ho is proficient and e?perienced in the system !nder test/ 'ond!ctin) the test in#ol#es test r!ns of the sim!lation by the e?pert and then the e?pert e#al!ates and #alidates the res!lts based on some criteria/ Kne ad#anta)e of this approach o#er objecti#e testin) is that it can test those conditions .hich cannot be tested objecti#ely/ For e?ample, an e?pert can determine .hether the joystic handlin) of the fli)ht feels Pri)htP/ Kne disad#anta)e is that the e#al!ation of the system is based on the Pe?pertQsP opinion, .hich may differ from e?pert to e?pert/ Also, if the system is #ery lar)e then it is bo!nd to ha#e many e?perts/ <ach e?pert may #ie. it differently and can )i#e conflictin) opinions/ 0his ma es it diffic!lt to determine the #alidity of the system/ Despite all these disad#anta)es, s!bjecti#e testin) is necessary for testin) systems .ith h!man interaction/ >b=ective Testing Kbjecti#e testin) is mainly !sed in systems .here the data can be recorded .hile the sim!lation is r!nnin)/ 0his testin) techni>!e relies on the application of statistical and a!tomated methods to the data collected/ Statistical methods are !sed to pro#ide an insi)ht into the acc!racy of the sim!lation/ 0hese methods incl!de hypothesis testin), data plots, principle component analysis and cl!ster analysis/ A!tomated testin) re>!ires a no.led)e base of #alid o!tcomes for #ario!s r!ns of sim!lation/ 0his no.led)e base is created by domain e?perts of the sim!lation system bein) tested/ 0he data collected in #ario!s test r!ns is compared a)ainst this no.led)e base to a!tomatically #alidate the system !nder test/ An ad#anta)e of this Statistical ;ethods Statistical methods are !sed to pro#ide an insi)ht into the acc!racy of the sim!lation/ 0hese methods incl!de hypothesis testin), data plots, principle component analysis and cl!ster analysis/ 'utomated Testing A!tomated testin) re>!ires a no.led)e base of #alid o!tcomes for #ario!s r!ns of sim!lation/ 0his no.led)e base is created by domain e?perts of the sim!lation system bein) tested/ 0he data collected in #ario!s test r!ns is compared a)ainst this no.led)e base to a!tomatically #alidate the system !nder test/ An ad#anta)e of this ind of testin) is that the system can contin!ally be re)ression tested as it is bein) de#eloped/
3: of 367
ind of
testin) is that the system can contin!ally be re)ression tested as it is bein) de#eloped/
0#). %atabase ;anagement S!stems

As the name denotes, the Database Mana)ement Systems ADBMSB handles the mana)ement of databases/ 1t is basically a collection of pro)rams that enable the stora)e, modification and e?traction from the Database/ 0he DBMS, as they are often referred to as, can be of #ario!s different types ran)in) from small systems that r!n on $'Fs to MainframeFs/ 0he follo.in) can be cate)oriCed as e?ample of DBMS, 'omp!teriCed Library Systems/ A!tomated 0eller Machines/ $assen)er &eser#ation Systems/ 1n#entory Systems/
0#)) %ata 'c7uisition

Data Ac>!isition systems, ta en in real time data and store them for f!t!re !se/ A simple e?ample of Data Ac>!isition system can be a A0' AAir 0raffic 'ontrolB Soft.are .hich ta es in real time data of the position and speed of the fli)ht and stores it in compressed form for later !se/
0#)+ %ata Presentation

Data $resentation soft.are stores data and displays the same to the !ser .hen re>!ired/ An e?ample is a 'ontent Mana)ement System/ %o! ha#e a .eb site and this is in <n)lish, yo! also ha#e yo!r .eb site in other lan)!a)es/ 0he !ser can select the lan)!a)e he .ishes to see and the system displays the same .eb site in the !ser chosen lan)!a)e/ %o! de#elop yo!r .eb site in #ario!s lan)!a)es and store them on the system/ 0he system displays the re>!ired lan)!a)e, the !ser chooses/
0#)/ %ecision and Planning S!stems

0hese systems !se Artificial 1ntelli)ence techni>!es to pro#ide decision5ma in) sol!tions to the !ser/
0#)0 Pattern and Image Processing S!stems

0hese systems are !sed for scannin), storin), modifyin) and displayin) )raphic ima)es/ 0he !se of s!ch systems is no. bein) increased as research tests are bein) cond!cted in #is!al modelin) and their !se in o!r daily li#es is increasin)/ 0hese systems are !sed for sec!rity re>!ests s!ch as dia)nosin) photo)raph, th!mb impression of the #isitor etc/
3; of 367
0#)1 (om&uter S!stem Software S!stems

0hese are the normal comp!ter soft.areFs, that can be !sed for #ario!s p!rposes/
0#)* Software %evelo&ment Tools

0hese systems ease the process of Soft.are De#elopment/
1#
euristics of Software Testing
Testabilit! Soft.are testability is ho. easily, completely and con#eniently a comp!ter pro)ram can be tested/ Soft.are en)ineers desi)n a comp!ter prod!ct, system or pro)ram eepin) in mind the prod!ct testability/ Good pro)rammers are .illin) to do thin)s that .ill help the testin) process and a chec list of possible desi)n points, feat!res and so on can be !sef!l in ne)otiatin) .ith them/ *ere are the t.o main he!ristics of soft.are testin)/ 3/ +isibility 7/ 'ontrol "isibilit! +isibility is o!r ability to obser#e the states and o!tp!ts of the soft.are !nder test/ Feat!res to impro#e the #isibility are Access to 'ode De#elopers m!st pro#ide f!ll access Aso!rce code, infrastr!ct!re, etcB to testers/ 0he 'ode, chan)e records and desi)n doc!ments sho!ld be pro#ided to the testin) team/ 0he testin) team sho!ld read and !nderstand the code/ <#ent lo))in) 0he e#ents to lo) incl!de (ser e#ents, System milestones, <rror handlin) and completed transactions/ 0he lo)s may be stored in files, rin) b!ffers in memory, and-or serial ports/ 0hin)s to be lo))ed incl!de description of e#ent, timestamp, s!bsystem, reso!rce !sa)e and se#erity of e#ent/ Lo))in) sho!ld be adj!sted by s!bsystem and type/ Lo) file report internal errors, help in isolatin) defects, and )i#e !sef!l information abo!t conte?t, tests, c!stomer !sa)e and test co#era)e/ 0he more readable the Lo) &eports are, the easier it becomes to identify the defect ca!se and .or to.ards correcti#e meas!res/
72 of 367
<rror detection mechanisms Data inte)rity chec in) and System le#el error detection Ae/)/ Microsoft App#ie.erB are !sef!l here/ 1n addition, Assertions and probes .ith the follo.in) feat!res are really helpf!l 'ode is added to detect internal errors/ Assertions abort on error/ $robes lo) errors/ Desi)n by 'ontract theory5550his techni>!e re>!ires that
assertions be defined for f!nctions/ $reconditions apply to inp!t and #iolations implicate callin) f!nctions .hile post5conditions apply to o!tp!ts and #iolations implicate called f!nctions/ 0his effecti#ely sol#es the oracle problem for testin)/ &eso!rce Monitorin) Memory !sa)e sho!ld be monitored to find memory lea s/ States of r!nnin) methods, threads or processes sho!ld be .atched A$rofilin) interfaces may be !sed for this/B/ 1n addition, the confi)!ration #al!es sho!ld be d!mped/ &eso!rce monitorin) is of partic!lar concern in applications .here the load on the application in real time is estimated to be considerable/ (ontrol 'ontrol refers to o!r ability to pro#ide inp!ts and reach states in the soft.are !nder test/ 0he feat!res to impro#e controllability are, 0est $oints Allo. data to be inspected, inserted or modified at points in the soft.are/ 1t is especially !sef!l for dataflo. applications/ 1n addition, a pipe and filters architect!re pro#ides many opport!nities for test points/ '!stom (ser 1nterface controls '!stom (1 controls often raise serio!s testability problems .ith G(1 test dri#ers/ <ns!rin) testability !s!ally re>!ires, Addin) methods to report necessary information '!stomiCin) test tools to ma e !se of these methods Gettin) a tool e?pert to ad#ise de#elopers on testability and to b!ild the re>!ired s!pport/
73 of 367
As in) third party control #endors re)ardin) s!pport by test tools/
0est 1nterfaces 1nterfaces may be pro#ided specifically for testin) e/)/ <?cel and Lcon> etc/ <?istin) interfaces may be able to s!pport si)nificant testin) e/)/ 1nstallSheild, A!to'AD, 0i#oli, etc/
Fa!lt injection <rror seedin)555instr!mentin) lo. le#el 1-K code to sim!late errors555ma es it m!ch easier to test error handlin)/ 1t can be handled at both system and application le#el, 0i#oli, etc/
1nstallation and set!p 0esters sho!ld be notified .hen installation has completed s!ccessf!lly/ 0hey sho!ld be able to #erify installation, pro)rammatically create sample records and r!n m!ltiple clients, daemons or ser#ers on a sin)le machine/
' BR>'%9R "I9, Belo. are )i#en a broader set of characteristics A!s!ally he!risticsB that lead to testable soft.are/ no.n as James Bach
'ate)ories of *e!ristics of soft.are testin) Kperability The better it +or1s, the more efficient)y it can be tested. 0he system sho!ld ha#e fe. b!)s, no b!)s sho!ld bloc and testin)B/ Kbser#ability What .e see is .hat .e test/ Distinct o!tp!t sho!ld be )enerated for each inp!t '!rrent and past system states and #ariables sho!ld be #isible d!rin) testin) All factors affectin) the o!tp!t sho!ld be #isible/ 1ncorrect o!tp!t sho!ld be easily identified/ So!rce code sho!ld be easily accessible/
77 of 367
the e?ec!tion of tests
and the prod!ct sho!ld e#ol#e in f!nctional sta)es Asim!ltaneo!s de#elopment
1nternal errors sho!ld be a!tomatically detected Athro!)h self5testin) mechanismsB and reported/
'ontrollability The better +e contro) the soft+are, the more the testing process can be a tomated and optimi6ed. 'hec that All o!tp!ts can be )enerated and code can be e?ec!ted thro!)h some combination of inp!t/ Soft.are and hard.are states can be controlled directly by the test en)ineer/ 1np!ts and o!tp!t formats are consistent and str!ct!red/ 0est can be con#eniently, specified, a!tomated and reprod!ced/
Decomposability 0y contro))ing the scope of testing, +e can 7 ic1)y iso)ate prob)ems and perform effecti*e and efficient testing. 0he soft.are system sho!ld be b!ilt from independent mod!les .hich can be tested independently/
Simplicity The )ess there is to test, the more 7 ic1)y +e can test it. 0he points to consider in this re)ard are f!nctional Ae/)/ minim!m set of feat!resB, str!ct!ral Ae/)/ architect!re is mod!lariCedB and code Ae/)/ a codin) standard is adoptedB simplicity/
Stability The fe+er the changes, the fe+er are the disr ptions to testing. 0he chan)es to soft.are sho!ld be infre>!ent, controlled and not in#alidatin) e?istin) tests/ 0he soft.are sho!ld be able to reco#er .ell from fail!res/
(nderstandability The more information +e +i)) ha*e, the smarter +e +i)) test. 0he testers sho!ld be able to !nderstand .ell the desi)n, chan)es to the desi)n and the dependencies bet.een internal, e?ternal and shared components/ 0echnical doc!mentation sho!ld be instantly accessible, acc!rate, .ell or)aniCed, specific and detailed/
S!itability The more +e 1no+ abo t the intended organi6e o r testing to find important b gs. se of the soft+are, the better +e can
78 of 367
0he abo#e he!ristics can be !sed by a soft.are en)ineer to de#elop a soft.are confi)!ration Ai/e/ pro)ram, data and doc!mentationB that is con#enient to test and #erify/
*# ,hen Testing should occur,rong 'ssum&tion 0estin) is sometimes incorrectly tho!)ht as an after5the5fact acti#ityR performed after pro)rammin) is done for a prod!ct/ 1nstead, testin) sho!ld be performed at e#ery de#elopment sta)e of the prod!ct /0est data sets m!st be deri#ed and their correctness and consistency sho!ld be monitored thro!)ho!t the de#elopment process/ 1f .e di#ide the lifecycle of soft.are de#elopment into G&e>!irements AnalysisH, GDesi)nH, G$ro)rammin)-'onstr!ctionH and GKperation and MaintenanceH, then testin) sho!ld accompany each of the abo#e phases/ 1f testin) is isolated as a sin)le phase late in the cycle, errors in the problem statement or desi)n may inc!r e?orbitant costs/ "ot only m!st the ori)inal error be corrected, b!t the entire str!ct!re b!ilt !pon it m!st also be chan)ed/ 0herefore, testin) sho!ld not be isolated as an inspection acti#ity/ &ather testin) sho!ld be in#ol#ed thro!)ho!t the SDL' in order to brin) o!t a >!ality prod!ct/ Testing 'ctivities in 9ach Phase 0he follo.in) testin) acti#ities sho!ld be performed d!rin) the phases &e>!irements Analysis 5 A3B Determine correctness A7B Generate f!nctional test data/ Desi)n 5 A3B Determine correctness and consistency A7B Generate
str!ct!ral and f!nctional test data/ $ro)rammin)-'onstr!ction 5 A3B Determine correctness and consistency A7B Generate str!ct!ral and f!nctional test data A8B Apply test data A6B &efine test data/ Kperation and Maintenance 5 A3B &etest/
"o. .e consider these in detail/ Re7uirements 'nal!sis 0he follo.in) test acti#ities sho!ld be performed d!rin) this sta)e/
76 of 367
-n*est in ana)ysis at the beginning of the pro5ect 5 *a#in) a clear, concise and formal statement of the re>!irements facilitates pro)rammin),
comm!nication, error analysis an d test data )eneration/ 0he re>!irements statement sho!ld record the follo.in) information and decisions, 3/ $ro)ram f!nction 5 What the pro)ram m!st do@ 7/ 0he form, format, data types and !nits for inp!t/ 8/ 0he form, format, data types and !nits for o!tp!t/ 6/ *o. e?ceptions, errors and de#iations are to be handled/ =/ For scientific comp!tations, the n!merical method or at least the re>!ired acc!racy of the sol!tion/ 4/ 0he hard.are-soft.are en#ironment re>!ired or ass!med Ae/)/ the machine, the operatin) system, and the implementation lan)!a)eB/ Decidin) the abo#e iss!es is one of the acti#ities related to testin) that sho!ld be performed d!rin) this sta)e/
Start de*e)oping the test set at the re7 irements ana)ysis phase 5 Data sho!ld be )enerated that can be !sed to determine .hether the re>!irements ha#e been met/ 0o do this, the inp!t domain sho!ld be partitioned into classes of #al!es that the pro)ram .ill treat in a similar manner and for each class a representati#e element sho!ld be incl!ded in the test data/ 1n addition, follo.in) sho!ld also be incl!ded in the data set, A3B bo!ndary #al!es A7B any non5e?treme inp!t #al!es that .o!ld re>!ire special handlin)/ 0he o!tp!t domain sho!ld be treated similarly/ 1n#alid inp!t re>!ires the same analysis as #alid inp!t/
The correctness, consistency and comp)eteness of the re7 irements sho )d a)so be ana)y6ed 5 'onsider .hether the correct problem is bein) sol#ed, chec for conflicts and inconsistencies amon) the re>!irements and consider the possibility of missin) cases/
%esign 0he desi)n doc!ment aids in pro)rammin), comm!nication, and error analysis and test data )eneration/ 0he re>!irements statement and the desi)n doc!ment sho!ld to)ether
7= of 367
)i#e the problem and the or)aniCation of the sol!tion i/e/ .hat the pro)ram .ill do and ho. it .ill be done/ 0he desi)n doc!ment sho!ld contain, $rincipal data str!ct!res/ F!nctions, al)orithms, he!ristics or special techni>!es !sed for processin)/ 0he pro)ram or)aniCation, ho. it .ill be mod!lariCed and cate)oriCed into e?ternal and internal interfaces/ Any additional information/
*ere the testin) acti#ities sho!ld consist of,
2na)ysis of design to chec1 its comp)eteness and consistency 5 the total process sho!ld be analyCed to determine that no steps or special cases ha#e been o#erloo ed/ 1nternal interfaces, 1-K handlin) and data str!ct!res sho!ld specially be chec ed for inconsistencies/
2na)ysis of design to chec1 +hether it satisfies the re7 irements 5 chec
.hether
both re>!irements and desi)n doc!ment contain the same form, format, !nits !sed for inp!t and o!tp!t and also that all f!nctions listed in the re>!irement doc!ment ha#e been incl!ded in the desi)n doc!ment/ Selected test data .hich is )enerated d!rin) the re>!irements analysis phase sho!ld be man!ally sim!lated to determine .hether the desi)n .ill yield the e?pected #al!es/
"eneration of test data based on the design 5 0he tests )enerated sho!ld co#er the str!ct!re as .ell as the internal f!nctions of the desi)n li e the data str!ct!res, al)orithm, f!nctions, he!ristics and )eneral pro)ram str!ct!re etc/ Standard e?treme and special #al!es sho!ld be incl!ded and e?pected o!tp!t sho!ld be recorded in the test data/
&ee?amination and refinement of the test data set )enerated at the re>!irements analysis phase/
0he first t.o steps sho!ld also be performed by some collea)!e and not only the desi)ner-de#eloper/ $ro)rammin)-'onstr!ction
74 of 367
*ere the main testin) points are,
Chec1 the code for consistency +ith design 5 the areas to chec
incl!de mod!lar
str!ct!re, mod!le interfaces, data str!ct!res, f!nctions, al)orithms and 1-K handlin)/
Perform the Testing process in an organi6ed and systematic manner .ith test r!ns dated, annotated and sa#ed/ A plan or sched!le can be !sed as a chec list to help the pro)rammer or)aniCe testin) efforts/ 1f errors are fo!nd and chan)es made to the pro)ram, all tests in#ol#in) the erroneo!s se)ment Aincl!din) those .hich res!lted in s!ccess pre#io!slyB m!st be rer!n and recorded/
2s1s some co))eag e for assistance 5 Some independent party, other than the pro)rammer of the specific part of the code, sho!ld analyCe the de#elopment prod!ct at each phase/ 0he pro)rammer sho!ld e?plain the prod!ct to the party .ho .ill then >!estion the lo)ic and search for errors .ith a chec list to )!ide the search/ 0his is needed to locate errors the pro)rammer has o#erloo ed/
$se a*ai)ab)e too)s 5 the pro)rammer sho!ld be familiar .ith #ario!s compilers and interpreters a#ailable on the system for the implementation lan)!a)e bein) !sed beca!se they differ in their error analysis and code )eneration capabilities/
2pp)y Stress to the Program 5 0estin) sho!ld e?ercise and stress the pro)ram str!ct!re, the data str!ct!res, the internal f!nctions and the e?ternally #isible f!nctions or f!nctionality/ Both #alid and in#alid data sho!ld be incl!ded in the test set/
Test one at a time 5 $ieces of code, indi#id!al mod!les and small collections of mod!les sho!ld be e?ercised separately before they are inte)rated into the total pro)ram, one by one/ <rrors are easier to isolate .hen the no/ of potential interactions sho!ld be ept small/ 1nstr!mentation5insertion of some code into loop control #ariables, the pro)ram solely to meas!re #ario!s pro)ram characteristics I can be !sef!l here/ A tester sho!ld perform array bo!nd chec s, chec determine .hether ey data #al!es are .ithin permissible ran)es, trace pro)ram e?ec!tion, and co!nt the no/ of times a )ro!p of statements is e?ec!ted/
79 of 367
8eas re testing co*erage!9hen sho )d testing stop@ 5 1f errors are still fo!nd e#ery time the pro)ram is e?ec!ted, testin) sho!ld contin!e/ Beca!se errors tend to cl!ster, mod!les appearin) partic!larly error5prone re>!ire special scr!tiny/ 0he metrics !sed to meas!re testin) thoro!)hness incl!de statement testin) A.hether each statement in the pro)ram has been e?ec!ted at least onceB, branch testin) A.hether each e?it from each branch has been e?ec!ted at least onceB and path testin) A.hether all lo)ical paths, .hich may in#ol#e repeated e?ec!tion of #ario!s se)ments, ha#e been e?ec!ted at least onceB/ Statement testin) is the co#era)e metric most fre>!ently !sed as it is relati#ely simple to implement/ 0he amo!nt of testin) depends on the cost of an error/ 'ritical pro)rams or f!nctions re>!ire more thoro!)h testin) than the less si)nificant f!nctions/
>&erations and maintenance 'orrections, modifications and e?tensions are bo!nd to occ!r e#en for small pro)rams and testin) is re>!ired e#ery time there is a chan)e/ 0estin) d!rin) maintenance is termed re)ression testin)/ 0he test set, the test plan, and the test res!lts for the ori)inal pro)ram sho!ld e?ist/ Modifications m!st be made to accommodate the pro)ram chan)es, and then all portions of the pro)ram affected by the modifications m!st be re5 tested/ After re)ression testin) is complete, the pro)ram and test doc!mentation m!st be !pdated to reflect the chan)es/
2# The Test %evelo&ment Life (!cle 3T%L(4

(s!ally, 0estin) is considered as a part of the System De#elopment Life 'ycle/ With o!r practical e?perience, .e framed this 0est De#elopment Life 'ycle/ 0he dia)ram does not depict .here and .hen yo! .rite yo!r 0est $lan and Strate)y doc!ments/ B!t, it is !nderstood that before yo! be)in yo!r testin) acti#ities these doc!ments sho!ld be ready/ 1deally, .hen the $roject $lan and $roject Strate)y are bein) made, this is the time .hen the 0est $lan and 0est Strate)y doc!ments are also made/
7: of 367
0est De#elopment Life 'ycle A0DL'B
Requirement Study
Requirement Checklist
Software Requirement Specification
Software Requirement Specification
Functional Specification Checklist
Functional Specification Document
Functional Specification Document
Architecture Design
Architecture Design
Detailed Design Document
Coding Functional Specification Document
Unit Test Case Documents
Unit Test Case Document Design Document Functional Specification Document System Test Case Document Integration Test Case Document Regression Test Case Document
Unit Integration System Test Case Documents
Functional Specification Document !erformance Criteria Software Requirement Specification Regression Test Case Document !erformance Test Cases and Scenarios
!erformance Test Cases and Scenarios
User Acceptance Test Case Documents Scenarios
7; of 367
5# ,hen should Testing sto&PWhen to stop testin)P is one of the most diffic!lt >!estions to a test en)ineer/ 0he follo.in) are fe. of the common 0est Stop criteria, 3/ All the hi)h priority b!)s are fi?ed/ 7/ 0he rate at .hich b!)s are fo!nd is too small/ 8/ 0he testin) b!d)et is e?ha!sted/ 6/ 0he project d!ration is completed/ =/ 0he ris in the project is !nder acceptable limit/ $ractically, .e feel that the decision of stoppin) testin) is based on the le#el of the ris acceptable to the mana)ement/ As testin) is a ne#er endin) process .e can ne#er ass!me that 322 O testin) has been done, .e can only minimiCe the ris the prod!ct to client .ith L testin) done/ 0he ris simply, 5 Meas!rin) 0est 'o#era)e/ "!mber of test cycles/ "!mber of hi)h priority b!)s/ b!t for small d!ration - lo. b!d)et - lo. reso!rces project, ris of shippin) analysis can be meas!red by &is
can be ded!ced by
6# "erification Strategies
What is D+erificationF@ +erification is the process of e#al!atin) a system or component to determine .hether the prod!cts of a )i#en de#elopment phase satisfy the conditions imposed at the start of that phase/3 What is the importance of the +erification $hase@ +erification process helps in detectin) defects early, and pre#entin) their lea a)e do.nstream/ 0h!s, the hi)her cost of later detection and re.or is eliminated/
6#) Review
A process or meetin) d!rin) .hich a .or for comment or appro#al/ prod!ct, or set of .or prod!cts, is
presented to project personnel, mana)ers, !sers, c!stomers, or other interested parties
82 of 367
0he main )oal of re#ie.s is to find defects/ &e#ie.s are a )ood compliment to testin) to help ass!re >!ality/ A fe. p!rposesF of SEA re#ie.s can be as follo.s, Ass!re the >!ality of deli#erableFs before the project mo#es to the ne?t sta)e/ Knce a deli#erable has been re#ie.ed, re#ised as re>!ired, and appro#ed, it can be !sed as a basis for the ne?t sta)e in the life cycle/ What are the #ario!s types of re#ie.s@ 0ypes of re#ie.s incl!de Mana)ement &e#ie.s, 0echnical &e#ie.s, 1nspections, Wal thro!)hs and A!dits/ ;anagement Reviews Mana)ement re#ie.s are performed by those directly responsible for the system in order to monitor pro)ress, determine stat!s of plans and sched!les, confirm re>!irements and their system allocation/ 0herefore the main objecti#es of Mana)ement &e#ie.s can be cate)oriCed as follo.s, +alidate from a mana)ement perspecti#e that the project is ma in) pro)ress accordin) to the project plan/ <ns!re that deli#erables are ready for mana)ement appro#als/ &esol#e iss!es that re>!ire mana)ementFs attention/ 1dentify any project bottlenec s/ Keepin) project in 'ontrol/
S!pport decisions made d!rin) s!ch re#ie.s incl!de 'orrecti#e actions, 'han)es in the allocation of reso!rces or chan)es to the scope of the project 1n mana)ement re#ie.s the follo.in) Soft.are prod!cts are re#ie.ed, A!dit &eports 'ontin)ency plans 1nstallation plans &is mana)ement plans Soft.are E-A 0he participants of the re#ie. play the roles of Decision5Ma er, &e#ie. Leader, &ecorder, Mana)ement Staff, and 0echnical Staff/ Technical Reviews 0echnical re#ie.s confirm that prod!ct 'onforms to specifications, adheres to re)!lations, standards, )!idelines, plans, chan)es are properly implemented, chan)es affect only those system areas identified by the chan)e specification/
83 of 367
0he main objecti#es of 0echnical &e#ie.s can be cate)oriCed as follo.s, <ns!re that the soft.are confirms to the or)aniCation standards/ <ns!re that any chan)es in the de#elopment proced!res Adesi)n, codin), testin)B are implemented per the or)aniCation pre5defined standards/ 1n technical re#ie.s, the follo.in) Soft.are prod!cts are re#ie.ed Soft.are re>!irements specification Soft.are desi)n description Soft.are test doc!mentation Soft.are !ser doc!mentation 1nstallation proced!re &elease notes
0he participants of the re#ie. play the roles of Decision5ma er, &e#ie. leader, &ecorder, 0echnical staff/ ,hat is Re7uirement ReviewA process or meetin) d!rin) .hich the re>!irements for a system, hard.are item, or soft.are item are presented to project personnel, mana)ers, !sers, c!stomers, or other interested parties for comment or appro#al/ 0ypes incl!de system re>!irements re#ie., soft.are re>!irements re#ie./ ,ho is involved in Re7uirement Review $rod!ct mana)ement leads &e>!irement &e#ie./ Members from e#ery affected department participates in the re#ie. In&ut (riteria Soft.are re>!irement specification is the essential doc!ment for the re#ie./ A chec list can be !sed for the re#ie./ 9:it (riteria <?it criteria incl!de the filled S completed chec list .ith the re#ie.ersF comments S s!))estions and the re5#erification .hether they are incorporated in the doc!ments/ ,hat is %esign ReviewA process or meetin) d!rin) .hich a system, hard.are, or soft.are desi)n is presented to project personnel, mana)ers, !sers, c!stomers, or other interested parties for
87 of 367
comment or appro#al/ 0ypes incl!de critical desi)n re#ie., preliminary desi)n re#ie., and system desi)n re#ie./ ,ho involve in %esign Review EA team member leads desi)n re#ie./ Members from de#elopment team and EA team participate in the re#ie./ In&ut (riteria Desi)n doc!ment is the essential doc!ment for the re#ie./ A chec list can be !sed for the re#ie./ 9:it (riteria <?it criteria incl!de the filled S completed chec list .ith the re#ie.ersF comments S s!))estions and the re5#erification .hether they are incorporated in the doc!ments/
,hat is (ode ReviewA meetin) at .hich soft.are code is presented to project personnel, mana)ers, !sers, c!stomers, or other interested parties for comment or appro#al/ ,ho is involved in (ode Review EA team member A1n case the EA 0eam is only in#ol#ed in Blac from de#elopment team and EA team participate in the re#ie./ In&ut (riteria 0he 'odin) Standards Doc!ment and the So!rce file are the essential doc!ments for the re#ie./ A chec list can be !sed for the re#ie./ 9:it (riteria <?it criteria incl!de the filled S completed chec list .ith the re#ie.ersF comments S s!))estions and the re5#erification .hether they are incorporated in the doc!ments/ Bo? 0estin), then
the De#elopment team lead chairs the re#ie. teamB leads code re#ie./ Members
6#+ ,alkthrough
A static analysis techni>!e in .hich a desi)ner or pro)rammer leads members of the de#elopment team and other interested parties thro!)h a se)ment of doc!mentation or
88 of 367
code, and the participants as
>!estions and ma e comments abo!t possible errors,
#iolation of de#elopment standards, and other problems/ 0he objecti#es of Wal thro!)h can be s!mmariCed as follo.s, Detect errors early/ <ns!re AreBestablished standards are follo.ed, 0rain and e?chan)e technical information amon) project teams .hich participate in the .al thro!)h/ 1ncrease the >!ality of the project, thereby impro#in) morale of the team members/
0he participants in Wal thro!)hs ass!me one or more of the follo.in) roles, aB Wal 5thro!)h leader bB &ecorder cB A!thor dB 0eam member 0o consider a re#ie. as a systematic .al 5thro!)h, a team of at least t.o members shall be assembled/ &oles may be shared amon) the team members/ 0he .al 5thro!)h leader or the a!thor may ser#e as the recorder/ 0he .al 5thro!)h leader may be the a!thor/ 1ndi#id!als holdin) mana)ement positions o#er any member of the .al 5thro!)h team shall not participate in the .al 5thro!)h/ 1np!t to the .al 5thro!)h shall incl!de the follo.in), aB A statement of objecti#es for the .al 5thro!)h bB 0he soft.are prod!ct bein) e?amined cB Standards that are in effect for the ac>!isition, s!pply, de#elopment, operation, and-or maintenance of the soft.are prod!ct 1np!t to the .al 5thro!)h may also incl!de the follo.in), dB Any re)!lations, standards, )!idelines, plans, and proced!res a)ainst .hich the soft.are prod!ct is to be inspected eB Anomaly cate)ories 0he .al 5thro!)h shall be considered complete .hen aB 0he entire soft.are prod!ct has been e?amined bB &ecommendations and re>!ired actions ha#e been recorded cB 0he .al 5thro!)h o!tp!t has been completed
86 of 367
6#/ Ins&ection
A static analysis techni>!e that relies on #is!al e?amination of de#elopment prod!cts to detect errors, #iolations of de#elopment standards, and other problems/ 0ypes incl!de code inspectionR desi)n inspection, Architect!ral inspections, 0est .are inspections etc/ 0he participants in 1nspections ass!me one or more of the follo.in) roles, aB 1nspection leader bB &ecorder cB &eader dB A!thor eB 1nspector All participants in the re#ie. are inspectors/ 0he a!thor shall not act as inspection leader and sho!ld not act as reader or recorder/ Kther roles may be shared amon) the team members/ 1ndi#id!al participants may act in more than one role/ 1ndi#id!als holdin) mana)ement positions o#er any member of the inspection team shall not participate in the inspection/ 1np!t to the inspection shall incl!de the follo.in), aB A statement of objecti#es for the inspection bB 0he soft.are prod!ct to be inspected cB Doc!mented inspection proced!re dB 1nspection reportin) forms eB '!rrent anomalies or iss!es list 1np!t to the inspection may also incl!de the follo.in), fB 1nspection chec lists )B Any re)!lations, standards, )!idelines, plans, and proced!res a)ainst .hich the soft.are prod!ct is to be inspected hB *ard.are prod!ct specifications iB *ard.are performance data jB Anomaly cate)ories 0he indi#id!als may ma e additional reference material a#ailable responsible for the soft.are prod!ct .hen re>!ested by the inspection leader/ 0he p!rpose of the e?it criteria is to brin) an !nambi)!o!s clos!re to the inspection meetin)/ 0he e?it decision shall determine if the soft.are prod!ct meets the inspection e?it criteria and shall prescribe any appropriate re.or follo.in),
8= of 367
and #erification/ Specifically,
the inspection team shall identify the soft.are prod!ct disposition as one of the
aB 2ccept +ith no or minor re+or1. 0he soft.are prod!ct is accepted as is or .ith only minor re.or / AFor e?ample, that .o!ld re>!ire no f!rther #erificationB/ bB 2ccept +ith re+or1 *erification. 0he soft.are prod!ct is to be accepted after the inspection leader or a desi)nated member of the inspection team Aother than the a!thorB #erifies re.or / cB Re.inspect. Sched!le a re5inspection to #erify re.or / At a minim!m, a re5inspection shall e?amine the soft.are prod!ct areas chan)ed to resol#e anomalies identified in the last inspection, as .ell as side effects of those chan)es/
).# Testing T!&es and Techni7ues

Testing t!&es 0estin) types refer to different approaches to.ards testin) a comp!ter pro)ram, system or prod!ct/ 0he t.o types of testin) are b)ac1 bo/ testing and +hite bo/ testing, .hich .o!ld both be disc!ssed in detail in this chapter/ Another type, termed as gray bo/ testing or hybrid testing is e#ol#in) presently and it combines the feat!res of the t.o types/ Testing Techni7ues 0estin) techni>!es refer to different methods of testin) partic!lar feat!res a comp!ter pro)ram, system or prod!ct/ <ach testin) type has its o.n testin) techni>!es .hile some techni>!es combine the feat!re of both types/ Some techni>!es are <rror and anomaly detection techni>!e 1nterface chec in) $hysical !nits chec in) Loop testin) A Disc!ssed in detail in this chapterB Basis $ath testin)-Mc'abeFs cyclomatic n!mberA Disc!ssed in detail in this chapterB 'ontrol str!ct!re testin)A Disc!ssed in detail in this chapterB <rror G!essin)A Disc!ssed in detail in this chapterB Bo!ndary +al!e analysis A Disc!ssed in detail in this chapterB Graph based testin)A Disc!ssed in detail in this chapterB <>!i#alence partitionin)A Disc!ssed in detail in this chapterB 1nstr!mentation based testin) &andom testin) Domain testin)
84 of 367
*alsteadFs soft.are science And many more
Some of these and many others .o!ld be disc!ssed in the later sections of this chapter/
%ifference between Testing T!&es and Testing Techni7ues 0estin) types deal .ith .hat aspect of the comp!ter soft.are .o!ld be tested, .hile testin) techni>!es deal .ith ho. a specific part of the soft.are .o!ld be tested/ 0hat is, testin) types mean .hether .e are testin) the f!nction or the str!ct!re of the soft.are/ 1n other .ords, .e may test each f!nction of the soft.are to see if it is operational or .e may test the internal components of the soft.are to chec internal .or in)s are accordin) to specification/ Kn the other hand, D0estin) techni>!eF means .hat methods or .ays .o!ld be applied or calc!lations .o!ld be done to test a partic!lar feat!re of a soft.are ASometimes .e test the interfaces, sometimes .e test the se)ments, sometimes loops etc/B ow to (hoose a Black Bo: or ,hite Bo: TestWhite bo? testin) is concerned only .ith testin) the soft.are prod!ctR it cannot )!arantee that the complete specification has been implemented/ Blac implementation ha#e been tested/ 0h!s blac bo? testin) is concerned only .ith testin) the specificationR it cannot )!arantee that all parts of the bo? testin) is testin) a)ainst the specification and .ill disco#er fa!lts of omission, indicatin) that part of the specification has not been f!lfilled/ White bo? testin) is testin) a)ainst the implementation and .ill disco#er fa!lts of commission, indicatin) that part of the implementation is fa!lty/ 1n order to completely test a soft.are prod!ct both blac and .hite bo? testin) are re>!ired/ if its
White bo? testin) is m!ch more e?pensi#e A1n terms of reso!rces and timeB than blac bo? testin)/ 1t re>!ires the so!rce code to be prod!ced before the tests can be planned and is m!ch more laborio!s in the determination of s!itable inp!t data and the determination if the soft.are is or is not correct/ 1t is ad#ised to start test plannin) .ith a blac bo? testin) approach as soon as the specification is a#ailable/ White bo? tests are to be planned as soon as the Lo. Le#el Desi)n ALLDB is complete/ 0he Lo. Le#el Desi)n .ill address all the al)orithms and codin) style/ 0he paths sho!ld then be
89 of 367
chec ed a)ainst the blac be
bo? test plan and any additional re>!ired test cases sho!ld and applied/
determined
0he conse>!ences of test fail!re at initiati#e-re>!irements sta)e are #ery e?pensi#e/ A fail!re of a test case may res!lt in a chan)e, .hich re>!ires all blac bo? testin) to be repeated and the re5determination of the .hite bo? paths/ 0he cheaper option is to re)ard the process of testin) as one of >!ality ass!rance rather than >!ality control/ 0he intention is that s!fficient >!ality is p!t into all pre#io!s desi)n and prod!ction sta)es so that it can be e?pected that testin) .ill project the presence of #ery fe. fa!lts, rather than testin) bein) relied !pon to disco#er any fa!lts in the soft.are, as in case of >!ality control/ A combination of blac bo? and .hite bo? test considerations is still not a completely ade>!ate test rationale/
).#) ,hite Bo: Testing

,hat is ,BTWhite bo? testin) in#ol#es loo in) at the str ct re of the code/ When yo! no. the
internal str!ct!re of a prod!ct, tests can be cond!cted to ens!re that the internal operations performed accordin) to the specification/ And all internal components ha#e been ade>!ately e?ercised/ 1n other .ord WB0 tends to in#ol#e the co#era)e of the specification in the code/ 'ode co#era)e is defined in si? types as listed belo./ Se)ment co#era)e I <ach se)ment of code b-. control str!ct!re is e?ec!ted at least once/ Branch 'o#era)e or "ode 0estin) I <ach branch in the code is ta en in each possible direction at least once/ 'ompo!nd 'ondition 'o#era)e I When there are m!ltiple conditions, yo! m!st test not only each direction b!t also each possible combinations of conditions, .hich is !s!ally done by !sin) a D0r!th 0ableF Basis $ath 0estin) I <ach independent path thro!)h the code is ta en in a pre5 determined order/ 0his point .ill f!rther be disc!ssed in other section/ Data Flo. 0estin) ADF0B I 1n this approach yo! trac the specific #ariables
thro!)h each possible calc!lation, th!s definin) the set of intermediate paths thro!)h the code i/e/, those based on each piece of code chosen to be trac ed/
8: of 367
<#en tho!)h the paths are considered independent, dependencies across m!ltiple paths are not really tested for by this approach/ DF0 tends to reflect dependencies b!t it is mainly thro!)h se>!ences of data manip!lation/ 0his approach tends to !nco#er b!)s li e #ariables !sed b!t not initialiCe, or declared b!t not !sed, and so on/ $ath 0estin) I $ath testin) is .here all possible paths thro!)h the code are defined and co#ered/ 0his testin) is e?tremely laborio!s and time cons!min)/ Loop 0estin) I 1n addition top abo#e meas!res, there are testin) strate)ies based on loop testin)/ 0hese strate)ies relate to testin) sin)le loops, concatenated loops, and nested loops/ Loops are fairly simple to test !nless dependencies e?ist amon) the loop or b-. a loop and the code it contains/ ,hat do we do in ,BT1n WB0, .e !se the contro) str ct re of the proced!ral desi)n to deri#e test cases/ (sin) WB0 methods a tester can deri#e the test cases that G!arantee that all independent paths .ithin a mod!le ha#e been e?ercised at least once/ <?ercise all lo)ical decisions on their tr!e and false #al!es/ <?ec!te all loops at their bo!ndaries and .ithin their operational bo!nds <?ercise internal data str!ct!res to ens!re their #alidity/
White bo? testin) AWB0B is also called Str ct ra) or ")ass bo/ testin)/ ,h! ,BTWe do WB0 beca!se Blac bo? testin) is !nli ely to !nco#er n!mero!s sorts of
defects in the pro)ram/ 0hese defects can be of the follo.in) nat!re,
Logic errors and incorrect ass mptions are in#ersely proportional to the probability that a pro)ram path .ill be e?ec!ted/ <rror tend to creep into o!r .or .hen .e desi)n and implement f!nctions, conditions or controls that are o!t of the pro)ram
0he lo)ical flo. of the pro)ram is sometimes co!nterint!iti#e, meanin) that o!r !nconscio!s ass!mptions abo!t flo. of control and data may lead to design errors that are !nco#ered only .hen path testin) starts/
8; of 367
Typographica) errors are random, some of .hich .ill be !nco#ered by synta? chec in) mechanisms b!t others .ill )o !ndetected !ntil testin) be)ins/
62 of 367
Skills Re7uired 0al in) theoretically, all .e need to do in WB0 is to define all lo)ical paths, de#elop test cases to e?ercise them and e#al!ate res!lts i/e/ )enerate test cases to e?ercise the pro)ram lo)ic e?ha!sti#ely/ For this .e need to no. the pro)ram .ell i/e/ We sho!ld no. the specification
and the code to be testedR related doc!ments sho!ld be a#ailable too !s /We m!st be able to tell the e?pected stat!s of the pro)ram #ers!s the act!al stat!s fo!nd at any point d!rin) the testin) process/ Limitations (nfort!nately in WB0, e?ha!sti#e testin) of a code presents certain lo)istical problems/ <#en for small pro)rams, the n!mber of possible lo)ical paths can be #ery lar)e/ For instance, a 322 line ' Lan)!a)e pro)ram that contains t.o nested loops e?ec!tin) 3 to 72 times dependin) !pon some initial inp!t after some basic data declaration/ 1nside the interior loop fo!r if5then5else constr!cts are re>!ired/ 0hen there are appro?imately 3236 lo)ical paths that are to be e?ercised to test the pro)ram e?ha!sti#ely/ Which means that a ma)ic test processor de#elopin) a sin)le test case, e?ec!te it and e#al!ate res!lts in one millisecond .o!ld re>!ire 8392 years .or in) contin!o!sly for this e?ha!sti#e testin) .hich is certainly impractical/ <?ha!sti#e WB0 is impossible for lar)e soft.are systems/ B!t that doesnFt mean WB0 sho!ld be considered as impractical/ Limited WB0 in .hich a limited no/ of important lo)ical paths are selected and e?ercised and important data str!ct!res are probed for #alidity, is both practical and WB0/ 1t is s!))ested that .hite and blac bo? testin) techni>!es can be co!pled to pro#ide an approach that that #alidates the soft.are interface selecti#ely ens!rin) the correction of internal .or in) of the soft.are/ Tools used for ,hite Bo: testing: Fe. 0est a!tomation tool #endors offer .hite bo? testin) tools .hich, 3B $ro#ide r!n5time error and memory lea detectionR 7B &ecord the e?act amo!nt of time the application spends in any )i#en bloc of code for the p!rpose of findin) inefficient code bottlenec sR and 8B $inpoint areas of the application that ha#e and ha#e not been e?ec!ted/
63 of 367
).#)#) Basis Path Testing Basis path testin) is a +hite bo/ testing techni>!e first proposed by 0om Mc'abe/ 0he Basis path method enables to deri#e a lo)ical comple?ity meas!re of a proced!ral desi)n and !se this meas!re as a )!ide for definin) a basis set of e?ec!tion paths/ 0est 'ases deri#ed to e?ercise the basis set are )!aranteed to e?ec!te e#ery statement in the pro)ram at least one time d!rin) testin)/ ).#)#+ Flow Gra&h $otation 0he flo. )raph depicts lo)ical control flo. !sin) a dia)rammatic notation/ <ach str!ct!red constr!ct has a correspondin) flo. )raph symbol/ ).#)#/ (!clomatic (om&le:it! Cyc)omatic comp)e/ity is a soft+are metric that pro*ides a 7 antitati*e meas re of the )ogica) comp)e/ity of a program. When !sed in the conte?t of a basis path testin) method, the #al!e comp!ted for 'yclomatic comple?ity defines the n!mber for independent paths in the basis set of a pro)ram and pro#ides !s an !pper bo!nd for the n!mber of tests that m!st be cond!cted to ens!re that all statements ha#e been e?ec!ted at least once/ An independent path is any path thro!)h the pro)ram that introd!ces at least one ne. set of processin) statements or a ne. condition/
(om&uting (!clomatic (om&le:it! 'yclomatic comple?ity has a fo!ndation in )raph theory and pro#ides !s .ith e?tremely !sef!l soft.are metric/ 'omple?ity is comp!ted in one of the three .ays, 3/ 0he n!mber of re)ions of the flo. )raph corresponds to the 'yclomatic comple?ity/ 7/ 'yclomatic comple?ity, +AGB, for a flo. )raph, G is defined as + AGB T <5"J7 Where <, is the n!mber of flo. )raph ed)es, " is the n!mber of flo. )raph nodes/ 8/ 'yclomatic comple?ity, + AGB for a flo. )raph, G is also defined as, + AGB T $J3 Where $ is the n!mber of predicate nodes contained in the flo. )raph G/ ).#)#0 Gra&h ;atrices 0he proced!re for deri#in) the flo. )raph and e#en determinin) a set of basis paths is amenable to mechaniCation/ 0o de#elop a soft.are tool that assists in basis path testin), a data str!ct!re, called a graph matri/ can be >!ite !sef!l/
67 of 367
A "raph 8atri/ is a s>!are matri? .hose siCe is e>!al to the n!mber of nodes on the flo. )raph/ <ach ro. and col!mn corresponds to an identified node, and matri? entries correspond to connections bet.een nodes/ ).#)#1 (ontrol Structure Testing Described belo. are some of the #ariations of 'ontrol Str!ct!re 0estin)/ (ondition Testing 'ondition testin) is a test case desi)n method that e?ercises the lo)ical conditions contained in a pro)ram mod!le/ %ata Flow Testing 0he data flo. testin) method selects test paths of a pro)ram accordin) to the locations of definitions and !ses of #ariables in the pro)ram/ ).#)#* Loo& Testing Loop 0estin) is a .hite bo? testin) techni>!e that foc!ses e?cl!si#ely on the #alidity of loop constr!cts/ Fo!r classes of loops can be defined, Simple loops, 'oncatenated loops, nested loops, and !nstr!ct!red loops/ Sim&le Loo&s 0he follo.in) sets of tests can be applied to simple loops, .here DnF is the ma?im!m n!mber of allo.able passes thro!)h the loop/ 3/ S ip the loop entirely/ 7/ Knly one pass thro!)h the loop/ 8/ 0.o passes thro!)h the loop/ 6/ DmF passes thro!)h the loop .here mUn/ =/ n53, n, nJ3 passes thro!)h the loop/ $ested Loo&s 1f .e e?tend the test approach from simple loops to nested loops, the n!mber of possible tests .o!ld )ro. )eometrically as the le#el of nestin) increases/ 3/ Start at the innermost loop/ Set all other loops to minim!m #al!es/ 7/ 'ond!ct simple loop tests for the innermost loop .hile holdin) the o!ter loops at their minim!m iteration parameter #al!es/ Add other tests for o!t5of5 ran)e or e?cl!de #al!es/ 8/ Wor o!t.ard, cond!ctin) tests for the ne?t loop, b!t eep all other o!ter loops at minim!m #al!es and other nested loops to GtypicalH #al!es/ 6/ 'ontin!e !ntil all loops ha#e been tested/
68 of 367
(oncatenated Loo&s 'oncatenated loops can be tested !sin) the approach defined for simple loops, if each of the loops is independent of the other/ *o.e#er, if t.o loops are concatenated and the loop co!nter for loop 3 is !sed as the initial #al!e for loop 7, then the loops are not independent/ 8nstructured Loo&s Whene#er possible, this class of loops sho!ld be redesi)ned to reflect the !se of the str!ct!red pro)rammin) constr!cts/
).#+ Black Bo: Testing

Blac bo? is a test desi)n method/ Blac bo? testin) treats the system as a Pblac 5bo?P, so it doesnQt e?plicitly !se Kno.led)e of the internal str!ct!re/ Kr in other .ords the 0est en)ineer need not no. the internal .or in) of the GBlac bo?H/ 1t foc!ses on the f!nctionality part of the mod!le/ Some people li e to call blac bo? testin) as beha#ioral, f!nctional, opa>!e5bo?, and bo? is most pop!larly !se, many people prefer the
closed5bo?/ While the term blac
terms Pbeha#ioralP and Pstr!ct!ralP for blac bo? and .hite bo? respecti#ely/ Beha#ioral test desi)n is sli)htly different from blac 5bo? test desi)n beca!se the !se of internal no.led)e isnQt strictly forbidden, b!t itQs still disco!ra)ed/ $ersonally .e feel that there is a trade off bet.een the approaches !sed to test a prod!ct !sin) .hite bo? and blac bo? types/ 0here are some b!)s that cannot be fo!nd !sin) only blac bo? or only .hite bo?/ 1f the test cases are e?tensi#e and the test inp!ts are also from a lar)e sample space then it is al.ays possible to find majority of the b!)s thro!)h blac bo? testin)/ Tools used for Black Bo: testing: Many tool #endors ha#e been prod!cin) tools for a!tomated blac capt!re the res!lts of blac bo? and a!tomated .hite bo? testin) for se#eral years/ 0he basic f!nctional or re)ression testin) tools bo? tests in a script format/ Knce capt!red, these scripts can be e?ec!ted a)ainst f!t!re b!ilds of an application to #erify that ne. f!nctionality hasnQt disabled pre#io!s f!nctionality/ 'dvantages of Black Bo: Testing 5 0ester can be non5technical/ 5 0his testin) is most li ely to find those b!)s as the !ser .o!ld find/
66 of 367
5 0estin) helps to identify the #a)!eness and contradiction in f!nctional specifications/ 5 0est cases can be desi)ned as soon as the f!nctional specifications are complete %isadvantages of Black Bo: Testing 5 'hances of ha#in) repetition of tests that are already done by pro)rammer/ 5 0he test inp!ts needs to be from lar)e sample space/ 5 1t is diffic!lt to identify all possible inp!ts in limited testin) time/ So .ritin) test cases is slo. and diffic!lt 'hances of ha#in) !nidentified paths d!rin) this testin) ).#+#) Gra&h Based Testing ;ethods Soft.are testin) be)ins by creatin) a )raph of important objects and their relationships and then de#isin) a series of tests that .ill co#er the )raph so that each objects and their relationships and then de#isin) a series of tests that .ill co#er the )raph so that each object and relationship is e?ercised and error is !nco#ered/ ).#+#+ 9rror Guessing <rror G!essin) comes .ith e?perience .ith the technolo)y and the project/ <rror G!essin) is the art of )!essin) .here errors can be hidden/ 0here are no specific tools and techni>!es for this, b!t yo! can .rite test cases dependin) on the sit!ation, <ither .hen readin) the f!nctional doc!ments or .hen yo! are testin) and find an error that yo! ha#e not doc!mented/ ).#+#/ Boundar! "alue 'nal!sis Bo!ndary +al!e Analysis AB+AB is a test data selection techni>!e AF!nctional 0estin) techni>!eB .here the e?treme #al!es are chosen/ Bo!ndary #al!es incl!de ma?im!m, minim!m, j!st inside-o!tside bo!ndaries, typical #al!es, and error #al!es/ 0he hope is that, if a system .or s correctly for these special #al!es then it .ill .or correctly for all #al!es in bet.een/ <?tends e>!i#alence partitionin) 0est both sides of each bo!ndary Loo at o!tp!t bo!ndaries for test cases too 0est min, min53, ma?, ma?J3, typical #al!es
B+A foc!ses on the bo!ndary of the inp!t space to identify test cases &ational is that errors tend to occ!r near the e?treme #al!es of an inp!t #ariable
6= of 367
There are two wa!s to generali?e the B"' techni7ues: 3/ By the n!mber of #ariables o o For n #ariables, B+A yields 6n J 3 test cases/
7/ By the inds of ran)es GeneraliCin) ran)es depends on the nat!re or type of #ariables "e?tDate has a #ariable Month and the ran)e co!ld be defined as VJan, Feb, WDecX Min T Jan, Min J3 T Feb, etc/ 0rian)le had a declared ran)e of V3, 72,222X Boolean #ariables ha#e e?treme #al!es 0r!e and False b!t there is no clear choice for the remainin) three #al!es 'dvantages of Boundar! "alue 'nal!sis 3/ &ob!stness 0estin) 5 Bo!ndary +al!e Analysis pl!s #al!es that )o beyond the limits 7/ Min 5 3, Min, Min J3, "om, Ma? 53, Ma?, Ma? J3 8/ Forces attention to e?ception handlin) 6/ For stron)ly typed lan)!a)es rob!st testin) res!lts in r!n5time errors that abort normal e?ec!tion Limitations of Boundar! "alue 'nal!sis B+A .or s best .hen the pro)ram is a f!nction of se#eral independent #ariables that represent bo!nded physical >!antities 3/ 1ndependent +ariables o o o o "e?tDate test cases deri#ed from B+A .o!ld be inade>!ate, foc!sin) on the bo!ndary .o!ld not lea#e emphasis on Febr!ary or leap years Dependencies e?ist .ith "e?tDateQs Day, Month and %ear 0est cases deri#ed .itho!t consideration of the f!nction An e?ample of physical #ariables bein) tested, telephone n!mbers 5 .hat fa!lts mi)ht be re#ealed by n!mbers of 22252222, 22252223, ===5====, ;;;5;;;:, ;;;5;;;;@ ).#+#0 97uivalence Partitioning <>!i#alence partitionin) is a blac bo? testin) method that di#ides the inp!t domain of a pro)ram into classes of data from .hich test cases can be deri#ed/
64 of 367
7/ $hysical E!antities
<$ can be defined accordin) to the follo.in) )!idelines, 3/ 1f an inp!t condition specifies a ran)e, one #alid and one t.o in#alid classes are defined/ 7/ 1f an inp!t condition re>!ires a specific #al!e, one #alid and t.o in#alid e>!i#alence classes are defined/ 8/ 1f an inp!t condition specifies a member of a set, one #alid and one in#alid e>!i#alence class is defined/ 6/ 1f an inp!t condition is Boolean, one #alid and one in#alid class is defined/ ).#+#1 (om&arison Testing 0here are sit!ations .here independent #ersions of soft.are be de#eloped for critical applications, e#en .hen only a sin)le #ersion .ill be !sed in the deli#ered comp!ter based system/ 1t is these independent #ersions .hich form the basis of a blac testin) techni>!e called 'omparison testin) or bac 5to5bac testin)/ ).#+#* >rthogonal 'rra! Testing 0he Krtho)onal Array 0estin) Strate)y AKA0SB is a systematic, statistical .ay of testin) pair5.ise interactions by deri#in) a s!itable small set of test cases Afrom a lar)e n!mber of possibilitiesB/ bo?
))# %esigning Test (ases

0here are #ario!s techni>!es in .hich yo! can desi)n test cases/ For e?ample, the belo. ill!strated )i#es yo! an o#er#ie. as to ho. yo! deri#e test cases !sin) the basis path method, 0he basis path testin) method can be applied to a proced!ral desi)n or to so!rce code/ 0he follo.in) steps can be applied to deri#e the basis set, 3/ (se the desi)n or code as a fo!ndation, dra. correspondin) flo. )raph/ 7/ Determine the 'yclomatic comple?ity of the res!ltant flo. )raph/ 8/ Determine a basis set of linearly independent paths/ 6/ $repare test cases that .ill fore e?ec!tion of each path in the basis set/ Let !s no. see ho. to desi)n test cases in a )eneric manner, 3/ (nderstand the re>!irements doc!ment/ 7/ Brea the re>!irements into smaller re>!irements Aif it impro#es yo!r testabilityB/
69 of 367
8/ For each &e>!irement, decide .hat techni>!e yo! sho!ld !se to deri#e the test cases/ For e?ample, if yo! are testin) a Lo)in pa)e, yo! need to .rite test cases basin) on error )!essin) and also ne)ati#e cases for handlin) fail!res/ 6/ *a#e a 0raceability Matri? as follo.s, &e>!irement "o A1n &DB &e>!irement 0est 'ase "o
What this 0raceability Matri? pro#ides yo! is the co#era)e of 0estin)/ Keep fillin) in the 0raceability matri? .hen yo! complete .ritin) test caseFs for each re>!irement/
)+# "alidation Phase

0he +alidation $hase falls into pict!re after the soft.are is ready or .hen the code is bein) .ritten/ 0here are #ario!s techni>!es and testin) types that can be appropriately !sed .hile performin) the testin) acti#ities/ Let !s e?amine a fe. of them/
)+#) 8nit Testing

0his is a typical scenario of Man!al (nit 0estin) acti#ity5 A (nit is allocated to a $ro)rammer for pro)rammin)/ $ro)rammer has to !se DF!nctional SpecificationsF doc!ment as inp!t for his .or / $ro)rammer prepares D$ro)ram SpecificationsF for his (nit from the F!nctional Specifications/ $ro)ram Specifications describe the pro)rammin) approach, codin) tips for the (nitFs codin)/ (sin) these D$ro)ram specificationsF as inp!t, $ro)rammer prepares D(nit 0est 'asesF doc!ment for that partic!lar (nit/ A D(nit 0est 'ases 'hec listF may be !sed to chec the completeness of (nit 0est 'ases doc!ment/ D$ro)ram SpecificationsF and D(nit 0est 'asesF are re#ie.ed and appro#ed by E!ality Ass!rance Analyst or by peer pro)rammer/ 0he pro)rammer implements some f!nctionality for the system to be de#eloped/ 0he same is tested by referrin) the !nit test cases/ While testin) that f!nctionality if any defects ha#e been fo!nd, they are recorded !sin) the defect lo))in) tool .hiche#er is applicable/ 0he pro)rammer fi?es the b!)s fo!nd and tests the same for any errors/ Stubs and %rivers A soft.are application is made !p of a n!mber of D(nitsF, .here o!tp!t of one D(nitF )oes as an D1np!tF of another (nit/ e/)/ A DSales Krder $rintin)F pro)ram ta es a DSales KrderF as an inp!t, .hich is act!ally an o!tp!t of DSales Krder 'reationF pro)ram/
6: of 367
D!e to s!ch interfaces, independent testin) of a (nit becomes impossible/ B!t that is .hat .e .ant to doR .e .ant to test a (nit in isolationM So here .e !se DSt!bF and DDri#er/ A DDri#erF is a piece of soft.are that dri#es Ain#o esB the (nit bein) tested/ A dri#er creates necessary D1np!tsF re>!ired for the (nit and then in#o es the (nit/ A (nit may reference another (nit in its lo)ic/ A DSt!bF ta es place of s!ch s!bordinate !nit d!rin) the (nit 0estin)/ A DSt!bF is a piece of soft.are that .or s similar to a !nit .hich is referenced by the (nit bein) tested, b!t it is m!ch simpler that the act!al !nit/ A St!b .or s as a DStand5inF for the s!bordinate !nit and pro#ides the minim!m re>!ired beha#ior for that !nit/ $ro)rammer needs to create s!ch DDri#ersF and DSt!bsF for carryin) o!t (nit 0estin)/ Both the Dri#er and the St!b are ept at a minim!m le#el of comple?ity, so that they do not ind!ce any errors .hile testin) the (nit in >!estion/ <?ample 5 For (nit 0estin) of DSales Krder $rintin)F pro)ram, a DDri#erF pro)ram .ill ha#e the code .hich .ill create Sales Krder records !sin) hardcoded data and then call DSales Krder $rintin)F pro)ram/ S!ppose this printin) pro)ram !ses another !nit .hich calc!lates Sales disco!nts by some comple? calc!lations/ 0hen call to this !nit .ill be replaced by a DSt!bF, .hich .ill simply ret!rn fi? disco!nt data/ 8nit Test (ases 1t m!st be clear by no. that preparin) (nit 0est 'ases doc!ment Areferred to as (0' hereafterB is an important tas in (nit 0estin) acti#ity/ *a#in) an (0', .hich is complete .ith e#ery possible test case, leads to comp)ete (nit 0estin) and th!s )i#es an ass!rance of defect5free (nit at the end of (nit 0estin) sta)e/ So let !s disc!ss abo!t ho. to prepare a (0'/ 0hin of follo.in) aspects .hile preparin) (nit 0est 'ases I <?pected F!nctionality, Write test cases a)ainst each f!nctionality that is e?pected to be pro#ided from the (nit bein) de#eloped/ e/)/ 1f an SEL script contains commands for creatin) one table and alterin) another table then test cases sho!ld be .ritten for testin) creation of one table and alteration of another/ 1t is important that (ser &e>!irements sho!ld be traceable to F!nctional Specifications, F!nctional Specifications be traceable to $ro)ram Specifications and $ro)ram Specifications be traceable to (nit 0est 'ases/ Maintainin) s!ch traceability ens!res that the application f!lfills (ser &e>!irements/ 1np!t #al!es,
6; of 367
<#ery inp!t #al!e, Write test cases for each of the inp!ts accepted by the (nit/ e/)/ 1f a Data <ntry Form has 32 fields on it, .rite test cases for all 32 fields/
+alidation of inp!t, <#ery inp!t has certain #alidation r!le associated .ith it/ Write test cases to #alidate this r!le/ Also, there can be cross5field #alidations in .hich one field is enabled dependin) !pon inp!t of another field/ 0est cases for these sho!ld not be missed/ e/)/ A combo bo? or list bo? has a #alid set of #al!es associated .ith it/ A n!meric field may accept only positi#e #al!es/ An email address field m!st ha#e ampersand AYB and period A/B in it/ A DSales ta? codeF entered by !ser m!st belon) to the DStateF specified by the !ser/
Bo!ndary conditions, 1np!ts often ha#e minim!m and ma?im!m possible #al!es/ Do not for)et to .rite test cases for them/ e/)/ A field that accepts Dpercenta)eF on a Data <ntry Form sho!ld be able to accept inp!ts only from 3 to 322/
Limitations of data types, +ariables that hold the data ha#e their #al!e limits dependin) !pon their data types/ 1n case of comp!ted fields, it is #ery important to .rite cases to arri#e at an !pper limit #al!e of the #ariables/
'omp!tations, 1f any calc!lations are in#ol#ed in the processin), .rite test cases to chec #al!es/ the arithmetic e?pressions .ith all possible combinations of
K!tp!t #al!es, Write test cases to )enerate scenarios, .hich .ill prod!ce all types of o!tp!t #al!es that are e?pected from the (nit/ e/)/ A &eport can display one set of data if !ser chooses a partic!lar option and another set of data if !ser chooses a different option/ Write test cases to chec each of these o!tp!ts/ When the o!tp!t is a res!lt of some calc!lations bein) performed or some form!lae bein) !sed, then appro?imations play a major role and m!st be chec ed/ Screen - &eport Layo!t, Screen Layo!t or .eb pa)e layo!t and &eport layo!t m!st be tested a)ainst the re>!irements/ 1t sho!ld not happen that the screen or the report loo s bea!tif!l and perfect, b!t !ser .anted somethin) entirely differentM 1t sho!ld ens!re that pa)es and screens are consistent/ $ath co#era)e, A (nit may ha#e conditional processin) .hich res!lts in #ario!s paths the control can tra#erse thro!)h/ 0est case m!st be .ritten for each of these paths/
=2 of 367
Ass!mptions, A (nit may ass!me certain thin)s for it to f!nction/ For e?ample, a (nit may need a database to be open/ 0hen test case m!st be .ritten to chec that the (nit reports error if s!ch ass!mptions are not met/ 0ransactions, 1n case of database applications, it is important to ma e s!re that transactions are properly desi)ned and no .ay inconsistent data )ets sa#ed in the database/ Abnormal terminations, Beha#ior of the (nit in case of abnormal termination sho!ld be tested/ <rror messa)es, <rror messa)es sho!ld be short, precise and self5e?planatory/ 0hey sho!ld be properly phrased and sho!ld be free of )rammatical mista es/ 8T( %ocument Gi#en belo. is a simple format for (0' doc!ment/ Test (ase $o# 1D .hich can be referred to in other doc!ments li e D0raceability Matri?F, &oot 'a!se Analysis of Defects etc/ Test (ase &ur&ose What to test Procedure *o. to test 9:&ected Result What sho!ld happen 'ctual result What act!ally happened@ 0his col!mn can be omitted .hen Defect &ecordin) 0ool is !sed/
#ote that as this is a samp)e, +e ha*e not pro*ided co) mns for Pass!%ai) and Remar1s. <?ample, Let !s say .e .ant to .rite (0' for a Data <ntry Form belo.,
Item No. Item No. Item Name Item Item No.Price .. Item No. Item No.
Item Master Form
Gi#en belo. are some of the (nit 0est 'ases for the abo#e Form, Test Test (ase Procedure 9:&ected Result (ase &ur&ose $o#
'ctual result
=3 of 367
1tem no/ to start by DAF or DBF/
3/'reate a ne. record/ 7/0ype 1tem no/ startin) .ith DAF/ 8/0ype item no/ startin) .ith DBF/ 6/0ype item no/ startin) .ith any character other than DAF and DBF/ 3/'reate a ne. record .ith 1tem no/ startin) .ith DAF/ 7/Specify price U 3222 8/Specify price Z7222/ 6/Specify price T 3222/ =/Specify price T 7222/ 4/Specify price bet.een 3222 and 7222/
7/
1tem $rice to be bet.een 3222 to 7222 if 1tem no/ starts .ith DAF/
7,8/ Sho!ld )et accepted and control sho!ld mo#e to ne?t field/ 6/ Sho!ld not )et accepted/ An error messa)e sho!ld be displayed and control sho!ld remain in 1tem no/ field/ 7,8/<rror sho!ld )et displayed and control sho!ld remain in $rice field/ 6,=,4/Sho!ld )et accepted and control sho!ld mo#e to ne?t field/
8T( (hecklist (0' chec list may be !sed .hile re#ie.in) the (0' prepared by the pro)rammer/ As any other chec list, it contains a list of >!estions, .hich can be ans.ered as either a D%esF or a D"oF/ 0he DAspectsF list )i#en in Section 6/8 abo#e can be referred to .hile preparin) (0' chec list/ e/)/ Gi#en belo. are some of the chec points in (0' chec list I 3/ Are test cases present for all form field #alidations@ 7/ Are bo!ndary conditions considered@ 8/ Are <rror messa)es properly phrased@
%efect Recording Defect &ecordin) can be done on the same doc!ment of (0', in the col!mn of D<?pected &es!ltsF/ 0his col!mn can be d!plicated for the ne?t iterations of (nit 0estin)/ Defect &ecordin) can also be done !sin) some tools li e B!)Cilla, in .hich defects are stored in the database/ Defect &ecordin) needs to be done .ith care/ 1t sho!ld be able to indicate the problem in clear, !nambi)!o!s manner, and reprod!cin) of the defects sho!ld be easily possible from the defect information/ (onclusion
=7 of 367
<?ha!sti#e (nit 0estin) filters o!t the defects at an early sta)e in the De#elopment Life 'ycle/ 1t pro#es to be cost effecti#e and impro#es E!ality of the Soft.are before the smaller pieces are p!t to)ether to form an application as a .hole/ (nit 0estin) sho!ld be done sincerely and metic!lo!sly, the efforts are paid .ell in the lon) r!n/
)+#+ Integration Testing

1nte)ration testin) is a systematic techni>!e for constr!ctin) the pro)ram str!ct!re .hile at the same time cond!ctin) tests to !nco#er errors associated .ith interfacin)/ 0he objecti#e is to ta e !nit tested components and b!ild a pro)ram str!ct!re that has been dictated by desi)n/ (s!ally, the follo.in) methods of 1nte)ration testin) are follo.ed, 3/ 0op5do.n 1nte)ration approach/ 7/ Bottom5!p 1nte)ration approach/ )+#+#) To&@%own Integration 0op5do.n inte)ration testin) is an incremental approach to constr!ction of pro)ram str!ct!re/ Mod!les are inte)rated by mo#in) do.n.ard thro!)h the control hierarchy, be)innin) .ith the main control mod!le/ Mod!les s!bordinate to the main control mod!le are incorporated into the str!ct!re in either a depth5first or breadth5first manner/ 3/ 0he 1nte)ration process is performed in a series of fi#e steps, 7/ 0he main control mod!le is !sed as a test dri#er and st!bs are s!bstit!ted for all components directly s!bordinate to the main control mod!le/ 8/ Dependin) on the inte)ration approach selected s!bordinate st!bs are replaced one at a time .ith act!al components/ 6/ 0ests are cond!cted as each component is inte)rated/ =/ Kn completion of each set of tests, st!b is replaced .ith the real component/ 4/ &e)ression testin) may be cond!cted to ens!re that ne. errors ha#e not been introd!ced/ )+#+#+ Bottom@8& Integration Bottom5!p inte)ration testin) be)ins constr!ction and testin) .ith atomic mod!les Ai/e/ components at the lo.est le#els in the pro)ram str!ct!reB/ Beca!se components are inte)rated from the bottom !p, processin) re>!ired for components s!bordinate to a )i#en le#el is al.ays a#ailable and the need for st!bs is eliminated/ 3/ A Bottom5!p inte)ration strate)y may be implemented .ith the follo.in) steps,
=8 of 367
7/ Lo. le#el components are combined into cl!sters that perform a specific soft.are s!b f!nction/ 8/ A dri#er is .ritten to coordinate test case inp!t and o!tp!t/ 6/ 0he cl!ster is tested/ Dri#ers are remo#ed and cl!sters are combined mo#in) !p.ard in the pro)ram str!ct!re/
)+#/ S!stem Testing

System testin) concentrates on testin) the complete system .ith a #ariety of techni>!es and methods/ System 0estin) comes into pict!re after the (nit and 1nte)ration 0ests/ )+#/#) (om&atibilit! Testing 'ompatibility 0estin) concentrates on testin) .hether the )i#en application )oes .ell .ith third party tools, soft.are or hard.are platform/ For e?ample, yo! ha#e de#eloped a .eb application/ 0he major compatibility iss!e is, the .eb site sho!ld .or .ell in #ario!s bro.sers/ Similarly .hen yo! de#elop if the application .or s on other applications on one platform, yo! need to chec
operatin) systems as .ell/ 0his is the main )oal of 'ompatibility 0estin)/ Before yo! be)in compatibility tests, o!r sincere s!))estion is that yo! sho!ld ha#e a cross reference matri? bet.een #ario!s soft.areFs, hard.are based on the application re>!irements/ For e?ample, let !s s!ppose yo! are testin) a .eb application/ A sample list can be as follo.s, ardware $enti!m I 11, 37: MB &AM $enti!m I 111, 7=4 MB &AM $enti!m I 1+, =37 MB &AM Software 1< 6/?, Kpera, "etscape 1< =/?, "etscape MoCilla >&erating S!stem Windo.s ;= Windo.s L$ Lin!?
'ompatibility tests are also performed for #ario!s client-ser#er based applications .here the hard.are chan)es from client to client/ 'ompatibility 0estin) is #ery cr!cial to or)aniCations de#elopin) their o.n prod!cts/ 0he prod!cts ha#e to be chec ed for compliance .ith the competitors of the third party tools, hard.are, or soft.are platform/ </)/ A 'all center prod!ct has been b!ilt for a sol!tion .ith L prod!ct b!t there is a client interested in !sin) it .ith % prod!ctR then the iss!e of compatibility arises/ 1t is of importance that the prod!ct is compatible .ith #aryin) platforms/ Within the same platform, the or)aniCation has to be .atchf!l that .ith each ne. release the prod!ct has to be tested for compatibility/ A )ood .ay to eep !p .ith this .o!ld be to ha#e a fe. reso!rces assi)ned alon) .ith their ro!tine tas s to eep !pdated abo!t s!ch compatibility iss!es and plan for testin) .hen and if the need arises/
=6 of 367
By the abo#e e?ample it is not intended that companies .hich are not de#elopin) prod!cts do not ha#e to cater for this type of testin)/ 0here case is e>!ally e?istent, if an application !ses standard soft.are then .o!ld it be able to r!n s!ccessf!lly .ith the ne.er #ersions too@ Kr if a .ebsite is r!nnin) on 1< or "etscape, .hat .ill happen .hen it is opened thro!)h Kpera or MoCilla/ *ere a)ain it is best to eep these iss!es in mind and plan for compatibility testin) in parallel to a#oid any catastrophic fail!res and delays/ )+#/#+ Recover! Testing &eco#ery testin) is a system test that foc!ses the soft.are to fall in a #ariety of .ays and #erifies that reco#ery is properly performed/ 1f it is a!tomatic reco#ery then re5 initialiCation, chec pointin) mechanisms, data reco#ery and restart sho!ld be e#al!ated for correctness/ 1f reco#ery re>!ires h!man inter#ention, the mean5time5to5 repair AM00&B is e#al!ated to determine .hether it is .ithin acceptable limits/ )+#/#/ 8sabilit! Testing (sability is the de)ree to .hich a !ser can easily learn and !se a prod!ct to achie#e a )oal/ (sability testin) is the system testin) .hich attempts to find any h!man5factor problems/ A simpler description is testin) the soft.are from a !sersF point of #ie./ <ssentially it means testin) soft.are to pro#e-ens!re that it is !ser5friendly, as distinct from testin) the f!nctionality of the soft.are/ 1n practical terms it incl!des er)onomic considerations, screen desi)n, standardiCation etc/ 0he idea behind !sability testin) is to ha#e act!al !sers perform the tas s for .hich the prod!ct .as desi)ned/ 1f they canQt do the tas s or if they ha#e diffic!lty performin) the tas s, the (1 is not ade>!ate and sho!ld be redesi)ned/ 1t sho!ld be remembered that !sability testin) is j!st one of the many techni>!es that ser#e as a basis for e#al!atin) the (1 in a !ser5centered approach/ Kther techni>!es for e#al!atin) a (1 incl!de inspection methods s!ch as he!ristic e#al!ations, e?pert re#ie.s, card5sortin), matchin) test or 1con int!iti#eness e#al!ation, co)niti#e .al thro!)hs/ 'onf!sion re)ardin) !sa)e of the term can be a#oided if .e !se D!sability e#al!ationF for the )eneric term and reser#e D!sability testin)F for the specific e#al!ation method based on !ser performance/ *e!ristic <#al!ation and (sability 1nspection or co)niti#e .al thro!)h does not in#ol#e real !sers/ 1t often in#ol#es b!ildin) prototypes of parts of the !ser interface, ha#in) representati#e !sers perform representati#e tas s and seein) if the appropriate !sers can perform the tas s/ 1n other techni>!es s!ch as the inspection methods, it is not performance, b!t
== of 367
someoneQs opinion of ho. !sers mi)ht perform that is offered as e#idence that the (1 is acceptable or not/ 0his distinction bet.een performance and opinion abo!t performance is cr!cial/ Kpinions are s!bjecti#e/ Whether a sample of !sers can accomplish .hat they .ant or not is objecti#e/ (nder many circ!mstances it is more !sef!l to find o!t if !sers can do .hat they .ant to do rather than as in) someone/ P9RF>R;I$G
T 9 T9ST
3/ Get a person .ho fits the !ser profile/ Ma e s!re that yo! are not )ettin) someone .ho has .or ed on it/
2. Sit them do.n in front of a comp!ter, )i#e them the application, and tell them a
small scenario, li e, G0han yo! for #ol!nteerin) ma in) it easier for !sers to find .hat they are loo in) for/ We .o!ld li e yo! to ans.er se#eral >!estions/ 0here is no ri)ht or .ron) ans.ers/ What .e .ant to learn is .hy yo! ma e the choices yo! do, .hat is conf!sin), .hy choose one thin) and not another, etc/ J!st tal !s thro!)h yo!r search and let !s no. .hat yo! are thin in)/ We ha#e a recorder .hich is )oin) to capt!re .hat yo! say, so yo! .ill ha#e to tell !s .hat yo! are clic in) on as yo! also tell !s .hat yo! are thin in)/ Also thin alo!d .hen yo! are st!c some.hereH 8/ "o. donFt spea anythin)/ So!nds easy, b!t see if yo! act!ally can sh!t !p/ 6/ Watch them !se the application/ 1f they as yo! somethin), tell them yo!Qre not there/ 0hen sh!t !p a)ain/ =/ Start notin) all the thin)s yo! .ill ha#e to chan)e/ 4/ After.ards as them .hat they tho!)ht and note them do.n/ 9/ Knce the .hole thin) is done than the #ol!nteer/ 8S'BILITA T9STI$G for
T>>LS '"'IL'BL9 F>R
9rgoLight 8sabilit! Software offers comprehensi#e G(1 >!ality sol!tions
the professional Windo.s application de#eloper/ <r)oLi)ht offers sol!tions for de#elopers of Windo.s applications for testin) and e#al!atin) their !sability/
,eb;etrics Tool Suite from $ational Institute of Standards and Technolog! contains rapid, remote, and a!tomated tools to help in prod!cin) !sable .eb sites/ 0he Web Static AnalyCer 0ool AWebSA0B chec s the html of a .eb pa)e a)ainst n!mero!s !sability )!idelines/ 0he o!tp!t from WebSA0 consists of identification of potential !sability problems, .hich sho!ld be in#esti)ated f!rther thro!)h !ser testin)/ 0he Web 'ate)ory Analysis 0ool AWeb'A0B lets the !sability en)ineer >!ic ly constr!ct and cond!ct a simple cate)ory analysis across the .eb/
=4 of 367
Bobb! from (enter for '&&lied S&ecial Technolog! is a .eb5based p!blic ser#ice offered by 'AS0 that analyCes .eb pa)es for their accessibility to people .ith disabilities as .ell as their compatibility .ith #ario!s bro.sers/
%R8; from Serco 8sabilit! Services is a tool, .hich has been de#eloped by close cooperation bet.een *!man Factors professionals and soft.are en)ineers to pro#ide a broad ran)e of s!pport for #ideo5assisted obser#ational st!dies/
Form Testing Suite from (or&orate Research and 'dvanced %evelo&mentB %igital 97ui&ment (or&oration $ro#ides a test s!ite de#eloped to test #ario!s .eb bro.sers/ 0he test res!lts section pro#ides a description of the tests/
8S'BILITA L'BS
0he 8sabilit! (enter A(LABB is a f!ll ser#ice or)aniCation, .hich pro#ides a PStreet5WiseP approach to !sability ris mana)ement and prod!ct !sability e?cellence/ 1t has c!stom desi)ned (LAB facilities/
8sabilit! Sciences (or&oration has a !sability lab in Dallas consistin) of t.o lar)e offices separated by a one .ay mirror/ 0he test room in each lab is e>!ipped .ith m!ltiple #ideo cameras, a!dio e>!ipment, as .ell as e#erythin) a !ser needs to operate the pro)ram/ 0he #ideo control and obser#ation room feat!res fi#e monitors, a #ideo recorder .ith special effects s.itchin), t.o5.ay a!dio system, remote camera controls, a $' for test lo) p!rposes, and a telephone for !se as a help des /
8ser,orksB Inc/ Aformerly Man5Made SystemsB is a cons!ltin) firm in the Washin)ton, D' area specialiCin) in the desi)n of !ser5prod!ct interfaces/ (serWor s does analyses, mar et research, !ser interface desi)n, rapid prototypin), prod!ct !sability e#al!ations, competiti#e testin) and analyses, er)onomic analyses, and h!man factors contract research/ (serWor s offers se#eral portable !sability labs Aa!dio5#ideo data collection systemsB for sale or rent and an obser#ational data lo))in) soft.are prod!ct for sale/
Lodestone Research has !sability5testin) laboratory .ith state of the art a!dio and #is!al recordin) and testin) e>!ipment/ All e>!ipment has been desi)ned to be portable so that it can be ta en on the road/ 0he lab consists of a test room and an obser#ation-control room that can seat as many as ten obser#ers/ A5+ e>!ipment incl!des t.o Asoon to be 8B f!lly controllable S+*S cameras, capt!re-feed capabilities for test participantQs $' #ia scan con#erter and direct split si)nal Ato +GA Psla#eP monitors in obser#ation roomB, !p to ei)ht #ideo monitors and fo!r +'A monitors for obser#er #ie.in), mi?in)-editin)
=9 of 367
e>!ipment, and P.iretapP capabilities to monitor and record both sides of telephone con#ersation Ae/)/, if participant calls c!stomer s!pportB/
>nline (om&uter Librar! (enterB Inc pro#ides insi)ht into the !sability test laboratory/ 1t )i#es an o#er#ie. of the infrastr!ct!re as .ell as the process bein) !sed in the laboratory/
9$%
G>'LS >F
8S'BILITA T9STI$G
0o s!mmariCe the )oals, it can be said that it ma es the soft.are more !ser friendly/ 0he end res!lt .ill be, Better >!ality soft.are/ Soft.are is easier to !se/ Soft.are is more readily accepted by !sers/ Shortens the learnin) c!r#e for ne. !sers/
)+#/#0 Securit! Testing Sec!rity testin) attempts to #erify that protection mechanisms b!ilt into a system .ill, in fact, protect it from improper penetration/ D!rin) Sec!rity testin), pass.ord crac in), !na!thoriCed entry into the soft.are, net.or consideration/ sec!rity are all ta en into
)+#/#1 Stress Testing Stress testin) e?ec!tes a system in a manner that demands reso!rces in abnormal >!antity, fre>!ency, or #ol!me/ 0he follo.in) types of tests may be cond!cted d!rin) stress testin)R Special tests may be desi)ned that )enerate ten interr!pts per second, .hen one or t.o is the a#era)e rate/ 1np!t data rates may be increases by an order of ma)nit!de to determine ho. inp!t f!nctions .ill respond/ 0est 'ases that re>!ire ma?im!m memory or other reso!rces/ 0est 'ases that may ca!se e?cessi#e h!ntin) for dis 5resident data/ 0est 'ases that my ca!se thrashin) in a #irt!al operatin) system/
)+#/#* Performance Testing $erformance testin) of a Web site is basically the process of !nderstandin) ho. the Web application and its operatin) en#ironment respond at #ario!s !ser load le#els/ 1n
=: of 367
)eneral, .e .ant to meas!re the Res&onse TimeB Through&ut, and 8tili?ation of the Web site .hile sim!latin) attempts by #irt!al !sers to sim!ltaneo!sly access the site/ Kne of the main objecti#es of performance testin) is to maintain a Web site .ith lo. response time, hi)h thro!)hp!t, and lo. !tiliCation/ Res&onse Time &esponse 0ime is the delay e?perienced .hen a re>!est is made to the ser#er and the ser#erQs response to the client is recei#ed/ 1t is !s!ally meas!red in !nits of time, s!ch as seconds or milliseconds/ Generally spea in), &esponse 0ime increases as the in#erse of !n!tiliCed capacity/ 1t increases slo.ly at lo. le#els of !ser load, b!t increases rapidly as capacity is !tiliCed/ Fi)!re 3 demonstrates s!ch typical characteristics of &esponse 0ime #ers!s !ser load/
Figure)# T!&ical characteristics of latenc! versus user load 0he s!dden increase in response time is often ca!sed by the ma?im!m !tiliCation of one or more system reso!rces/ For e?ample, most Web ser#ers can be confi)!red to start !p a fi?ed n!mber of threads to handle conc!rrent !ser re>!ests/ 1f the n!mber of conc!rrent re>!ests is )reater than the n!mber of threads a#ailable, any incomin) re>!ests .ill be placed in a >!e!e and .ill .ait for their t!rn to be processed/ Any time spent in a >!e!e nat!rally adds e?tra .ait time to the o#erall &esponse 0ime/ 0o better !nderstand .hat &esponse 0ime means in a typical Web farm, .e can di#ide response time into many se)ments and cate)oriCe these se)ments into t.o major types, net.or response time and application response time/ "et.or response time refers to the time it ta es for data to tra#el from one ser#er to another/ Application response time is the time re>!ired for data to be processed .ithin a ser#er/ Fi)!re 7 sho.s the different response time in the entire process of a typical Web re>!est/
=; of 367
Fi)!re 7 sho.s the different response time in the entire process of a typical Web re>!est/ 0otal &esponse 0ime T A"3 J "7 J "8 J "6B J AA3 J A7 J A8B Where "/ represents the net.or &esponse 0ime/ 1n )eneral, the &esponse 0ime is mainly constrained by "3 and "6/ 0his &esponse 0ime represents the method yo!r clients are !sin) to access the 1nternet/ 1n the most common scenario, e5commerce clients access the 1nternet !sin) relati#ely slo. dial5!p connections/ Knce 1nternet access is achie#ed, a clientQs re>!est .ill spend an indeterminate amo!nt of time in the 1nternet clo!d sho.n in Fi)!re 7 as re>!ests and responses are f!nneled from ro!ter to ro!ter across the 1nternet/ 0o red!ce these net.or s &esponse 0ime A"3 and "6B, one common sol!tion is to mo#e the ser#ers and-or Web contents closer to the clients/ 0his can be achie#ed by hostin) yo!r farm of ser#ers or replicatin) yo!r Web contents .ith major 1nternet hostin) pro#iders .ho ha#e red!ndant hi)h5speed connections to major p!blic and pri#ate 1nternet e?chan)e points, th!s red!cin) the n!mber of net.or the clients and the ser#ers/ "et.or &esponse 0imes "7 and "8 !s!ally depend on the performance of the s.itchin) e>!ipment in the ser#er farm/ When traffic to the bac 5end database )ro.s, consider !p)radin) the s.itches and net.or adapters to boost performance/ &ed!cin) application &esponse 0imes AA3, A7, and A8B is an art form !nto itself beca!se the comple?ity of ser#er applications can ma e analyCin) performance data and performance t!nin) >!ite challen)in)/ 0ypically, m!ltiple soft.are components interact on the ser#er to ser#ice a )i#en re>!est/ &esponse time can be introd!ced by any of the components/ 0hat said, there are .ays yo! can approach the problem, First, yo!r application desi)n sho!ld minimiCe ro!nd trips .here#er possible/ M!ltiple ro!nd trips Aclient to ser#er or application to databaseB m!ltiply transmission and reso!rce ac>!isition &esponse time/ (se a sin)le ro!nd trip .here#er possible/
42 of 367
&esponse 0ime and A/ represents the application
ro!tin) hops bet.een
%o! can optimiCe many ser#er components to impro#e performance for yo!r confi)!ration/ Database t!nin) is one of the most important areas on .hich to foc!s/ KptimiCe stored proced!res and inde?es/
Loo
for contention amon) threads or components competin) for common
reso!rces/ 0here are se#eral methods yo! can !se to identify contention bottlenec s/ Dependin) on the specific problem, eliminatin) a reso!rce contention bottlenec may in#ol#e restr!ct!rin) yo!r code, applyin) ser#ice pac s, or !p)radin) components on yo!r ser#er/ "ot all reso!rce contention problems can be completely eliminated, b!t yo! sho!ld stri#e to red!ce them .here#er possible/ 0hey can become bottlenec s for the entire system/ Finally, to increase capacity, yo! may .ant to !p)rade the ser#er hard.are Ascalin) !pB, if system reso!rces s!ch as '$( or memory are stretched o!t and ha#e become the bottlenec / (sin) m!ltiple ser#ers as a cl!ster Ascalin) o!tB may help to lessen the load on an indi#id!al ser#er, th!s impro#in) system performance and red!cin) application latencies/ Through&ut 0hro!)hp!t refers to the n!mber of client re>!ests processed .ithin a certain !nit of time/ 0ypically, the !nit of meas!rement is re>!ests per second or pa)es per second/ From a mar etin) perspecti#e, thro!)hp!t may also be meas!red in terms of #isitors per day or pa)e #ie.s per day, altho!)h smaller time !nits are more !sef!l for performance testin) beca!se applications typically see pea a#era)e load in a day/ As one of the most !sef!l metrics, the thro!)hp!t of a Web site is often meas!red and analyCed at different sta)es of the desi)n, de#elop, and deploy cycle/ For e?ample, in the process of capacity plannin), thro!)hp!t is one of the ey parameters for determinin) the hard.are and system re>!irements of a Web site/ 0hro!)hp!t also plays an important role in identifyin) performance bottlenec s and impro#in) application and system performance/ Whether a Web farm !ses a sin)le ser#er or m!ltiple ser#ers, thro!)hp!t statistics sho. similar characteristics in reactions to #ario!s !ser load le#els/ Fi)!re 8 demonstrates s!ch typical characteristics of thro!)hp!t #ers!s !ser load/ loads of se#eral times the
43 of 367
Figure /# T!&ical characteristics of through&ut versus user load As Fi)!re 8 ill!strates, the thro!)hp!t of a typical Web site increases proportionally at the initial sta)es of increasin) load/ *o.e#er, d!e to limited system reso!rces, thro!)hp!t cannot be increased indefinitely/ 1t .ill e#ent!ally reach a pea , and the o#erall performance of the site .ill start de)radin) .ith increased load/ Ma?im!m thro!)hp!t, ill!strated by the pea of the )raph in Fi)!re 8, is the ma?im!m n!mber of !ser re>!ests that can be s!pported conc!rrently by the site in the )i#en !nit of time/ "ote that it is sometimes conf!sin) to compare the thro!)hp!t metrics for yo!r Web site to the p!blished metrics of other sites/ 0he #al!e of ma?im!m thro!)hp!t #aries from site to site/ 1t mainly depends on the comple?ity of the application/ For e?ample, a Web site consistin) lar)ely of static *0ML pa)es may be able to ser#e many more re>!ests per second than a site ser#in) dynamic pa)es/ As .ith any statistic, thro!)hp!t metrics can be manip!lated by selecti#ely i)norin) some of the data/ For e?ample, in yo!r meas!rements, yo! may ha#e incl!ded separate data for all the s!pportin) files on a pa)e, s!ch as )raphic files/ Another siteQs p!blished meas!rements mi)ht consider the o#erall pa)e as one !nit/ As a res!lt, thro!)hp!t #al!es are most !sef!l for comparisons .ithin the same site, !sin) a common meas!rin) methodolo)y and set of metrics/ 1n many .ays, thro!)hp!t and &esponse time are related, as different approaches to thin in) abo!t the same problem/ 1n )eneral, sites .ith hi)h latency .ill ha#e lo. thro!)hp!t/ 1f yo! .ant to impro#e yo!r thro!)hp!t, yo! sho!ld analyCe the same criteria as yo! .o!ld to red!ce latency/ Also, meas!rement of thro!)hp!t .itho!t consideration of latency is misleadin) beca!se latency often rises !nder load before thro!)hp!t pea s/ 0his means that pea thro!)hp!t may occ!r at a latency that is !nacceptable from an application !sability standpoint/ 0his s!))ests that $erformance reports incl!de a c!t5off #al!e for &esponse time, s!ch as,7=2 re>!ests-second Y = seconds ma?im!m &esponse time
47 of 367
8tili?ation (tiliCation refers to the !sa)e le#el of different system reso!rces, s!ch as the ser#erQs '$(AsB, memory, net.or band.idth, and so forth/ 1t is !s!ally meas!red as a percenta)e of the ma?im!m a#ailable le#el of the specific reso!rce/ (tiliCation #ers!s !ser load for a Web ser#er typically prod!ces a c!r#e, as sho.n in Fi)!re 6/
Figure 0# T!&ical characteristics of utili?ation versus user load As Fi)!re 6 ill!strates, !tiliCation !s!ally increases proportionally to increasin) !ser load/ *o.e#er, it .ill top off and remain at a constant .hen the load contin!es to b!ild !p/ 1f the specific system reso!rce tops off at 3225percent !tiliCation, itQs #ery li ely that this reso!rce has become the performance bottlenec of the site/ (p)radin) the reso!rce .ith hi)her capacity .o!ld allo. )reater thro!)hp!t and lo.er latency[th!s better performance/ 1f the meas!red reso!rce does not top off close to 3225percent !tiliCation, it is probably beca!se one or more of the other system reso!rces ha#e already reached their ma?im!m !sa)e le#els/ 0hey ha#e become the performance bottlenec of the site/ 0o locate the bottlenec , yo! may need to )o thro!)h a lon) and painsta in) process of r!nnin) performance tests a)ainst each of the s!spected reso!rces, and then #erifyin) if performance is impro#ed by increasin) the capacity of the reso!rce/ 1n many cases, performance of the site .ill start deterioratin) to an !nacceptable le#el .ell before the major system reso!rces, s!ch as '$( and memory, are ma?imiCed/ For e?ample, Fi)!re = ill!strates a case .here response time rises sharply to 6= seconds .hen '$( !tiliCation has reached only 42 percent/
48 of 367
Figure 1# 'n e:am&le of Res&onse Time versus utili?ation As Fi)!re = demonstrates, monitorin) the '$( or memory !tiliCation alone may not al.ays indicate the tr!e capacity le#el of the ser#er farm .ith acceptable performance/ '&&lications While most traditional applications are desi)ned to respond to a sin)le !ser at any time, most Web applications are e?pected to s!pport a .ide ran)e of conc!rrent !sers, from a doCen to a co!ple tho!sand or more/ As a res!lt, performance testin) has become a critical component in the process of deployin) a Web application/ 1t has pro#en to be most !sef!l in Ab!t not limited toB the follo.in) areas, 'apacity plannin) B!) fi?in)
(a&acit! Planning *o. do yo! no. if yo!r ser#er confi)!ration is s!fficient to s!pport t.o million #isitors per day .ith a#era)e response time of less than fi#e seconds@ 1f yo!r company is projectin) a b!siness )ro.th of 722 percent o#er the ne?t t.o months, ho. do yo! no. if yo! need to !p)rade yo!r ser#er or add more ser#ers to the Web farm@ 'an yo!r ser#er and application s!pport a si?5fold traffic increase d!rin) the 'hristmas shoppin) season@ 'apacity plannin) is abo!t bein) prepared/ %o! need to set the hard.are and soft.are re>!irements of yo!r application so that yo!Qll ha#e s!fficient capacity to meet anticipated and !nanticipated !ser load/ Kne approach in capacity plannin) is to load5test yo!r application in a testin) Asta)in)B ser#er farm/ By sim!latin) different load le#els on the farm !sin) a Web application performance testin) tool s!ch as WAS, yo! can collect and analyCe the test res!lts to better !nderstand the performance characteristics of the application/ $erformance charts s!ch as those sho.n in Fi)!res 3, 8, and 6 can then be )enerated to sho. the e?pected &esponse 0ime, thro!)hp!t, and !tiliCation at these load le#els/
46 of 367
1n addition, yo! may also .ant to test the scalability of yo!r application .ith different hard.are confi)!rations/ For e?ample, load testin) yo!r application on ser#ers .ith one, t.o, and fo!r '$(s respecti#ely .o!ld help to determine ho. .ell the application scales .ith symmetric m!ltiprocessor ASM$B ser#ers/ Li e.ise, yo! sho!ld load test yo!r application .ith different n!mbers of cl!stered ser#ers to confirm that yo!r application scales .ell in a cl!ster en#ironment/ Altho!)h performance testin) is as important as f!nctional testin), itFs often o#erloo ed /Since the re>!irements to ens!re the performance of the system is not as strai)htfor.ard as the f!nctionalities of the system, achie#in) it correctly is more diffic!lt/ 0he effort of performance testin) is addressed in t.o .ays, Load testin) Stress testin)
Load testing Load testin) is a m!ch !sed ind!stry term for the effort of performance testin)/ *ere load means the n!mber of !sers or the traffic for the system/ Load testin) is defined as the testin) to determine .hether the system is capable of handlin) anticipated n!mber of !sers or not/ 1n Load 0estin), the #irt!al !sers are sim!lated to e?hibit the real !ser beha#ior as m!ch as possible/ <#en the !ser thin time s!ch as ho. !sers .ill ta e time to thin before inp!ttin) data .ill also be em!lated/ 1t is carried o!t to j!stify .hether the system is performin) .ell for the specified limit of load/ For e?ample, Let !s say an online5shoppin) application is anticipatin) 3222 conc!rrent !ser hits at pea period/ 1n addition, the pea period is e?pected to stay for 37 hrs/ inds of tests 0hen the system is load tested .ith 3222 #irt!al !sers for 37 hrs/ 0hese
are carried o!t in le#els, first 3 !ser, =2 !sers, and 322 !sers, 7=2 !sers, =22 !sers and so on till the anticipated limit are reached/ 0he testin) effort is closed e?actly for 3222 conc!rrent !sers/ 0he objecti#e of load testin) is to chec .hether the system can perform .ell for
specified load/ 0he system may be capable of accommodatin) more than 3222 conc!rrent !sers/ B!t, #alidatin) that is not !nder the scope of load testin)/ "o attempt is made to determine ho. many more conc!rrent !sers the system is capable of ser#icin)/ 0able 3 ill!strates the e?ample specified/
4= of 367
Stress testing Stress testin) is another ind!stry term of performance testin)/ 0ho!)h load testin) S Stress testin) are !sed synonymo!sly for performanceIrelated efforts, their )oal is different/ (nli e load testin) .here testin) is cond!cted for specified n!mber of !sers, stress testin) is cond!cted for the n!mber of conc!rrent !sers beyond the specified limit/ 0he objecti#e is to identify the ma?im!m n!mber of !sers the system can handle before brea in) do.n or de)radin) drastically/ Since the aim is to p!t more stress on system, thin time of the !ser is i)nored and the system is e?posed to e?cess load/ 0he )oals of load and stress testin) are listed in 0able 7/ &efer to table 8 for the inference dra.n thro!)h the $erformance 0estin) <fforts/ Let !s ta e the same e?ample of online shoppin) application to ill!strate the objecti#e of stress testin)/ 1t determines the ma?im!m n!mber of conc!rrent !sers an online system can ser#ice .hich can be beyond 3222 !sers Aspecified limitB/ *o.e#er, there is a possibility that the ma?im!m load that can be handled by the system may fo!nd to be same as the anticipated limit/ 0he 0ableU\\Zill!strates the e?ample specified/ Stress testin) also determines the beha#ior of the system as !ser base increases/ 1t chec s .hether the system is )oin) to de)rade )racef!lly or crash at a shot .hen the load )oes beyond the specified limit/ Table ): Load and stress testing of illustrative e:am&le T!&es of Testing Load 0estin) Stress 0estin) 3 (ser =2 (sers 322 (sers 7=2 (sers =22 (sersWWWW/ 3222(sers 3 (ser =2 (sers 322 (sers 7=2 (sers =22 (sersWWWW/ 3222(sers Beyond 3222 (sersWWW// Ma?im!m (sers 37 *o!rs 37 *o!rs $umber of (oncurrent users %uration
44 of 367
Table +: Goals of load and stress testing T!&es of testing Load testin) Goals 0estin) for anticipated !ser base +alidates .hether system is
capable of handlin) load !nder Stress testin) specified limit 0estin) beyond the anticipated !ser base 1dentifies the ma?im!m load a system can handle 'hec s at a shot Table /: Inference drawn b! load and stress testing T!&e of Testing Load 0estin) Stress 0estin) Inference Whether system A#ailable@ 1f yes, is the a#ailable system is stable@ Whether system is A#ailable@ 1f yes, is the a#ailable system is stable@ 1f %es, is it mo#in) to.ards (nstable state@ When the system is )oin) to brea drastically@ do.n or de)rade .hether the system
de)rades )racef!lly or crashes
'ond!ctin) performance testin) man!ally is almost impossible/ Load and stress tests are carried o!t .ith the help of a!tomated tools/ Some of the pop!lar tools to a!tomate performance testin) are listed in table 6/ Table 0: Load and stress testing tools Tools Load&!nner Astra load test Sil performer WebLoad EALoad e5Load e+alid WebSpray 0estMana)er
"endor Merc!ry 1nteracti#e 1nc Merc!ry 1nteracti#e 1nc Se)!e &ad#ie. Soft.are 'omp!Ware <mpiri? Soft.are Soft.are research 1nc 'A1 net.or &ational
49 of 367
Web application center test KpenLoad A"0S KpenS0A Astra Loadtest WA$0 Sitestress E!ati!mpro <asy WebLoad
Microsoft technolo)ies KpenDemand &ed Gate Soft.are Kpen so!rce Merc!ry interacti#e 1nc "o#asoft 1nc Webmaster sol!tions E!ati!m technolo)ies $rimeMail 1nc
Bug Fi:ing Some errors may not occ!r !ntil the application is !nder hi)h !ser load/ For <?ample, memory lea s can e?acerbate ser#er or application problems s!stainin) hi)h load/ $erformance testin) helps to detect and fi? s!ch problems before la!nchin) the application/ 1t is therefore recommended that de#elopers ta e an acti#e role in performance testin) their applications, especially at different major milestones of the de#elopment cycle/ )+#/#2 (ontent ;anagement Testing D'ontent Mana)ementF has )ained a predominant importance after the Web applications too a major part of o!r li#es/ What is 'ontent Mana)ement@ As the name denotes, it is mana)in) the content/ *o. do they .or @ Let !s ta e a common e?ample/ %o! are in 'hina and yo! .anted to open the %ahooM 'hinese #ersion/ When yo! choose Chinese #ersion on the main pa)e of %ahooM %o! )et to see the entire content in 'hinese/ %ahooM Wo!ld strate)ically plan and ha#e #ario!s ser#ers for #ario!s lan)!a)es/ When yo! choose a partic!lar #ersion of the pa)e, the re>!est is redirected to the ser#er .hich mana)es the 'hinese content pa)e/ 0he 'ontent Mana)ement systems help is placin) content for #ario!s p!rposes and also help in displayin) .hen the re>!est comes in/ 'ontent Mana)ement 0estin) in#ol#es, 3/ 0estin) the distrib!tion of the content/ 7/ &e>!est, &esponse 0imeFs/ 8/ 'ontent display on #ario!s bro.sers and operatin) systems/ 6/ Load distrib!tion on the ser#ers/ 1n fact all the performance related testin) sho!ld be performed for each #ersion of the .eb application .hich !ses the content mana)ement ser#ers/ )+#/#5 Regression Testing &e)ression testin) as the name s!))ests is !sed to test - chec the effect of chan)es made in the code/
4: of 367
Most of the time the testin) team is as ed to chec last min!te chan)es in the code j!st before ma in) a release to the client, in this sit!ation the testin) team needs to chec only the affected areas/ So in short for the re)ression testin) the testin) team sho!ld )et the inp!t from the de#elopment team abo!t the nat!re - amo!nt of chan)e in the fi? so that testin) team can first chec the fi? and then the side effects of the fi?/ 1n my present or)aniCation .e too faced the same problem/ So .e made a re)ression b!c et Athis is a simple e?cel sheet containin) the test cases that .e need thin ass!re !s of bare minim!m f!nctionalityB this b!c et is r!n e#ery time before the release/ 1n fact the re)ression testin) is the testin) in .hich ma?im!m a!tomation can be done/ 0he reason bein) the same set of test cases .ill be r!n on different b!ilds m!ltiple times/ B!t a)ain the e?tent of a!tomation depends on .hether the test cases .ill remain applicable o#er the time, 1n case the a!tomated test cases do not remain applicable for some amo!nt of time then test en)ineers .ill end !p in .astin) time to a!tomate and donFt )et eno!)h o!t of a!tomation/ What is &e)ression testin)@ &e)ression 0estin) is retestin) !nchan)ed se)ments of application/ 1t in#ol#es rer!nnin) tests that ha#e been pre#io!sly e?ec!ted to ens!re that the same res!lts can be achie#ed c!rrently as .ere achie#ed .hen the se)ment .as last tested/ 0he selecti#e retestin) of a soft.are system that has been modified to ens!re that any b!)s ha#e been fi?ed and that no other pre#io!sly .or in) f!nctions ha#e failed as a res!lt of the reparations and that ne.ly added feat!res ha#e not created problems .ith pre#io!s #ersions of the soft.are/ Also referred to as #erification testin), re)ression testin) is initiated after a pro)rammer has attempted to fi? a reco)niCed problem or has added so!rce code to a pro)ram that may ha#e inad#ertently introd!ced errors/ 1t is a >!ality control meas!re to ens!re that the ne.ly modified code still complies .ith its specified re>!irements and that !nmodified code has not been affected by the maintenance acti#ity/ What do yo! do d!rin) &e)ression testin)@ o o &er!nnin) of pre#io!sly cond!cted tests &e#ie.in) pre#io!sly prepared man!al proced!res
4; of 367
'omparin) the c!rrent test res!lts .ith the pre#io!sly e?ec!ted test res!lts
What are the tools a#ailable for &e)ression testin)@ Altho!)h the process is simple i/e/ the test cases that ha#e been prepared can be !sed and the e?pected res!lts are also no.n, if the process is not a!tomated it can be #ery time5cons!min) and tedio!s operation/ Some of the tools a#ailable for re)ression testin) are, &ecord and $laybac tools I *ere the pre#io!sly e?ec!ted scripts can be rer!n to #erify .hether the same set of res!lts are obtained/ </)/ &ational &obot What are the end )oals of &e)ression testin)@ o o o 0o ens!re that the !nchan)ed system se)ments f!nction properly 0o ens!re that the pre#io!sly prepared man!al proced!res remain correct after the chan)es ha#e been made to the application system 0o #erify that the data dictionary of data elements that ha#e been chan)ed is correct the effect of chan)es
&e)ression testin) as the name s!))ests is !sed to test - chec made in the code/
Most of the time the testin) team is as ed to chec the last min!te chan)es in the code j!st before ma in) a release to the client, in this sit!ation the testin) team needs to chec only the affected areas/ So in short for the re)ression testin) the testin) team sho!ld )et the inp!t from the de#elopment team abo!t the nat!re - amo!nt of chan)e in the fi? so that testin) team can first chec the fi? and then the affected areas/ 1n my present or)aniCation .e too faced the same problem/ So .e made a re)ression b!c et Athis is a simple e?cel sheet containin) the test cases that .e need thin ass!re !s of bare minim!m f!nctionalityB this b!c et is r!n e#ery time before the release/ 1n fact the re)ression testin) is the testin) in .hich ma?im!m a!tomation can be done/ 0he reason bein) the same set of test cases .ill be r!n on different b!ilds m!ltiple times/
But again the extent of automation depends on whether the test cases will remain applicable over the time, In case the
92 of 367
automated test cases do not remain applicable for some amount of time then test engineers will end up in wasting time to automate and dont get enough out of automation.
)+#0 'l&ha Testing

A soft.are prototype sta)e .hen the soft.are is first a#ailable for r!n/ *ere the soft.are has the core f!nctionalities in it b!t complete f!nctionality is not aimed at/ 1t .o!ld be able to accept inp!ts and )i#e o!tp!ts/ (s!ally the most !sed f!nctionalities Aparts of codeB are de#eloped more/ 0he test is cond!cted at the de#eloperFs site only/ 1n a soft.are de#elopment cycle, dependin) on the f!nctionalities the n!mber of alpha phases re>!ired is laid do.n in the project plan itself/ D!rin) this, the testin) is not a thro!)h one, since only the prototype of the soft.are is a#ailable/ Basic installation I !ninstallation tests, the completed core f!nctionalities are tested/ 0he f!nctionality complete area of the Alpha sta)e is )ot from the project plan doc!ment/ 'im is to identify any serio!s errors to j!d)e if the indented f!nctionalities are implemented to pro#ide to the c!stomer the feel of the soft.are
A thro!)h !nderstandin) of the prod!ct is done no./ D!rin) this phase, the test plan and test cases for the beta phase Athe ne?t sta)eB is created/ 0he errors reported are doc!mented internally for the testers and de#elopers reference/ "o iss!es are !s!ally reported and recorded in any of the defect mana)ement-b!) trac ers
Role of test lead (nderstand the system re>!irements completely/ 1nitiate the preparation of test plan for the beta phase/
Role of the tester
93 of 367
to pro#ide inp!t .hile there is still time to ma e si)nificant chan)es as the desi)n e#ol#es/ &eport errors to de#elopers
)+#1 8ser 'cce&tance Testing

(ser Acceptance testin) occ!rs j!st before the soft.are is released to the c!stomer/ 0he end5!sers alon) .ith the de#elopers perform the (ser Acceptance 0estin) .ith a certain set of test cases and typical scenarios/
)+#* Installation Testing

1nstallation testin) is often the most !nder tested area in testin)/ 0his type of testin) is performed to ens!re that all 1nstalled feat!res and options f!nction properly/ 1t is also performed to #erify that all necessary components of the application are, indeed, installed/ 1nstallation testin) sho!ld ta e care of the follo.in) points, 5 3/ 0o chec if .hile installin) prod!ct chec s for the dependent soft.are - patches say Ser#ice pac 8/ 7/ 0he prod!ct sho!ld chec #ersion/ 8/ 1nstaller sho!ld )i#e a defa!lt installation path say G',]pro)rams]/H 6/ 1nstaller sho!ld allo. !ser to install at location other then the defa!lt installation path/ =/ 'hec if the prod!ct can be installed GK#er the "et.or H 4/ 1nstallation sho!ld start a!tomatically .hen the 'D is inserted/ 9/ 1nstaller sho!ld )i#e the remo#e - &epair options/ :/ When !ninstallin), chec that all the re)istry eys, files, Dll, shortc!ts, acti#e L components are remo#ed from the system/ ;/ 0ry to install the soft.are .itho!t administrati#e pri#ile)es Alo)in as )!estB/ 32/ 0ry installin) on different operatin) system/ 0ry installin) on system ha#in) non5compliant confi)!ration s!ch as less memory &AM - *DD/ for the #ersion of the same prod!ct on the tar)et machine, say the pre#io!s #ersion sho!ld not be o#er installed on the ne.er
97 of 367
)+#2 Beta Testing

0he Beta testin) is cond!cted at one or more c!stomer sites by the end5!ser of the soft.are/ 0he beta test is a li#e application of the soft.are in an en#ironment that cannot be controlled by the de#eloper/ 0he Soft.are reaches beta sta)e .hen most of the f!nctionalities are operatin)/ 0he soft.are is tested in c!stomerFs en#ironment, )i#in) !ser the opport!nity to e?ercise the soft.are, find the errors so that they co!ld be fi?ed before prod!ct release/ Beta testin) is a detailed testin) and needs to co#er all the f!nctionalities of the prod!ct and also the dependent f!nctionality testin)/ 1t also in#ol#es the (1 testin) and doc!mentation testin)/ *ence it is essential that this is planned .ell and the tas accomplished/ 0he test plan doc!ment has to be prepared before the testin) phase is started, .hich clearly lays do.n the objecti#es, scope of test, tas s to be performed and the test matri? .hich depicts the sched!le of testin)/ Beta Testing >b=ectives <#al!ate soft.are technical content <#al!ate soft.are ease of !se <#al!ate !ser doc!mentation draft 1dentify errors &eport errors-findin)s
Role of a Test Lead
$ro#ide Test Instruction Sheet that describes items s!ch as testin) objecti#es, steps to follo., data to enter, f!nctions to in#o e/ $ro#ide feedbac forms and comments/
Role of a tester (nderstand the soft.are re>!irements and the testin) objecti#es/ 'arry o!t the test cases
&eport defects
)/# 8nderstanding 9:&lorator! Testing

"E plorator! testing involves simultaneousl! learning" planning" running tests" and reporting # trou$leshooting
98 of 367
Results%" & 'r% (em )aner% "E plorator! testing is an interactive process of concurrent product e ploration" test design and test e ecution% To the e tent that the ne t test we do is influenced $! the result of the last test we did" we are doing e plorator! testing%* & +ames ,ach% <?ploratory testin) is defined as sim!ltaneo!s test desi)n, test e?ec!tion and b!) reportin)/ 1n this approach the tester e?plores the system Afindin) o!t .hat it is and then testin) itB .itho!t ha#in) any prior test cases or test scripts/ Beca!se of this reason it also called as ad hoc testin), )!errilla testin) or int!iti#e testin)/ B!t there is some difference bet.een them/ 1n operational terms, e?ploratory testin) is an interacti#e process of conc!rrent prod!ct e?ploration, test desi)n, and test e?ec!tion/ 0he o!tcome of an e?ploratory testin) session is a set of notes abo!t the prod!ct, fail!res fo!nd, and a concise record of ho. the prod!ct .as tested/ When practiced by trained testers, it yields consistently #al!able and a!ditable res!lts/ <#ery tester performs this type of testin) at one point or the other/ 0his testin) totally depends on the s ill and creati#ity of the tester/ Different testers can e?plore the system in different .ays dependin) on their s ills/ 0h!s the tester has a #ery #ital role to play in e?ploratory testin)/ 0his approach of testin) has also been ad#ised by SW<BKK for testin) since it mi)ht !nco#er the b!)s, .hich the normal testin) mi)ht not disco#er/ A systematic approach of e?ploratory testin) can also be !sed .here there is a plan to attac the system !nder test/ 0his systematic approach of e?plorin) the system is termed FormaliCed e?ploratory testin)/ <?ploratory testin) is a po.erf!l approach in the field of testin)/ %et this approach has not )ot the reco)nition and is often mis!nderstood and not )ained the respect it needs/ 1n many sit!ations it can be more prod!cti#e than the scripted testin)/ B!t the real fact is that all testers do practice this methodolo)y sometime or the other, most often !n no.in)lyM <?ploratory testin) belie#es in conc!rrent phases of prod!ct e?ploration, test desi)n and test e?ec!tion/ 1t is cate)oriCed !nder Blac 5bo? testin)/ 1t is basically a free5style testin) approach .here yo! do not be)in .ith the !s!al proced!res of elaborate test plans and test steps/ 0he test plan and strate)y is #ery .ell in the testerFs mind/ 0he tester as s the ri)ht >!estion to the prod!ct - application and j!d)es the
96 of 367
o!tcome/ D!rin) this phase he is act!ally learnin) the prod!ct as he tests it/ 1t is interacti#e and creati#e/ A conscio!s plan by the tester )i#es )ood res!lts/ *!man bein)s are !ni>!e and thin differently, .ith a ne. set of ideas and report/ <?ploratory emer)in)/ A tester has the basic s ills to listen, read, thin
testin) is j!st tryin) to e?ploit this and str!ct!re it do.n/ 0he richness of this process is only limited to the breadth and depth of o!r ima)ination and the insi)ht into the prod!ct !nder test/ ow does it differ from the normal test &rocedures0he definition of e?ploratory testin) con#eys the difference/ 1n the normal testin) style, the test process is planned .ell in ad#ance before the act!al testin) be)ins/ *ere the test desi)n is separated from the test e?ec!tion phase/ Many a times the test desi)n and test e?ec!tion is entr!sted on different persons/ <?ploratory testin) sho!ld not be conf!sed .ith the dictionary meanin) of Gad5 hocH/ Ad hoc testin) normally refers to a process of impro#ised, imprompt! b!) searchin)/ By definition, anyone can do ad hoc testin)/ 0he term Ge?ploratory testin)H55 by Dr/ 'em Kaner, in 0estin) 'omp!ter Soft.are55refers to a sophisticated, systematic, tho!)htf!l approach to ad hoc testin)/ ,hat is formali?ed 9:&lorator! TestingA str!ct!red and reasoned approach to e?ploratory testin) is termed as FormaliCed <?ploratory 0estin)/ 0his approach consists of specific tas s, objecti#es, and deli#erables that ma e it a systematic process/ (sin) the systematic approach Ai/e/ the formaliCe approachB an o!tline of .hat to attac first, its scope, the time re>!ired to be spent etc is achie#ed/ 0he approach mi)ht be !sin) simple notes to more descripti#e charters to some #a)!e scripts/ By !sin) the systematic approach the testin) can be more or)aniCed foc!sin) at the )oal to be reached/ 0h!s sol#in) the problem .here the p!re <?ploratory 0estin) mi)ht drift a.ay from the )oal/ When .e apply <?ploratory $lannin) to 0estin), .e create <?ploratory plannin)/ 0he formaliCed approach !sed for the <?ploratory 0estin) can #ary dependin) on the #ario!s criteria li e the reso!rce, time, the
no.led)e of the application the

9= of 367
a#ailable etc/ Dependin) on these criteria, the approach !sed to attac
system .ill also #ary/ 1t may in#ol#e creatin) the o!tlines on the notepad to more sophisticated .ay by !sin) charters etc/ Some of the formal approaches !sed for <?ploratory 0estin) can be s!mmariCed as follo.s/ 1dentify the application domain/ 0he e?ploratory testin) can be performed by identifyin) the application domain/ 1f the tester has )ood no.led)e of domain, then it .o!ld be easier to test the system .itho!t ha#in) any test cases/ 1f the tester .ere .ell a.are of the domain, it .o!ld help analyCin) the system faster and better/ *is no.led)e .o!ld help in identifyin) the #ario!s .or flo.s that !s!ally e?ist in that domain/ *e .o!ld also be able to decide .hat are the different scenarios and .hich are most critical for that system/ *ence he can foc!s his testin) dependin) on the scenarios re>!ired/ 1f a EA lead is tryin) to assi)n the tester to a tas , it is ad#isable that the tester identifies the person .ho has the domain system for <0/ For e?ample, consider soft.are has been b!ilt to )enerate the in#oices for its c!stomers dependin) on the n!mber of the !nits of po.er that has been cons!med/ 1n s!ch a case e?ploratory testin) can be done by identifyin) the domain of the application/ A tester .ho has e?perience of the billin) systems for the ener)y domain .o!ld fit better than one .ho does not ha#e any application domain no.led)e/ 0he tester .ho has no.led)e in the no.s the terminolo)y !sed as .ell the scenarios no. the .ays in .hich no.led)e of that
that .o!ld be critical to the system/ *e .o!ld
#ario!s comp!tations are done/ 1n s!ch a case, tester .ith )ood no.led)e .o!ld be familiar to the terms li e to line item, billin) rate, billin) cycle and the .ays in .hich the comp!tation of in#oice .o!ld be done/ *e .o!ld e?plore the system to the best and ta es lesser time/ 1f the tester does not ha#e domain no.led)e re>!ired, then it .o!ld ta e time to !nderstand the #ario!s .or flo.s as .ell the terminolo)y !sed/ *e mi)ht not be able to foc!s on critical areas rather foc!s on the other areas/ 1dentify the p!rpose/ Another approach to <?ploratory 0estin) is by identifyin) the p!rpose of the system i/e/ What is that system !sed for/ By identifyin) the p!rpose
94 of 367
try to analyCe to .hat e?tent it is !sed/ 0he effort can be more foc!sed by identifyin) the p!rpose/ For e?ample, consider soft.are de#eloped to be !sed in Medical operations/ 1n s!ch case care sho!ld be ta en that the soft.are b!ild is 322O defect free/ *ence the effort that needs to be foc!sed is more and care sho!ld be ta en that the #ario!s .or flo.s in#ol#ed are co#ered/ Kn the other hand, if the soft.are b!ild is to pro#ide some entertainment then the criticality is lesser/ 0h!s effort that needs to be foc!sed #aries/ 1dentifyin) the p!rpose of the system or the application to be tested helps to a )reat e?tent/ 1dentify the primary and secondary f!nctions/ $rimary F!nction, Any f!nction so important that, in the estimation of a normal !ser, its inoperability or impairment .o!ld render the prod!ct !nfit for its p!rpose/ A f!nction is primary if yo! can associate it .ith the p!rpose of the prod!ct and it is essential to that p!rpose/ $rimary f!nctions define the prod!ct/ For e?ample, the f!nction of addin) te?t to a doc!ment in Microsoft Word is certainly so important that the prod!ct .o!ld be !seless .itho!t it/ Gro!ps of f!nctions, ta en to)ether, may constit!te a primary f!nction, too/ For e?ample, .hile perhaps no sin)le f!nction on the dra.in) toolbar of Word .o!ld be considered primary, the entire toolbar mi)ht be primary/ 1f so, then most of the f!nctions on that toolbar sho!ld be operable in order for the prod!ct to pass 'ertification/ Secondary F!nction or contrib!tin) f!nction, Any f!nction that
contrib!tes to the !tility of the prod!ct, b!t is not a primary f!nction/ <#en tho!)h contrib!tin) f!nctions are not primary, their inoperability may be )ro!nds for ref!sin) to )rant 'ertification/ For e?ample, !sers may be technically able to do !sef!l thin)s .ith a prod!ct, e#en if it has an G(ndoH f!nction that ne#er .or s, b!t most !sers .ill find that intolerable/ S!ch a fail!re .o!ld #iolate f!ndamental e?pectations abo!t ho. Windo.s prod!cts sho!ld .or /
99 of 367
0h!s by identifyin) the primary f!nction and secondary f!nctions for the system, testin) can be done .here more foc!s and effort can be )i#en to $rimary f!nctions compared to the secondary f!nctions/ <?ample, 'onsider a .eb based application de#eloped for online shoppin)/ For s!ch an application .e can identify the primary f!nctions and secondary f!nctions and )o ahead .ith <?ploratory 0estin)/ 0he main f!nctionality of that application is that the items selected by the !ser need to be properly added to the shoppin) cart and price to be paid is properly calc!lated/ 1f there is online payment, then sec!rity is also an aspect/ 0hese can be considered as the primary f!nctions/ Whereas the b!lletin board pro#ided or the mail f!nctionality pro#ided are considered as the secondary f!nctions/ 0h!s testin) to be performed is more foc!sed at the primary f!nctions rather than on the secondary f!nctions/ 1f the primary f!nctions do not .or main intention of ha#in) the application is lost/ 1dentify the .or flo.s/ 1dentifyin) the .or flo.s for testin) any system .itho!t any scripted test cases can be considered as one of the best approaches !sed/ 0he .or flo.s are nothin) b!t a #is!al representation of the scenarios as the system .o!ld beha#e for any )i#en inp!t/ 0he .or flo.s can be simple flo. charts or Data Flo. Dia)ramFs ADFDB or the somethin) li e state dia)rams, !se cases, models etc/ 0he .or flo.s .ill also help to identify the scope for that scenario/ 0he .or flo.s .o!ld help the tester to eep trac of the scenarios for testin)/ 1t is s!))ested that the tester na#i)ates thro!)h the application before he starts e?plorin)/ 1t helps the tester in identifyin) the #ario!s possible .or flo.s and iss!es any fo!nd .hich he is comfortable can be disc!ssed .ith the concerned team/ <?ample, 'onsider a .eb application !sed for online shoppin)/ 0he application has #ario!s lin s on the .eb pa)e/ 1f tester is tryin) to test if the items that he is addin) to cart are properly bein) added, then he sho!ld no. the flo. for the same/ *e sho!ld first identify the .or flo. for s!ch a scenario/ *e needs to lo)in and then select a cate)ory and identify the items and then add the item he .o!ld re>!ire/ 0h!s .itho!t as re>!ired then the
9: of 367
no.in) the .or flo. for s!ch a scenario .o!ld not help the tester and in the process loses his time/ 1n case he is not a.are of the system, try to na#i)ate thro!)h the application once and )et comfortable/ Knce the application is d!lly !nderstood, it is easier to test and e?plore more b!)s/ 1dentify the brea points/ Brea points are the sit!ations .here the system starts beha#in) abnormally/ 1t does not )i#e the o!tp!t it is s!pposed to )i#e/ So by identifyin) s!ch sit!ations also testin) can be done/ (se bo!ndary #al!es or in#ariance for findin) the brea points of the application/ 1n for normal most of the cases it is obser#ed that system .o!ld .or the .orse sit!ation/ <?ample, consider an application b!ild to )enerate the reports for the acco!nts department of a company dependin) on the criteria )i#en/ 1n s!ch cases try to select a .orse case of report )eneration for all the employees for their ser#ice/ 0he system mi)ht not beha#e normally in the sit!ation/ 0ry to inp!t a lar)e inp!t file to the application that pro#ides the !ser to !pload and sa#e the data )i#en/ 0ry to inp!t =22 characters in the te?t bo? of the .eb application/ 0h!s by tryin) to identify the e?treme conditions or the brea points .o!ld help the tester to !nco#er the hidden b!)s/ S!ch cases mi)ht not be co#ered in the normal scripted testin)/ *ence this helps in findin) the b!)s .hich mi)ht not co#ered in the normal testin)/ 'hec (1 a)ainst Windo.s interface etc standards/ 0he <?ploratory 0estin) can be performed by identifyin) the (ser interface standards/ 0here are set standards laid do.n for the !ser interfaces that need to be de#eloped/ 0hese !ser standards are nothin) b!t the loo and feel aspects of the interfaces the !ser interacts .ith/ 0he !ser sho!ld be comfortable .ith any of the screens that AsBhe .or in)/ 0hese aspects help the end !ser to accept the system faster/
9; of 367
inp!ts or o!tp!ts/ 0ry to )i#e inp!t that mi)ht be the ideal sit!ation or
<?ample, For Web application, o o o o 1s the bac )ro!nd as per the standards@ 1f the bri)ht bac )ro!nd is !sed, the !ser mi)ht not feel comfortable/ What is siCe of the font !sed@ Are the b!ttons of the re>!ired siCe and are they placed in the comfortable location/ Sometimes the applications are de#eloped to a#oid !sa)e of the scroll bar/ 0he content can be seen .ith o!t the need to scroll/ By identifyin) the (ser standards, define an approach to test beca!se the application de#eloped sho!ld be !ser friendly for the !serFs !sa)e/ *e sho!ld feel comfortable .hile !sin) the system/ 0he more familiar and easier the application for !sa)e, faster the !ser feels comfortable to the system/ 1dentify e?pected res!lts/ 0he tester sho!ld no. .hat he is testin) for and e?pected o!tp!t for the )i#en inp!t/ (ntil and !nless the aim of his testin) is not no.n, there is no !se of the testin) done/ Beca!se the tester may not s!cceed in distin)!ishin) the real error and normal .or flo./ First he needs to analyCe .hat is the e?pected o!tp!t for the scenario he is testin)/ <?ample, 'onsider soft.are !sed to pro#ide the !ser .ith an interface to search for the employee name in the or)aniCation )i#en some of the inp!ts li e the first name or last name or his id etc/ For s!ch a scenario, the tester sho!ld identify the e?pected o!tp!t for any combination of inp!t #al!es/ 1f the inp!t pro#ided does not res!lt in any data and sho.s a messa)e H<rror not data fo!ndH/ 0he tester sho!ld not misinterpret this as an error, beca!se this mi)ht be as per re>!irement .hen no data is fo!nd/ 1nstead for a )i#en inp!t, the messa)e sho.n is G 6265 File not fo!ndH, the tester sho!ld identify it as an error not a re>!irement/ 0h!s he sho!ld be able to distin)!ish bet.een an error and normal .or flo./ 1dentify the interfaces .ith other interfaces-e?ternal applications/ 1n the a)e of component de#elopment and ma?im!m re!sability, de#elopers try to pic !p the already de#eloped components and inte)rate them/ 0h!s, achie#in) the desired res!lt in short time/ 1n some
:2 of 367
cases it .o!ld help the tester e?plore the areas .here the components are co!pled/ 0he o!tp!t of one component sho!ld be correctly sent to other component/ *ence s!ch scenarios or .or flo.s need to be identified and e?plored more/ More foc!s on some of the sho.n areas that are more error prone/ <?ample, consider the online shoppin) application/ 0he !ser adds the items to his cart and proceeds to the payments details pa)e/ *ere the items added, their >!antity etc sho!ld be properly sent to the ne?t mod!le/ 1f there is any error in any of the data transfer process, the pay details .ill not be correct and the !ser .ill be billed .ron)/ 0here by leadin) to a major error/ 1n s!ch a scenario, more foc!s is re>!ired in the interfaces/ 0here may be e?ternal interfaces, li e the application is inte)rated .ith another application for the data/ 1n s!ch cases, foc!s sho!ld be more on the interface bet.een the t.o applications/ *o. data is bein) passed, is correct data bein) passed, if there is lar)e data, is transfer of entire data done or is system beha#in) abnormally .hen there is lar)e data are fe. points .hich sho!ld be addressed/
&ecord fail!res 1n e?ploratory testin), .e do the testin) .itho!t ha#in) any doc!mented test cases/ 1f a b!) has been fo!nd, it is #ery diffic!lt for !s to test it after fi?/ 0his is beca!se there are no doc!mented steps to na#i)ate to that partic!lar scenario/ *ence .e need to eep trac of the flo. re>!ired to reach .here a b!) has been fo!nd/ So .hile testin), it is important that at least the b!)s that ha#e been disco#ered are doc!mented/ *ence by recordin) fail!res .e are able to eep trac of .or that has been done/ 0his .o!ld also help e#en if the tester .ho .as act!ally doin) <0 is not a#ailable/ Since the doc!ment can be referred and list all the b!)s that ha#e been reported as .ell the flo.s for the same can be identified/ <?ample, for e?ample consider the online shoppin) site/ A b!) has been fo!nd .hile tryin) to add the items of )i#en cate)ory into the cart/ 1f the tester can j!st doc!ment the flo. as .ell as the error that has occ!rred,
:3 of 367
it .o!ld help the tester himself or any other tester/ 1t can be referred .hile testin) the application after a fi?/ Doc!ment iss!es and >!estion/ 0he tester tryin) to test an application !sin) <?ploratory 0estin) methodolo)y sho!ld feel comfortable to test/ *ence it is ad#isable that the tester na#i)ates thro!)h the application once and notes any ambi)!ities or >!eries he mi)ht feel/ *e can e#en )et the clarification on the .or flo.s he is not comfortable/ *ence by doc!mentin) all the iss!es and >!estions that ha#e been fo!nd .hile scannin) or na#i)atin) the application can help the tester ha#e testin) done .itho!t any loss in time/ Decompose the main tas acti#ities/ 1t is al.ays easier to .or .ith the smaller tas s .hen compared to lar)e tas s/ 0his is #ery !sef!l in performin) <?ploratory 0estin) beca!se lac of test cases mi)ht lead !s to different ro!tes/ By ha#in) a smaller tas , the scope as .ell as the bo!ndary are confined .hich .ill help the tester to foc!s on his testin) and plan accordin)ly/ 1f a bi) tas is ta en !p for testin), as .e e?plore the system, .e mi)ht )et de#iated from o!r main )oal or tas / 1t mi)ht be hard to define bo!ndaries if the application is a ne. one/ With smaller tas s, the )oal is no.n and hence the foc!s and the effort re>!ired can be properly planned/ <?ample, An application that pro#ides email facility/ 0he ne. !sers can re)ister and !se the application for the email/ 1n s!ch a scenario, the main tas itself can be di#ided into smaller tas s/ Kne tas to chec if the (1 standards are met and it is !ser friendly/ 0he other tas is to test if the ne. !sers are able to re)ister .ith the application and !se email facility/ 0h!s the t.o tas s are smaller .hich .ill the correspondin) )ro!ps to foc!s their testin) process/ 'harter5 states the )oal and the tactics to be !sed/ into smaller tas s /0he smaller ones to still smaller
:7 of 367
'harter S!mmary, o o o o o o o o o o o o GArchitectin) the 'hartersH i/e/ 0est $lannin) Brief information - )!idelines on, Mission, Why do .e test this@ What sho!ld be tested@ *o. to test AapproachB@ What problems to loo for@ Mi)ht incl!de )!idelines on, 0ools to !se Specific 0est 0echni>!es or tactics to !se What ris s are in#ol#ed Doc!ments to e?amine Desired o!tp!t from the testin)/
A charter can be simple one to more descripti#e )i#in) the strate)ies and o!tlines for the testin) process/ <?ample, 0est the application for report )eneration/ Kr/ 0est the application if the report is bein) )enerated for the date before 23-23-7222/(se the !se cases models for identifyin) the .or flo.s/
Session Based 0est Mana)ementASB0MB, Session Based 0est Mana)ement is a formaliCed approach that !ses the concept of charters and the sessions for performin) the <0/ A session is not a test case or b!) report/ 1t is the re#ie.able prod!ct prod!ced by chartered and !ninterr!pted test effort/ A session can last from 42 to ;2 min!tes, b!t there is no hard and fast r!le on the time spent for testin)/ 1f a session lasts closer to 6= min!tes, .e call it a short session/ 1f it lasts closer to t.o ho!rs, .e call it a )ong session/ <ach
:8 of 367
session desi)ned depends on the tester and the charter/ After the session is completed, each session is debriefed/ 0he primary objecti#e in the debriefin) is to !nderstand and accept the session report/ Another objecti#e is to pro#ide feedbac and coachin) to the tester/ 0he debriefin)s .o!ld help the mana)er to plan the sessions in f!t!re and also to estimate the time re>!ired for testin) the similar f!nctionality/ 0he debriefin) session is based on a)enda called $&KKF/ Past4 What happened d!rin) the session@ Res )ts4 What .as achie#ed d!rin) the session@ : t)oo14 What still needs to be done@ :bstac)es4 What )ot in the .ay of )ood testin)@ %ee)ing4 *o. does the tester feel abo!t all this@ 0he time spent Gon charterH and Gon opport!nityH is also noted/ Kpport!nity testin) is any testin) that doesnFt fit the charter of the session/ 0he tester is not restricted to his charter, and hence allo.ed to de#iate from the )oal specified if there is any scope of findin) an error/ A session can be broadly classified into three tas s Anamely the 0BS metricsB/ Session test !p, 0ime re>!ired in settin) !p the application !nder test/ 0est desi)n and e?ec!tion, 0ime re>!ired scannin) the prod!ct and test/ B!) in#esti)ation and reportin), 0ime re>!ired findin) the b!)s and reportin) to the concerned/ 0he entire session report consists of these sections, Session charter Aincl!des a mission statement, and areas to be testedB 0ester nameAsB Date and time started
:6 of 367
0as brea do.n Athe 0BS metricsB Data files 0est notes 1ss!es B!)s For each session, a session sheet is made/ 0he session sheet consist of the mission of testin), the tester details, d!ration of testin), the 0BS metrics alon) .ith the data related to testin) li e the b!)s, notes, iss!es etc/ Data files if any !sed in the testin) .o!ld also be enclosed/ 0he data collected d!rin) different testin) sessions are collected and e?ported to <?cel or some database/ All the sessions, the b!)s reported etc can be trac ed !sin) the !ni>!e id associated .ith each/ 1t is easy for the client as .ell to eep trac / 0h!s this concept of testers testin) in sessions and prod!cin) the re>!ired o!tp!t .hich are trac able is called as Session based test mana)ement/ Defect Dri#en <?ploratory 0estin), Defect dri#en e?ploratory testin) is another formaliCed approach !sed for <0/ &efect &ri*en ;/p)oratory Testing (&&;T) is a goa).oriented approach foc sed on the critica) areas identified on the &efect ana)ysis st dy based on Proced ra) Testing res )ts. 1n $roced!ral testin), the tester e?ec!tes readily a#ailable test cases, .hich are .ritten based on the re>!irement specifications/ Altho!)h the test cases are e?ec!ted completely, defects .ere fo!nd in the soft.are .hile doin) e?ploratory testin) by j!st .anderin) thro!)h the prod!ct blindly/ J!st e?plorin) the prod!ct .itho!t si)ht .as a in to )ropin) in the dar and did not help the testers !nearth all the hidden b!)s in the soft.are as they .ere not #ery s!re abo!t the areas that needed to be e?plored in the soft.are/ A reliable basis .as needed for e?plorin) the soft.are/ 0h!s Defect dri#en e?ploratory testin) is an idea of e?plorin) that part of the prod!ct based on the res!lts obtained d!rin) proced!ral testin)/ After analyCin) the defects fo!nd d!rin) the DD<0 process,
:= of 367
it .as fo!nd that these .ere the most critical b!)s, .hich .ere camo!fla)ed in the soft.are and .hich if present co!ld ha#e made the soft.are D"ot fit for (seF/ 0here are some pre re>!isites for DD<0, o o o o o / o "o .asta)e of time/ 1n5depth no.led)e of the prod!ct/ $roced!ral 0estin) has to be carried o!t/ Defect Analysis based on Scripted 0ests/ 0ester has clear cl!es on the areas to be e?plored/ Goal oriented approach , hence better res!lts
Ad#anta)es of DD<0,
,here does 9:&lorator! Testing Fit: 1n )eneral, <0 is called for in any sit!ation .here itFs not ob#io!s .hat the ne?t test sho!ld be, or .hen yo! .ant to )o beyond the ob#io!s tests/ More specifically, freestyle <?ploratory 0estin) fits in any of the follo.in) sit!ations, %o! need to pro#ide rapid feedbac on a ne. prod!ct or feat!re/ %o! need to learn the prod!ct >!ic ly/ %o! ha#e already tested !sin) scripts, and see to di#ersify the testin)/ %o! .ant to find the sin)le most important b!) in the shortest time/ %o! .ant to chec the .or of another tester by doin) a brief independent 1n#esti)ation/ %o! .ant to in#esti)ate and isolate a partic!lar defect/ %o! .ant to in#esti)ate the stat!s of a partic!lar ris , in order to e#al!ate the need for scripted tests in that area/ Pros and (ons: Pros Does not re>!ire e?tensi#e doc!mentation/ &esponsi#e to chan)in) scenarios/ (nder ti)ht sched!les, testin) can be more foc!sed dependin) on the b!) rate or ris s/ 1mpro#ed co#era)e/ (ons Dependent on the testerFs s ills/ 0est trac in) not concrete/
:4 of 367
More prone to h!man error/
"o contin)ency plan if the tester is !na#ailable/

,hat s&ecifics affect 9:&lorator! Testing*ere is a list that affects e?ploratory testin), ;ission 0he )oal of testin) needs to be !nderstood first before the .or be)ins/ 0his co!ld be the o#erall mission of the test project or co!ld be a partic!lar f!nctionality scenario/ 0he mission is achie#ed by as in) the ri)ht >!estions abo!t the prod!ct, desi)nin) tests to ans.er these >!estions and e?ec!tin) tests to )et the ans.ers/ Kften the tests do not completely ans.er, in s!ch cases .e need to e?plore/ 0he test proced!re is recorded A.hich co!ld later form part of the scripted testin)B and the res!lt stat!s too/ Tester 0he tester needs to ha#e a )eneral plan in mind, tho!)h may not be #ery constrained/ 0he tester needs to ha#e the ability to desi)n )ood test strate)y, e?ec!te )ood tests, find important problems and report them/ *e simply has to thin o!t of the bo?/ Time 0ime a#ailable for testin) is a critical factor/ 0ime falls short d!e to the follo.in) reasons, o Many a time in project life cycles, the time and reso!rces re>!ired in creatin) the test strate)y, test plan and desi)n, e?ec!tion and reportin) is o#erloo ed/ <?ploratory testin) becomes !sef!l since the test plan, desi)n and e?ec!tion happen to)ether/ o o Also .hen testin) is essential on a short period of notice A ne. feat!re is implemented 0he mission of the partic!lar test session 0he tester s ills, talents, preferences A#ailable time and other reso!rces 0he stat!s of other testin) cycles for the prod!ct *o. m!ch the tester no.s abo!t the prod!ct
:9 of 367
'han)e re>!est come in m!ch later sta)e of the cycle .hen m!ch of the testin) is done .ith
1n s!ch sit!ations e?ploratory testin) comes handy/ Practicing 9:&lorator! Testing A basic strate)y of e?ploratory testin) is to ha#e a )eneral plan of attac , b!t also allo. yo!rself to de#iate from it for short period of time/ 1n a session of e?ploratory testin), a set of test ideas, .ritten notes Asimple <n)lish or scriptsB and b!) reports are the res!lts/ 0his can be re#ie.ed by the test lead - test mana)er/ Test Strateg!
1t is important to identify the scope of the test to be carried/ 0his is dependent on the project approach to testin)/ 0he test mana)er - test lead can decide the scope and con#ey the same to the test team/ Test design and e:ecution
0he tester crafts the test by systematically e?plorin) the prod!ct/ *e defines his approach, analyCe the prod!ct, and e#al!ate the ris %ocumentation
0he .ritten notes - scripts of the tester are re#ie.ed by the test lead - mana)er/ 0hese later form into ne. test cases or !pdated test materials/ ,here does 9:&lorator! Testing Fit<?ploratory testin) fits almost in any ind of testin) projects, projects .ith
ri)oro!s test plans and proced!res or in projects .here testin) is not dictated completely in ad#ance/ 0he sit!ations .here e?ploratory testin) co!ld fit in are, "eed to pro#ide a rapid feedbac on a ne. feat!re implementation - prod!ct Little prod!ct no.led)e and need to learn it >!ic ly $rod!ct analysis and test plannin) Done .ith scripted testin) and need to di#ersify more 1mpro#e the >!ality of e?istin) test scripts Write ne. scripts
0he basic r!le is this, e?ploratory testin) is called for any time the ne?t test yo! sho!ld perform is not ob#io!s, or .hen yo! .ant to )o beyond the ob#io!s/
:: of 367
' Good 9:&lorator! Tester <?ploratory testin) approach relies a lot on the tester himself/ 0he tester acti#ely controls the desi)n of tests as they are performed and !ses the information )ained to desi)n ne. and better ideas/ A )ood e?ploratory tester sho!ld *a#e the ability to desi)n )ood tests, e?ec!te them and find important problems Sho!ld doc!ment his ideas and !se them in later cycles/ M!st be able to e?plain his .or Be a caref!l obser#er, <?ploratory testers are more caref!l obser#ers than no#ices and e?perienced scripted testers/ Scripted testers need only obser#e .hat the script tells/ <?ploratory tester m!st .atch for anythin) !n!s!al or mysterio!s/ Be a critical thin er, 0hey are able to re#ie. and e?plain their lo)ic, loo in) o!t for errors in their o.n thin in)/ *a#e di#erse ideas so as to ma e ne. test cases and impro#e e?istin) ones/
A )ood e?ploratory tester al.ays as s himself, +hat<s the best test - can perform no+= 0hey remain alert for ne. opport!nities/ 'dvantages <?ploratory testin) is ad#anta)eo!s .hen &apid testin) is essential 0est case de#elopment time not a#ailable "eed to co#er hi)h ris areas .ith more inp!ts "eed to test soft.are .ith little no.led)e abo!t the specifications De#elop ne. test cases or impro#e the e?istin) Dri#e o!t monotony of normal step I by 5 step test e?ec!tion
%rawbacks S illed tester re>!ired Diffic!lt to >!antiCe
Balancing 9:&lorator! Testing with Scri&ted Testing <?ploratory testin) relies on the tester and the approach he proceeds .ith/ $!re scripted testin) doesnFt !nder)o m!ch chan)e .ith time and hence the po.er fades a.ay/ 1n test scenarios .here in repeatability of tests are re>!ired, a!tomated scripts
:; of 367
ha#in) an ed)e o#er e?ploratory approach/ *ence it is important to achie#e a balance bet.een the t.o approaches and combine the t.o to )et the best of both/
)0# 8nderstanding Scenario Based Testing

Scenario Based 0ests ASB0B are best s!ited .hen yo! need to tests need to concentrate on the f!nctionality of the application, than anythin) else/ Let !s ta e an e?ample, .here yo! are testin) an application .hich is >!ite old Aa le)acy applicationB and it is a ban in) application/ 0his application has been b!ilt based on the re>!irements of the or)aniCation for #ario!s ban in) p!rposes/ "o., this application .ill ha#e contin!o!s !p)rades in the .or in) Atechnolo)y .ise and b!siness .iseB/ What do yo! do to test the application@ Let !s ass!me that the application is !nder)oin) only f!nctional chan)es and not the (1 chan)es/ 0he test cases sho!ld be !pdated for e#ery release/ K#er a period of time, maintainin) the test .are becomes a major set bac / 0he Scenario Based 0ests .o!ld help yo! here/ As per the re>!irements, the base f!nctionality is stable and there are no (1 chan)es/ 0here are only chan)es .ith respect to the b!siness f!nctionality/ As per the re>!irements and the sit!ation, .e clearly !nderstand that only re)ression tests need to be r!n contin!o!sly as part of the testin) phase/ K#er a period of time, the indi#id!al test cases .o!ld become diffic!lt to mana)e/ 0his is the sit!ation .here .e !se Scenarios for testin)/ ,hat do !ou do for deriving ScenariosWe can !se the follo.in) as the basis for deri#in) scenarios, =/ From the re>!irements, list o!t all the f!nctionalities of the application/ 4/ (sin) a )raph notation, dra. depictions of #ario!s transactions .hich pass thro!)h #ario!s f!nctionalities of the application/ 9/ 'on#ert these depictions into scenarios/ :/ &!n the scenarios .hen performin) the testin)/ ,ill !ou use Scenario Based Tests onl! for Legac! a&&lication testing"o/ Scenario Based 0ests are not only for le)acy application testin), b!t for any application .hich re>!ires yo! to concentrate more on the f!nctional re>!irements/ 1f yo! can plan o!t a perfect test strate)y, then the Scenario Based 0ests can be !sed for any application testin) and for any re>!irements/ Scenario Based tests .ill be a )ood choice .ith a combination of #ario!s test types and techni>!es .hen yo! are testin) projects .hich adopt (ML A(nified Modelin) Lan)!a)eB based de#elopment strate)ies/
;2 of 367
%o! can deri#e scenarios based on the (se 'aseFs/ (se 'aseFs pro#ide )ood co#era)e of the re>!irements and f!nctionality/
)1# 8nderstanding 'gile Testing

0he concept of A)ile testin) rests on the #al!es of the A)ile Alliance +al!es, .hich states that, GWe ha#e come to #al!e, 1ndi#id!als and interactions o*er processes and tools Wor in) soft.are o*er comprehensi#e doc!mentation '!stomer collaboration o*er contract ne)otiation &espondin) to chan)e o*er follo.in) a plan 0hat is, .hile there is #al!e in the items on the ri)ht, .e #al!e the items on the left more/P 5 http,--.../a)ilemanifesto/or)What is A)ile testin)@
16 A)ile testers treat the de#elopers as their c!stomer and follo. the a)ile
manifesto/ 0he Conte/t dri*en testing princip)es Ae?plained in later partB act as a set of principles for the a)ile tester/
7B
Kr it can be treated as the testin) methodolo)y follo.ed by testin) team .hen an entire project follo.s a)ile methodolo)ies/ 1f so .hat is the role of a tester in s!ch a fast paced methodolo)y@B
0raditional EA seems to be totally at lo))erheads .ith the A)ile manifesto in the follo.in) re)ard .here, $rocess and tools are a ey part of EA and testin)/ EA people seem to lo#e doc!mentation/ EA people .ant to see the .ritten specification/ And .here is testin) .itho!t a $LA"@
So the >!estion arises is there a role for EA in A)ile projects@ 0here ans.er is maybe b!t the roles and tas s are different/
;3 of 367
1n the first definition of A)ile testin) .e described it as one follo.in) the 'onte?t dri#en principles/ 0he conte?t dri#en principles .hich are )!idelines for the a)ile tester are, 3/ 0he #al!e of any practice depends on its conte?t/ 7/ 0here are )ood practices in conte?t, b!t there are no best practices/ 8/ $eople, .or in) to)ether, are the most important part of any projectFs conte?t/ 6/ $rojects !nfold o#er time in .ays that are often not predictable/ =/ 0he prod!ct is a sol!tion/ 1f the problem isnFt sol#ed, the prod!ct doesnFt .or / 4/ Good soft.are testin) is a challen)in) intellect!al process/ 9/ Knly thro!)h j!d)ment and s ill, e?ercised cooperati#ely thro!)ho!t the entire project, are .e able to do the ri)ht thin)s at the ri)ht times to effecti#ely test o!r prod!cts/ http,--.../conte?t5dri#en5testin)/com1n the second definition .e described A)ile testin) as a testin) methodolo)y adopted .hen an entire project follo.s A)ile Ade#elopmentB Methodolo)y/ We shall ha#e a loo at the A)ile de#elopment methodolo)ies bein) practiced c!rrently, 'gile %evelo&ment ;ethodologies <?treme $ro)rammin) AL$B 'rystal Adapti#e Soft.are De#elopment AASDB Scr!m Feat!re Dri#en De#elopment AFDDB Dynamic Systems De#elopment Method ADSDMB Lbreed
1n a fast paced en#ironment s!ch as in A)ile de#elopment the >!estion then arises as to .hat is the GRoleH of testin)@ 0estin) is as rele#ant in an A)ile scenario if not more than a traditional soft.are de#elopment scenario/
;7 of 367
0estin) is the >ead)ight of the a)ile project sho.in) .here the project is standin) no. and the direction it is headed/ 0estin) pro#ides the re>!ired and rele#ant information to the teams to ta e informed and precise decisions/ 0he testers in a)ile frame.or s )et in#ol#ed in m!ch more than findin) Gsoft.are b!)sH, anythin) that can Gb gH the potential !ser is a iss!e for them b!t testers donFt ma e the final call, itFs the entire team that disc!sses o#er it and ta es a decision o#er a potential iss!es/ A firm belief of A)ile practitioners is that any testin) approach does not ass!re >!ality itFs the team that does Aor doesnFtB do it, so there is a hea#y emphasis on the s ill and attit!de of the people in#ol#ed/ A)ile 0estin) is not a )ame of G)otchaH, itFs abo!t findin) .ays to set )oals rather than foc!s on mista es/ Amon) these A)ile methodolo)ies mentioned .e shall loo at L$ A<?treme
$ro)rammin)B in detail, as this is the most commonly !sed and pop!lar one/ 0he basic components of the CP practices are, 0est5 First $ro)rammin) $air $ro)rammin) Short 1terations S &eleases &efactorin) (ser Stories Acceptance 0estin)
We shall disc!ss these factors in detail/
Test@First Programming De#elopers .rite !nit tests before codin)/ 1t has been noted that this ind of approach moti#ates the codin), speeds codin) and also and impro#es desi)n res!lts in better desi)ns A.ith less co!plin) and more cohesionB 1t s!pports a practice called &efactorin) Adisc!ssed later onB/
;8 of 367
A)ile practitioners prefer 0ests AcodeB to 0e?t A.ritten doc!mentsB for describin) system beha#ior/ 0ests are more precise than h!man lan)!a)e and they are also a lot more li ely to be !pdated .hen the desi)n chan)es/ *o. many times ha#e yo! seen desi)n doc!ments that no lon)er acc!rately described the c!rrent .or in)s of the soft.are@ K!t5of5date desi)n doc!ments loo fail/ pretty m!ch li e !p5to5date doc!ments/ K!t5of5date tests
Many open so!rce tools li e ?(nit ha#e been de#eloped to s!pport this methodolo)y/
Refactoring &efactorin) is the practice chan)in) a soft.are system in s!ch a .ay that it does not alter the e?ternal beha#ior of the code yet impro#es its internal str!ct!re/ 0raditional de#elopment tries to !nderstand ho. all the code .ill .or to)ether in ad#ance/ 0his is the desi)n/ With a)ile methods, this diffic!lt process of ima)inin) .hat code mi)ht loo li e before it is .ritten is a#oided/ 1nstead, the code is restr!ct!red as needed to maintain a coherent desi)n/ Fre>!ent refactorin) allo.s less !p5front plannin) of desi)n/ A)ile methods replace hi)h5le#el desi)n .ith fre>!ent redesi)n Arefactorin)B/ S!ccessf!l refactorin) B!t it also re>!ires a .ay of ens!rin) chec in) .hether that the beha#ior .asnFt inad#ertently chan)ed/ 0hatFs .here the tests come in/ Ma e the simplest desi)n that .ill .or needed and refactor as necessary/ &efactorin) re>!ires !nit tests to ens!re that desi)n chan)es Arefactorin)sB donFt brea e?istin) code/ 'cce&tance Testing Ma e !p !ser e?periences or (ser stories, .hich are short descriptions of the feat!res to be coded/ Acceptance tests #erify the completion of !ser stories/ 1deally they are .ritten before codin)/ and add comple?ity only .hen
;6 of 367
With all these feat!res and process incl!ded .e can define a practice for A)ile testin) encompassin) the follo.in) feat!res/ 'on#ersational 0est 'reation 'oachin) 0ests $ro#idin) 0est 1nterfaces <?ploratory Learnin)
Loo in) deep into each of these practices .e can describe each of them as, 'on#ersational 0est 'reation 0est case .ritin) sho!ld be a collaborati#e acti#ity incl!din) majority of the entire team/ As the c!stomers .ill be b!sy .e sho!ld ha#e someone representin) the c!stomer/ Definin) tests is a DonQt do it alone/ ey acti#ity that sho!ld incl!de pro)rammers and c!stomer representati#es/
'oachin) 0ests A .ay of thin in) abo!t Acceptance 0ests/ 0!rn !ser stories into tests/ 0ests sho!ld pro#ide Goals and )!idance, 1nstant feedbac meas!rement 0ests sho!ld be in specified in a format that is clear eno!)h that !sersc!stomers can !nderstand and that is specific eno!)h that it can be e?ec!ted Specification sho!ld be done by e?ample/ and $ro)ress
$ro#idin) 0est 1nterfaces De#elopers are responsible for pro#idin) the fi?t!res that a!tomate coachin) tests 1n most cases L$ teams are addin) test interfaces to their prod!cts, rather than !sin) e?ternal test tools Test Interaction ;odel
;= of 367
<?ploratory Learnin) $lan to e?plore, learn and !nderstand the prod!ct .ith each iteration/ Loo for b!)s, missin) feat!res and opport!nities for impro#ement/ We donFt !nderstand soft.are !ntil .e ha#e !sed it/
We belie#e that A)ile 0estin) is a major step for.ard/ %o! may disa)ree/ B!t re)ardless A)ile $ro)rammin) is the .a#e of the f!t!re/ 0hese practices .ill de#elop and some of the e?treme ed)es may be .orn off, b!t itFs only )ro.in) in infl!ence and attraction/ Some testers may not li e it, b!t those .ho donFt fi)!re o!t ho. to li#e .ith it are simply )oin) to be left behind/ Some testers are still !pset that they donFt ha#e the a!thority to bloc they thin that they no. ha#e the a!thority to bloc the release/ Do
the adoption of these ne.
de#elopment methods@ 0heyFll need to )et on this ship and if they .ant to try to eep it from the shoals/ Stay on the doc if yo! .ish/ Bon +oya)eM
)*# 'PI Testing

2pp)ication programmab)e -nterfaces (2P-s) are collections of soft.are f!nctions or proced!res that can be !sed by other applications to f!lfill their f!nctionality/ A$1s pro#ide an interface to the soft.are component/ 0hese form the critical elements for the de#elopin) the applications and are !sed in #aried applications from )raph dra.in)
;4 of 367
pac a)es, to speech en)ines, to .eb5based airline reser#ation systems, to comp!ter sec!rity components/ <ach A$1 is s!pposed to beha#e the .ay it is coded, i/e/ it is f!nctionality specific/ 0hese A$1s may offer different res!lts for different type of the inp!t pro#ided/ 0he errors or the e?ceptions ret!rned may also #ary/ *o.e#er once inte)rated .ithin a prod!ct, the common f!nctionality co#ers a #ery minimal code path of the A$1 and the f!nctionality testin) - inte)ration testin) may co#er only those paths/ By considerin) each A$1 as a blac bo?, a )eneraliCed approach of testin) can be applied/ B!t, there may be some paths .hich are not tested and lead to b!)s in the application/ Applications can be #ie.ed and treated as A$1s from a testin) perspecti#e/ 0here are some distincti#e attrib!tes that ma e testin) of A$1s sli)htly different from testin) other common soft.are interfaces li e G(1 testin)/ Testing 2P-s re7 ires a thoro gh 1no+)edge of its inner +or1ings . Some A$1s may interact .ith the KS ernel, other A$1s, .ith other soft.are to offer their f!nctionality/ 0h!s an !nderstandin) of the inner .or in)s of the interface .o!ld help in analyCin) the call se>!ences and detectin) the fail!res ca!sed/ 2de7 ate programming s1i))s . A$1 tests are )enerally in the form of se>!ences of calls, namely, pro)rams/ <ach tester m!st possess e?pertise in the pro)rammin) lan)!a)eAsB that are tar)eted by the A$1/ 0his .o!ld help the tester to re#ie. and scr!tiniCe the interface !nder test .hen the so!rce code is a#ailable/ Lac1 of &omain 1no+)edge I Since the testers may not be .ell trained in !sin) the A$1, a lot of time mi)ht be spent in e?plorin) the interfaces and their !sa)e/ 0his problem can be sol#ed to an e?tent by in#ol#in) the testers from the initial sta)e of de#elopment/ 0his .o!ld help the testers to ha#e some !nderstandin) on the interface and a#oid e?plorin) .hile testin)/ #o doc mentation I <?perience has sho.n that it is hard to create precise and readable doc!mentation/ 0he A$1s de#eloped .ill hardly ha#e any proper doc!mentation a#ailable/ Witho!t the doc!mentation, it is diffic!lt for the test desi)ner to !nderstand the p!rpose of calls, the parameter types and possible #alid-in#alid #al!es, their ret!rn #al!es, the calls it ma es to other f!nctions,
;9 of 367
and !sa)e scenarios/ *ence ha#in) proper doc!mentation .o!ld help test desi)ner desi)n the tests faster/ 2ccess to so rce code I 0he a#ailability of the so!rce code .o!ld help tester to !nderstand and analyCe the implementation mechanism !sedR and can identify the loops or #!lnerabilities that may ca!se errors/ 0h!s if the so!rce code is not a#ailable then the tester does not ha#e a chance to find anomalies that may e?ist in the code/ Time constraints I 0horo!)h testin) of A$1s is time cons!min), re>!ires a learnin) o#erhead and reso!rces to de#elop tools and desi)n tests/ Keepin) !p .ith deadlines and ship dates may become a ni)htmare/
0estin) of A$1 calls can be done in isolation or in Se>!ence to #ary the order in .hich the f!nctionality is e?ercised and to ma e the A$1 prod!ce !sef!l res!lts from these tests/ Desi)nin) tests is essentially desi)nin) se>!ences of A$1 calls that ha#e a potential of satisfyin) the test objecti#es/ 0his in t!rn boils do.n to desi)nin) each call .ith specific parameters and to b!ildin) a mechanism for handlin) and e#al!atin) ret!rn #al!es/ 0h!s desi)nin) of the test cases can depend on some of the )eneral >!estions li e Which #al!e sho!ld a parameter ta e@ What #al!es to)ether ma e sense@ What combination of parameters .ill ma e A$1s .or in a desired manner@ What combination .ill ca!se a fail!re, a bad ret!rn #al!e, or an anomaly in the operatin) en#ironment@
Which se>!ences are the best candidates for selection@ etc/
Some interestin) problems for testers bein) 3/ <ns!rin) that the test harness #aries parameters of the A$1 calls in .ays that #erify f!nctionality and e?pose fail!res/ 0his incl!des assi)nin) common parameter #al!es as .ell as e?plorin) bo!ndary conditions/ 7/ Generatin) interestin) parameter #al!e combinations for calls .ith t.o or more parameters/
;: of 367
8/ Determinin) the content !nder .hich an A$1 call is made/ 0his mi)ht incl!de settin) e?ternal en#ironment conditions Afiles, peripheral de#ices, and so forthB and also internal stored data that affect the A$1/ 6/ Se>!encin) A$1 calls to #ary the order in .hich the f!nctionality is e?ercised and to ma e the A$1 prod!ce !sef!l res!lts from s!ccessi#e calls/ By analyCin) the problems listed abo#e, a strate)y needs to be form!lated for testin) the A$1/ 0he A$1 to be tested .o!ld re>!ire some en#ironment for it to .or / *ence it is re>!ired that all the conditions and prere>!isites !nderstood by the tester/ 0he ne?t step .o!ld be to identify and st!dy its points of entry/ 0he G(1s .o!ld ha#e items li e men!s, b!ttons, chec bo?es, and combo lists that .o!ld tri))er the e#ent or action to be ta en/ Similarly, for A$1s, the inp!t parameters, the e#ents that tri))er the A$1 .o!ld act as the point of entry/ S!bse>!ently, a chief tas is to analyCe the points of entry as .ell as si)nificant o!tp!t items/ 0he inp!t parameters sho!ld be tested .ith the #alid and in#alid #al!es !sin) strate)ies li e the bo!ndary #al!e analysis and e>!i#alence partitionin)/ 0he fo!rth step is to !nderstand the p!rpose of the ro!tines, the conte?ts in .hich they are to be !sed/ Knce all this parameter selections and combinations are desi)ned, different call se>!ences need to be e?plored/ 0he steps can be s!mmariCed as follo.in) 3/ 1dentify the initial conditions re>!ired for testin)/ 7/ 1dentify the parameters I 'hoosin) the #al!es of indi#id!al parameters/ 8/ 1dentify the combination of parameters I pic o!t the possible and applicable parameter combinations .ith m!ltiple parameters/ 6/ 1dentify the order to ma e the calls I decidin) the order in .hich to ma e the calls to force the A$1 to e?hibit its f!nctionality/ =/ Kbser#e the o!tp!t/ )#Identif! the initial condition: 0he testin) of an A$1 .o!ld depend lar)ely on the en#ironment in .hich it is to be tested/ *ence initial condition plays a #ery #ital role in !nderstandin) and #erifyin) the beha#ior of the A$1 !nder test/ 0he initial conditions for testin) A$1s can be classified as ;andator! &re@setters# Behavioral &re@setters#
;; of 367
;andator! Pre@setters 0he e?ec!tion of an A$1 .o!ld re>!ire some minimal state, en#ironment/ 0hese type of initial conditions are classified !nder the mandatory initialiCation AMandatory pre5 settersB for the A$1/ For e?ample, a non5static member f!nction A$1 re>!ires an object to be created before it co!ld be called/ 0his is an essential acti#ity re>!ired for in#o in) the A$1/ Behavioral &re@setters 0o test the specific beha#ior of the A$1, some additional en#ironmental state is re>!ired/ 0hese types of initial conditions are called the beha#ioral pre5setters cate)ory of 1nitial condition/ 0hese are optional conditions re>!ired by the A$1 and need to be set before in#o in) the A$1 !nder test th!s infl!encin) its beha#ior/ Since these inf) ence the beha*ior of the 2Pparameters 0h!s to test any A$1, the en#ironment re>!ired sho!ld also be clearly !nderstood and set !p/ Witho!t these criteria, A$1 !nder test mi)ht not f!nction as re>!ired and lea#e the testerFs job !ndone/ +#In&ut<Parameter Selection: 0he list of #alid inp!t parameters need to be identified to #erify that the interface act!ally performs the tas s that it .as desi)ned for/ While there is no method that ens!res this beha#ior .ill be tested completely, !sin) inp!ts that ret!rn >!antifiable and #erifiable res!lts is the ne?t best thin)/ 0he different possible inp!t #al!es A#alid and in#alidB need to be identified and selected for testin)/ 0he techni>!es li e the bo!ndary #al!es analysis and e>!i#alence5partitionin) need to be !sed .hile tryin) to consider the inp!t parameter #al!es/ 0he bo!ndary #al!es or the limits that .o!ld lead to errors or e?ceptions need to be identified/ 1t .o!ld also be helpf!l if the data str!ct!res and other components that !se these data str!ct!res apart from the A$1 are analyCed/ 0he data str!ct!re can be loaded by !sin) the other components and the A$1 can be tested .hile the other component is accessin) these data str!ct!res/ +erify that all other dependent components f!nctionality are not affected .hile the A$1 accesses and manip!lates the data str!ct!res nder test, they are considered as additiona) inp ts other than the
322 of 367
0he a#ailability of the so!rce code to the testers .o!ld help in analyCin) the #ario!s inp!ts #al!es that co!ld be possible for testin) the A$1/ 1t .o!ld also help in !nderstandin) the #ario!s paths .hich co!ld be tested/ 0herefore, not only are testers re>!ired to !nderstand the calls, b!t also all the constants and data types !sed by the interface/ /# Identif! the combination of &arameters: $arameter combinations are e?tremely important for e?ercisin) stored data and comp!tation/ 1n A$1 calls, t.o independently #alid #al!es mi)ht ca!se a fa!lt .hen !sed to)ether .hich mi)ht not ha#e occ!rred .ith the other combinational #al!es/ 0herefore, a ro!tine called .ith t.o parameters re>!ires selection of #al!es for one based on the #al!e chosen for the other/ Kften the response of a ro!tine to certain data combinations is incorrectly pro)rammed d!e to the !nderlyin) comple? lo)ic/ 0he A$1 needs to be tested ta in) into consideration the combination of different parameter/ 0he n!mber of possible combinations of parameters for each call is typically lar)e/ For a )i#en set of parameters, if only the bo!ndary #al!es ha#e been selected, the n!mber of combinations, .hile relati#ely diminished, may still be prohibiti#ely lar)e/ For e?ample, consider an A$1 .hich ta es three parameters as inp!t/ 0he #ario!s combinations of different #al!es for the inp!t #al!es and their combinations needs to be identified/ $arameter combination is f!rther complicated by the f!nction o#erloadin) capabilities of many modern pro)rammin) lan)!a)es/ 1t is important to isolate the differences bet.een s!ch f!nctions and ta e into acco!nt that their !se is conte?t dri#en/ 0he A$1s can also be tested to chec that there are no memory lea s after they are called/ 0his can be #erified by contin!o!sly callin) the A$1 and obser#in) the memory !tiliCation/ 0#(all Se7uencing: When combinations of possible ar)!ments to each indi#id!al call are !nmana)eable, the n!mber of possible call se>!ences is infinite/ $arameter selection and combination iss!es f!rther complicate the problem call5se>!encin) problem/ Fa!lts ca!sed by improper call se>!ences tend to )i#e rise to some of the most dan)ero!s problems in soft.are/ Most sec!rity #!lnerabilities are ca!sed by the e?ec!tion of some s!ch seemin)ly improbable se>!ences/ 1#>bserve the out&ut: 0he o!tcome of an e?ec!tion of an A$1 depends !pon the beha#ior of that A$1, the test condition and the en#ironment/ 0he o!tcome of an A$1 can
323 of 367
be at different .ays i/e/, some co!ld )enerally ret!rn certain data or stat!s b!t for some of the A$1Qs, it mi)ht not ret!rn or shall be j!st .aitin) for a period of time, tri))erin) another e#ent, modifyin) certain reso!rce and so on/ 0he tester sho!ld be a.are of the o!tp!t that needs to be e?pected for the A$1 !nder test/ 0he o!tp!ts ret!rned for #ario!s inp!t #al!es li e #alid-in#alid, bo!ndary #al!es etc needs to be obser#ed and analyCed to #alidate if they are as per the f!nctionality/ All the error codes ret!rned and e?ceptions ret!rned for all the inp!t combinations sho!ld be e#al!ated/ 'PI Testing Tools: 0here are many testin) tools a#ailable/ Dependin) on the le#el of testin) re>!ired, different tools co!ld be !sed/ Some of the A$1 testin) tools a#ailable are mentioned here/ D"erif!: 0his is from Man Machine Systems/ J+erify is a Ja#a class-A$1 testin) tool that s!pports a !ni>!e in#asi#e testin) model/ 0he in#asi#e model allo.s access to the internals Apri#ate elementsB of any Ja#a object from .ithin a test script/ 0he ability to in#ade class internals facilitates more effecti#e testin) at class le#el, since controllability and obser#ability are enhanced/ 0his can be #ery #al!able .hen a class has not been desi)ned for testability/ DavaS&ec: Ja#aSpec is a S!n0estQs A$1 testin) tool/ 1t can be !sed to test Ja#a applications and libraries thro!)h their A$1/ Ja#aSpec )!ides the !sers thro!)h the entire test creation process and lets them foc!s on the most critical aspects of testin)/ Knce the !ser has entered the test data and assertions, Ja#aSpec a!tomatically )enerates self5chec in) tests, *0ML test doc!mentation, and detailed test reports/ *ere is an e?ample of ho. to a!tomate the A$1 testin)/ Ass!mptions, 5 3/ 0est en)ineer is s!pposed to test some A$1/ 7/ 0he A$1Fs are a#ailable in form of library A/libB/ 8/ 0est en)ineer has the A$1 doc!ment/ 0here are mainly t.o thin)s to test in A$1 testin), 5 3/ Blac bo? testin) of the A$1Fs 7/ 1nteraction - inte)ration testin) of the A$1Fs/ By blac bo? testin) of the A$1 mean that .e ha#e to test the A$1 for o!tp!ts/ 1n simple .ords .hen .e )i#e a no.n inp!t Aparameters to the A$1B then .e also no. the ideal o!tp!t/ So .e ha#e to chec for the act!al o!t p!t a)ainst the idle o!tp!t/
327 of 367
For this .e can .rite a simple c pro)ram that .ill do the follo.in), 5 aB 0a e the parameters from a te?t file Athis file .ill contain many of s!ch inp!t parametersB/ bB 'all the A$1 .ith these parameters/
c6 Match the act!al and idle o!tp!t and also chec the parameters for )ood
#al!es that are passed .ith reference ApointersB/ dB Lo) the res!lt/ Secondly .e ha#e test the inte)ration of the A$1Fs/ For e?ample there are t.o A$1Fs say andle h E handle createconte:t 3void4F When the handle to the de#ice is to be closed then the correspondin) f!nction Bool bishandledeleted E bool deleteconte:t 3handle Gh4F *ere .e ha#e to call the t.o A$1Fs and chec if they are handled by the created createconte?t AB and are deleted by the deleteconte?t AB/ 0his .ill ens!re that these t.o A$1Fs are .or in) fine/ For this .e can .rite a simple c pro)ram that .ill do the follo.in), 5 aB 'all the t.o A$1Fs in the same order/ bB $ass the o!tp!t parameter of the first as the inp!t of the second cB 'hec for the o!tp!t parameter of the second A$1 dB Lo) the res!lt/ 0he e?ample is o#er simplified b!t this .or s beca!se .e are !sin) this ind of test tool for e?tensi#e re)ression testin) of o!r A$1 library/
)2# 8nderstanding Ra&id Testing

&apid testin) is the testin) soft.are faster than !s!al, .itho!t compromisin) on the standards of >!ality/ 1t is the techni>!e to test as thoro!)h as reasonable .ithin the constraints/ 0his techni>!e loo s at testin) as a process of he!ristic in>!iry and lo)ically spea in) it sho!ld be based on e?ploratory testin) techni>!es/ Altho!)h most projects !nder)o contin!o!s testin), it does not !s!ally prod!ce the
328 of 367
information re>!ired to deal .ith the sit!ations .here it is necessary to ma e an instantaneo!s assessment of the prod!ctQs >!ality at a partic!lar moment/ 1n most cases the testin) is sched!led for j!st prior to la!nch and con#entional testin) techni>!es often cannot be applied to soft.are that is incomplete or s!bject to constant chan)e/ At times li e these &apid 0estin) can be !sed/ 1t can be said that rapid testin) has a str!ct!re that is b!ilt on a fo!ndation of fo!r components namely, $eople 1nte)rated test process Static 0estin) and Dynamic 0estin)
0here is a need for people .ho can handle the press!re of ti)ht sched!les/ 0hey need to be prod!cti#e contrib!tors e#en thro!)h the early phases of the de#elopment life cycle/ Accordin) to James Bach, a core s ill is the ability to thin critically/ 1t sho!ld also be noted that dynamic testin) lies at the heart of the soft.are testin) process, and the plannin), desi)n, de#elopment, and e?ec!tion of dynamic tests sho!ld be performed .ell for any testin) process to be efficient/ T R'PI% T9STI$G
PR'(TI(9
1t .o!ld help !s if .e scr!tiniCe each phase of a de#elopment process to see ho. the efficiency, speed and >!ality of testin) can be impro#ed, bearin) in mind the follo.in) factors, Actions that the test team can ta e to pre#ent defects from escapin)/ For e?ample, practices li e e?treme pro)rammin) and e?ploratory testin)/ Actions that the test team can ta e to mana)e ris to the de#elopment sched!le/ 0he information that can be obtained from each phase so that the test team can speed !p the acti#ities/ 1f a test process is desi)ned aro!nd the ans.ers to these >!estions, both the speed of testin) and the >!ality of the final prod!ct sho!ld be enhanced/ Some of the aspects that can be !sed .hile rapid testin) are )i#en belo., )# 0est for lin inte)rity +# 0est for disabled accessibility /# 0est the defa!lt settin)s 0# 'hec the na#i)ationFs
326 of 367
1# 'hec data
for inp!t constraints by injectin) special characters
at the so!rces of
*# &!n M!ltiple instances 2# 'hec for interdependencies and stress them 5# 0est for consistency of desi)n 6# 0est for compatibility ).# 0est for !sability ))# 'hec for the possible #ariabilityFs and attac them )+# Go for possible stress and load tests )/# And o!r fa#orite I ban)in) the eyboard
)5# Test ,are %evelo&ment

0est Ware de#elopment is the ey role of the 0estin) 0eam/ What comprises 0est Ware and some )!idelines to b!ild the test .are is disc!ssed belo.,
)5#) Test Strateg!

Before startin) any testin) acti#ities, the team lead .ill ha#e to thin a lot S arri#e at a strate)y/ 0his .ill describe the approach, .hich is to be adopted for carryin) o!t test acti#ities incl!din) the plannin) acti#ities/ 0his is a formal doc!ment and the #ery first doc!ment re)ardin) the testin) area and is prepared at a #ery early sta) in SDL'/ 0his doc!ment m!st pro#ide )eneric test approach as .ell as specific details re)ardin) the project/ 0he follo.in) areas are addressed in the test strate)y doc!ment/ )5#)#) Test Levels 0he test strate)y m!st tal abo!t .hat are the test le#els that .ill be carried o!t for that partic!lar project/ (nit, 1nte)ration S System testin) .ill be carried o!t in all projects/ B!t many times, the inte)ration S system testin) may be combined/ Details li e this may be addressed in this section/ )5#)#+ Roles and Res&onsibilities 0he roles and responsibilities of test leader, indi#id!al testers, project mana)er are to be clearly defined at a project le#el in this section/ 0his may not ha#e names associated, b!t the role has to be #ery clearly defined/ 0he re#ie. and appro#al mechanism m!st be stated here for test plans and other test doc!ments/ Also, .e ha#e to state .ho re#ie.s the test cases, test records and .ho appro#ed them/ 0he doc!ments may )o thr! a series of re#ie.s or m!ltiple appro#als and they ha#e to be mentioned here/
32= of 367
)5#)#/ Testing Tools Any testin) tools, .hich are to be !sed in different test le#els, m!st be, clearly identified/ 0his incl!des j!stifications for the tools bein) !sed in that partic!lar le#el also/ )5#)#0 Risks and ;itigation Any ris s that .ill affect the testin) process m!st be listed alon) .ith the miti)ation/ By doc!mentin) the ris s in this doc!ment, .e can anticipate the occ!rrence of it .ell ahead of time and then .e can proacti#ely pre#ent it from occ!rrin)/ Sample ris s are dependency of completion of codin), .hich is done by s!b5contractors, capability of testin) tools etc/ )5#)#1 Regression Test '&&roach When a partic!lar problem is identified, the pro)rams .ill be deb!))ed and the fi? .ill be done to the pro)ram/ 0o ma e s!re that the fi? .or s, the pro)ram .ill be tested a)ain for that criteria/ &e)ression test .ill ma e s!re that one fi? does not create some other problems in that pro)ram or in any other interface/ So, a set of related test cases may ha#e to be repeated a)ain, to ma e s!re that nothin) else is affected by a partic!lar fi?/ *o. this is )oin) to be carried o!t m!st be elaborated in this section/ 1n some companies, .hene#er there is a fi? in one !nit, all !nit test cases for that !nit .ill be repeated, to achie#e a hi)her le#el of >!ality/ )5#)#* Test Grou&s From the list of re>!irements, .e can identify related areas, .hose f!nctionality is similar/ 0hese areas are the test )ro!ps/ For e?ample, in a rail.ay reser#ation system, anythin) related to tic et boo in) is a f!nctional )ro!pR anythin) related .ith report )eneration is a f!nctional )ro!p/ Same .ay, .e ha#e to identify the test )ro!ps based on the f!nctionality aspect/ )5#)#2 Test Priorities Amon) test cases, .e need to establish priorities/ While testin) soft.are projects, certain test cases .ill be treated as the most important ones and if they fail, the prod!ct cannot be released/ Some other test cases may be treated li e cosmetic and if they fail, .e can release the prod!ct .itho!t m!ch compromise on the f!nctionality/ 0his priority le#els m!st be clearly stated/ 0hese may be mapped to the test )ro!ps also/
324 of 367
)5#)#5 Test Status (ollections and Re&orting When test cases are e?ec!ted, the test leader and the project mana)er m!st .here e?actly .e stand in terms of testin) acti#ities/ 0o no., no. .here .e stand, the
inp!ts from the indi#id!al testers m!st come to the test leader/ 0his .ill incl!de, .hat test cases are e?ec!ted, ho. lon) it too , ho. many test cases passed and ho. many5 failed etc/ Also, ho. often .e collect the stat!s is to be clearly mentioned/ Some companies .ill ha#e a practice of collectin) the stat!s on a daily basis or .ee ly basis/ 0his has to be mentioned clearly/ )5#)#6 Test Records ;aintenance When the test cases are e?ec!ted, .e need to eep trac of the e?ec!tion details li e .hen it is e?ec!ted, .ho did it, ho. lon) it too , .hat is the res!lt etc/ 0his data m!st be a#ailable to the test leader and the project mana)er, alon) .ith all the team members, in a central location/ 0his may be stored in a specific directory in a central ser#er and the doc!ment m!st say clearly abo!t the locations and the directories/ 0he namin) con#ention for the doc!ments and files m!st also be mentioned/
)5#)#).
Re7uirements Traceabilit! ;atri:
1deally each soft.are de#eloped m!st satisfy the set of re>!irements completely/ So, ri)ht from desi)n, each re>!irement m!st be addressed in e#ery sin)le doc!ment in the soft.are process/ 0he doc!ments incl!de the *LD, LLD, so!rce codes, !nit test cases, inte)ration test cases and the system test cases/ &efer the follo.in) sample table .hich describes &e>!irements 0raceability Matri? process/ 1n this matri?, the ro.s .ill ha#e the re>!irements/ For e#ery doc!ment V*LD, LLD etcX, there .ill be a separate col!mn/ So, in e#ery cell, .e need to state, .hat section in *LD addresses a partic!lar re>!irement/ 1deally, if e#ery re>!irement is addressed in e#ery sin)le doc!ment, all the indi#id!al cells m!st ha#e #alid section ids or names filled in/ 0hen .e no. that e#ery re>!irement is addressed/ 1n case of any missin) of re>!irement, .e need to )o bac to the doc!ment and correct it, so that it addressed the re>!irement/ For testin) at each le#el, .e may ha#e to address the re>!irements/ Kne inte)ration and the system test case may address m!ltiple re>!irements/
%TP Scenario
%T( Id
(ode
LL% Section
329 of 367
&e>!irement &e>!irement &e>!irement &e>!irement W W W W W W W &e>!irement
3 7 8 6
$o J#e-5#e J#e-5#e J#e-5#e J#e-5#e
3,7,8,6 3,7,8,6 3,7,8,6 3,7,8,6
"
J#e-5#e 0<S0<&
3,7,8,6 0<S0<&
D<+<LK$<&
0<S0 L<AD
)5#)#))
Test Summar!
0he senior mana)ement may li e to ha#e test s!mmary on a .ee ly or monthly basis/ 1f the project is #ery critical, they may need it on a daily basis also/ 0his section m!st address .hat ind of test s!mmary reports .ill be prod!ced for the senior mana)ement alon) .ith the fre>!ency/ 0he test strate)y m!st )i#e a clear #ision of .hat the testin) team .ill do for the .hole project for the entire d!ration/ 0his doc!ment .ill-may be presented to the client also, if needed/ 0he person, .ho prepares this doc!ment, m!st be f!nctionally stron) in the prod!ct domain, .ith a #ery )ood e?perience, as this is the doc!ment that is )oin) to dri#e the entire team for the testin) acti#ities/ 0est strate)y m!st be clearly e?plained to the testin) team members ti)ht at the be)innin) of the project/
)5#+ Test Plan

0he test strate)y identifies m!ltiple test le#els, .hich are )oin) to be performed for the project/ Acti#ities at each le#el m!st be planned .ell in ad#ance and it has to be formally doc!mented/ Based on the indi#id!al plans only, the indi#id!al test le#els are carried o!t/ 0he plans are to be prepared by e?perienced people only/ 1n all test plans, the <0+L V<ntry50as 5+alidation5<?itX criteria are to be mentioned/ <ntry means the entry point to that phase/ For e?ample, for !nit testin), the codin) m!st be complete and then only one can start !nit testin)/ 0as is the acti#ity that is performed/ +alidation is the .ay in .hich the pro)ress and correctness and compliance are #erified for that phase/ <?it
32: of 367
tells the completion criteria of that phase, after the #alidation is done/ For e?ample, the e?it criterion for !nit testin) is all !nit test cases m!st pass/ <0+L is a modelin) techni>!e for de#elopin) .orldly and atomic le#el models/ 1t sands for <ntry, 0as , +erification and <?it/ 1t is a tas 5based model .here the details of each tas are e?plicitly defined in a specification table a)ainst each phase i/e/ <ntry, <?it, 0as , Feedbac 1n, Feedbac K!t, and meas!res/ 0here are t.o types of cells, !nit cells and implementation cells/ 0he implementation cells are basically !nit cells containin) the f!rther tas s/ For e?ample if there is a tas of siCe estimation, then there .ill be a !nit cell of siCe has f!rther tas s namely, define meas!res, estimate estimation/ 0hen since this tas
siCe/ 0he !nit cell containin) these f!rther tas s .ill be referred to as the implementation cell and a separate table .ill be constr!cted for it/ A p!rpose is also stated and the #ie.er of the model may also be defined e/)/ top mana)ement or c!stomer/ 3:/7/3 (nit 0est $lan V(0$X 0he !nit test plan is the o#erall plan to carry o!t the !nit test acti#ities/ 0he lead tester prepares it and it .ill be distrib!ted to the indi#id!al testers, .hich contains the follo.in) sections/ 3:/7/3/3 What is to be tested@
0he !nit test plan m!st clearly specify the scope of !nit testin)/ 1n this, normally the basic inp!t-o!tp!t of the !nits alon) .ith their basic f!nctionality .ill be tested/ 1n this case mostly the inp!t !nits .ill be tested for the format, ali)nment, acc!racy and the totals/ 0he (0$ .ill clearly )i#e the r!les of .hat data types are present in the system, their format and their bo!ndary conditions/ 0his list may not be e?ha!sti#eR b!t it is better to ha#e a complete list of these details/ 3:/7/3/7Se>!ence of 0estin) 0he se>!ences of test acti#ities that are to be carried o!t in this phase are to be listed in this section/ 0his incl!des, .hether to e?ec!te positi#e test cases first or ne)ati#e test cases first, to e?ec!te test cases based on the priority, to e?ec!te test cases based on test )ro!ps etc/ $ositi#e test cases pro#e that the system performs .hat is s!pposed to doR ne)ati#e test cases pro#e that the system does not perform .hat is not s!pposed to do/ 0estin) the screens, files, database etc/, are to be )i#en in proper se>!ence/
32; of 367
3:/7/3/6 Basic F!nctionality of (nits *o. the independent f!nctionalities of the !nits are tested .hich e?cl!des any comm!nication bet.een the !nit and other !nits/ 0he interface part is o!t of scope of this test le#el/ Apart from the abo#e sections, the follo.in) sections are addressed, #ery specific to !nit testin)/ (nit 0estin) 0ools $riority of $ro)ram !nits "amin) con#ention for test cases Stat!s reportin) mechanism &e)ression test approach <0+L criteria
3:/7/7 1nte)ration 0est $lan 0he inte)ration test plan is the o#erall plan for carryin) o!t the acti#ities in the inte)ration test le#el, .hich contains the follo.in) sections/ 7/7/3 What is to be tested@ inds of interfaces fall !nder the scope of testin)
0his section clearly specifies the
internal, e?ternal interfaces, .ith re>!est and response is to be e?plained/ 0his need not )o deep in terms of technical details b!t the )eneral approach ho. the interfaces are tri))ered is e?plained/ 3:/7/7/3Se>!ence of 1nte)ration When there are m!ltiple mod!les present in an application, the se>!ence in .hich they are to be inte)rated .ill be specified in this section/ 1n this, the dependencies bet.een the mod!les play a #ital role/ 1f a !nit B has to be e?ec!ted, it may need the data that is fed by !nit A and !nit L/ 1n this case, the !nits A and L ha#e to be inte)rated and then !sin) that data, the !nit B has to be tested/ 0his has to be stated to the .hole set of !nits in the pro)ram/ Gi#en this correctly, the testin) acti#ities .ill lead to the prod!ct, slo.ly b!ildin) the prod!ct, !nit by !nit and then inte)ratin) them/ 3:/7/7/7 List of Mod!les and 1nterface F!nctions
0here may be " n!mber of !nits in the application, b!t the !nits that are )oin) to comm!nicate .ith each other, alone are tested in this phase/ 1f the !nits are desi)ned in s!ch a .ay that they are m!t!ally independent, then the interfaces do not come into pict!re/ 0his is almost impossible in any system, as the !nits ha#e to comm!nicate to other !nits, in order to )et different types of f!nctionalities e?ec!ted/ 1n this section, .e
332 of 367
need to list the !nits and for .hat p!rpose it tal s to the others need to be mentioned/ 0his .ill not )o into technical aspects, b!t at a hi)her le#el, this has to be e?plained in plain <n)lish/ Apart from the abo#e sections, the follo.in) sections are addressed, #ery specific to inte)ration testin)/ 1nte)ration 0estin) 0ools $riority of $ro)ram interfaces "amin) con#ention for test cases Stat!s reportin) mechanism &e)ression test approach <0+L criteria B!ild-&efresh criteria VWhen m!ltiple pro)rams or objects are to be lin ed to arri#ed at sin)le prod!ct, and one !nit has some modifications, then it may need to reb!ild the entire prod!ct and then load it into the inte)ration test en#ironment/ When and ho. often, the prod!ct is reb!ilt and refreshed is to be mentionedX/ 3:/7/8 System 0est $lan VS0$X 0he system test plan is the o#erall plan carryin) o!t the system test le#el acti#ities/ 1n the system test, apart from testin) the f!nctional aspects of the system, there are some special testin) acti#ities carried o!t, s!ch as stress testin) etc/ 0he follo.in) are the sections normally present in system test plan/ 3:/7/8/3What is to be tested@ 0his section defines the scope of system testin), #ery specific to the project/ "ormally, the system testin) is based on the re>!irements/ All re>!irements are to be #erified in the scope of system testin)/ 0his co#ers the f!nctionality of the prod!ct/ Apart from this .hat special testin) is performed are also stated here/ 3:/7/8/7 F!nctional Gro!ps and the Se>!ence 0he re>!irements can be )ro!ped in terms of the f!nctionality/ Based on this, there may be priorities also amon) the f!nctional )ro!ps/ For e?ample, in a ban in) application, anythin) related to c!stomer acco!nts can be )ro!ped into one area, anythin) related to inter5branch transactions may be )ro!ped into one area etc/ Same .ay for the prod!ct bein) tested, these areas are to be mentioned here and the
333 of 367
s!))ested se>!ences of testin) of these areas, based on the priorities are to be described/ 3:/7/8/8Special 0estin) Methods 0his co#ers the different special tests li e load-#ol!me testin), stress testin), interoperability testin) etc/ 0hese testin) are to be done based on the nat!re of the prod!ct and it is not mandatory that e#ery one of these special tests m!st be performed for e#ery prod!ct/ Apart from the abo#e sections, the follo.in) sections are addressed, #ery specific to system testin)/ System 0estin) 0ools $riority of f!nctional )ro!ps "amin) con#ention for test cases Stat!s reportin) mechanism &e)ression test approach <0+L criteria B!ild-&efresh criteria
3:/7/6 Acceptance 0est $lan VA0$X 0he client at their place performs the acceptance testin)/ 1t .ill be #ery similar to the system test performed by the Soft.are De#elopment (nit/ Since the client is the one .ho decides the format and testin) methods as part of acceptance testin), there is no specific cl!e on the .ay they .ill carry o!t the testin)/ B!t it .ill not differ m!ch from the system testin)/ Ass!me that all the r!les, .hich are applicable to system test, can be implemented to acceptance testin) also/ Since this is j!st one le#el of testin) done by the client for the o#erall prod!ct, it may incl!de test cases incl!din) the !nit and inte)ration test le#el details/ ' sam&le Test Plan >utline along with their descri&tion is as shown below: 0est $lan K!tline 3/ BA'KG&K("D I 0his item s!mmariCes the f!nctions of the application system and the tests to be performed/ 7/ 1"0&KD('01K"
337 of 367
8/ ASS(M$01K"S I 1ndicates any anticipated ass!mptions .hich .ill be made .hile testin) the application/ 6/ 0<S0 10<MS 5 List each of the items Apro)ramsB to be tested/ =/ F<A0(&<S 0K B< 0<S0<D 5 List each of the feat!res Af!nctions or re>!irementsB .hich .ill be tested or demonstrated by the test/ 4/ F<A0(&<S "K0 0K B< 0<S0<D 5 <?plicitly lists each feat!re, f!nction, or re>!irement .hich .onQt be tested and .hy not/ 9/ A$$&KA'* 5 Describe the data flo.s and test philosophy/ Sim!lation or Li#e e?ec!tion, <tc/ 0his section also mentions all the approaches .hich .ill be follo.ed at the #ario!s sta)es of the test e?ec!tion/ :/ 10<M $ASS-FA1L '&10<&1A Blan et statement 5 1temiCed list of e?pected o!tp!t and tolerances ;/ S(S$<"S1K"-&<S(M$01K" '&10<&1A 5 M!st the test r!n from start to completion@ (nder .hat circ!mstances it may be res!med in the middle@ <stablish chec 5points in lon) tests/ 32/ 0<S0 D<L1+<&ABL<S 5 What, besides soft.are, .ill be deli#ered@ 0est report 0est soft.are 33/ 0<S01"G 0ASKS F!nctional tas s Ae/)/, e>!ipment set !pB Administrati#e tas s 37/ <"+1&K"M<"0AL "<<DS Sec!rity clearance Kffice space S e>!ipment *ard.are-soft.are re>!irements 38/ &<S$K"S1B1L101<S Who does the tas s in Section 32@ What does the !ser do@ 36/ S0AFF1"G S 0&A1"1"G 3=/ S'*<D(L<
338 of 367
34/ &<SK(&'<S 39/ &1SKS S 'K"01"G<"'1<S 3:/ A$$&K+ALS 0he sched!le details of the #ario!s test pass s!ch as (nit tests, 1nte)ration tests, System 0ests sho!ld be clearly mentioned alon) .ith the estimated efforts/
)5#/ Test (ase %ocuments

Desi)nin) )ood test cases is a comple? art/ 0he comple?ity comes from three so!rces, 0est cases help !s disco#er information/ Different types of tests are more effecti#e for different classes of information/ 0est cases can be G)oodH in a #ariety of .ays/ "o test case .ill be )ood in all of them/ $eople tend to create test cases accordin) to certain testin) styles, s!ch as domain testin) or ris 5based testin)/ Good domain tests are different from )ood ris 5based tests/ WhatFs a test case@ GA test case specifies the pretest state of the 1(0 and its en#ironment, the test inp!ts or conditions, and the e?pected res!lt/ 0he e?pected res!lt specifies .hat the 1(0 sho!ld prod!ce from the test inp!ts/ 0his specification incl!des messa)es )enerated by the 1(0, e?ceptions, ret!rned #al!es, and res!ltant state of the 1(0 and its en#ironment/ 0est cases may also specify initial and res!ltin) conditions for other objects that constit!te the 1(0 and its en#ironment/H WhatFs a scenario@ A scenario is a hypothetical story, !sed to help a person thin problem or system/ (haracteristics of Good Scenarios A scenario test has fi#e ey characteristics/ 1t is AaB a story that is AbB moti#atin), AcB thro!)h a comple?
credible, AdB comple?, and AeB easy to e#al!ate/ 0he primary objecti#e of test case desi)n is to deri#e a set of tests that ha#e the hi)hest attit!de of disco#erin) defects in the soft.are/ 0est cases are desi)ned based on the
336 of 367
analysis of re>!irements, !se cases, and technical specifications, and they sho!ld be de#eloped in parallel .ith the soft.are de#elopment effort/ A test case describes a set of actions to be performed and the res!lts that are e?pected/ A test case sho!ld tar)et specific f!nctionality or aim to e?ercise a #alid path thro!)h a !se case/ 0his sho!ld incl!de in#alid !ser actions and ille)al inp!ts that are not necessarily listed in the !se case/ A test case is described depends on se#eral factors, e/)/ the n!mber of test cases, the fre>!ency .ith .hich they chan)e, the le#el of a!tomation employed, the s ill of the testers, the selected testin) methodolo)y, staff t!rno#er, and ris / 0he test cases .ill ha#e a )eneric format as belo./ Test case I% @ 0he test case id m!st be !ni>!e across the application Test case descri&tion @ 0he test case description m!st be #ery brief/ Test &rere7uisite @ 0he test pre5re>!isite clearly describes .hat sho!ld be present in the system, before the test can be e?ec!tes/ Test In&uts @ 0he test inp!t is nothin) b!t the test data that is prepared to be fed to the system/ Test ste&s @ 0he test steps are the step5by5step instr!ctions on ho. to carry o!t the test/ 9:&ected Results @ 0he e?pected res!lts are the ones that say .hat the system m!st )i#e as o!tp!t or ho. the system m!st react based on the test steps/ 'ctual Results H 0he act!al res!lts are the ones that say o!tp!ts of the action for the )i#en inp!ts or ho. the system reacts for the )i#en inp!ts/ Pass<Fail 5 1f the <?pected and Act!al res!lts are same then test is $ass other.ise Fail/ 0he test cases are classified into positi#e and ne)ati#e test cases/ $ositi#e test cases are desi)ned to pro#e that the system accepts the #alid inp!ts and then process them correctly/ S!itable techni>!es to desi)n the positi#e test cases are Specification deri#ed tests, <>!i#alence partitionin) and State5transition testin)/ 0he ne)ati#e test cases are desi)ned to pro#e that the system rejects in#alid inp!ts and does not process them/ S!itable techni>!es to desi)n the ne)ati#e test cases are <rror )!essin), Bo!ndary #al!e analysis, internal bo!ndary #al!e testin) and State5transition testin)/ 0he test cases details m!st be #ery clearly specified, so that a ne. person can )o thro!)h the test cases step and step and is able to e?ec!te it/ 0he test cases .ill be e?plained .ith specific e?amples in the follo.in) section/
33= of 367
For e?ample consider online shoppin) application/ At the !ser interface le#el the client re>!est the .eb ser#er to display the prod!ct details by )i#in) email id and (sername/ 0he .eb ser#er processes the re>!est and .ill )i#e the response/ For this application .e .ill desi)n the !nit, 1nte)ration and system test cases/
Fi)!re 4/Web based application 8nit Test (ases 38T(4 0hese are #ery specific to a partic!lar !nit/ 0he basic f!nctionality of the !nit is to be !nderstood based on the re>!irements and the desi)n doc!ments/ Generally, Desi)n doc!ment .ill pro#ide a lot of information abo!t the f!nctionality of a !nit/ 0he Desi)n doc!ment has to be referred before (0' is .ritten, beca!se it pro#ides the act!al f!nctionality of ho. the system m!st beha#e, for )i#en inp!ts/ For e?ample, 1n the Knline shoppin) application, 1f the !ser enters #alid <mail id and (sername #al!es, let !s ass!me that Desi)n doc!ment says, that the system m!st display a prod!ct details and sho!ld insert the <mail id and (sername in database table/ 1f !ser enters in#alid #al!es the system .ill display appropriate error messa)e and .ill not store it in database/
Figure 2: Sna&shot of Login Screen Test (onditions for the fields in the Login screen <mail51t sho!ld be in this format AFor <) clic meYyahoo/comB/ (sername I 1t sho!ld accept only alphabets not )reater than 4/"!merics and special type of characters are not allo.ed/
334 of 367
0est $rere>!isite, 0he !ser sho!ld ha#e access to '!stomer Lo)in screen form screen $egative Test (ase $roject "ame5Knline shoppin) +ersion53/3 Mod!le5'atalo) T est I 3 'hec #al!es field for inp!ttin) in <mail %escri&tion Test In&uts 9:&ected Results 'ctual results Pass<Fa il
'mail78eerthi9re&i5 5mail
(sernameTLa#ier
1np!ts sho!ld not be accepted/ 1t sho!ld messa)e display G<nter
'hec #al!es field
for inp!ttin) in <mail
<mailTjohn74\rediff mail/com (sernameTJohn
#alid <mailH 1np!ts sho!ld not be accepted/ 1t sho!ld messa)e display G<nter
'hec field
for inp!ttin)
'mail7shilpa9yahoo .com
(sernameTMar 76
#alid <mailH 1np!ts sho!ld not be accepted/ 1t sho!ld messa)e display G<nter
#al!es in (sername
correct (sernameH Positive Test (ase Test I 3 %escri&tion 'hec <mail field 'hec <mail field 8 'hec (sername field
Test In&uts for <mailTshanYyahoo/com (sernameTda#e <mailT n iYrediffmail/co m (sernameTjohn for <mailT?a#Yyahoo/com (sernameTmar
9:&ected Results 1np!ts sho!ld accepted/ 1np!ts sho!ld accepted/ 1np!ts sho!ld accepted/ be be be
'ctual results
Pass<Fail
inp!ttin) #al!es in 7 for
inp!ttin) #al!es in
inp!ttin) #al!es in
339 of 367
Integration Test (ases Before desi)nin) the inte)ration test cases the testers sho!ld )o thro!)h the 1nte)ration test plan/ 1t .ill )i#e complete idea of ho. to .rite inte)ration test cases/ 0he main aim of inte)ration test cases is that it tests the m!ltiple mod!les to)ether/ By e?ec!tin) these test cases the !ser can find o!t the errors in the interfaces bet.een the Mod!les/ For e?ample, in online shoppin), there .ill be 'atalo) and Administration mod!le/ 1n catalo) section the c!stomer can trac information related to it/ the list of prod!cts and can b!y the prod!cts online/ 1n administration mod!le the admin can enter the prod!ct name and
Table/: Integration Test (ases Test I 3 %escri&tion 'hec Screen for Lo)in Test In&uts <nter For <), <mail TshilpaYyahoo/com Bac end +erification (sernameTshilpa Select emailB username from (usF 0he <mail (sername sho!ld displayed 7 'hec $rod!ct 1nformation for 'lic prod!ct information lin be at entered and #al!es in <mail 9:&ected Results 1np!ts sho!ld be accepted/ 'ctual results Pass<Fail
and (ser"ame/
s>lprompt/ 1t sho!ld display complete details of the prod!ct
33: of 367
'hec screen
for admin
<nter #al!es in $rod!ct 1d and $rod!ct name fields/ For <), $rod!ct 1d576= $rod!ct name5"orton Anti#ir!s Select &id B &name from ProductF
1np!ts
sho!ld
be accepted/
Bac end #erification
0he $rod!ct sho!ld
entered name be
$rod!ct id and
displayed at the s>l prompt/ $>T9: The tester has to e:ecute above unit and Integration test cases after coding# 'nd e<She has to fill the actual results and Pass<fail columns# If the test cases fail then defect re&ort should be &re&ared#
S!stem Test (ases: @ 0he system test cases meant to test the system as per the re>!irementsR end5to end/ 0his is basically to ma e s!re that the application .or s as per S&S/ 1n system test cases, A)enerally in system testin) itselfB, the testers are s!pposed to act as an end !ser/ So, system test cases normally do concentrate on the f!nctionality of the system, inp!ts are fed thro!)h the system and each and e#ery chec is performed !sin) the system itself/ "ormally, the #erifications done by chec in) the database tables directly or r!nnin) pro)rams man!ally are not enco!ra)ed in the system test/ 0he system test m!st foc!s on f!nctional )ro!ps, rather than identifyin) the pro)ram !nits/ When it comes to system testin), it is ass!me that the interfaces bet.een the mod!les are .or in) fine Ainte)ration passedB/ 1deally the test cases are nothin) b!t a !nion of the f!nctionalities tested in the !nit testin) and the inte)ration testin)/ 1nstead of testin) the system inp!ts o!tp!ts thro!)h database or e?ternal pro)rams, e#erythin) is tested thro!)h the system itself/ For e?ample, in a online shoppin) application, the catalo) and administration screens Apro)ram !nitsB .o!ld ha#e been independently !nit tested and the test res!lts .o!ld be #erified thro!)h the database/ 1n system testin), the tester .ill mimic as an end !ser and hence chec s the application thro!)h its o!tp!t/
33; of 367
0here are occasions, .here some-many of the inte)ration and !nit test cases are repeated in system testin) alsoR especially .hen the !nits are tested .ith test st!bs before and not act!ally tested .ith other real mod!les, d!rin) system testin) those cases .ill be performed a)ain .ith real mod!les-data in
)6# %efect ;anagement

Defects determine the effecti#eness of the 0estin) .hat .e do/ 1f there are no defects, it directly implies that .e donFt ha#e o!r job/ 0here are t.o points .orth considerin) here, either the de#eloper is so stron) that there are no defects arisin) o!t, or the test en)ineer is .ea / 1n many sit!ations, the second is pro#in) correct/ 0his implies that .e lac the nac / 1n this section, let !s !nderstand Defects/
)6#) ,hat is a %efectFor a test engineerB a defect is following: @ Any de#iation from specification Anythin) that ca!ses !ser dissatisfaction 1ncorrect o!tp!t Soft.are does not do .hat it intended to do/
Bug < %efect < 9rror: @
Soft.are is said to ha#e bug if it feat!res de#iates from specifications/ Soft.are is said to ha#e defect if it has !n.anted side effects/ Soft.are is said to ha#e 9rror if it )i#es incorrect o!tp!t/
B!t as for a test en)ineer all are same as the abo#e definition is only for the p!rpose of doc!mentation or indicati#e/
)6#+ %efect Ta:onomies

'ate)ories of Defects, All soft.are defects can be broadly cate)oriCed into the belo. mentioned types, : : : : <rrors of commission, somethin) .ron) is done <rrors of omission, somethin) left o!t by accident <rrors of clarity and ambi)!ity, different interpretations <rrors of speed and capacity
*o.e#er, the abo#e is a broad cate)oriCationR belo. .e ha#e for yo! a host of #aried types of defects that can be identified in different soft.are applications,
372 of 367
3/ 'oncept!al b!)s - Desi)n b!)s 7/ 'odin) b!)s 8/ 1nte)ration b!)s 6/ (ser 1nterface <rrors =/ F!nctionality 4/ 'omm!nication 9/ 'ommand Str!ct!re :/ Missin) 'ommands ;/ $erformance 32/ K!tp!t 33/ <rror *andlin) <rrors 37/ Bo!ndary5&elated <rrors 38/ 'alc!lation <rrors 36/ 1nitial and Later States 3=/ 'ontrol Flo. <rrors 34/ <rrors in *andlin) Data 39/ &ace 'onditions <rrors 3:/ 3;/ 72/ 73/ 77/ Load 'onditions <rrors *ard.are <rrors So!rce and +ersion 'ontrol <rrors Doc!mentation <rrors 0estin) <rrors
)6#/ Life (!cle of a %efect

0he follo.in) self e?planatory fi)!re e?plains the life cycle of a defect,
Defer Assign Fix/Change
Submit Defect
Review, Verify and Qualify
Validate
Du licate, Re!ect "r #"re $nf" % date Defect
Cl"se
373 of 367
Cancel
+.# ;etrics for Testing

,hat is a ;etricDMetricF is a meas!re to >!antify soft.are, soft.are de#elopment reso!rces, and-or the soft.are de#elopment process/ A Metric can >!antify any of the follo.in) factors, Sched!le, Wor <ffort, $rod!ct SiCe, $roject Stat!s, and E!ality $erformance
-easuring ena$les.% Metrics enables estimation of f!t!re .or / 0hat is, considerin) the case of testin) 5 Decidin) the prod!ct is fit for shipment or deli#ery depends on the rate the defects are fo!nd and fi?ed/ Defect collected and fi?ed is one ind of metric/ A.../processimpact/comB As defined in the M1S&A &eport, 1t is beneficial to classify metrics accordin) to their !sa)e/ 1<<< ;7:/3 ^6_ identifies t.o classes, iB iiB $rocess I Acti#ities performed in the prod!ction of the Soft.are $rod!ct I An o!tp!t of the $rocess, for e?ample the soft.are or its doc!mentation/
Defects are analyCed to identify .hich are the major ca!ses of defect and .hich is the phase that introd!ces most defects/ 0his can be achie#ed by performin) $areto analysis of defect ca!ses and defect introd!ction phases/ 0he main re>!irements for any of these analysis is Soft.are Defect Metrics/ Fe. of the Defect Metrics are, %efect %ensit!: A"o/ Kf Defects &eported by SEA J "o/ Defects &eported By $eer &e#ie.B-Act!al SiCe/ 0he SiCe can be in KLK', SLK', or F!nction $oints/ 0he method !sed in the Kr)aniCation to meas!re the siCe of the Soft.are $rod!ct/ 0he SEA is considered to be the part of the Soft.are testin) team/
377 of 367
Test effectiveness:
Dt - AtJ(atB .here tTtotal no/ of defects reported d!rin) testin)
and (at T total no/ of defects reported d!rin) (ser acceptance testin) (ser Acceptance 0estin) is )enerally carried o!t !sin) the Acceptance 0est 'riteria accordin) to the Acceptance 0est $lan/ %efect Removal 9fficienc!: A0otal "o Kf Defects &emo#ed -0otal "o/ Kf Defects 1njectedB`322 at #ario!s sta)es of SDL' %escri&tion 0his metric .ill indicate the effecti#eness of the defect identification and remo#al in sta)es for a )i#en project Formula &e>!irements, D&< T Â&e>!irement defects corrected d!rin) &e>!irements phaseB - A&e>!irement defects injected d!rin) &e>!irements phaseB_ ` 322 Desi)n, D&< T ÂDesi)n defects corrected d!rin) Desi)n phaseB - ADefects identified d!rin) &e>!irements phase J Defects injected d!rin) Desi)n phaseB_ ` 322 'ode, D&< T Â'ode defects corrected d!rin) 'odin) phaseB - ADefects identified d!rin) &e>!irements phase J Defects identified d!rin) Desi)n phase J Defects injected d!rin) codin) phaseB_ ` 322 K#erall, D&< T Â0otal defects corrected at all phases before deli#eryB - A0otal defects detected at all phases before and after deli#eryB_ ` 322 ;etric Re&resentation $ercenta)e (alculated at Sta)e completion or $roject 'ompletion (alculated from B!) &eports and $eer &e#ie. &eports %efect %istribution: $ercenta)e of 0otal defects Distrib!ted across &e>!irements
Analysis, Desi)n &e#ie.s, 'ode &e#ie.s, (nit 0ests, 1nte)ration 0ests, System 0ests, (ser Acceptance 0ests, &e#ie. by $roject Leads and $roject Mana)ers/ Software Process ;etrics are meas!res .hich pro#ide information abo!t the performance of the de#elopment process itself/ Pur&ose:
378 of 367
3/ $ro#ide an 1ndicator to the (ltimate E!ality of Soft.are bein) $rod!ced 7/ Assists to the Kr)aniCation to impro#e its de#elopment process by *i)hli)htin) areas of 1nefficiency or error5prone areas of the process/ Software Product ;etrics are meas!res of some attrib!te of the Soft.are $rod!ct/ A<?ample, So!rce 'odeB/ Pur&ose: 3/ (sed to assess the >!ality of the o!tp!t ,hat are the most general metricsRe7uirements ;anagement ;etrics (ollected 3/ 7/ 8/ &e>!irements by state I Accepted, &ejected, $ostponed "o/ of baselined re>!irements "!mber of re>!irements modified after base linin) 3/ 7/ &e>!irements Stability 1nde? A&S1B &e>!irements to Desi)n 0raceability %erived ;etrics 3/ Sched!le +ariance 3/ <ffort +ariance 3/ 'ost +ariance 3/ SiCe +ariance
%erived ;etrics
Pro=ect ;anagement ;etrics (ollected 3/ $lanned "o/ of days 7/ Act!al "o/ of days 3/ <stimated effort 7/ Act!al <ffort 3/ <stimated 'ost 7/ Act!al 'ost 3/ <stimated SiCe 7/ Act!al SiCe Testing G Review ;etrics (ollected 3/ 7/ 8/ 6/ "o/ of defects fo!nd by &e#ie.s "o/ of defects fo!nd by 0estin) "o/ of defects fo!nd by 'lient 0otal "o/ of defects fo!nd by &e#ie.s
%erived ;etrics 3/ K#erall &e#ie. <ffecti#eness AK&<B 7/ K#erall 0est <ffecti#eness Peer Reviews ;etrics (ollected
376 of 367
3/ 7/ 8/ 6/ =/ 4/ 9/ :/ ;/
KLK' - F$ per person ho!r ALan)!a)eB for $reparation KLK' - F$ per person ho!r ALan)!a)eB for &e#ie. Meetin) "o/ of pa)es - ho!r re#ie.ed d!rin) preparation A#era)e n!mber of defects fo!nd by &e#ie.er d!rin) $reparation "o/ of pa)es - ho!r re#ie.ed d!rin) &e#ie. Meetin) A#era)e n!mber of defects fo!nd by &e#ie.er d!rin) &e#ie. Meetin) &e#ie. 0eam SiCe +s Defects &e#ie. speed +s Defects Major defects fo!nd d!rin) &e#ie. Meetin)
32/ Defects +s &e#ie. <ffort %erived ;etrics 3/ &e#ie. <ffecti#eness AMajorB 7/ 0otal n!mber of defects fo!nd by re#ie.s for a project >ther ;etrics ;etrics (ollected 3/ "o/ of &e>!irements Desi)ned 7/ "o/ of &e>!irements not Desi)ned 8/ "o/ of Desi)n elements matchin) &e>!irements 6/ "o/ of Desi)n elements not matchin) &e>!irements =/ "o/ of &e>!irements 0ested 4/ "o/ of &e>!irements not 0ested 9/ "o/ of 0est 'ases .ith matchin) &e>!irements :/ "o/ of 0est 'ases .itho!t matchin) &e>!irements ;/ "o/ of Defects by Se#erity 32/ "o/ of Defects by sta)e of 5 Kri)in, Detection, &emo#al %erived ;etrics 3/ Defect Density 7/ "o/ of &e>!irements Desi)ned +s not Desi)ned 8/ "o/ of &e>!irements 0ested +s not 0ested 6/ Defect &emo#al <fficiency AD&<B Some ;etrics 9:&lained Schedule /ariance 0S/1 %escri&tion 0his metric )i#es the #ariation of act!al sched!le #s/ the planned sched!le/ 0his is calc!lated for each project I sta)e .ise Formula S+ T ÂAct!al no/ of days I $lanned no/ of daysB - $lanned no/ of days_ ` 322
37= of 367
;etric Re&resentation $ercenta)e (alculated at Sta)e completion (alculated from Soft.are $roject $lan for planned n!mber of days for completin) each sta)e and for act!al n!mber of days ta en to complete each sta)e 'efect Removal Efficienc! 0'RE1 %escri&tion 0his metric .ill indicate the effecti#eness of the defect identification and remo#al in sta)es for a )i#en project Formula &e>!irements, D&< T Â&e>!irement defects corrected d!rin) &e>!irements phaseB - A&e>!irement defects injected d!rin) &e>!irements phaseB_ ` 322 Desi)n, D&< T ÂDesi)n defects corrected d!rin) Desi)n phaseB - ADefects identified d!rin) &e>!irements phase J Defects injected d!rin) Desi)n phaseB_ ` 322 'ode, D&< T Â'ode defects corrected d!rin) 'odin) phaseB - ADefects identified d!rin) &e>!irements phase J Defects identified d!rin) Desi)n phase J Defects injected d!rin) codin) phaseB_ ` 322 K#erall, D&< T Â0otal defects corrected at all phases before deli#eryB - A0otal defects detected at all phases before and after deli#eryB_ ` 322 ;etric Re&resentation $ercenta)e (alculated at Sta)e completion or $roject 'ompletion (alculated from B!) &eports and $eer &e#ie. &eports 2verall Review Effectiveness %escri&tion 0his metric .ill indicate the effecti#eness of the &e#ie. process in identifyin) the defects for a )i#en project Formula K#erall &e#ie. <ffecti#eness, K&< T Â"!mber of defects fo!nd by re#ie.sB A0otal n!mber of defects fo!nd by re#ie.s J "!mber of defects fo!nd d!rin) 0estin) J "!mber of defects fo!nd d!rin) post5deli#eryB_ ` 322
374 of 367
;etric Re&resentation $ercenta)e
(alculated at Monthly Sta)e completion or $roject 'ompletion
(alculated from $eer re#ie.s, Formal &e#ie.s 0est &eports '!stomer 1dentified Defects
2verall Test Effectiveness 02TE1 %escri&tion 0his metric .ill indicate the effecti#eness of the 0estin) process in identifyin) the defects for a )i#en project d!rin) the testin) sta)e Formula K#erall 0est <ffecti#eness, K0< T Â"!mber of defects fo!nd d!rin) testin)B A0otal n!mber of defects fo!nd d!rin) 0estin) J "!mber of defects fo!nd d!rin) post deli#eryB_ ` 322 ;etric Re&resentation $ercenta)e
(alculated at Monthly B!ild completion or $roject 'ompletion
(alculated from 0est &eports '!stomer 1dentified Defects
Effort /ariance 0E/1 %escri&tion 0his metric )i#es the #ariation of act!al effort #s/ the estimated effort/ 0his is calc!lated for each project Sta)e .ise Formula <+ T ÂAct!al person ho!rs I <stimated person ho!rsB - <stimated person ho!rs_ ` 322 ;etric Re&resentation $ercenta)e
(alculated at Sta)e completion as identified in S$$

379 of 367
(alculated from <stimation sheets for estimated #al!es in person ho!rs, for each acti#ity .ithin a )i#en sta)e and Act!al Wor ed *o!rs #al!es in person ho!rs/ (ost /ariance 0(/1 %escri&tion 0his metric )i#es the #ariation of act!al cost +s the estimated cost/ 0his is calc!lated for each project Sta)e .ise Formula '+ T ÂAct!al 'ost I <stimated 'ostB - <stimated 'ost_ ` 322
(alculated at Sta)e completion
(alculated from <stimation sheets for estimated #al!es in dollars or r!pees, for each acti#ity .ithin a )i#en sta)e Act!al cost inc!rred
Si3e /ariance %escri&tion 0his metric )i#es the #ariation of act!al siCe +s/ the estimated siCe/ 0his is calc!lated for each project sta)e .ise Formula SiCe +ariance T ÂAct!al SiCe I <stimated SiCeB - <stimated SiCe_ ` 322
(alculated at Sta)e completion $roject 'ompletion
(alculated from <stimation sheets for estimated #al!es in F!nction $oints or KLK' Act!al siCe
Productivit! on Review Preparation Technical %escri&tion

37: of 367
0his metric .ill indicate the effort spent on preparation for &e#ie./ (se this to calc!late for lan)!a)es !sed in the $roject Formula For ever! language 3such as (B (JJB DavaB CSLB etcW4 usedB calculate
AKLK' or F$ B - ho!r A` Lan)!a)eB
`Lan)!a)e I ', 'JJ, Ja#a, LML, etcW ;etric Re&resentation KLK' or F$ per ho!r
(alculated at Monthly B!ild completion
(alculated from $eer &e#ie. &eport
$umber of defects found &er Review ;eeting %escri&tion 0his metric .ill indicate the n!mber of defects fo!nd d!rin) the &e#ie. Meetin) across #ario!s sta)es of the $roject Formula "!mber of defects per &e#ie. Meetin)
;etric Re&resentation Defects - &e#ie. Meetin)
(alculated at Monthly 'ompletion of &e#ie.
(alculated from $eer &e#ie. &eport $eer &e#ie. Defect List
Review Team 9fficienc! 3Review Team Si?e "s %efects Trend4 %escri&tion 0his metric .ill indicate the &e#ie. 0eam siCe and the defects trend/ 0his .ill help to determine the efficiency of the &e#ie. 0eam Formula &e#ie. 0eam SiCe to the Defects trend
;etric Re&resentation &atio

37; of 367
(alculated at
Monthly 'ompletion of &e#ie.
Review 9ffectiveness %escri&tion 0his metric .ill indicate the effecti#eness of the &e#ie. process Formula &e#ie. <ffecti#eness T Â"!mber of defects fo!nd by &e#ie.sB - AA0otal n!mber of defects fo!nd by re#ie.sB J 0estin)B_ ` 322 ;etric Re&resentation $ercenta)e
(alculated at 'ompletion of &e#ie. or 'ompletion of 0estin) sta)e
(alculated from $eer &e#ie. &eport $eer &e#ie. Defect List B!)s &eported by 0estin)
Total number of defects found b! Reviews %escri&tion 0his metric .ill indicate the total n!mber of defects identified by the &e#ie. process/ 0he defects are f!rther cate)oriCed as *i)h, Medi!m or Lo. Formula 0otal n!mber of defects identified in the $roject ;etric Re&resentation Defects per Sta)e
(alculated at 'ompletion of &e#ie.s
%efects vs# Review effort H Review Aield %escri&tion
382 of 367
0his metric .ill indicate the effort e?pended in each sta)e for re#ie.s to the defects fo!nd Formula Defects - &e#ie. effort
;etric Re&resentation Defects - &e#ie. effort
(alculated at 'ompletion of &e#ie.s
Re7uirements Stabilit! Inde: 3RSI4 %escri&tion 0his metric )i#es the stability factor of the re>!irements o#er a period of time, after the re>!irements ha#e been m!t!ally a)reed and baselined bet.een 1#esia Sol!tions and the 'lient Formula &S1 T 322 ` ^ A"!mber of baselined re>!irementsB I A"!mber of chan)es in re>!irements after the re>!irements are baselinedB _ - A"!mber of baselined re>!irementsB ;etric Re&resentation $ercenta)e
(alculated at Sta)e completion and $roject completion
(alculated from 'han)e &e>!est Soft.are &e>!irements Specification
(hange Re7uests b! State %escri&tion 0his metric pro#ides the analysis on state of the re>!irements Formula "!mber of accepted re>!irements "!mber of rejected re>!irements "!mber of postponed re>!irements
383 of 367
;etric Re&resentation "!mber
(alculated from 'han)e &e>!est Soft.are &e>!irements Specification
Re7uirements to %esign Traceabilit! %escri&tion 0his metric pro#ides the analysis on the n!mber of re>!irements desi)ned to the n!mber of re>!irements that .ere not desi)ned Formula 0otal "!mber of &e>!irements "!mber of &e>!irements Desi)ned "!mber of &e>!irements not Desi)ned
(alculated from S&S Detail Desi)n
%esign to Re7uirements Traceabilit! %escri&tion 0his metric pro#ides the analysis on the n!mber of desi)n elements matchin) re>!irements to the n!mber of desi)n elements not matchin) re>!irements Formula "!mber of Desi)n elements "!mber of Desi)n elements matchin) &e>!irements "!mber of Desi)n elements not matchin) &e>!irements
(alculated from S&S

387 of 367
Detail Desi)n
Re7uirements to Test case Traceabilit! %escri&tion 0his metric pro#ides the analysis on the n!mber of re>!irements tested +s the n!mber of re>!irements not tested Formula "!mber of &e>!irements "!mber of &e>!irements 0ested "!mber of &e>!irements not 0ested
(alculated from S&S Detail Desi)n 0est 'ase Specification
Test cases to Re7uirements traceabilit! %escri&tion 0his metric pro#ides the analysis on the n!mber of test cases matchin) re>!irements +s the n!mber of test cases not matchin) re>!irements Formula "!mber of &e>!irements "!mber of 0est cases .ith matchin) &e>!irements "!mber of 0est cases not matchin) &e>!irements
(alculated from S&S 0est 'ase Specification
$umber of defects in coding found during testing b! severit! %escri&tion 0his metric pro#ides the analysis on the n!mber of defects by the se#erity Formula
388 of 367
"!mber of Defects "!mber of defects of lo. priority "!mber of defects of medi!m priority "!mber of defects of hi)h priority
(alculated from B!) &eport
%efects H Stage of originB detectionB removal %escri&tion 0his metric pro#ides the analysis on the n!mber of defects by the sta)e of ori)in, detection and remo#al/ Formula "!mber of Defects Sta)e of ori)in Sta)e of detection Sta)e of remo#al
(alculated from B!) &eport
%efect %ensit! %escri&tion 0his metric pro#ides the analysis on the n!mber of defects to the siCe of the .or prod!ct Formula Defect Density T ^0otal no/ of Defects - SiCe AF$ - KLK'B_ ` 322 ;etric Re&resentation $ercenta)e

386 of 367
(alculated from
Defects List B!) &eport
ow do !ou determine metrics for !our a&&licationKbjecti#es of Metrics are not only to meas!re b!t also !nderstand the pro)ress to the Kr)aniCational Goal/ 0he $arameters for determinin) the Metrics for an application, D!ration 'omple?ity 0echnolo)y 'onstraints $re#io!s <?perience in Same 0echnolo)y B!siness Domain 'larity of the scope of the project
Kne interestin) and !sef!l approach to arri#e at the s!itable metrics is !sin) the Goal5E!estion5Metric 0echni>!e/ As e#ident from the name, the GEM model consists of three layersR a Goal, a Set of E!estions, and lastly a Set of 'orrespondin) Metrics/ 1t is th!s a hierarchical str!ct!re startin) .ith a )oal Aspecifyin) p!rpose of meas!rement, object to be meas!red, iss!e to be meas!red, and #ie.point from .hich the meas!re is ta enB/ 0he )oal is refined into se#eral >!estions that !s!ally brea do.n the iss!e into its major components/ <ach >!estion is then refined into metrics, some of them objecti#e, some of them s!bjecti#e/ 0he same metric can be !sed in order to ans.er different >!estions !nder the same )oal/ Se#eral GEM models can also ha#e >!estions and metrics in common, ma in) s!re that, .hen the meas!re is act!ally ta en, the different #ie.points are ta en into acco!nt correctly Ai/e/, the metric mi)ht ha#e different #al!es .hen ta en from different #ie.pointsB/ 1n order to )i#e an e?ample of application of the model,
Goal Pur&ose Issue >b=ect "iew Point
1mpro#e the timeliness of 'han)e &e>!est $rocessin) from the $roject Mana)erFs #ie.point What is the c!rrent 'han)e &e>!est $rocessin) Speed@ A#era)e 'ycle 0ime Standard De#iation
Kuestion ;etric
38= of 367
Kuestion ;etric
O cases o!tside of the !pper limit 1s the performance of the process impro#in)@ '!rrent a#era)e cycle time Baseline a#era)e cycle time 322 S!bjecti#e ratin) of mana)erQs satisfaction
,hen do !ou determine ;etricsWhen the re>!irements are !nderstood in a hi)h5le#el, at this sta)e, the team siCe, project siCe m!st be no.n to an e?tent, in .hich the project is at a PdefinedP sta)e/
384 of 367
References
<ffecti#e Methods of Soft.are 0estin), William < $erry/ Soft.are <n)ineerin) I A $ractitioners Approach, &o)er $ressman/ An A$1 0estin) Method by Alan A Jor)ensen and James A Whitta er/ A$1 0estin) Methodolo)y by Anoop K!mar $, .or in) for "o#ell Soft.are De#elopment A1B $#t Ltd/, Ban)alore/ GWhy is A$1 0estin) Different Gby "i hil "ila antan, *e.lett $ac ard and 1brahim K/ <l5Far, Florida 1nstit!te of 0echnolo)y/ 0est Strate)y S 0est $lan $reparation I 0rainin) co!rse attended Y SoftSmith Desi)nin) 0est 'ases 5 'em Kaner, J/D/, $h/D/ Scenario 0estin) 5 'em Kaner, J/D/, $h/D/ <?ploratory 0estin) <?plained, #/3/8 6-34-28 by James Bach/ <?plorin) <?ploratory 0estin) by Andy 0in ham and 'em Kaner/ Session5Based 0est Mana)ement by Jonathan Bach Afirst p!blished in Soft.are 0estin) and E!ality <n)ineerin) ma)aCine, 33-22B/ Defect Dri#en <?ploratory 0estin) ADD<0B by Ananthala shmi/ Soft.are <n)ineerin) Body of Kno.led)e #3/2 Ahttp,--.../sei/cm!/ed!-p!blicationsB (nit 0estin) )!idelines by Scott *i)het Ahttp,--.../Stic yminds/comB http,--.../sasystems/com http,--.../soft.are>atest/com http,--.../en)/m!/ed!-corliss)-3;:/7223-KF"ach335tools/html http,--.../ics/!ci/ed!-bjrobbins-ics37=.26-nona#-ho.to5re#ie.s/html 1<<< SKF0WA&< &<+1<WS Std 327:53;;9 http,--.../a)ilemanifesto/or) http,--.../processimpact/com 0he Goal E!estion Metric Approach, +ictor &/ Basili3 Gianl!i)i 'aldiera3 */ Dieter &ombach7 http,--.../.ebopedia/com
389 of 367
G$8 Free %ocumentation License

+ersion 3/7, "o#ember 7227 'opyri)ht A'B 7222,7223,7227 Free Soft.are Fo!ndation, 1nc/ =; 0emple $lace, S!ite 882, Boston, MA 2733353829 (SA <#eryone is permitted to copy and distrib!te #erbatim copies of this license doc!ment, b!t chan)in) it is not allo.ed/ .# PR9';BL9 0he p!rpose of this License is to ma e a man!al, te?tboo , or other f!nctional and !sef!l doc!ment PfreeP in the sense of freedom, to ass!re e#eryone the effecti#e freedom to copy and redistrib!te it, .ith or .itho!t modifyin) it, either commercially or noncommercially/ Secondarily, this License preser#es for the a!thor and p!blisher a .ay to )et credit for their .or , .hile not bein) considered responsible for modifications made by others/ 0his License is a ind of PcopyleftP, .hich means that deri#ati#e .or s of the doc!ment m!st themsel#es be free in the same sense/ 1t complements the G"( General $!blic License, .hich is a copyleft license desi)ned for free soft.are/ We ha#e desi)ned this License in order to !se it for man!als for free soft.are, beca!se free soft.are needs free doc!mentation, a free pro)ram sho!ld come .ith man!als pro#idin) the same freedoms that the soft.are does/ B!t this License is not limited to soft.are man!alsR it can be !sed for any te?t!al .or , re)ardless of s!bject matter or .hether it is p!blished as a printed boo / We recommend this License principally for .or s .hose p!rpose is instr!ction or reference/ )# 'PPLI('BILITA '$% %9FI$ITI>$S 0his License applies to any man!al or other .or , in any medi!m, that contains a notice placed by the copyri)ht holder sayin) it can be distrib!ted !nder the terms of this License/ S!ch a notice )rants a .orld5.ide, royalty5free license, !nlimited in d!ration, to !se that .or !nder the conditions stated herein/ 0he PDoc!mentP, belo., refers to any s!ch man!al or .or / Any member of the p!blic is a licensee, and is addressed as Pyo!P/ %o! accept the license if yo! copy, modify or distrib!te the .or in a .ay re>!irin) permission !nder copyri)ht la./ A PModified +ersionP of the Doc!ment means any .or containin) the Doc!ment or a portion of it, either copied #erbatim, or .ith modifications and-or translated into another lan)!a)e/ A PSecondary SectionP is a named appendi? or a front5matter section of the Doc!ment that deals e?cl!si#ely .ith the relationship of the p!blishers or a!thors of the Doc!ment to the Doc!mentQs o#erall s!bject Aor to related mattersB and contains nothin) that co!ld fall directly .ithin that o#erall s!bject/ A0h!s, if the Doc!ment is in part a te?tboo of mathematics, a Secondary Section may not e?plain any mathematics/B 0he relationship co!ld be a matter of historical connection .ith the s!bject or .ith related matters, or of le)al, commercial, philosophical, ethical or political position re)ardin) them/ 0he P1n#ariant SectionsP are certain Secondary Sections .hose titles are desi)nated, as bein) those of 1n#ariant Sections, in the notice that says that the Doc!ment is released !nder this License/ 1f a section does not fit the abo#e definition of Secondary then it is not allo.ed to be desi)nated as 1n#ariant/ 0he Doc!ment may contain Cero 1n#ariant Sections/ 1f the Doc!ment does not identify any 1n#ariant Sections then there are none/ 0he P'o#er 0e?tsP are certain short passa)es of te?t that are listed, as Front5'o#er 0e?ts or Bac 5'o#er 0e?ts, in the notice that says that the Doc!ment is released !nder this License/ A Front5'o#er 0e?t may be at most = .ords, and a Bac 5'o#er 0e?t may be at most 7= .ords/ A P0ransparentP copy of the Doc!ment means a machine5readable copy, represented in a format .hose specification is a#ailable to the )eneral p!blic, that is s!itable for re#isin) the doc!ment strai)htfor.ardly .ith )eneric te?t editors or Afor ima)es composed of pi?elsB )eneric paint pro)rams or Afor dra.in)sB some .idely a#ailable dra.in) editor, and that is s!itable for inp!t to te?t formatters or for a!tomatic
38: of 367
translation to a #ariety of formats s!itable for inp!t to te?t formatters/ A copy made in an other.ise 0ransparent file format .hose mar !p, or absence of mar !p, has been arran)ed to th.art or disco!ra)e s!bse>!ent modification by readers is not 0ransparent/ An ima)e format is not 0ransparent if !sed for any s!bstantial amo!nt of te?t/ A copy that is not P0ransparentP is called PKpa>!eP/ <?amples of s!itable formats for 0ransparent copies incl!de plain AS'11 .itho!t mar !p, 0e?info inp!t format, La0eL inp!t format, SGML or LML !sin) a p!blicly a#ailable D0D, and standard5conformin) simple *0ML, $ostScript or $DF desi)ned for h!man modification/ <?amples of transparent ima)e formats incl!de $"G, L'F and J$G/ Kpa>!e formats incl!de proprietary formats that can be read and edited only by proprietary .ord processors, SGML or LML for .hich the D0D and-or processin) tools are not )enerally a#ailable, and the machine5)enerated *0ML, $ostScript or $DF prod!ced by some .ord processors for o!tp!t p!rposes only/ 0he P0itle $a)eP means, for a printed boo , the title pa)e itself, pl!s s!ch follo.in) pa)es as are needed to hold, le)ibly, the material this License re>!ires to appear in the title pa)e/ For .or s in formats .hich do not ha#e any title pa)e as s!ch, P0itle $a)eP means the te?t near the most prominent appearance of the .or Qs title, precedin) the be)innin) of the body of the te?t/ A section P<ntitled L%NP means a named s!b!nit of the Doc!ment .hose title either is precisely L%N or contains L%N in parentheses follo.in) te?t that translates L%N in another lan)!a)e/ A*ere L%N stands for a specific section name mentioned belo., s!ch as PAc no.led)ementsP, PDedicationsP, P<ndorsementsP, or P*istoryP/B 0o P$reser#e the 0itleP of s!ch a section .hen yo! modify the Doc!ment means that it remains a section P<ntitled L%NP accordin) to this definition/ 0he Doc!ment may incl!de Warranty Disclaimers ne?t to the notice .hich states that this License applies to the Doc!ment/ 0hese Warranty Disclaimers are considered to be incl!ded by reference in this License, b!t only as re)ards disclaimin) .arranties, any other implication that these Warranty Disclaimers may ha#e is #oid and has no effect on the meanin) of this License/ +# "9RB'TI; (>PAI$G %o! may copy and distrib!te the Doc!ment in any medi!m, either commercially or noncommercially, pro#ided that this License, the copyri)ht notices, and the license notice sayin) this License applies to the Doc!ment are reprod!ced in all copies, and that yo! add no other conditions .hatsoe#er to those of this License/ %o! may not !se technical meas!res to obstr!ct or control the readin) or f!rther copyin) of the copies yo! ma e or distrib!te/ *o.e#er, yo! may accept compensation in e?chan)e for copies/ 1f yo! distrib!te a lar)e eno!)h n!mber of copies yo! m!st also follo. the conditions in section 8/ %o! may also lend copies, !nder the same conditions stated abo#e, and yo! may p!blicly display copies/ /# (>PAI$G I$ K8'$TITA 1f yo! p!blish printed copies Aor copies in media that commonly ha#e printed co#ersB of the Doc!ment, n!mberin) more than 322, and the Doc!mentQs license notice re>!ires 'o#er 0e?ts, yo! m!st enclose the copies in co#ers that carry, clearly and le)ibly, all these 'o#er 0e?ts, Front5'o#er 0e?ts on the front co#er, and Bac 5'o#er 0e?ts on the bac co#er/ Both co#ers m!st also clearly and le)ibly identify yo! as the p!blisher of these copies/ 0he front co#er m!st present the f!ll title .ith all .ords of the title e>!ally prominent and #isible/ %o! may add other material on the co#ers in addition/ 'opyin) .ith chan)es limited to the co#ers, as lon) as they preser#e the title of the Doc!ment and satisfy these conditions, can be treated as #erbatim copyin) in other respects/ 1f the re>!ired te?ts for either co#er are too #ol!mino!s to fit le)ibly, yo! sho!ld p!t the first ones listed Aas many as fit reasonablyB on the act!al co#er, and contin!e the rest onto adjacent pa)es/ 1f yo! p!blish or distrib!te Kpa>!e copies of the Doc!ment n!mberin) more than 322, yo! m!st either incl!de a machine5readable 0ransparent copy alon) .ith each Kpa>!e copy, or state in or .ith each Kpa>!e copy a comp!ter5net.or location from .hich the
38; of 367
)eneral net.or 5!sin) p!blic has access to do.nload !sin) p!blic5standard net.or protocols a complete 0ransparent copy of the Doc!ment, free of added material/ 1f yo! !se the latter option, yo! m!st ta e reasonably pr!dent steps, .hen yo! be)in distrib!tion of Kpa>!e copies in >!antity, to ens!re that this 0ransparent copy .ill remain th!s accessible at the stated location !ntil at least one year after the last time yo! distrib!te an Kpa>!e copy Adirectly or thro!)h yo!r a)ents or retailersB of that edition to the p!blic/ 1t is re>!ested, b!t not re>!ired, that yo! contact the a!thors of the Doc!ment .ell before redistrib!tin) any lar)e n!mber of copies, to )i#e them a chance to pro#ide yo! .ith an !pdated #ersion of the Doc!ment/ 0# ;>%IFI('TI>$S %o! may copy and distrib!te a Modified +ersion of the Doc!ment !nder the conditions of sections 7 and 8 abo#e, pro#ided that yo! release the Modified +ersion !nder precisely this License, .ith the Modified +ersion fillin) the role of the Doc!ment, th!s licensin) distrib!tion and modification of the Modified +ersion to .hoe#er possesses a copy of it/ 1n addition, yo! m!st do these thin)s in the Modified +ersion, '# (se in the 0itle $a)e Aand on the co#ers, if anyB a title distinct from that of the Doc!ment, and from those of pre#io!s #ersions A.hich sho!ld, if there .ere any, be listed in the *istory section of the Doc!mentB/ %o! may !se the same title as a pre#io!s #ersion if the ori)inal p!blisher of that #ersion )i#es permission/ B# List on the 0itle $a)e, as a!thors, one or more persons or entities responsible for a!thorship of the modifications in the Modified +ersion, to)ether .ith at least fi#e of the principal a!thors of the Doc!ment Aall of its principal a!thors, if it has fe.er than fi#eB, !nless they release yo! from this re>!irement/ (# State on the 0itle pa)e the name of the p!blisher of the Modified +ersion, as the p!blisher/ %# $reser#e all the copyri)ht notices of the Doc!ment/
9# Add an appropriate copyri)ht notice for yo!r modifications adjacent to the other copyri)ht notices/ F# 1ncl!de, immediately after the copyri)ht notices, a license notice )i#in) the p!blic permission to !se the Modified +ersion !nder the terms of this License, in the form sho.n in the Addend!m belo./ G# $reser#e in that license notice the f!ll lists of 1n#ariant Sections and re>!ired 'o#er 0e?ts )i#en in the Doc!mentQs license notice/ # 1ncl!de an !naltered copy of this License/ I# $reser#e the section <ntitled P*istoryP, $reser#e its 0itle, and add to it an item statin) at least the title, year, ne. a!thors, and p!blisher of the Modified +ersion as )i#en on the 0itle $a)e/ 1f there is no section <ntitled P*istoryP in the Doc!ment, create one statin) the title, year, a!thors, and p!blisher of the Doc!ment as )i#en on its 0itle $a)e, then add an item describin) the Modified +ersion as stated in the pre#io!s sentence/ D# $reser#e the net.or location, if any, )i#en in the Doc!ment for p!blic access to a 0ransparent copy of the Doc!ment, and li e.ise the net.or locations )i#en in the Doc!ment for pre#io!s #ersions it .as based on/ 0hese may be placed in the P*istoryP section/ %o! may omit a net.or location for a .or that .as p!blished at least fo!r years before the Doc!ment itself, or if the ori)inal p!blisher of the #ersion it refers to )i#es permission/ L# For any section <ntitled PAc no.led)ementsP or PDedicationsP, $reser#e the 0itle of the section, and preser#e in the section all the s!bstance and tone of each of the contrib!tor ac no.led)ements and-or dedications )i#en therein/
362 of 367
L# $reser#e all the 1n#ariant Sections of the Doc!ment, !naltered in their te?t and in their titles/ Section n!mbers or the e>!i#alent are not considered part of the section titles/ ;# Delete any section <ntitled P<ndorsementsP/ S!ch a section may not be incl!ded in the Modified +ersion/ $# Do not retitle any e?istin) section to be <ntitled P<ndorsementsP or to conflict in title .ith any 1n#ariant Section/ ># $reser#e any Warranty Disclaimers/ 1f the Modified +ersion incl!des ne. front5matter sections or appendices that >!alify as Secondary Sections and contain no material copied from the Doc!ment, yo! may at yo!r option desi)nate some or all of these sections as in#ariant/ 0o do this, add their titles to the list of 1n#ariant Sections in the Modified +ersionQs license notice/ 0hese titles m!st be distinct from any other section titles/ %o! may add a section <ntitled P<ndorsementsP, pro#ided it contains nothin) b!t endorsements of yo!r Modified +ersion by #ario!s parties55for e?ample, statements of peer re#ie. or that the te?t has been appro#ed by an or)aniCation as the a!thoritati#e definition of a standard/ %o! may add a passa)e of !p to fi#e .ords as a Front5'o#er 0e?t, and a passa)e of !p to 7= .ords as a Bac 5'o#er 0e?t, to the end of the list of 'o#er 0e?ts in the Modified +ersion/ Knly one passa)e of Front5'o#er 0e?t and one of Bac 5'o#er 0e?t may be added by Aor thro!)h arran)ements made byB any one entity/ 1f the Doc!ment already incl!des a co#er te?t for the same co#er, pre#io!sly added by yo! or by arran)ement made by the same entity yo! are actin) on behalf of, yo! may not add anotherR b!t yo! may replace the old one, on e?plicit permission from the pre#io!s p!blisher that added the old one/ 0he a!thorAsB and p!blisherAsB of the Doc!ment do not by this License )i#e permission to !se their names for p!blicity for or to assert or imply endorsement of any Modified +ersion/ 1# (>;BI$I$G %>(8;9$TS %o! may combine the Doc!ment .ith other doc!ments released !nder this License, !nder the terms defined in section 6 abo#e for modified #ersions, pro#ided that yo! incl!de in the combination all of the 1n#ariant Sections of all of the ori)inal doc!ments, !nmodified, and list them all as 1n#ariant Sections of yo!r combined .or in its license notice, and that yo! preser#e all their Warranty Disclaimers/ 0he combined .or need only contain one copy of this License, and m!ltiple identical 1n#ariant Sections may be replaced .ith a sin)le copy/ 1f there are m!ltiple 1n#ariant Sections .ith the same name b!t different contents, ma e the title of each s!ch section !ni>!e by addin) at the end of it, in parentheses, the name of the ori)inal a!thor or p!blisher of that section if no.n, or else a !ni>!e n!mber/ Ma e the same adj!stment to the section titles in the list of 1n#ariant Sections in the license notice of the combined .or / 1n the combination, yo! m!st combine any sections <ntitled P*istoryP in the #ario!s ori)inal doc!ments, formin) one section <ntitled P*istoryPR li e.ise combine any sections <ntitled PAc no.led)ementsP, and any sections <ntitled PDedicationsP/ %o! m!st delete all sections <ntitled P<ndorsements/P *# (>LL9(TI>$S >F %>(8;9$TS %o! may ma e a collection consistin) of the Doc!ment and other doc!ments released !nder this License, and replace the indi#id!al copies of this License in the #ario!s doc!ments .ith a sin)le copy that is incl!ded in the collection, pro#ided that yo! follo. the r!les of this License for #erbatim copyin) of each of the doc!ments in all other respects/ %o! may e?tract a sin)le doc!ment from s!ch a collection, and distrib!te it indi#id!ally !nder this License, pro#ided yo! insert a copy of this License into the e?tracted
363 of 367
doc!ment, and follo. this License in all other respects re)ardin) #erbatim copyin) of that doc!ment/ 2# 'GGR9G'TI>$ ,IT I$%9P9$%9$T ,>RLS A compilation of the Doc!ment or its deri#ati#es .ith other separate and independent doc!ments or .or s, in or on a #ol!me of a stora)e or distrib!tion medi!m, is called an Pa))re)ateP if the copyri)ht res!ltin) from the compilation is not !sed to limit the le)al ri)hts of the compilationQs !sers beyond .hat the indi#id!al .or s permit/ When the Doc!ment is incl!ded in an a))re)ate, this License does not apply to the other .or s in the a))re)ate .hich are not themsel#es deri#ati#e .or s of the Doc!ment/ 1f the 'o#er 0e?t re>!irement of section 8 is applicable to these copies of the Doc!ment, then if the Doc!ment is less than one half of the entire a))re)ate, the Doc!mentQs 'o#er 0e?ts may be placed on co#ers that brac et the Doc!ment .ithin the a))re)ate, or the electronic e>!i#alent of co#ers if the Doc!ment is in electronic form/ Kther.ise they m!st appear on printed co#ers that brac et the .hole a))re)ate/ 5# TR'$SL'TI>$ 0ranslation is considered a ind of modification, so yo! may distrib!te translations of the Doc!ment !nder the terms of section 6/ &eplacin) 1n#ariant Sections .ith translations re>!ires special permission from their copyri)ht holders, b!t yo! may incl!de translations of some or all 1n#ariant Sections in addition to the ori)inal #ersions of these 1n#ariant Sections/ %o! may incl!de a translation of this License, and all the license notices in the Doc!ment, and any Warranty Disclaimers, pro#ided that yo! also incl!de the ori)inal <n)lish #ersion of this License and the ori)inal #ersions of those notices and disclaimers/ 1n case of a disa)reement bet.een the translation and the ori)inal #ersion of this License or a notice or disclaimer, the ori)inal #ersion .ill pre#ail/ 1f a section in the Doc!ment is <ntitled PAc no.led)ementsP, PDedicationsP, or P*istoryP, the re>!irement Asection 6B to $reser#e its 0itle Asection 3B .ill typically re>!ire chan)in) the act!al title/ 6# T9R;I$'TI>$ %o! may not copy, modify, s!blicense, or distrib!te the Doc!ment e?cept as e?pressly pro#ided for !nder this License/ Any other attempt to copy, modify, s!blicense or distrib!te the Doc!ment is #oid, and .ill a!tomatically terminate yo!r ri)hts !nder this License/ *o.e#er, parties .ho ha#e recei#ed copies, or ri)hts, from yo! !nder this License .ill not ha#e their licenses terminated so lon) as s!ch parties remain in f!ll compliance/ ).# F8T8R9 R9"ISI>$S >F T IS LI(9$S9 0he Free Soft.are Fo!ndation may p!blish ne., re#ised #ersions of the G"( Free Doc!mentation License from time to time/ S!ch ne. #ersions .ill be similar in spirit to the present #ersion, b!t may differ in detail to address ne. problems or concerns/ See http,--.../)n!/or)-copyleft-/ <ach #ersion of the License is )i#en a distin)!ishin) #ersion n!mber/ 1f the Doc!ment specifies that a partic!lar n!mbered #ersion of this License Por any later #ersionP applies to it, yo! ha#e the option of follo.in) the terms and conditions either of that specified #ersion or of any later #ersion that has been p!blished Anot as a draftB by the Free Soft.are Fo!ndation/ 1f the Doc!ment does not specify a #ersion n!mber of this License, yo! may choose any #ersion e#er p!blished Anot as a draftB by the Free Soft.are Fo!ndation/
367 of 367

Software Testing Guide Book Part 1

Uploaded by

Copyright:

Available Formats

You might also like

Software Testing Guide Book Part 1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Software Testing Guide Book Part 1

Uploaded by

Copyright:

Available Formats

Software Testing Guide Book

Part I: Fundamentals of Software Testing

Software Testing Research Lab http,--.../Sof0&eL/or)

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

%escri&tion 1nitial doc!ment creation 1ncorporated &e#ie. 'omments Draft &elease

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

Software Testing Guide Book################################################################################) )# The Software Testing Guide Book######################################################################*

+# ,hat is Software Testing and ,h! is it Im&ortant-#########################################). /# T!&es of %evelo&ment S!stems#######################################################################)+

0# T!&es of Software S!stems##############################################################################)/

8/3 8/7 8/8 8/6

;/3 &e#ie.///////////////////////////////////////////////////////////////////////////////////////////////////82 ;/7 Wal thro!)h//////////////////////////////////////////////////////////////////////////////////////////88

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

32/3 White Bo? 0estin)////////////////////////////////////////////////////////////////////////////////8:

37/3 (nit 0estin)/////////////////////////////////////////////////////////////////////////////////////////6: 37/7 1nte)ration 0estin)///////////////////////////////////////////////////////////////////////////////=8

12.2.1 Top./own 0ntegration.....................................................................................!3 12.2.2 Bottom.1p 0ntegration....................................................................................!3

)6# %efect ;anagement####################################################################################)+.

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

)# The Software Testing Guide Book

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

Pur&ose of this %ocument

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

ow to use this %ocument

,hat this Guide Book is not

can #isit http,--.../softrel/or)-st)b/html for !pdates on the $roject/

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

/# T!&es of %evelo&ment S!stems

/#) Traditional %evelo&ment S!stems

/#+ Iterative %evelo&ment

/#/ ;aintenance S!stem

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

/#0 Purchased<(ontracted Software

0# T!&es of Software S!stems

0#) Batch S!stems

0#+ 9vent (ontrol S!stems

0#/ Process (ontrol S!stems

Soft.are 0estin) G!ide Boo

$art 1, F!ndamentals of Soft.are 0estin)

0#0 Procedure (ontrol S!stems

0#1 'dvanced ;athematical ;odels

0#* ;essage Processing S!stems

0#2 %iagnostic Software S!stems

0#5 Sensor and Signal Processing S!stems