Professional Documents
Culture Documents
Black Box Testing PDF
Black Box Testing PDF
Black Box Testing PDF
Readings:RonPattonchap5,DanielGalinchap9.5.1 Withmaterialfrom:R.Binder,TestingObjectOrientedSystems
WhatisBlackBoxtesting?
Testingwithouthavinganinsightintothedetails ofunderlyingcode
input Unit output
Testingbasedonspecification
WhatisBlackBoxtesting?
Advantages noneedforsourcecode canbeusedforunitaswellassystemleveltesting Disadvantages don'ttesthiddenfunctions Approaches: Errorguessing EquivalencePartitioning BoundaryValueAnalysis Decisiontables CauseEffectGraphing Testsderivedfromfunctionalrequirements(e.g.usecases)
ErrorGuessing(1)
Adhocapproachbasedonexperience Approach:
1.Makelistofpossibleerrorsorerrorpronesituations
errormodel
2.Designtestcasestocovererrormodel
Developandmaintainyourownerrormodels
ErrorGuessing(2)
Examplesortingarrayfunction Errormodel:
Generatetestcasesforthesesituations
EquivalencePartitioning
DifficultinpracticetofindperfectECs
HeuristicsforIdentifyingEquivalence Classes
Foreachexternalinput:
1. ifspecifiesrangeofvalidvalues,define
2. ifspecifiesanumber(N)ofvalidvalues,define
3. ifspecifiesasetofvalidvalues,define
HeuristicsforIdentifyingEquivalence Classes
Foreachexternalinput:
4. ifthereisareasontobelievethatprogramhandlevalidinput differently,define
5. ifspecifiesamustbesituation,define
6. ifthereisareasontobelievethatelementsinanECarenot handledinanidenticalmannerbytheprogram,
subdivideECintosmallerEcs.
EquivalenceClassPartitioning
EquivalenceClassesPartitioning TriangleExample(1)
Specification
inputthreeintegers(sidesofatriangle:a,b,c)
outputtypeofthetriangle:
EquivalenceClassesPartitioning TriangleExample(2)
Accordingtoheuristic#5
InputCondition Sides(a,b,c) ValidEC All>0&<=20(1) InvalidEC a>20(2) b>20(3) c>20(4) a<=0(5) b<=0(6) c<=0(7)
EC#1toobroadcanbesubdivided(heuristic#6)
EquivalenceClassesPartitioning TriangleExample(3)
Basedonthetreatmenttodatahandlingofdata
Basedoninput
Basedontriangleproperty
EquivalenceClassesPartitioning Identifyingtestcases
1.UntilallvalidECshavebeencoveredbytestcases,
writeanewtestcasethatcoverasmanyofthe uncoveredvalidECsaspossible
2.UntilallinvalidECshavebeencoveredbytestcases,
writeatestcasethatcoversone,andonlyoneofthe uncoveredinvalidECs
EquivalenceClassesPartitioning TriangleExample(4)
Testcasesfortriangle
EC 1.1,1.5 1.2,1.6 1.3,1.9 1.4,1.10 2 3 4 5 6 7 a 12 6 9 8 30 10 7 5 12 8 b 12 6 3 2 15 100 14 10 10 5 c 12 7 7 3 6 99 21 11 10 1 ExpectedOutput Equilateral Isosceles Scalene Notatriangle
EquivalenceClassesPartitioning Nextdate(1)
Input:month,day,yearrepresentingadate
Output:dateofthedayaftertheinputdate
EquivalenceClassesPartitioning Nextdate(2)
Equivalenceclasses
InputCondition month day year ValidEcs 1<=month<=12 1<=day<=31 1812<=year<=2012 InvalidEcs month<1 month>12 day<1 day>31 year<1812 year>2012
EquivalenceClassesPartitioning Nextdate(3)
DecompositionofvalidECs
InputCondition month ValidEcs 30days(1) 31days(2) February(3) 1<=day<=28(4) day=29(5) day=30(6) day=31(7) year=1900(8) 1812<=year<=2012and year<>1900and yearmod4=0(9) 1812<=year<=2012and yearmod4<>0(10)
day
year
EquivalenceClassesPartitioning Nextdate(4)
Testcases
ID 1 2 3 4 5 6 7 8 9 10 11 12 Ecs month 1,4,8 6 1,4,9 6 1,4,10 6 1,5,8 6 1,5,9 6 1,5,10 6 1,6,8 6 1,6,9 6 1,6,10 6 1,7,8 6 1,7,9 6 1,7,10 6 day 14 14 14 29 29 29 30 30 30 31 31 31 year 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913 output 15/06/1900 15/06/1912 15/06/1913 30/06/1900 30/06/1912 30/06/1913 01/07/1900 01/07/1912 01/07/1913 Error Error Error
EquivalenceClassesPartitioning Problems
BoundaryValueAnalysis
BoundaryConditions
Boundaryconditioncharacteristics
BoundaryValueAnalysis Example
Examplesofboundaryvaluesfornextdate
BoundaryValueAnalysis Guidelines
Foreachboundarycondition
Usesameguidelinesforeachoutputcondition
includetestcaseswithinputsuchthatoutputatboundariesare produced
BoundaryValueAnalysis
SubBoundaryConditions
internaltosoftware(notdefinedinspecification) examples:
Powersoftwo:bit,nibble,byte,word,kilo,mega,giga,tera ASCIItable
BoundaryValueAnalysis
Character / lowerbound 0 1 2 3 4 5 6 7 8 upperbound 9 : A a ASCIICode 47 48 49 50 51 52 53 54 55 56 57 58 65 97
DecisionTable
Idealforsituationswhere:
DecisionTableFormat
Conditions
Combinationof conditions(variants)
Actions
Selectedactions
DecisionTableDevelopment
1. Identifydecisionvariablesandconditions 2. Identifyresultantactionstobeselectedorcontrolled 3. Identifywhichactionshouldbeproducedinresponseto particularcombinationsofactions
Example
Supposethefollowinginsurancerenewalrules
DecisionTableTriangleExample
Decisionvariables:sidesa,b,c Conditions
Actions
DecisionTableTriangleExample
c1:a<b+c c2:b<a+c c3:c<a+b c4:a=b c5:a=c c6:b=c c7:a>20 c8:b>20 c9:c>20 a1:notatriangle a2:Scalene a3:Isosceles a4:Equilateral a5:impossible a6:error
1 N N N N X
2 Y N N N N X
3 Y Y N N N N X
4 Y Y Y Y Y Y N N N
5 Y Y Y Y Y N N N N
6 Y Y Y Y N Y N N N
7 Y Y Y Y N N N N N
8 Y Y Y N Y Y N N N
9 Y Y Y N Y N N N N
10 Y Y Y N N Y N N N
11 Y Y Y N N N N N N X
12 Y
13 Y
14 Y
X X X X X
Attentionforinconsistencies
DecisionTableDon'tCare condition
Don'tCarecondition
DecisionTableCan'tHappen& Don'tknowconditions
AttentionnottoconfuseaDon'tCareconditionwith
Can'tHappenConditionreflectsassumptionthat
Don'tKnowConditionreflectsanincompletemodel
testsneededtoexercisetheseundefinedcases
Usuallyindicationofmisspecification
AllExplicitvariants
DecisionTableTestgeneration strategies
atestcaseforeachexplicitvariant atestcaseforeachimplicitvariant
2n variantsfor nconditions
AllVariants
AllTrue
atestcaseforeachvariantwithanoutcome(action) atestcaseforeachvariantwithoutoutcome
AllFalse
DecisionTableTestgeneration Triangleexample
AllExplicitVariants
Variant a 1 4 2 1 3 1 4 5 5 6 7 2 8 9 2 10 3 11 3 12 21 13 6 b 1 4 2 5 b 2 2 4 5 Expectedoutput NotaTriangle NotaTriangle NotaTriangle Equilateral Impossible Impossible Isosceles Impossible Isosceles Isosceles Scalene Error Error
2 3 2 4 6 45
3 2 2 5 12 17
EachCondition/AllConditions
DecisionTableTestgeneration strategies
Heuristictoreducenumberofvariants Foreachvariable,avariantsuchthatthevariableismade truewithallothervariablesbeingfalse,and
Setofvariantsincludes
Onevariantsuchthatallvariablesaretrue(andlogic),or Onevariantsuchthatallvariablesarefalse(orlogic)
DecisionTableEachCondition/All Conditions
S=PorQorR
P F F T F
Q F T F F
R S (action) T x F x F x F
DecisionTableEachCondition/All Conditions
S=PandQandR
P F F T T
Q F T F T
R S (action) T F F T x
DecisionTableEachCondition/All Conditions
Z=(AandBand(notC))or(AandD)
Developvariantsfor(AandBand(notC))andvariantsfor(AandD)
A T F F T T F T
B F T F T -
C T T F F -
D F T T
Z (action)
Valuesmustbeassignedto
DecisionTableEachCondition/All Conditions
Z=(AandBand(notC))or(AandD)
A T F F T T F T B F T F T T T T C T T F F F T T D T F F F F T T Z (action)
fortestsuitegeneration
Exampleofvariantsused
DecisionTableEachCondition/All Conditions
Providesacompacttable
numberoftestsincreaselinearlywithnumberof productterms
Assumesindependenceofconditionevaluation, andabsenceofdefaultactions
DecisionTableBinaryDecision DiagramDeterminants
BinaryDecisionDiagram(BDD)compactrepresentationofa truthtable 1.ConstructBDDfromthetruthtable a)Createadecisiontreefromthetruthtable
DecisionTableBinaryDecision DiagramDeterminants
Example:Z=Aand(BorC) DecisionTree A B C 0 0 0 0 C 1 1 C 1
B C 0
DecisionTableBinaryDecision DiagramDeterminants
b)ReducedecisiontreetoaBDD
Fromlefttoright,replaceleafnodeswithequivalent constantsorvariablesandprunethebranches A B C C 0 0 0 0 C 1 1 B C 1
DecisionTableBinaryDecision DiagramDeterminants
A 0 C 0 1 1 B C 1
DecisionTableBinaryDecision DiagramDeterminants
A 0 C 1 B C 1
DecisionTableBinaryDecision DiagramDeterminants
DecisionTableBinaryDecision DiagramDeterminants
2.MapBDDintoBDDdeterminanttable
BDDdeterminantroottoleafpathinBDD
BDD Variant 1 2 3 A F T T B F T C C Z F C T
DecisionTableBinaryDecision DiagramDeterminants
3.GenerateBDDtestsuite
BDD Variant 1 2 2 3 A F T T T B F F T C F T Z
x x
Valuesmustbeassignedto
DecisionTableVariableNegation
StrategydesignedtorevealfaultsresultingfromDon'tCare implementation
BDDdeterminantstrategydoesnotexplicitlyaddressDon't Carevariables
DecisionTableVariableNegation
Example:Z=Aand(BorC)=(AandB)or(AandC)
1. Uniquetruepoints
(AandB)true,(AandC)false{A=T,B=T,C=F} (AandB)false,(AandC)true{A=T,B=F,C=T}
DecisionTableVariableNegation
Example:Z=Aand(BorC)=(AandB)or(AandC)
2. Nearfalsepoints
(AandB)
(AandC)
Causeeffectgraphing
Systematicwaytoaidselectingahighyieldset oftestcases
Nodes:
Causeeffectgraphinglayout
Causeeffectgraphingtypeoflinks
directcause ^ AND
OR
NOT
Causeeffectgraphingexample
Afileupdatedependsonthevalueintwofields.Thevalue
infield1mustbean``A''or``B''.Thevalueinfield2must be a digit. In this situation the file update is made. If the value in field 1 is incorrect, error message X12 is issued. If thevalueinfield2isincorrect,errormessageX13isissued.
Causeeffectgraphingexample
Causes
Effects
Causeeffectgraphingconstraints betweencauses
c1 Exclusivec1andc2cannotbetruesimultaneously E c2 c1 c2 c1 c2 c1 c2 OneandonlyOneoneandonlyoneofc1andc2canbe trueatagiventime Exclusiveoneofc1andc2mustbetrue(can'tbeboth false) Requirec2mustbetrueforc1tobetrue
Causeeffectgraphingconstraints betweeneffects
e1 e2 M Maske1forcedtofalseife2
TestgenerationfromCauseEffect graphs(1)
Dividethespecificationintoworkablepieces. Defineseparategraphsforeachpiece.
1. IdentifyCausesandEffects 2. DeduceLogicalRelationshipsandConstraints representasgraph 3. Drawadecisiontable 4. Converteachvariantinthedecisiontableintoatest case.
TestgenerationfromCauseEffect graphs(2)
Tofindtheuniquecombinationofconditions Workwithasingleeffect(output)atatime
1. Seteffecttotrue(1)state 2. Lookbackwardforallcombinationofinputswhich willforceeffecttothetruestate(constraintslimit numberofcombination) 3. Createacolumnindecisiontableforeach combination 4. Determinestatesofallothereffectsforeach combination
BooleanFormulaGenerationfrom CauseEffectgraphs
Workingfromeffectstowardcauses
1. Transcribenodetonodeformulasfromthegraph
writeformulaforeacheffectanditspredecessors Foreachintermediatenode
writeformulaforintermediatenodeanditspredecessors
2. DerivethecompleteBooleanformula
BooleanFormulaGenerationfrom CauseEffectgraphs
A B C D E
~
~
^ F
v H v I
^ G
I=(~AB)C(D~E )
TestCasesderivationfrom requirements
Involveclarificationandrestatementoftherequirements
Obtainapointformformulation
Foreachrequirement
e.g.buttryingsomethingforbidden
Testboundariesandconstraintswhenpossible
TestCasesderivationfrom requirements
Example:RequirementsforaVideoRentalsystem
1.Thesystemshallallowhiringandreturningoffilms
1.1. Ifafilmisavailableforhirethenitmaybelenttoacustomer.
1.1.1Afilmisavailableforhireuntilallcopieshavebeensimultaneously borrowed.
TestCasesderivationfrom requirements
Testsituationsforrequirement1.1
Testsituationsforrequirement1.1.1
TestCasesderivationfrom requirements
Testsituationsforrequirement1.2
Borrowanunavailablefilm. Returnafilmandborrowit
Testsituationsforrequirement1.3
Borrowafilm,returnitandcheckdates Checkdateonanonreturnedfilm
TestCasesderivationfrom requirements
Testsituationsforrequirement1.4
Testsituationsforrequirement1.5
TestCasesderivationfrom UseCases
Forallusecases:
1. DevelopaGraphofscenarios 2. Determineallpossiblescenarios 3. Analyzeandrankscenarios 4. Generatetestcasesfromscenariostomeetacoverage goal 5. Executetestcases
ScenarioGraph
Generatedfromausecase
Nodescorrespondtopointwheresystemwaitsforanevent
environmentevent,systemreaction
Mayincludeconditions
Specialloopingedge
Scenario:
pathfromstartingnodetoafinishnode
Title:Userlogin Actors:User Precondition:SystemisON 1:UserinsertsaCard 2:SystemasksforPersonalIdentificationNumber(PIN) 3:UsertypesPIN 4:SystemvalidatesUSERidentification 5:SystemdisplaysawelcomemessagetoUSER 6:SystemejectsCard Alternatives: 1a:Cardisnotregular 1a1:Systememitsalarm 1a2:SystemejectsCard 4a:Useridentificationisinvalid ANDnumberofattempts<4 4a.1GobacktoStep2 4b:Useridentificationisinvalid ANDnumberofattempts>=4 4b.1:Systememitsalarm 4b.2:SystemejectsCard Postcondition:Userisloggedin
ScenarioGraph
Scenarios
ScenarioRanking
shouldbetestedfirst
TestCasesgeneration
Accordingtocoveragegoal.E.g.
ExampleofTestCase
TestCase:TC1 Goal:TestthemaincourseofeventsfortheATMsystem. ScenarioReference:1 Setup:CreateaCard#2411withPIN#5555asvaliduser identification,SystemisON Courseoftestcase
1 2 External Event Reaction Comment User insert Card #2411 System asks for Personal Identification Number (PIN) User types PIN #5555 System validates USER identification System displays a welcome message to USER
Passcriteria:Userisloggedin