pce Vat} a) Software Testing Basics ce ‘as an engineering activity, Role of process in software quality, me ions, Software testing principles, The tester’s role in a software development fects, Defect classes, The defect repository and test desi support for developing a defect repository. —— Syllabus Topic : Testing as an Engineering Activity 1.1__Testing as an Engineering Activity — In today’s world, it becomes very challenging to build the quality software systems, thus human resources with ‘great software development skills are in demand. — This is an energizing time to be a software product development engineer, as software systems are playing an important role in society. In order to support software development and software maintenance task various genuine strategies, methods, techniques, and tools are available. Software systems has noticeable effect on economical and social life. This enables software experts to focus on software quality issues. Poor quality software are not acceptable in the society- Failures within software systems can result in disastrous misfortunes. — oftware development people shall be trained for software product and process quality. This will ensure that developed software products will be of good quality, timely completed, within allocated budget. = Te will also guarantees = The education and tral Testing as a process, Basic organization, Origins of ign, Defect examples, Developer / Tester the presence of quality atributes such as reliability oF dependability, Coren: or accuracy, usability within developed Product and = ultimately meet all ification requirements. user's engineering discipline is based on the felated scientific. principles, engineering processess tandards, methods, tools, measurement and best practices as shown in Fig. 1-1 jnootng 7 ‘Gomputor ow on engineering == { Workin progress [Testing software engineering] Fig. 1.1.1 : Elements of the engineering disciplines ‘Scanned with CamScannee— The software development process is defined as a series of phases, procedures, and steps that result in the production of a software product. — Thus, software development should be viewed and taught as an engineering discipline. = Along with this, professionals must co important entities in this field viz - process nsider two extremely i {quality and the product quality. Using an engineering approach (0 software development implies that: © Well understood development process © Planning of Projects ‘o Well defined life cycle models Product and Process standards are defined o. Metrics to evaluate product and process quality ‘0 _ Reuse of components ‘> Validation and verification processes (0 measure quality © Education, training, and cert “These things are used by test specialist who is focusing a discipline in software fication to engineers on software testing - engineering. Test specialist oF software Jean and understand software testing related practices, processes, measurements, standards, plans, tools, and methods. He should learn how activities. ‘Testing is an essential component of software quality process. Its considered to be a challenging, and costly ((o some extent) task during software development and ‘maintenance phases. In software testing area, two process models Test Process Improvement (TPI) model and Test Maturity Model (TMM) are used. ‘These two models are developed at the Illinois Institute. of Technology and helps organizations to assess test process, identify improvements in it and suggest action tester need to principles, to apply them in the testing plan for improvement. LY = For example, - Due t ad 1.2 _ Role of Process. be defined in terms of quality -The Software Quality can factors and quality criteria. sparacteristies of a system re correctness, reliability, efficiency, re-usability etc. Attributes of to software development presents quality Behavioral ct factors for example, testability, maintainability, quality factors related represents a quality criterion. fan attribute of the architecture of software system is known as modularity. ISO 9126, CMM (Capability Maturity Model) are the well known ‘Software quality models. 10 need for high quality software products, oftware professionals in this field are given with the responsibility to identify and quantify the quality factors such as usability, testability, maintainability, and reliability, and to identify engineering practices that support the production of quality products having these favourable attributes. Following are the few best technical and managerial practices identified during the development of bigh- quality software. © Planning of project, © Managing requirements, Developing formal specifications, Structured design by using information hiding and encapsulation, Reuse of design and code, ‘0 Review and Inspections, Metrics for Product and process, Education and training to software professionals, Development of CASE tools and their application, Making use of effective testing techniques, ° Integration of testing activities into the entire SDLC. ‘Scanned with CamScannee'STOA (SPPU- Sem. 7-11) Further, researchers fro _tescarcers from this Geld realized that it was, important to integrate them within the high-quality sofware development pees = Process, in the software enginering domain can be defined as the set of methods, practices, standards documents, activities, policies, and. procedures Software engineers ot wineers use these processes to develop and maintain a software system and its associated art if acts, like project plan, test plan, design documents, code, manuals etc. Fig. 1.2.1 illustrates the components of process. - It is not a good idea to add individual practices temporarily into an existing software development process. It must integrate the best technical and managerial practices in a systematic way. This process must be engineered ie. it must be designed, implemented, evaluated, and maintained. Further the process must be evolved in a consistent and predictable way. Fig. 1.2.1: Components of engineered process Many organization have used Capability Maturity Model (CMM) and SPICE models to evaluate its current software process. ‘What are the differences between testing and ‘debugging ? (Refer section 1.3) (4 Marks) @.1.3.2 What are the differences between verification a ‘and validation? How does your organization handle each of these activites ? (Reter section 1.3) (6 Marks) — Testing is one of the process from several other Software development processes. = Testing can be used to reveals defects from developed software which is under test = hig abo used to evaluate quality attributes of te software such as accuracy, dependability, ease-of USS- —pifferent processes in software development ST defined in Fig. 1.3.1 Fig. 13:1; Processes involved in Software development POSS _ Testing process is comprised with two ether prODesScS called as verification and validation. = = Itcovers various activities in the following: domain such as, technical reviews, test planning, test tracking, (eS ‘case design, unit test, integration test, system tes acceptance test, usability test and so on. — Verification answers the question-"Are we developit this product. by firmly following all desi ‘These high-level models focusing on the whole specifications?” (Are we building the product right?) software process as instead of supporting to evaluate and improve sub processes such as design and testing. is the process of evaluating products in the develop phase to determine whether they are meeting specified requirements. ‘Scanned with CamScannee- ~~ a FP sto, XY XQ ) A ee 14 Software Testing Basic, ~ Wehecks ; cone a abe of een development | ~ It is considered that complete testing is Practically y,, possible due to sme economical constraint (ime yg Phase satisfy “St the conditions imposed at the start of tha pos f that In the process of verification, Tquirement specification, design specification, code, the product is referred as, ‘user manual, or even the final product. such as, requirement It comprises various activiti reviews, design reviews, code reviews that check the correctness of a development phase. Verification is concerned with term quality assurance, Validation answers the question ~"Are we developing the product which attempts all that user needs from this software?" (Are we building the right product), Validation process is focusing on the final product. It evaluates software during the or at the end of the development process to check whether it specified business requirements or client's needs. It also includes comparing test results to expected results. It comprises various activities such as, Unit testing, Integration testing, System testing, User acceptance testing. Validation is concerned with term quality control. There is a difference between testing and debugging activities. ‘The debugging process starts once testing has been carried out. This will enable the tester to understand that the software is not behaving as per provided Specifications. Debugging activity is also known as fault localization process. This process helps in locating fault or defect, repairing of code, and retesting of code. Testing activity has limited resources and time to complete. Every organization must prepare testing plan 80 that software product can be tested for satisfying user's requirements and delivered within the specified 1.4 frame of time and budget. 4 4 other resources) of organization. ‘The testing plan must define and document all the jy procedures, policies. steps, tools, methods, metrics, ang test goals very clearly. Testing people must be identified and trained to y. things specified in the test plan in order to engyn release of defect free and reliable software, Al the set testing goals must be quantifiable so thay an be measured and monitored. Further testing proces should be exile enough to evolve and adapt necessay continuous improvements. Syllabus Topic : Basic Definitions Basic Definitions Explain the differences between errors, faults and, failures. (Refer section 1.4) (6 Marka) Define following terms: (8 Markey i) Test Oracie Test Bed fil) Defects jv) Software Quality (Refer section 1.4) In software testing, following few terms are refereed very often viz. error, fault, defect and failure. These terms are closely related to each other, but there are some key differences between these four concepts. ‘Many of these definitions used in the below section are adapted from IEEE Standards Collection for Software Engineering which includes the standard glossary of Software Engineering Terminology. In the following * section, we present these basic terms. ‘Scanned with CamScanneeFig. 1.4.1 : Software Engineering Terminologies > @ Error - ih ‘The human action can be typing mistake also. It may also be caused due to misunderstanding of software developer. ‘a human action that produces an incorrect result misconception or — For example, an incorrect instruction to addition ‘computer program which could give a result as 15 whereas it is expected to produce addition result as 10. > aD Fautt = It is nothing but manifestation or introduction of an cerror in software under development. = It is sometimes also referred as bugs. A fault in the system may remain hidden and undetected for a long time, until some event activates Upon activating a fault, it first brings the program into a state called as intermediate error state. If the system is allowing to proceed without any suitable corrective action, then it brings a program failure. = There are several types of faults that may occur in the = Some possible reasons of failure includes, hut — The failure state of a software system results in 10s software system viz. Logical, functional, GUI, security ‘etc, Professional may adopt fault preventing techniques such as peer review, code analysis, good programming style et > (itt Defects Defect can be defined as an imperfection or deficiency in a work product. Such a defective work product does ‘not meet its requirements or specifications. It will be either repaired or replaced. Some Common types of defects includes, arithmetic defects, logical defects, syntax defects, multithreading defects, interface defects, performance defects etc. > (CV) Faiture = It is a deviation of the software from its existing delivery or expected service. = The failed software does not able to perform a required function within specified limits. yman Exror by keying in wrong inputs, some user operations with intention of breaking the system, producing incorrect result etc. sof Joss of money, loss of business reputation time, ig. 1.4.2 represent injury(sometimes death as well). Fi behavior chain. Fig. 1.4.2 : Defect ‘behavior chain > (V) Test Cases = To find defects in a software product, tester is exect this software by applying certain input data under conditions. In order to decide that software has p or failed the test, tester must know the ex] outcome of that test. = Attest case is a document that consists a set of tes (test data), preconditions, expected results an conditions. It is developed for a particular test s to verify specific requirement. ‘Scanned with CamScanneeG1 stoa (sppu-sem 7. Test Case applied at the begi with a set of input values, that produces an ome! point (known as of test exe me and leaves the system at some end ‘execution post condition) = TEEE has given the teat case documentation wi minimum required contents. However. every organization decides form of "4 1 required information, 0 est case document with some additional that these developed test cases can be reused. Typical test case parameters can be: 1, Test Case ID 2. Test Scenario 3. Test Case Description 4. Test Steps 5S. Prerequisite Test Data 6. 7. Expected Result & Actual Result 9. Comments > (VD Test Suite Test suite acts as a container that consists a set of tests. A Test case can be added to multiple test plans and test suites. [At first a test plan is created followed by test suites, which in turn can have any number of tests. Fig. 1.4.3 represents typical structure of test suite containing set of test case. Fig. 14.3 : Test suit creation process Th) Test Oracle rhe term was first introduced by William E. Howe a In the field of software testing, OACTE OF Let oracle, othing but a mechanism for determining Whether, ced) or failed based on result of applied 4, st * has pa cases. rhe test orace is used to compare the oUtpat(s of ystem under test, for a gIVER TES-CaS6 input 1 yy utput(s) thatthe oracle determines that Product heyyy have. Fig. 14 represents typical structure of a Test orate, — A test designed can create different types of test oraces by varying the level of details of oracle information ang changing the oracle procedure. When a product is switched to new environment then its own oracle can be ported as it is, so that functional behavior would not change in new environment. > (VII) Test Bed In order to perform or execute designed test cases on a software product under test, the test-designer must specify the test execution environment. Test bed is a collection of hardware, software (system software, application software), operating system, network configuration, and the product under test. Test bed forms a combination of hardware and software environment to support to execution of the test. For example, a test bed for a web application is defined as follows: ‘Scanned with CamScannerfOA(SPPU Sem, 7-1 Opera Web Server~ Apache Tomcat Web server Database - MySQL System: Linux Browser — Chrome IDK version: version 9 4X) Software Quality Quality of a software product is given by measuring quality tributes possessed by that product, ‘The typical quality atributes are usability (ease-of-use), portability, accuracy (correctness), reliability, ‘maintainability, interoperability ele, Software quality defined in IEEE Standard Glossary of Software Engineering Terminology as ‘Quality relates tothe degre to which a system, sytem component, oF process meets specified requirement 2. Quality relates 1 the degre to which a system, system ‘component, or process meets customer or user needs, oF expectations. = The quality stributes are described in IEEE Standards for Software Quality Metrics Methodology. Some examples of quality attributes are as follows vil Interoperability Fig 45 > (Correctness ‘Quality Attributes ‘The degree to which the system performs its intended function > GH) Retiabity to which the software is expected to perform “The degree defined conditions for 3 fac required fonetions under defined prio of tie. > cy Ueabitty “The degree of effort needed to Fearn. Operate, Prepare np, and interpret output ofthe software. > 0) Integrity “Abity to remain unchanged against intentional and sccideta attacks. > (0) Portability [Ability to teanser a software from one environment 1° another. > (oh) Matntainability smount of efforts needed 10 make changes 19 The a software. > (i) Interoperability — The amount of efforts needed system to another. “The quality attributes are measured Ws which gives quantitative measure of attribute. ‘Two commonly used metrics are “The product metic called as size of a SOM measured in term of lines of codes (LOC) “The time and cost can be treated as commen Process Snould be considered as @ metres, Further, testability st Guat atsibue that shows the ability ofthe sofware 1° reveal defects under testing conditions ‘Testability represents the amount of effort required 10 test the software that ensures it performance as per the specified requirements (relates to number of test cases needed), It is more useful to developers/testers than to to tink or couple on sing quality metic presence of a product and process yware can be clients > 0 Software Quality Assurance Group Software quality assurance (SQA) responsible fo monitoring the software engineering. processes. an methods used to ensure quality. YS EBT stan sreu-som.7-m SQA group is a set of skilled and carries out a supporting process that provides assurance related to compliance of all the work products, activities land processes with the predefined plans. ied people which SQA group develops quality policies and quality assurance plans with the help of project managers and testers for every project. SQA activities are organized into goals, commitments, and_ verification, abilities, activities, measurements, Typical activities carried out by SQA group can be - ing, analysis, record keeping, data collection, au reporting, configuration management etc. SQA group may check conformance to one or more ‘standards, such as ISO 9000 or. model such as CMMI. (XD Reviews A review is a static analysis tool and a manual activity aimed at finding and removing defects early in the software development life cycle. Reviews are used to verify documents such as such as a requirements document, a test plan, a design document, ‘a code component and test cases. Review activity is carried out by managers, clients, developers, testers and other personnel depending on the type of document under review. ‘Audit known as special type of review is carried out by SQA group to check compliance with agreements, specifications, and standards. Use of Review Process ‘Sufficient time is spent during the initial phase which reduces testing time in later phases. Reduction in cost because of fewer defects in the final software. — Improved productivity of development team. Correction of defects in early stages ensures clear and unambiguous product. 138 Sofware Testing bane, Syllabus Topic : Software Testing Principieg ~ 1.5 __ Software Testing Principles = White conducting sofware testing itis very ue *Y imporan to achieve an optimum test results without dey ining from the goal. Ie is very essential to know that right testing strategy is being followed by the testing team. To ensure this, some basic testing principles are wide practiced in the software industry are explainey in following section. Software Testing Principles Ty Allieste should be Waceable to customer requirements (@) Tests should be planned long before testing begins. {@) The Pareto principle applies to software testing. {(@) Testing should begin ‘nthe smalP and progress toward testing “in the large.” (6) Exhaustive testing is not possible, (6) Independent third party testing, (7) Keep software product static during the test. (© Testing time and resources are limited so avoid redundant test. Fig. 1.5.1 : Software Testing Principles > (D) All tests should be traceable to customer requirements If the developed product is of no use, does not fulfill he user's requirements and expectations then finding and fixing defects in such product does not help. ‘As said, if the product does not meet user's requirements — explicitly mentioned and implicitly implied, icc. if it is not fit for use, there is no point in testing, finding defects and fixing it. ‘Scanned with CamScanneeEf stan (sPPU-som.7- 17) > (2) Tests should be cael Planned long before testing — Testing activities should be planned (as per th objectives) and started inthe eatly phase of SDLC tot helps to uncover the defects early " = I the testing team is invotved right from the beginning of the requirement gathering and analysis phase they have better understanding and insight into the product and moreover the cost of quality will be much less if the defects are found as early as possible rather than later in the development life cycle. (3) The Pareto principle applies to software testing, — Testing effort should be focused to the expected areas of defects. Defect density is more in small modules that are discovered during pre-release testing, = Such modules are responsible for most of the operational failures. The Pareto principle of, 80:20 works here, that is 80 percent of defects are due to 20 percent of code! This bbe very helpful while testi particular module/area there is pretty high chance of getting many more there itself. formation could prove to we find one defect in a > (4) Testing should begin ‘in the small” and progress toward testing “in the large” — In the beginning, tests must be planned to test independent and single component of a system. As testing progresses, it must target integrated components and at last entire system. (5) Exhaustive testing is not possible = It is not a feasible idea to test everything with all possible combinations of inputs and preconditions — Testing efforts must directed after analyzing risks and priorities. — For example, if we are testing a text box that accepts numbers between 0 to 100, we would test for boundary values, one less than boundary value, one more than poundary values, few random numbers, middle number, that’s it and assume that if it is working fine for these numbers it will work for other numbers also. = We are not testing for each number from | to 100. Software Testing Basics > (© Independent third party testing = Testing should be carried out by @ group that is independent of the development group. not ~The software engineer who developed the system 1 good choice to test that system. = An independent tester who is all together new to the system is best suited. He must learn the system and attempt to break (1D) Keep software product static during the test = Functions or modules must not be modified during the implementation of test cases. (8) Testing time and resources are limited so avold redundant tests = Repeating same kinds of tests again and again will not be able to find any new bugs- “Pesticide Paradox” approach can be wed and revised on = To overcome thi used. The test cases must be revie' regular basis. = moder to uncover new bugs, different tests need 0 Pe ferent areas of the software. written that will focus om Syllabus Topic : The Tester’s Role in a Sofware Development Organization The Tester’s Role in a Software 1.6 Development Organization @.1.6.1 Given the many challenges facing @ 'e what types of skills do you believe should be required of a person being hired as a test specialist? (Refer section 1.6) (4 Marks) — Main goal of a tester is to work with the team of developers with an objective of producing high-quality software that meets the customers’ requirements. = Whenever software product does not work well then the tester’s job is to reveal defects, find weak points, inconsistent behaviour, and understand circumstances. = Tester shall possess good working relationships, communication and team-working skills that are essential things for a successful career. ‘Scanned with CamScanner1 STQA(SPPU - Sem.7 Testers performs functions like - plan, execute, record, and analyze tests They do not debug the software. The software with defects that are detected during testing, should be Feturmed to the developers. ~The developers performs the debugging task. They have 4 detailed understanding of the code so that they can Jocate the defect and repair the code. Its very difficult for the developers to test their own code because they may think that nothing could Possibly be wrong with it, = In order to become an effect to have extensive programming experience which will help to understand how the code is constructed?, where will be the possible presence of defects?, and what kind of defects are likely to occur etc. requires them tester, — The team of developer and tester is formed as per the availability of human resources, type of project, and — Consider for example, lower developeritester ratio will bbe required in an embedded real-time system (2/1), ‘whereas it will be 4/1 for a simple database application. In case of higher TMM levels, there is a well-defined testing group available. = Typical work set for a tester in an organization — Discuss with top management to have input for the development and maintenance of organizational testing standards, policies, and goals etc. — Design and develop test plans with the help of project managers. — Discuss with requirements engineers to ensure that provided requirements are testable or not. — Work with code developers to test the code ~ Planning for system test and acceptance test along with clients. Planning for integration test and unit test along with system designers ‘Software Test Lee ters may be a part gp In an organization, the development group (with special assignment i a, or they may be part of the SQA group. ", Tester should aways MVE magi indepen from developers. Testers should be well supported by managemen, yay respect to their efforts and contributions, They ys certain reasons listed incoming section ayy. developers, analysts, marketing staff, and maiteng, staf need to realize that testers add value 0a soya product. Testers are detecting defects and evaluating quay very cary stages of SDLC. Due to this, developers y. able to release product with less or m0 defecy Marketing staff can deliver such a well tested, acura, reliable and usable software that will satisfy ce, requirements The low defect software will reduce cost with respect y maintenance activities such as software repair, suppor calls etc. Syllabus Topic : Origins of Defects 1.7___ Origins of Defects ‘What are the typical origins of detects? From your own personal experiences what are the major sources of defects in the software artifacts that you have developed? (Refer section 1.7) (6 Marks) Software testers are working very hard and taking all required care to test the software fully, which results in high quality, defect free software. Relationship between errors, faults, defects and failures is discussed in ear section — Even after following best practices of softwar development, defects are being introduced in softwar Defects as shown in Fig. 1.7.1 originates from the following sources that are explained below. ‘Scanned with CamScanneeDefect sources 1. Lack of education 2. Poor communica 3. Oversight "on 4. Traneoniption 5. Immature process ‘Impact on software eritacte 1. Errore 2 Fauils/Dotect 3. Failures © (1) Education = The software engineer did not know how to prepare the software artifact due to improper educational background. For example, if a software engineer with lack of understanding of the operator precedence in a particular programming language could inject a defect, in an equation that calculates certain valve. (2) Communication = The software engineer was not informed about something by a colleague. For example, developer A and developer B are involved in an interfacing module. = Developer A had not informed developer B about the absence of error check sub-module in its own module. — Here developer B might make incorrect assumption considering presence or absence of error check sub- module. This could result in injection of defect. (3) Oversight — The software engineer omitted to do something. For example, omit to write initialization statement. (4) Transcription — The software engineer knows what to do, but makes a mistake in doing it. For example, simple mistake in writing (spelling) variable name in code. = The process used by the software engineer misdirected his or her actions. For example, if there is no sufficient time given for the development and review of specification document then it could lead to specification defects. ~ Main goal of a tester is to detect defects before using it ‘One approach to detect these defects is to design test ceases that will reveal defects. = Testing can be carried out as an experiment. Result of experiment is analyzed to check behavior of software. = This behavior could be correct or incorrect. Tester then develops a hypotheses about possible defects, followed by designing test cases based on hypotheses. = At last, tests are executed and result is analyzed t© prove or disprove of hypotheses. = One of the researchers, Myers has a similar approach testing. He describes the successful test as one that reveals the presence of a (hypothesized) defect. — Tester’s role is somewhat similar to Doctor checks patient's symptoms, used her knowledge of possible diseases, performs diagnosis and develops * hypotheses about possible illness and starts the treatment accordingly. — Thus, tester can imagine defective software as the ill patient. The tester as a doctor need to have knowledge about possible defects (illnesses) in order to develop defect hypotheses. = They use the hypotheses to design test cases and tes procedures, assemble test sets, select the appropriat testing levels (unit, integration, etc.), and evaluate th results of the tests. that of doctor. 1.8 Fault Model 1.8 Fault Model — The hypotheses is proved to be true after performin; successful testing experiment that means hypothesi: defect was present. ‘Scanned with CamScanner‘After that, the very useful concept related © ror mace anid (He diagnosis. described as @ faulUdefect in the sofware — The type of error can be, #0 misunderstood design clement: ® typographical too called as fault dictionary 'S gene tink between the ¢F jssing requirements error. A rated ult list, al ‘with the help of fault models “The faults can be selected f inputs are developed fora test ts revealed by the test dete and test rom dictionary. ‘The number of fal mines the effectiveness of a test. = Testers cat memory from years of experi and further diagnosis. 4 related to an incorrect value for & = For example, a mode! corte was observed because te precedence order [OF the arithmetic operators used to calculate its valve WAS incorrect. - By changing the order parentheses this fault can be fault hypotheses and test cases, tes fault model and the information about occurrence of this type of fault. ‘This would definitely ensure that adequate tests were performed to reveal such faults. of the operators or proper use of repaired. To generate a er may access this frequency of oe clasts Defect Repositor, ~ petect classes Pository i us TOP t Desig” testing @ code component ‘a defect: it calculates an ‘and you discover variable incorrectly. output (a) How would you classi © ly causes of this ity this defect? what are the like detect? what steps could have been taken to prevent this type of defect from propagating to the code? (Refer section 1.9) you mean by coding defects? (4 Marks) © (6 Marks) What do (Refer section 1.9) 7, Ik is very important to have storage and retrieval of defect data from all projects in a centrally accessible location. This is referred as defect repositories. Such ies may increase the effectiveness of their repositori testing and debugging processes. 'A defect classification scheme is used for developing the repository. Fig. 19.1: ig. 1.9.1 : Defect classes and the defect repository ‘Scanned with CamScannee[BF stan SePU-som.7-1 Ge o ‘The defect repository can be organized with respect to projects. Possible information stored can be - class of defects, frequency of their occurrence, impact on operation, and comments, me IL is suggested to capture defects from review activity and execution-based testing " Defects are cl : ified into four major classes based on their origin point in the SDLC. These classes ure: requirements/ specifications, design, code, and testing and are represented in Fig. 1.9.1 ‘These defect classes and associated sub-classes focus ‘mainly on execution-based testing. The defects related to conformance to styles and standards found in software reviews are not taken into consideration. Tis suggested to use any one classi apply it to entire project. sation scheme and However, some defects falls into more than one type- So it’s very important SQA group, developers and testers should record defect data in a consistent manner. ‘Test design and planning activities are based on defect, types and frequency of occurrence. Further, tests are carried out which has high probability of revealing defects. Following section is describing four major defect classes. Requirements and Specification Detects ‘Since many requirement and specification documents are written using a natural language representation, they may become ambiguous, contradictory, unclear, redundant, and imprecise. Defects injected in early phases of SDLC can be very difficult to remove in later phases. Following are some requirements/specification defects are: Functional Description Defects ‘This type of defects oocur when the description of product ic, what the product does, and how it should behave, its inputs and outputs, is incorrect, ambiguous, and/or incomplete. 1 software component or system such as performance and reliability. = This type of defects occur due to fi incomplete, or unnecessary description about the system features. Jing, incorrect, (Ul) Feature Interaction Defects = This type of defects occur due to an incorrect description of how the features should interact with each other. For example, a database having two important features which are, adding new customer and classification of customers. Both these features are interacting with each other. SO during testing both must be focused. (i) Interface Description Defects = This type of defects occur due to flaws in the description of how the target software is to interface with external software, hardware, and Users. 2. Design Defects = Design defects occur when the following are incorrectly designed, = System components, — Interactions between system components, — Interactions between the components and outside softwarefhardware, oF Users — Design defects are found in the algorithm design, data elements, control, logic, module interface descriptions, and external software/hardware/user interface descriptions. The six types of design defects are as follows : @ Algorithmic and Processing Defects = These occur when the processing steps in the algorithm as described by the pseudo code are incorrect. It may be caused due to missed steps, duplicated steps. ‘Scanned with CamScanneecommands, improper commands, Wrong STA (SPPU - Som. 7), eT anissing miss ea sequence of commands, lack of proper prompy feedback messages for the user etc, Example: not checking divide by er To get the average of two values ‘a’ and ‘b’, formult provided is a+b/2. Actually it should be (a+b)/2. (ii) Control, Logic, and Sequence Defects = Control defects possibly oceurs when logic flow in the pseudo code is incorrect. Control defects can be of tyPe+ branching to-late, use of an incorrect pseudo code procedure or branching to-soon, branching condition, elements, improper nesting, improper unreachable function calls = Logie defects occurred due to incorrect use of logic operators, for example, use of less than operator oF ‘greater than operator in Boolean expression controlling a branching instruction. (ii) Data Defects = This type of defects are associated with incorrect design of data structures. For example, improper memory allocation, undefined field in a record, incorrect data type used, incorrect size of data element. (iv) Module Interface Description Defects ‘This type of defects occur because of incorrect number of parameters or incorrect ordering of parameters, incorrect or inconsistent usage of parameter types. (v) Functional Description Defects The defects in this category include incorrect, missing, or unclear design elements. It could be also due to improper description of functionality of a module. ‘These defects are best detected during a design review For example: When the submit button will get activated in a sign-up form? Information about this must be clearly written. (vi) External Interface Description Defects Examples are, (@ defects are derived from incorrect design descriptions for interfaces with external software systems, hardware devices and databases. messages, lack of Coding Defects Coding defects are found through the errors in code implementation. Iris somewhat similar to design defect classes. Coding defects occur due t0 ing language constructs, miscommunication failure to understang programmi ‘with the designers, and omitting useful information, ithmic and Processing Defects @ Algori 1m and processing defects include Code related algorith Incorrect use of signs Comparing inappropriate data types Converting one data type to another Unchecked overflow and underflow conditions Incorrect ordering of arithmetic operators — Misuse of parentheses = omission of parentheses, ~ Precision loss, Control, Logic and Sequence Defects This type of defects include incorrect expression of case statements, incorrect iteration of loops, loop boundary @w problems, and missing paths. (ii) Typographical Defects ‘These are mainly syntax errors, for example, incorrectly spelled variable name detected by a compiler, self- reviews, or peer reviews. (iv) Initialization Defects — This type of defects occur when initialization statements are omitted or sofnetimes written incorrectly. ‘Common causes are - misunderstandings or lack of between programmers, of ‘communication and designer's, carelessness, oF programmer's misunderstanding of the programming environment. ‘Scanned with CamScanneeFT ston ser (v), Data-Flow Defects Som.7, Data-Flow defects occur when the code does not foll the necessary data-flow conditions, For example, consider a simple operational seq al sequences of data flow as, initialize a variable before itis used it calculation or a condition, " intermediate us should not be te itialized before there is an Before use, a variable should not be ignored. This suspicious use of a variable may or may not induce defect in code, but if it present needs to be addressed, (vi) Data Defects, These types of defects are indicated by incorrect implementation of data structures. For example, omitting a field in a record, an incorrect file type, incorrect access permission to a file, incorrect size allocation to an array, incorrect flags setting, wrong indices, and incorrectly set constants values ete. (vii) Module Interface Defects — This type of defects generally occurs due to use of incorrect parameter types, of parameters, or improper ordering of the parameters. It also occur due to incorrect sequence of calls, and calls incorrect umber to nonexistent modules. (viii) Code Documentation Defects It occurs due to incomplete, unclear, ambiguous, incorrect code documents. In such case, the code documentation does not describe what the program actually does. ‘This results in designing improper tests which are not appropriate with code, thus affects testing efforts. Code review may become remedial action for such defects. (op External Hardware, Software Interfaces Defects — These defects occur due to problems in system calls, input and output sequence, links to databases, interrupts 15 ‘and exception handling, memory usage, resource USABe: protocols, formats, data exchanges with hardware, timing sequences, interfaces with build files, ete. Testing Defects = Test plans, test cases, test harnesses, and test procedures can also contain defects = These defects are called testing defects. = Defects in test plans are best detected using review techniques. () Test Harness Detects ‘An auxiliary code or assistant code must be developed and integration levels. hamess or scaffolding designed, to test the software at the unit ‘This code is referred as the test code. The code should be carefully implemented, and tested. it is a work product code and this code can Be the software are However, reused when new releases of th developed. (i) Test Case Design and Test Procedure Defects _ hese consists of incorrect, incomplete, missing, inappropriate test cases, and test procedures. ttean be detected using test plan reviews or with careful analysis of test conditions and test results. i Syllabus Topic : Defect Examples 1.10 _Defect Examples ___—_— Consider a module of an interactive cash register system. This module calculates the total value of a set of coins. A sample informal specification is provided. = This program calculates the total rupees value for a set of coins. The user inputs the amount of 25p, SOp and Irs coins. There are size different denominations of ‘coins. The program outputs the total rupees and paise value of the coins to the user. Input number_of_coins is an integer. ‘Scanned with CamScanneeaaa EF ston (sPpu-sem.7-1D, Output ‘number_of rupees is an integer, number_of aise is a integer = Possible defect requirements/specification description defects, interface description defects. type nal are of funet shown defects, = The functional description defects are due to the incomplete and ambiguous functional description. It is not stating that the input and the output should be zero of greater and cannot accept negative values for number of coins, rupees and paise. = Because of these ambiguities and specification incompleteness, a checking routine may be omitted from the design. — A more formally stated set of preconditions and post- conditions is needed with the specification, AA precondition is a condition that must be true in order for a software component to operate properly. Here precondition is : number_of_coins >= 0 — A post-condition is a condition that must be true when a software component completes its operation properly. Here post-conditions are: number_of_rupees, snumber_of_paise >=0. - The functional description is unclear about the maximum number of coins of each denomination allowed, and the maximum number of rupees and paise allowed as output values. — It is not clear from the specification how the user interacts with the program to provide input, and how the output is to be reported. — The mentioned specification is transformed into a design description. — There are many design defects, due to the ambiguous and incomplete nature of the specification; others are newly introduced defects. 7 Design Description for the Coin Problem values teger total_coin_value is integer Program calculate_coit ‘number_of_coins is Software Testing Bay, number_of_paise is gers representing ccoin_values is array of ix i initialized to: 25,25,100 while loop_counter is less than six begin output “enter number of coins” read (number_of_coins ) otal_coin_value + total_coin_valu rnumber_of coins * coin_valuefloop_counter] increment loop_counter end rnumber_rupees = total_coin_value/100 ‘number_of_paise = total_coin_value — 100 * number_of_rupees ‘output (number of rupees, number_of_paise) end (1) Design Defects in the Coin Problem Design Defects In the Coin Problem Fig. 1.10.1; Design Defects in the Coin Problem > (Control, logic, and sequencing defects Incorrect “while” loop condition (should be less than ot equal to six). > (i) Algorithmic, and processing defects Lack of a path where users can correct erroneous inputs, lack of error checks for incorrect and/or invalid inputs, lack of a path for recovery from input errors. ‘Scanned with CamScannee[BF sta (SPPU- sem. 7-11) nn lg sae rast 2a > (ili) Data defects Software Testing Basics, ‘An incorrect value for one of the elements integer array, coin_values, which should be a te 25, 50, > (iv) Extern: interface description defects These are defects arising from the absence of input messages or prompts that introduce the program to the user and request inputs. (2) Coding Defects in the Coin Problem Coding Defects in the Coin Problem ( Control, logic, and sequence defects (i) Algorithmic and processing defects (ii) Data Flow defects (viata Defects San eee ig Fig. 1.10.2 : Coding Defects in the Coin Problems > (@ Control, logic, and sequence defects Found in loop variable increment step which is out of the scope of the loop. An incorrect loop condition (i<é6) is carried over from design and should be counted as @ design defect. (i) Algorithmic and processing defects “The division operator may create problems if negative values are divided, although this problem could be ‘eliminated with an input check. (ii) Data Flow defects ‘The variable total_coin_value is not i used before it is defined. > (iv) Data Defects ccoin_values is carried nted as a design “The error in initializing the array over from design and should be cou defect. ‘> (©) External Hardware, Software Interface Defects ‘The call to the external function “scant” is incorrect. ‘The address of the variable must be provided. ‘> (vi) Code Documentation Defects The documentation of code is incomplete and ambiguous, It shows the deficiencies in the external interface description and other defects that occurred during specification and design. ‘The poor quality of this small program is due to defects injected during several phases of the life cycle because of different reasons such as lack of education, @ poor process, and oversight by the designers and developers. Wt Developer / Tester Support for Developing a Defect Repository for defect (6 Marks) nats v Explain the role developeritester repository ? (Fiefer section 1.11) In the earlier sections we have seen some of the most common types of defects that occur during software development. It will be beneficial for any organization to develop a defect repository to store ‘defect information. Sofware engineers and test specialists must realize the usefulness of defect data. = Organization must plan the repository development during the preparation of testing and/or debugging policy statements. The development begin with a defect classification scheme and then initiate the collection of defect data from projects. Different forms and templates like test incident reports, defect fix reports etc. will need to be designed to collect the data. Testing team must record each defect and its frequency of occurrence after testing. Defect monitoring should be employed for each on- going project since the collected defect data is useful for following tasks- ‘Scanned with CamScanner/ EB sro srr Sotware Tostng 'STOA ‘Sem. toring of test (© to plan the test (referred as TMM level 2 maturity ‘9 Controlling and monitoring ‘0 _ Software quality evaluation and control Test measurement “Test process improvement goal, to select suitable testing techniques, © to design of testcase, to reuse of previously designed test cases, tw allocate the resources for detecting and removing, these defects, and (© toestimate testing schedules and costs. ~The defect data can support debugging activities also. A. defect repository can help in implementing several | Fig. 1.11.1 : The defect repository, and support for Ty maturity goals ag TMM maturity goals shown in Fig. 1.11.1 that includes; ‘Scanned with CamScannerTesting Techniques and evels of Testing syllabus Topics Using White Box Approach to Te and Control Flow Graphs, eer 4 - Stake Teng Vs, Sutra Terg, Code Functonal Testing Coreg vig, Decision tables, Sate-based testi aoe 188 t0 Test Case Design, Random Testing, Requirements based ‘Testing -Unit Testing, Integration Test ing, Cause-ettect graphing, Error guessing, Compatibility testing, Levels of ing, Defect Bash Elimination, System Testing - Usability and Accessibilty Testing, Configuration Testing, Compatibility Testing 21 TestCase Design Strategies 2 3. 4 5. 6 To improve test case design ? (Refer section 2.1) (@ Marks) @.2:1.2 Describe the difference between the white box and black box testing strategies. What is a test case? What is testcase (Refer section 2.1) (4 Marks) ‘Testing team main aim is to design test cases for testing to improve quality of software product. “Through the effective test design can achieve improved quality testing process and bug free software products. Effective test case design achieves: Increases defect detection probability. Efficiently use available organization recourse Increases reusability of test and test case, Deliver the project in ti ‘Avoid delaying the project means save the cost Deliver the higher quality software product design two strategies are used : Black Box also called functional or specification wite Box also called clear or glass Box ‘The black box approach only considers software behaviour and functionality. ‘The software can be a simple module, member function, or object cluster to a subsystem or a complete software system, In this approach the tester only has knowledge of what it does. It is also called as functional, or specification-based testing, This approach is especialy useful fr revealing requirements and specification defects. -The white box approach focuses on the inner strveture of the software to be tested. “The software can be a module or member function. The tester most have a knowledge of the strocture of software. Its also knows an glass box or open Box OF clear box testing. White box testing methods are especially useful for revealing design and ccode-based ‘control, logic and sequence defects, initialization defects, and data flow defects. White box testing is classified into “static” and setryctural” testing. The tester has to use both the strategies to design test cases smartly to provide low- defect, high-quality software. ‘Using any one approach will not guarantee the success in revealing all types of software defects. Either strategy is useful in revealing certain types of defects. Both strategies allows the tester to select the finite number of test cases that will be applied during test. ‘Scanned with CamScannerSTQA (SPPU - Som. ‘This will increases the chances of revealing the many different type of defects in the software-under-test = Iwill also provide large set of effective and reusable test cases for regression testing, and for testing new releases of the software. ‘Syllabus Topic : Using White Box Approach to Test Design Using White Box Approach to Test Design Q. 2.1.3 Write a short note on Test case design using white box, (Refer section 2.1.1) __ (6 Marks) 244 During the white box testing, tester is given with the responsibility of determining whether or not all the logical and data elements in the software unit are functioning properly. ‘The white-box test cases must exercise or cover the logic (source code) of the program. Tester gets the knowledge about test case design from detailed design phase of development. This is the reason so that white box test design follows black box test design. White box-based test design is ost usefal for testing small components. — Thus it requires very high level of detail. The size of the items required to develop the test data is very small. Following section will discuss on different types of approaches under white box test case design. White Box Test Case Design 2.0500. {1] Static Testing [2] Structural Testing (1) Static testing by] (2.1) Code Functional ‘Humans Testing (1.2) Static Analysis Tools | (2.2) Code Coverage Testing (23) Code Complexity Testing 2.1.1(A) Test Adequacy Criteria — It is also called as stopping rule. It is used to determine ‘whether or not sufficient testing has been carried out. Generally testing is focused on structural elements such ing Tochniques and Loves of Tes, 22 as statements and branches. Test adequacy rg. provides a framework to the testers which is used . For deciding which structural elements t0 select ayy, focus of testing. For choosing the appropriate test data, For setting testing objectives For deciding adequate testing is done, — For deciding when to terminate testing ‘Types of test adequacy criteria are - 1. Program based (focuses on program structure) ation based (focuses on program specification) 2. Speci 3. Random = Program-based adequacy criteria are commonly applied in white box testing. It focuses on the structural properties of a program, f uses logic and control structures, data flow, program Statements, etc. Test adequacy criteria is written as a statement. Some examples are as follows. — A test data set (statement or branch) is adequate, if a test set T for program P causes all the statements, or branches, to be executed respectively. = A test data set is adequate, if it executes a program segment from its entry to exit — Attest data set is adequate, if it executes a path segment from variable definition to its use. Test adequacy criteria is also referred as coverage criteria or coverage analysis. It gives information of which code properties are exercised by the test cases. — Generally it covers statements, paths, condition and function from a program, The coverage analysis is ‘measured in percentage, known as degree of coverage. 100% coverage is not possible because of following reasons- — Some statements/branches may not be reachable. — Timing, scheduling, and marketing constraints ~ Not enough trained testers = Lack of tools ‘Scanned with CamScannee2 = Syllabus Topic : static 7, sting 2..1(B) Static Testing tools. It checks whether, © The code works according to the purity functional ‘The code has been written in accordance with the design developed earlier in the project life cycle; ‘The code for any functionality has been missed out; © The code handles errors properly. Following section describes two ways of conducting static testing ‘Ways of performing static testing Gaeeay eee me 215: Wan EE es ‘> (A) Static testing by humans It works on the principle that humans reading the program code and detect errors instead of executing code on computers to find errors. ‘This method has certain advantages as follows - 1. Sometimes humans can find errors that computers cannot. 2. By making multiple humans read and evaluate the program, more problems can identified through multiple perspectives. 3. A human evaluation of the code can be compared against the specification docitment or design document to ensure that it is performing well. 4. Many problems can be detected in one attempt. It can find the root cause of problem as well. Thus, the overall me required to fix all the -problems can be reduced. a1 Computer resources can be saved through human read and test 1t helps to reduce delay in problem identification which ultimately reduces pressure from programmers in later Phases. ‘There are three activities to achieve static testing by humans which are as follows. ‘Activities to achieve etatie testing by humans 1. Desk checking ofthe code Re ea ‘These activities are not purely testing activities, instead they can be called as process-oriented or defect prevention-oriented or quality assurance-oriented activities. These activities can be found in well known ‘models like ISO 9000 and CMMI. Programs must be classified into three categories as - high, medium and low based on their complexity. ‘This can be done during the planning stages. The complexity measure will decide which method to apply. High or medium complex code should be subject 10 formal inspections, while "low" complex codes can be applied with either walkthroughs or desk checking. = Desk checking, walkthrough, review and inspection are not only used for code but can be used for all other deliverables in the project life cycle such as documents, binaries, and media, Desk Checking of the code = Itis performed by the programmer or author of a code. It will verify correctness of code by comparing ‘its result with specifications. It is done before actual compilation and execution 0 code. Instant correction is applied whenever error i spotted. This method is informal so no process i available to check its effectiveness. Programmers a ‘not maintaining any logs of spotted errors. ‘Scanned with CamScanner~ le detects and corrects the defects with minimum time ‘delay because programmer knows his code very well ~ This is done by one individual, there are fewer scheduling and logistics overheads. ‘7 Disadvantages ~ A developer is not the best person to detect problems in his or her own code, ~ Developers generally prefer to write new code rather than do any form of testing! — This method is person-dependent and informal thus ‘ay not work consistently for all developers. % 2 Code Walkthrough — This method is said to be better than the desk checking. Itis performed by the group of people (team members). — Author explains logic behind the formed code and then questions are raised by the remaining team members. Author takes those questions which are not answered and find their answers. This method is less formal ‘compared to code inspection. 7 Advantages — _ Itprovides multiple perspectives to the author. — _ lower debugging (error-correction) cost. ] Disadvantages - This method achieves completeness only in the area where questions are raised by the team. > 3. Code inspection Q.2.14 Whatis the importance of code inspection? (Refer section 2.1.1(B)) (3 Marks) Iti highly formal method. Objectives of this method is to detect all faults, violations, and other side-effects. It is also referred as Code inspection or Fagan Inspection. The stages followed in this method are: © Preparation before an inspection/review; a yr © Enlisting multiple diverse views; roles specific participants; and Going sequentially through the code im structured manner. ° a After performing several desk checking walkthroughs of code, author is suggested tg nis formal inspection. It is done when author feels that code is com ; ready. Code is presented in inspection meting 1 consists of four major roles- ‘Author - programmer Moderator - runs the inspection process Inspector - code reviewer (provides comments) pee a ide detail Scribe - notes taker (provide detailed nos 4, inspection team) Initially author or moderator selects inspection/review team based on their skills to uncoy, as many as possible defects. They provide all necessary documents to review tean such as requirement document, design document, coding standards document etc. Author also provides information about code portion which needs to be reviewed rigorously. Moderator informs about meeting date, time and venue. Inspection team studies all these documents during this timeline, The inspection meeting is referred as defect logging meeting. The moderator takes the team sequentially through the program code, asking each inspector for any defects in that part of the code. Defect is taken into consideration only when entire inspection team agrees to it. Further, that defect is classified it in two dimensions - minot/major and systemic/misexecution, where minot defects - does not affect the code; major defects - needs attention and corrective measures; systemic - related 0 development environment; and misexecution - happened due to error or from author. A formal document about defects identified during meeting is prepared by a scribe. ‘Scanned with CamScannee8 care of fixing these defects, In case of severe defects, a review meeting is called to inspect the fixes In any case, defects found through inspection need to be tracked till completion and someone in the team has to verify that the problems have been fixed properly © Disadvantages - These are time consuming since time required by preparation and formal meetings. ~The logistics and scheduling issues due to multiple people are involved, = Sometimes it's un-necessary to review entire code through formal inspection. > @) Static Analysis Tools — This method has reduced the manval work, which was ‘observed in reviews and inspections. Static analysis is a type of code review tools that can be implemented without actually executing, or running, the software. Static analysis tools look at applications in a non- runtime environment. = tan be viewed similar to (or extension to) compilers ‘There are some static analysis tool which checks coding standards for allowed data types, naming conventions, programming constructs etc. Syllabus Topic : Structural Testing, Code Functional Testing 2.1.1(C) Structural Testing, Code Functional Testing — In this approach, the tests are derived from the knowledge of the software's structure or internal implementation. These tests are actually run by the computer on the software. In this approach, code under testis exercised ‘as much as possible by running it against some pre designed test cases. = A given portion of the code is exercised if a test case causes the program to execute that portion of the code when running the test. Structural testing can be lassified into code functional testing and code ‘coverage testing, and code complexity testing. Following section gives details about all these types. (A) Code Functional Testing Fig. 2.1.3 : Types of structural testing > (A) Code Functional Testing —_Inthis typé of testing, developer is applying some quick check on the code. Then this code may undergo Coverage testing or complexity testing. — This method is focusing on debugging activities instead of testing. But all activities required knowledge of internal structure of program. Some of the examples of such activities are as follows - 1. Perform some obvious test with known input values and expected output values. This can be repeated forall inputs as well. It helps in finding some obvious mistakes and can be recovered easily. It will surely boost the confidence of developer. Due to such practice, much of review meeting time can be saved in discussing common/obvious error. 2. A debug method can be used for a program with complex logic and conditions. Here a debug version of program is created by writing print statements at obvious places in order to check whether it is passing through the loops and iteration for correct number of times or not. Once defect is detected and fixed, such print statements must be removed. 3. Developer can make use of IDE or debuggers to check program step by step. Due to this each instruction and variable contents can be observed easily. Developer can set breakpoints also at certain functions to check its working. > (B) Code Coverage Testing 2.2.1.5 Explain merits and demerits of function coverage. (Refer section 2.1.1(c)) (4 Marks) ‘Scanned with CamScanneraL STQA (SPPU - Sem. = Code Coverage testing is used to determine how much code is being tested. It involves designing and executing test cases and finding out the percentage of code that is covered by testing. A technique, called as instrumentation of code gives t - The percentage of code covered by a Instrumentation is achieved with the help of available specialized tools. Simple formula can be - test Code Coverage = ((No. of lines of code exercised) / (Total No. of lines of code) ) x 100% — A program with high code coverage means most of its source code executed during testing, which suggests it has a lower chance of containing undetected software bugs compared to a program with low code coverage. = To measure what percentage of code has been exercised by a test suite, one or more coverage criteria are used. Coverage criteria are usually defined as rules or requirements, which a test suite needs to satisfy. Code coverage testing is made up of the following types of coverage. 1. Statement coverage — Has cach statement in the program been executed? Path coverage — Has every possible route through a given part of the code been executed? 3. Condition coverage - Has each Boolean sub-expression evaluated both to true and false? 4. Function coverage —Has each function (or subroutine) in the program been called? ‘Types of Code coverage testing Fig. 2.1.4: Types of code coverage testing Statement coverage In statement converge testing structured programs are used. The structured of program consists of sequential assignment statements, decision conditions and control loops and this is called program primes. For statement coverage testing write a test cases to teg each program statement, Tester should prepare set op test cases to test all program statements at least oncg ‘when program code executed. It can be calculated as - Statement Coverage = (No. of executed statement) / (To) no, of Statements) x 100 In case of sequential statements, test cases can ty designed to run from top to bottom. A test case stats a the top, eventually exercises all statements and reachey to bottom of the section. However, this may not always be true because of exceptions like a divide by zero, ‘Thus, the test case may not cover all the statements in that section which means even in the case of sequentiay statements, coverage for all statements may not be achieved. = Incase of if-then-else statements, (at least) one testcase to test the Then part and (at least) one test case to test the else part. — Incase of switch case statements, to cover all possible cases, there would be multiple test cases. In case of looping statements, executing a set of statements repeatedly until certain conditions are satisfied. Major defects in programs found in the loops when they do not function properly. — Termination or boundary condition is common place of having defects. Loops can be covered using three steps such as - © Skip the loop completely - so the termination condition will be covered. © Exercise the loop - so the normal working of loop can be covered. © Cover the loop around the boundary - so conditions like, just below n, n, and just above 0 can be covered. Example of Statement Coverage testing : Printsum (Int a, Int b) { /fPrintsum i Int result = a+ b; If (result> 0) funetion ‘Scanned with CamScanneeTest case 1:10 A Number of executed. statements statements = 7 Suntement Coverage = 5/7 = 719% Test case 2: 1A = -3,B = 9 Number of executed statements = 6, Total number of statements Statement Coverage: 6/7 = 85% Consider another example (flow graph is represented in Fig. 2.1.5): Read X Read Y IFX+Y > 100 THEN Print “Large” ENDIF 1X > 50 THEN Print “X Large” ENDIF x50 enait —O- Flow graph for Statement coverage example Fig. 2. Test case | 0 and ¥=50, By traversing path (A1-B2-C4-5-D6-E8), all the nodes ‘are covered. So by traveling through only one path all the nodes (A, B, C, D and E) are covered Number of executed statements = 6, Total number of Statements = 6 Statement Coverage: 6/6 = 100% ‘Advantages 1k conforms quality code by executing each program statement at least once. = _ It checks flow of program statement ‘7 Disadvantages = Itis necessary but not sufficient way of testing. — Missing Statement in the source code are not covered = The number of test cases required to achieve statement coverage changes with the type of statement. = Incase of decision making statements, tester need to censure the correct path is chosen, this is not covered with statement coverage. = Itis costly due to time and money > 2 Path coverage — Path coverage considers all possible paths present in block code, Path coverage executes paths flow from start to end. — If program code contains loop then there is infinite number of paths. To measure path coverage by using following formu! Path Coverage = (Total path exercised) / (Total no. of Paths) x100 This metric reports whether each of the possible paths in each function have been followed. It is determined by tracing how many execution paths have been exercised by the proposed test cases. = In other words, the test case is executed in such a way that every path is executed at least once. In this type of testing every statement in the program is guaranteed to be executed at least one time. ‘Scanned with CamScanner[BF saa (spPu-sem.7- 11) — Path coverage is stronger criteria than the statement coven Consider the same example from Fig. 2.1.5. Available possible paths are: An.s.07 ALRD.C1S.D6ER ALR. ALRSS.Do.8R Path coverage (PC) ~ If designed test case covers only one path then path ‘coverage is calculated as -(1/4)*100 ~ Hence this Test Case, has 25% of Path Coverage Disadvantages - Number of paths is exponential to the number of branches. > 3% Condition coverage _Itis also known as Predicate Coverage. Condition coverage reports the true or false outcome of each condition or Boolean expression. — A condition is an operand of a logical operator that ‘does not contain logical operators. Condition coverage ‘measures the conditions independently of each other. — This metric has better sensitivity to the control flow. Sometimes correct path is selected to execute but it will ‘ot evaluate all the conditions in Boolean evaluation. For example, in OR statement, if first part/condition is. true then there is no need to check other arts/conditions. Similarly, in case of AND statement, if first parvcondition then other parts/conditions are eed not be evaluated at all. Condition coverage is stronger than the path coverage. ~ Condition coverage is calculated with below given formula. Condition Coverage = (Total decision exercised) / (Total no. of decisions in the program) x 100 ‘Consider an example. 28 Testing Techniques and Levels of To if (|| y) 8&2) { /istatement 1 statement 2 d else { Hstatement3 , In order to have 100% Condition coverage, x, y ang , should be evaluated at east once against “true” and "fal. Following are the thre test cases {0 ensure this. Througy these testcase all predicates are tested against tre oF faye value. xstue | y=not false else pant evaluated executed x=false [y=tue | z=tre if pan executed x=false | y=false | z=not else pant evaluated executed ~ Further, condition coverage can be extended to a type called as Multiple condition coverage. It tells whether every possible combination of conditions occurs The test cases required for full multiple condition coverage of a decision are given by the logical operator truth table for the decision. — For languages with short circuit operators such as C, C++, and Java, an advantage of multiple condition coverage is that it requires very thorough testing. For these languages, multiple condition coverage is very similar to condition coverage. 7 Disadvantages ~ It can be tedious to determine the minimum set of test cases required, especially for very complex Boolean expressions. — The number of test cases required could vary substantially among conditions that have similar complexity. ‘Scanned with CamScanner() WED&E (CHM KEG) To achieve full multiple condition cove condition requires 6 test cases, rage, the first No. aalilel && » a[3[3]5]5|=[> a[3]3]4/2[- Fl -/ w]- al=]=/=]- 2) (alld) && Cd) &Ke To achieve full multiple condition coverage, : : the first ‘condition requires 11 test cases. we late [u fot feat Te Ta To Ty Tas [e +] dri fe TT: 21 (et [et ee 3 Fl [tr Fl |t ? 4 G T Fi [tv T 5 el [rt TTI. - 6 Fl [t ileal T 7 it el |r : 8 T F T F ett f- | [t T 10 T : T F " T : T : un > 4, Function coverage — Requirements are mapped into function blocks and logical units of program are formed. A function can ‘embed many other functions into it. — This metric reports whether you invoked each function or procedure from the given program. - It is determined by tracing many functions/modules of a program have been exercised by the proposed set of test cases. It is calculated with the formula given below. how Function Coverage = (Total functions exercised) / (Total no. ‘of functions in the program) x 100 = As functions can be identified easily in the program so test case writing becomes easier. I is easy to measure how many times a func called in the program. It is possible to have 100 percent function coverage as it represents high level of code abstraction, It helps in improving performance and quality of & Droduct. Comparison between Static Testing and Structural Testing: ‘Statle Testing, ‘Structural Testing 1. | teis performed without | Iti performed with the execution ofthe code. | execution ofthe code. 2. | Process is examined Process is examined by ‘manually of some static | giving a set of inputs 80 analysis tool is used see if the output matches the expected results 3. _| Testing happens early on | Testing happens after before the development of | the product has been the product has even developed. begun, k “4. | Identifies the problems | Identifies the problems like, Missing like, Variables not requirements, Design _| constant, checking if defect, Syntax Error, ete. | output matches the expected values ete 3. | Some types are, Informal | Some types are, Reviews, Technical Coverage testing, unit Reviews, Walkthrough, | testiog. Inspection, Static code Review 6. | Ithelps to find bugs ‘helps to find bugs before compilation. after compilation. 7._| Ws Prevention measure _| Its a cure measure 8. | tris done in the tis done in the -verification stage. validation stage. 9. | Itis tess time consuming | It is more time process. ‘consuming process. 2.1.1(D) Control Flow Graphs (CFG) Q.2.1.6 Whats a control flow graph? How Is it used in white box test design (Refer section 2.1.1(0)) (4 Marka) = CRG represents graphical representation of program. It is used to show different paths in program. It work as framework for analyse control flow of program. ‘Scanned with CamScannee= This is done through the representation of the flow of ‘execution within the program. It is useful in program analysis required during test and improvement. = It addresses the sequence in which program instructions are executed. It gives the iterative as well as looping. nature of a program assessment = Itis usually modelled with directed graph, G = (N, B) of a program consists of a set of nodes N and a set of edge E, Following are the few terms used in CFG. ~ Nodes : Each node represents a set of program statements. There are five types of nodes. There is a ‘unique entry node and a unique exit node. ~ Edges : It represents flow of control from one statement to another. There is an edge from node al to node 2 means control may flow from the last statement in nl to the frst statement in n2 — Branch Nodes : nodes that have more than one edge exiting itself. — Branch Edge : edges which exit a branch. ~ Predicate Node : node which contains a condition and is characterized by two or more edges originating from it. It creates two or more control branches (e.g. if or switch statements). — Merge node : It usually does not contain any statement ‘and is used to represent a program point where multiple control branches merge. Following representation in Fig. 2.1.6 shows symbols used while drawing CFG. Srey Decision point Merge point Fig. 2.1.6: Representation of program primes Testing Techniques and Levels of To, = A rectangle represents sequen statement/computation. Multiple sequent statements/eomputations can be represented ether by, single rectangle or by many rectangles, cag, corresponding to one statement in the source code, = We label cach statement/computation and decision bo, with a unique notation (either integer “I” oF alphabe “AY of combination “AI"). The two branches of decision box are labelled with T and F which represen the true and false respectively, of the condition within the box. Following are the representation of few program constructs. Fig. 2.1.7 represents flow graphs for typica, 86 O) case statement Fig. 2.1. : Flow graph representation for program constrocts Example 1 i ity So 9. eouts <(w); ‘Scanned with CamScanneeSee snsernen mmm ih 211 Testin sone sting Techniques and Levels of seeks yente it { yexth yeyth axty: ss Syllabus Topic : Using Black Box Approaches to Test Case Design 21.2 Using Black Box Approaches to Test Case Design Write a short note on Test case design using white box. (Refer section 2.1.26 Marks) gar 2.21.8 Write black box test cases for coffee maker machine (Refer section 2.1.2) (6 Marks) ~ _Black-box testing strategy is also known as data-driven, 6 inpuoutput-driven testing. To use this method, view the program as a black box. ~ Tester is completely unconcerned and unaware about the internal behavior and structure of the program. Instead, he must concentrate on finding conditions in Which the program does not behave according to its ~ In this approach, test data are taken from the ‘specifications without looking at the internal structure of the program. Following are the different black box based test case design techniques are explained viz. © Equivalence Class Partitioning, © Boundary Value Analysis © Random Testing Requirements based testing © Decision tables © State-based testing © Cause-effect graphing © Error guessing Black box approach requires exhaustive input testing. i.e. making use of every possible input condition as a {est case in order to find all errors in the program. But keeping in mind the available time and resources, testers have to choose suitable set of inputs from the set of all possible v and invalid inputs. Consider the example of square root calculation of a number. One can start with testing all positive input values. This would give big number of possible test cases, Still there are some possible inputs like all negative numbers and fractions. After this, the number of test cases would definitely increase. The goal for the smart tester is to use available time, cost and other resources effectively and develop set of test cases that can focus on maximum number of defects. 2.1.2(A) Equivalence Class Partitioning @.2.1.9 Explain equivalence partitioning with an ‘example. (Refer section 2.1.2(A)) (6 Marks) Equivalence class partitioning is the process to reduce the number of test cases to save time but still testing is effective. It divides input data into two equivalent class ot partitions ie. valid and invalid class. T ‘The few test cases are identified from each partition. It reduces time required for testing because less number of test cases for testing software. This is type of black box testing method. It is very effective testing method for large range of data input. ‘Scanned with CamScanneesTQA (SPPU - Sem. 7 2. mostly used where Tester needs to run the test cases for several sets of data-sets. The following steps are used for designing of test cases: Step 1: Identify the partitions. and select representative value for cach partition to generate test case Step 2: Write few test cases to test both classes. Step 3: Identify the output parameters and partitions the output and generate possible test cases t0 test output For example: A program that edits credit limits within a given range (Rs. 20,000 to Rs. 50,000) would have three Low boundary plus or minus 2 | Om the boundary Rs. 19,999 and Rs. 20,091 Rs. 20,000 and Rs. 50,009 Rs, 49,999 and Rs. 50.001 3 | Upper boundary plus or = Example: Consider following loan application form Consists of five fields to be entered by user. Validation checks are applied and shown in the Fig. 2.1.8. Customer Name cromerere FH Loan amount requesied [——}_ | Wtematen (FT D Monthly repayment See equivalence classes: Repaymen Interest rate; 1 | Less than Rs. 20,000 Invalid Class Total paid back; 2.| Between Rs. 20,000 and Rs. $0,000 | Valid Class 3 | Greater than Rs. 50,000 Invalid Class. 2.1.2(B) Boundary Value Analysis (BVL) @.2.1.10 Define the concept of boundary values. (Refer section 2.1.2(8)) (6 Marks) BVL are used to test boundary value between valid and invalid partitions in test case design. It is black box test design method and it is used to find the defects at boundaries of input. ‘The test case conditions are based on under and over of the boundary class. Test cases are designed To test the edge of each Equivalence Class To test edges of both input and output classes In boundary value analysis identifies equivalency classes as : First select the value on the boundary ‘Second select the value under the boundary Lastly select value over the boundary For example: In same credit limit example, boundary analysis would test: ‘Scanned with CamScannee(5 sroA SPPU-Sem. 7m Positive Testing IC is the testing method where software application tested and validated for valid input data. tn this, test ‘cases are executed for valid set of input data and check the output is expected output of not. ‘The main goal of this testing is check whether software application does that what it is supposed to do. This {ype of testing is used to check software product meets the specified requirements oF not Example of Positive Testing : Test the age of employee In this example age is numerical so tester should enter only numerical value as age. It should not accept any input other than number. Requirement specified as only accept numerical value. Enter Ag +— Enter only Numbers conations | Valid | tnvatid | yang Paritions | Partions | Bo tnt 000 | 5000 | soo 78.00,000 | 9.00000 | 9.00001 | om | So.000 | prot | 900,000 | Zero | | Non | ume | | NULL Design test cases Test Description Case lean Name : ABC ‘Ace No = 224466 Loan : 50,000 ‘Term: | year Total Paid : 53,660 2 | Name: XYZ ‘Term :3 years ‘Acc No: 113355 | Repayment : 1,50,000 Loan : 6,00,000 _| Interest Rate : 10 % ‘Term :3 years Total Paid :5,50,000 2.1.2(C) Positive and Negative Testing — To test the software application carefully, test need to design more test cases to check these results are as per mentioned requirement or not. - Totest the software application, test cases are designed for: Positive and Negative testing. Negative Testing Negative Testing, commonly referred to as error path testing or failure testing is done to ensure the stability of the application. Negative testing is the process of applying as much creativity as possible and validating the application against invalid data. In Negative Testing the system is validated by providing invalid data as input. A negative test checks if an application behaves as expected with its negative inputs. This is to test the application that does not do anything that it is not supposed to do so. Such testing is to be carried out keeping negative point of view & only ‘execute the test cases for only invalid set of input data. ‘The main reason behind Negative testing is to check the stability of the software application against the influences of different variety of incorrect validation data set. — Negative testing helps to find more defects & improve the quality of the software application under test but it should be done once the positive testing is complete. ‘Scanned with CamScannerPE sam PP tom 7-11 214 Forting Tochrauon and Lares Tostny 7 Haumple of Neyative Testing Considering example ax we know phone mo field accepts only numbers and docs not accept the alphabets tnd special characters but if we type alphabets and special characters on phone number field to check it accepts the alphabets and special characters or not th iH negative testing. en ery arbors [aoc] = Here, expectation is that the text box will not accept invalid values and will display an error message for the wrong entry. Syllabus Topic : Rendom Testing 2.1.20), Mandom Testing 2.2.1.1 Explain the differences between random testing and testing using error guessing. (Reter section 2.1.2(0)) (4 Marks) = Each software module or system has an input domain from which test input data is selected. = In the random testing approach, test inputs are selected randomly from the input domain of the system. For example: A test module requires input from numbers between I to 50.Tester can select any random values from this input domain. Random testing uses a four-step procedure: Step ‘The input domain is identified. ‘Test inputs are selected from the domain, Selected inputs are executed on the system under test. This inputs forms a random test wet. The results are compared with the system ‘specification. The testis failure if any input gives incorrect results; otherwise test is a Step 4 success. {The random testing leads to few questions WAiCH ave gy follows 1 Ave the selected few values adequate 10 stirw that the module meets its specification? 2. Should additional or fewer values be wsed 10 make the inont effective use of rewurces? 4. Are there any input values, other than thone selecte imone Wikely to reveal defects? 4. Should any values outside the valid domain be wved ay {est inputs? Consider the following C++ code snippet to understand the concept of random testing intent Abstintnumm) 4 if (um >0) ( return num; J elwe { return nem; 1) big: should be ‘num’ ’ } Now the random tests for this function could be (93, -45, 0, 28}. Only the value '-45" triggers the bug. If there is no reference implementation to check the result, the bug still could go unnoticed. 7 Advantages = _Itis cheap to use: it does not need to be smart about the program under test. = This quick to find bug candidates — If software is properly specified, it finds real bugs. = Itis very useful atthe system level = It allows easy estimation software reliability from test outcomes. ‘7 Disadvantages - Itonly finds basic bugs. = [tis limited only to the specification and specifications are typically imprecise. ‘Scanned with CamScannerSTOA (SPPU - Sem 7-17) 245 If differen inputs are randomly selected on each test ron, this can create problems A large number of test inputs are typically required to et meaningful statistical results ‘Some kind of automation is required to generate a large ‘number of inputs 2.1.22) Requirements Based Testing ee 0.21.12 What's reaurarant based txing? Ean | | wth an example. | __Strscen 21286) aa | Requirements-based testing is a testing approach in ‘which test cases, conditions and data are derived from requirement documents. It is used to test the usability and functional requirements of software. This types of testing has two issues, are as follows: 1, Requirement Validation : To validate requirement ‘unambiguously and logically as correct. 2 Designing Test Cases : Design necessary and sufficient test cases to test code to ensure it meets all defined requirements. Consider an example of requirements about a web based application. ~The application should have a login page, and the user should be able to login based on their roles(either ‘admin, user, or vest) — The password should be encrypted. The application should be easy to use and fast. — The application should be able to store large number of records. ‘After reviewing these requirements next stages are planned. This is represented using Fig. 2.1.9. Fig. 2.1.9 : Process of RBT Testing Techniques and Levets of Tesel Requirement based testing process divided into eight sages Defining Test Completion Criteria - When all the functional and non-functional testing is complete ‘means testing is completed. Design Test Cases - It has five characteristics: initial state of precondition, data setup, the inputs, expected ‘outcomes and actual outcomes, Build Test Cases: Tso parameters are needed to build test cases from logical test cases. 1. Creation of required data. 2. Constructing the components 10 support testing. Execute Tests - Execute the test cases against the system under test and document the results. — Verify Test Results - Verify if the expected and actual results match each other. — Verify Test Coverage - Verify if the tests cover both functional and non-functional aspects of the requirement. 1. Track and Manage Defects - Any defects detected during the testing process goes through the defect life ‘cycle and are tracked to resolution. Defect Statisties are maintained which will give us the overall status of the project 2. Manage the Test Library. The test manager maintains the relationships between the test cases and the programs being tested. The test manager keeps track of what tests have or have not been executed, and whether the executed tests have passed or failed. ‘Advantages = Integrates the testing throughout the SDLC process. = Assures the quality ofthe Requirement Specification. — Aiming at defect prevention than defect detection. 2.1.2(F) Decision Tables = A decision table is an excellent tool to use in both testing and requirements management. It is also used to test system behaviour for different input combinations. Itis also called as a Cause-Effect table. ‘Scanned with CamScannerSTQA(SPPU Cause and effects for better «design technique 10 BE plex. = This technique captures the test coverage. It is block box tes ‘ used to determine the test scenarios fOr business logic. versus ision table contains inputs 1. Table the = The structure of deci si rules/eases/est conditions shown in Table 2+ Jef side consists of set of effets, conditions | form of ¥ of N or ~ (Dash) and right side consists set of rules with value column. (0 verify the combinations of uum is used as ( = The chect decision table. ‘Table 2.1.1: Decision table representation a | Condtions [ Vawes [ Rt | R2 | Aa | Re | AS | v6 | A7 | AB o_fyntyiyiy[v[w[w[w[w co fyn-tyty|uin[v[v[w[w 3 vyin{y[w[y[nu]y[n Ets et 1 2{4 2 ata ra 6 21/3 tht Cn a ce ce © Advantages = | The fequirements are easily mapped to a decision table. — The representation is simple so that it can be easily interpreted. = The table will help to make effective combinations and can ensure a better coverage for testing. ‘7 Disadvantages The main disadvantage is that when the number of inpot increases the table will become more complex Each rule of decision table represents testcase. The test case development steps of decision table techniques: 1 Read all columns and finalize the effect. ‘Then transfer columns into test cases. 8 See SS eel ‘Syllabus Topic : State-based Testing —e—EeEeeeeeSESeL 2.1.2(G) State-based Testing In this technique, software can be viewed as collection of stats, input to those states, transition between states and events that causes such transitions. State can be defined as an abstract situation or interna configuration of an entity. The state transition graphs (STG) are developed for entire system or modules during specification phase. It is referred as “state charts” in Object orienteg terminologies. States are represented as set of nodes (Circles, ovals, rounded rectangles) and are denoted name or number. Inputs/events and th a Outputvactions are represented with set of arrows between nodes indicate what will cause a transition or change between the two linked states. AA black dot indicates initial state. Final state also be referred as “done State”. There can be “error states” as well in STG. A simple STG is shown in Fig. 2.1.10. It has two states SI and $2. Because of inpuvevent B, a transition from SI to S2 occurs and it generates Action 3 in this state transition. This is represented by the symbol “B/act3.” ! Nact2 Alact-1 Blact-3 —<—__" — «Bias — Clact-s Clacté Fig. 2.1.10 : Simple State transition graph Validation for State Transition Graph 1. Identify the condition and effect. o 2. List all conditions and effects. 3. Find number of possible comt ions, 4. Fill the value in column with all possible combinations. | — ‘5. Find similar combination and reduce the rules. (One state is designated as the initial state with outgoing transitions At least one state is designated as a final state with only incoming transitions; if not, the conditions for termination shall be made explicit ‘Scanned with CamScannee= very state is reachable from the intial stat ial state action sequences) one Final states reachable from al the other sa least ~ Every defined event and action appears in ate in at least one transition (or state) = Except forthe initial and final states, every sate h state has at o ito inning tamston nda ea transition a The state table aids tester to des N Lest cases. It consist the inputs or events that cause state transitions, It shows each state and each input the neat state and action taken. Input can be be a command (example read or write), a signal from a device (example, hot or cold) or a button that is pushed (example, on or off). It is essential to describe the exact sequence of inputs and the expected sequence of state changes, and actions. It relax the tester to execute, interpret and maintain test cases. Further test specification tells when the test begins in the start state, covers intermediate states, and returns to the start state. Using software probes tester can - report the current state, report the incoming event, make state variable visible, monitor the tests, detect incorrect transitions ‘and. detect discrepancies in intermediate results. Following Table 2.1.2 shows state table for the example stated in above section. ‘Table 2.1.2: State table for states $1 and $2 INPUT SL S2 ‘A | Si (act-1) | $2 (act-2) B $1 (act-4) c 1-5) | $2 (act-6) Advantages Useful for object oriented and procedural based software testing. Gives additional views to write useful test cases. fosting Techni Lovols of Test Disndvantages For large systems and system components, state {ransition graphs can become very complex oO Syllabus Topic : Cause-etfect Graphing 2.1.2(H) Cause-ettect Graphing Q.2.1.13 Explain cause and effect diagram with an ‘example. (Refer section 2.1.2(H)) (8 Marks) ‘This technique mainly represents a graph which maps specifications written in natural language. It allows testers to use combination of inputs and derive useful, effective set of test cases that incompleteness and ambiguities in the specifications. reveal ‘The specification is transformed into a graph that resembles a digital logic circuit (a combinatorial logic network), This circuit uses a somewhat simpler notation instead of standard electronics notations. Tester is required only to understand the boolean logic and logic operators (and, or, not) thus no knowledge of, electronics is necessary. The graph is then converted into a decision table and tester uses this table to develop test cases. Following are steps followed to derive the test cases- ‘The specification is divided into workable pieces. For example, in a testing of a web page design, a piece can be Menu or Navigation Sequence, in a testing of e-commerce site, a piece can be cart value. Step2: The causes and effects in the specification are identified. A cause is a distinct input ‘condition or an equivalence class of input conditions. An effect is an output condition or transformation in system., For example, when a database is updated with some records (cause) then confirmation, alert message appears(effect). ‘Scanned with CamScannerFrom the step 2 information, a Boolean ise-and-effect graph is ereated. Causes and effects are represented as nodes in the graph. Causes are placed on the eft side and effects fon the right of the graph. Logical relationships are expressed using standard Iogical operators such as NOT, OR, AND. ‘and are associated with ares. Step 3: constrains 10 due 10 In Cause- effeet graph has some describe cause-effect combin environmental constraints. Step 4: Step: Inthis step converts cause-effect graph into & decision table. to number of test Lastly it is converted cases. Fig. 21.11 :The basic notation for the graph Effect 3 occurs if both causes 1 and 2 are present O—® Effect 2 occurs if causes 1 occurs O-® Effect 2 occurs it causes 1 does not occur Fig. 2.1.12 : Interpretation of cause-effect graph Testing Technique Level EET stan (sppu-sem.7 18. IS ot Toa, “The following section is focusing on detail g amp, to generate cause-effect graph and decision table Consider @ specification where User search, character occurrence in a string by entering inpuy 1. finite string length value and 2, search key (character) Possible effects are :Report postion of charac, string, otherwise show the error massage “Not Foungs ‘The input conditions or causes are as follows: 0 CI: Range of +ve integer is from 1 to 80 © C2: Search character in the string ‘The output conditions, or effects are: E1 : Integer out of range ° © E2: Position of character in string © E3: Character not found ‘The rules or relationships can be described as follows: If CI and C2, then E2. ° © IfCI and not C2, then E3. © IfnotCl, then El. — Based on the above causes, effects, and their relationships, a cause-and-effect graph to represent this information in Fig. 2.1.13. et? De © &) $2413: Cue and Et raph or exam = Consider an example with sample string “ABCDEFG™ As discussed it will lead to total three test cases as follows in Table 2.1.3. ‘Scanned with CamScannersoe (SPPU Som. 7-1) ent sen based on cause and effect uraph pt | Length sory Ovip 2 Non tnd Out of range | From the above graph, we need to develop a decisi table, The decision table shows the rules and shows th 8 and shows the effects for all possible combinations of ¢ men, Bach, ‘column represents a test case _ For n causes, there will be a decision table with (2¢ny nities. In this example, we have only two causes such » causes such as environmental constrains and mnlikely combinations _ Iewill reduce number of combination and subsequently reduces no of test cases shown in Table 2.14, A decision table is generated for same. ‘Table 2.14 + Cause and effeet combined with test eases [nt] cify |y [N aly [Nn un [NW wy [n [Ny Bn Ly [n | 2.1.2{) Error Guessing Q.2.1.14 Explain the differences between random testing and testing using error guessing. (Refer section 2.1.2(0) (4 Marks) 0.21.15 Explain error guessing testing with an 240 Tosting 1 ine ch Void uewwed easily by Meneavvet, 0 ean prepane a tint of types of errone that can uncovered, Candidates of uch Hat can be errors detected Me carlier test projects. ‘The following are the enitical afew Cl the code where defects ane mont Hibely tbe found Code portion with high complesity. Complenity can be found using the method of cyclomatic eonmplety Module which is recently added int ain cole Code onion with previews defect history Module coded with new ans unproven technokony Sefined functional Code portion with loosely specification, Code portion developed by budding engineer Code portion developed by a developer having, law ‘confidence about it Code portion developed by many developers 2.2 _ Levels of Testing = ‘Testing of large systems is usually carried out at ‘example. (Refer section 2.1.2()) (4 Marks) different levels. There are four levels of testing: unit ration test, system test, and acceptance Yes as shown in Fig. 2.2.1. ee | rc I Tee jfcnemnes] [Demrnereenne I | [ I Sh e Byrd vane = Error guessing is a test case design technique where @ test engineer makes use of & skil, inition and experience to identify the defects. Test engineer uses his experience to - guess the types and probable locations of defects = design tests specifically to reveal the defects, = For example, consider the testing of a code where memory is dynamically allocated. Possible error can Pe found if unused memory is not deallocated. This can be Pig. 22.1 : Levels of testing Bach level has its set of goals. In unit testing, a single ‘component is tested with the goal of detecting and structural defects in that unit integration testing, components in a group are tested to functional In investigate component interactions. = In system testing, system as a whole is tested with the goal of evaluating attribuies such as usability, ‘Scanned with CamScannerWE’ ston gop san. 7M ned and reliability, and performance, Both object-oreed Procedhural-hased software systems, testi PRE begins with the smallest units or compan — = Syllabus Topic : Unit Testing vk24 @.2.2.1 What aro drivers and stub modules in the context of unit tasting of a software product? (4 Marks) Unit Testing (Rotor section 2.2.1) = Anuit is the smattest testable part of any software. Wis used to test individual unity” components ofa software ‘The purpose of unit testing is 40 validate each unit oF module of software system, before integration testing ied during U ~ _ A:series of stand-alone tests are condu ‘Testing, Each test examines an individual component that is new oF has been modified A umit tes is also called « module test because it tess the individual units of code that comprise the application, Each test validates a single module that, based on the technical design documents, was built to perform a certain task with the expectation that it will behave ina specific way or produce specific results. ~ Unit tests focus on functionality and reliability, and the ‘entry and exit criteria can be the same for each module ‘or specific to a particular module. Unit testing is done in a test environment prior to system integration. If a defect is discovered during a unittest, the severity of the defect will dictate whether or not it will be fixed before the module is approved. — In unit testing we require drivers and stubs. The driver acts as a calling unit and the stub acts as a called unit ‘Steps for unit testing: 1. First test the module interfaces 2. Check local data structure of module 3. Check boundary conditions of input data 4, Check for independent paths 5S. Cheek for error handling paths Intorfaco Local data structuroy ‘Boundary conditions Indopendont pathy Error-handing pats Pig. 2.24 Wironment for Unit testing, In the Fig. 2.2.3 tpl presented. Inortace eal dat stu Boundary condiere Independent ane Erorhaning pa Test RESULTS Fig. 2.2.3: Unit test typical environment For example, testing a given module (X) may require = A driver module which transmits test cases in the form Of input arguments to X and either prints or interprets the result produced by X, Zero or more “stub” modules each of which simulates the function of a module called byx. - It is required for each module that is directly subordinate to X in the execution hierarchy. If X is @ terminal module (i.e. it ells no other modules), then no stubs are required. Stubs and drivers are code that are (temporarily) written in order to unit test a program = Driver ~ code that is executed to accept test case data pass it {0 the component being tested, obiain result (or pass/fail) ‘Scanned with CamScannermain () { foo(1.lala1sD)s foo 1-2-1,2.1); sign TA(prof, course) { print “You successfully called assign for,” prof J It allows for automation of the testing process, reduces difficulties of discovering errors contained in more complex pieces of the application, and test coverage is often enhanced because attention is given to each unit For example, if you have two units and decide it would bbe more cost effective to glue them together and initially test them as an integrated unit, an error could cccur in a variety of places © Isthe error due to a defect in unit 1? © Isthe error due to a defect in unit 27 ‘© _Isthe error due to defects in both tnits? © _Isthe error due to a defect in the interface between the units? © Isthe error due to a defect in the test? — Finding the error (or errors) in the integrated module is ‘much more complicated than first isolating the units, testing each, then integrating them and testing the whole. - — Drivers and stubs can be reused so the constant changes that occur during the development eycle can be retested frequently without writing large amounts of additional test code = Inefféct, this reduces the cost of writing the drivers and stubs ona per-use’ basis and the cost of retesting is better controlled. Why is unit testing useful? E& staa (sPPu-sem.7 ing Techniques and Levels of Testin ing unit tests helps produce higher quality code 0” ‘many levels ‘The availability of tests helps detect the introduction of bugs whenever the programmer adds new features or te-factors the code. This is called regression testing. Tests also serve as a source of documentation about what code is really expected to do. ‘The act of writing the tests challenges the programmer to consider possible edge cases and their consequences. = Last but not least, the act of writing tests encourages the programmer to write’code in small chunks that ean be tested independently. Limitations 1. Itean only show the presence of errors; it cannot show the absence of errors. 2. It is more effective if used with other software testing activities, 3. Tt may not catch integration errors, performance problems, or other system-wide issues. 4, High discipline is needed in unit testing process to keep all test records since beginning. 5, Use of a version control system is essential that can provide a list of the source code changes (if any) that have been applied to the unit from begining. 2.2.2 Integration Testing @. Discuss the advantages and disadvantages of top-down and bottom-up approaches to integration testing. (Refer section 2.2.2) (@ Marks) ,2.2.3 Define sandwich testing. (Refer section 2.2.2) (@ Marks) — Integration testing is a logical extension of unit testing. In its simplest form, two units that have already been tested are combined into a component and the interface between them is tested. — A component, in this sense, refers to an integrated aggregate of more than one unit. In a realistic scenario, ‘Scanned with CamScannerTooting Tach Lovols of Tost 2 TOA (SPPU - Bom. 7-11 fas [unit ] fare combined into component, whieh are ‘many un in tun aggregated into even larger parts of the proyaain The den 1s ta test combinations of pieces and eventually expand the process to test your modules With those of ether groups Eventually all the modules making up a process are tested together. Beyond that, if the program is ore than one process, they should be composed of tested in pairs rather than all at once. Integration testing identifies problems that occur when Units are combined, By using a test plan that requires {you (0 fest cach unit and ensure the viability of each before combining units, you know that any errors discovered when combining units are likely related to the interface between units, ~ This method reduces the number of possibilities 10 4 far simpler level of analysis. The integration testing process is divided into two main type which are - Non- incremental and Incremental (A) Non-neremental integration testing | (8) Incromental integration testing Fig. 2.2.4 : Types of integration testing 2.2.2(A) Non-Incremental Integration Testing ~ Its also known as big-bang or umbrella approach, All the software units are assembled into the entire Program. This assembly is then tested as a whole from the beginning to know whether all the integrated ‘modules are working fine or not, = I may create a critical situation, as the causes of defects are not easily isolated and corrected. This approach followed by developers can be termed as ‘Run it and see it" approuch. Fig. 2.2.5 shows big bang integration {esti approach. Fig, 2.25: Big bang approach Suppose a system consist of four units = "nit I nit 2, ‘unit 3 and nit 4” are integrated simultaneously ang then the testing is performed. Hence in this approach no individual integration testing is performed because of which the chances of critica failures increases. Disadvantages Itis very time consuming testing process It is very difficult to trace the root cause of failures because of this late integration. ‘There are high chances of having critical failures because of integrating all the components together at same time, In case if any bug is found then it is very difficult to dotach all the modules in order to find out the root ‘cause of it 2.2.2(B) Incremental Integration Testing ‘The program is constructed and tested in small increments by adding a minimum number of components at each interval. Therefore, the errors are easier to isolate and correct, and the interfaces are more likely to be tested completely. ‘This process is carried out by using dummy programs called Stubs and Drivers. Stubs and Drivers do not implement the entire programming logic of the software module but just simulate data communication with the calling module, Stub is called by the Module lunder Test, whereas driver calls the Module to be tested. ‘Scanned with CamScannerfd iy System (B) Crea ) ae Ming driver (C) Creating stubs Fig, 2.2.6: Integration testing with incremental approach i approact Ithas three subtypes as ‘Types of Incremental integration Testing 1. Top-down ineremental integration esting 11, Bottom-up incremental integration testing Il, Sandwich (Bi-directional) increm Integration testing oe Fig. 2.2.7 : Types of incremental integration testing. 4 (1) Top-Down Incremental Integration Testing, = The top-down approach to integration testing requires the highest-level modules to be tested and integrated first = Modules are integrated from the main module (main program) to the subordinate modules cither in the depth-first or breadth-first manner. ‘This approach uses a special program called as stubs. Integration testing starts with the highest-level, control ‘module, or main program, with all its subordinates replaced by stubs. — These stubs may be created by tester or provided by testing hamess. Fig. 228 ; Top-Down Incremental Integration Testing = Consider following example. The testing starts with Module 1. To. test Module 1 in isolation 8, Communications to modules Module 2, Module 3 needs to be sim wed by the tester by some means, since these modules may not be ready yet. To simulate their Fesponses of whenever they are to be invoked from Module 1, stubs are c ed for them, Mes 1 Module Modula 2 3 Module Modula 4 6 Example of Top-Down Incremental Integration Testing, Top Down Fig. 2. In the above illustration, Module 1 would require stubs to simulate the activities of Module 2, Module 3. The integration of Module 2 would require a stub or stubs for Module 4 and Module 5, whereas for the Module 3 would require stubs for Module 6. Advantages All major faults are captured at the beginning of testing, ‘There is a possibility to obtain an early prototype program that allows demonstrations and boosts the ‘morale. Critical Modules are tested on priority so that major design flaws could be found and fixed first. Disadvantages Needs many Stubs. ‘Writing stubs is somewhat difficult tsk. Modules at lower level are tested inadequately. Program correctness can be misleading as only simulated values will be used initially, Itis very difficult to create integration test conditions. ‘Scanned with CamScannerEE ston (gppu-sem 7-0 > (ID Bottomup Incremental Integration Jy starts to test the lowest-level Testing = The bottom-up approve units first and then integrate all are integrated and tested, and then the successively superior level components siting the hierarchy from the nits. = The lowest level sub-mextules are adie and tested, ra bottom side to upwand side = This approach uses a special program known as driver It involves development of all the elementary modules or child modules for integration with the corresponding parent modules. During integration of modules, if at any stage it Is found that some mandatory module is missing, then in such an event that particular module is replaced with driver. These drivers may be created by tester or provided by testing harness. Fig- 22.10 : Bottom-Up Incremental Integration Testing, = Consider following example. If Module 6 is ready, we ‘need to simulate the activities of its superior, Module 3. Such a “driver” for Module 6 would simulate the invocation activities of Module 3. The driver would be responsible for invoking the module under test, it could bbe responsible for passing test data and it might be responsible for receiving output data. fe] eof) [s] Fig. 2.2.11 : Example of Bottom-Up Incremental Integration ‘Testing Testing Techniques and Levels of, For the above-mentioned Bottom-Up example, 4, must be provided for modules Module 4, Mog, et ue Module 6. However there is no need fora driver, fe topmost Module | Advantages Locating faults is easier. <4 waiting for all modules 4) No time is waste developed unlike Big-bang approach. In this approach development and testing can be don, together so that the product or application wit 4, efficient and as per the customer specification, It is easier to create the test conditions. It uses the live data from beginning so that test outpy, becomes easier. Disadvantages ‘The program as an entity does not exist until the lag module is added. Critical modules (at the top level of software architecture) which control the flow of application ae tested last and may be prone to defects. Important interface defects can be found at the end of cycle. It is required to create the drivers for modules at all levels except the top control. (I) Sandwich(Bi-directional) Incremental Integration Testing It is also known as hybrid or bidirectional integration testing. It combines the working methodologies of both top down and bottom up integration testing. In this approach, advantages of both the methodologies are worked in combination that results in getting the verification of product for flaws. ‘The system is viewed as having three layers or levels as shown in the Fig. 2.2.12. Midale layer -A target layer in the middle. Top layer - A layer above the target. Bottom layer- A layer below the target. ‘Scanned with CamScanner‘Fig. 2.2.12 : Example of Sandwich integration tert ng gr Execution strategy in Sandwich Testing ~The bottom up way of the testing initiates fi rile oF UE YE a 56 vad mad upper levels in the application, ‘te ‘The top down way of the testing starts from the middle layer and proceeds downwards towards the lower levels inthe software product under test. ‘The Ul is tested with stubs in isolation, and the lower level functions are verified using drivers. The stubs and drivers are not necessary for the topmost and the bottommost level. ‘The middle or target layer is left for the performance of the final set of tests. ‘© Modified Sandwich Testing Mode! ‘The three layers of the original sandwich model are tested individually before their coming together for incremental testing with one another. ‘The individual tests is on the subdivided top layer(use of stubs), bottom layer(use of drivers) and the (aEst layer that combines action of stubs and iver: “The combined layer tests are modified i a way hat the top layer is substituted by drivers andthe bottom most layer replaced with stubs. Advantages 1 Itis usefol when a large seale projects © be tested needs ‘More resources and big teams to perform both bottom. YUP and top-dow eran ia ead Disadvantages ‘The size ie incre wih ti form of tng it igh due Combination of two methodologies. Not recommended for the system or the product having interdependence between the different modules. Individual modules or systems which form sub- divisions in isions in the product are not tested in the entirety before integration —___—_————- Syllabus Topic : Detect Bash Elimination 2.2.2(C) Detect Bash Elimination Defect bash is an ad hoc testing where people performing different roles (developers, analysts. managers, testers, usability researchers, designers ‘documentation team people, marketing staff and many ‘thers) in an organization test the product together at the same time, ‘The testing task is solely left to an individual's interpretation, decision and creativity. It may be possible that they can perform operations that are not specified in the product specifications. It is not based ‘of automated and manual testing lis (on written test cases process, instead itis an additional testing effor. also known as a technique used under black-box testing and can be used as unofficial demo. It is the most common QA techniques widely applied in software development companies where the product can be used by people who perform diferent ols Defect bash is a unique and unconventional testing method which can bring out both funetional and non- functional defects. Functional defects are those that are inthe product and are reported by the users. Non-functonal defe=ts such as memory lea, long timaround time, missed requess, igh impact an utilization of system resouees 13S found while monitoring the system resources, |__ tun wie meting ti sen ee _ ‘Scanned with CamScanneesTOA (SPPU - Som. 7-17) 7 Process Fig. 22.13 explains about the defect bash proc wash session and css. It shows three phases as: preparation, by post process data as shown in Fig. 2.2.13. Iheonsists of two kinds of roles, onganizer(s) - who plan the defect hash, mederate it and analyze the report from re and participants; and participator who test the softwar report the findings. Fig. 2.2.13 : Process of defect bash In phase 1. organizers perform all preparatory activities like- setting goals, when to have defect bash, deciding duration, target software system , testing environment, defect reporting system, participants, evaluation system, instructions etc. In phase 2, organizers compose a team having a good mix of experienced, new and untrained people to participate. They explain system in a brief way, show known issues and a good error report to participants, provide focus on testing areas etc. at the beginning. They are allowing participants to report things like - issues faced, crashes encountered, questions, comments, general {eed-backs, any suggestions or ‘enhancements etc. = In phase 3, organizers performs analysis of all the issues and suggestions and summarize them for the team. Issues are checked for duplicates, all duplicated issues grouped into one issue, and the known issues are fered out. The newly encountered defects, suggestions and ‘enhancement that will be used to improve the software further. Information about result is then send to all participants. Frequency and Duration a. = Frequency and duration should be well planned. With less frequent and too small duration bashes, objective of defect finding will not be achieved. (On the other hand, with frequent bashes not much effective outcome will be produced even at large investment. If bash duration is reasonably selected (even if it is small), it can reduce cost of testing as many people are involved in this. > 2 Selection of build This is very key step in entire process. It is suggested to select build tested with regression testing. With defect bash on such build can boost the confidence to produce 1 good quality product and helps for the future progress of it. If evolving product build is can lead to produce large number of defects (can be severe one), which results in losing confidence of test team. > 3. Communicating the Objectives Purpose and objectives of defect bash must be stated very clearly to the group of people involved. It is prepared with respect to uncovered, non-reproducible, and random defects that were not tracked with planned test cases. ‘Scanned with CamScanner«TOA (SPPU-Som.7 to the % nat all t6si98 MTOR CAN be Foc eq y towards Meeting. those ig &_ Setting up environment fore starting bash, the test environme software, hardware, human resources) m, oe wst be planned ra care 8 FeqUiTe 10 Uncover functional ~ SS ——_—L are very less chances of ash fai met then here ap & Fixing issues ‘After the bash large number of defects are re ‘There is a chance that many defects are depicts vi diferent interpretation and different impact levels ™ Fixing cach defect one by one is difficult, hence it should be classified into issues. An issue can be due 5 cone defect or it can be due to several defects. In other words, issue comprises defects from different components into one unit. __llreported defects are fixed with design analysis, code inspection and necessary preve actions are taken. + 6 Optimization of efforts _ Since large amount of efforts are involved, it is necessary to optimize it. It can be done by following practices like - keeping record of bash objectives and ‘outcome, selecting right build, setting up right ‘environment, sharing objectives etc. One approach to reduce efforts can be conducting @ micro level bash prior to a large scale bash. This can uncover many defects. It can be classified into component bash, integration bash and product bash. © Benefits = Different people from different roles in the organization ‘are coming together for testing that help in team building inside a company. = It saves money as there is no need 10 hire external group of people for testing process. = Ithelps wo raise morale and to sharpen the team’s ability to find and manage issues. Wb Ings all toget ees eee rete eae process ing different people since fresh eyes have less ‘bias and can uncover new defect From organi Fo% ortmizaion team point of view. defect tse Ips in lean arming the product and learning, from each Ee product and learning f Wean ac Wen co bor uty snd scence ih (or the application by having number of untrained People using it The defect life cycle can be minimized as the bash Feports can he verified and assigned during the defects bash, Ttean be used to break the system instead of tying to conclude the system works. ‘7 Limitations Large number of duplicate defect reports are generated, which can waste much time in investigating, diagnosing and logging the same problem several times. Defect reports quality may be low. Due to limited timeframe and many frst time users (OF system under test), there isn't much opportunity to Jeam from each other. However efficient and effective bash is done by both organizer and participants, after getting more experience and learning required things Itcan predict customer behavior forthe Fist few hours and not offering any information about product's long term usage. Te causes the extra load on available resources for setting test environment with a big Broup of people. However, itis overcome with careful planning. © Comparison between Unit Testing and Integration Testing ‘sr. Unit Testing Tntegration Testing | No. 1 | test each part ofthe to combine modules in the program and show thatthe | application ands $8 individual parts re group to see that they are correct. working fine. ‘Scanned with CamScanner