Week 8 Software Testing

<AFTER REF. BY STELLMAN & GREENE> Q : What is quality in software testing ? What is defect ? A : In software testing, quality is defined as conformance to requirements. Every use case, functional requirement, and other software requirement defines a specific behavior that the software must exhibit. When the software does not behave the way that the requirements say it must behave, that is a defect. This means that your software testers are responsible for figuring out whether the software that was produced by the team behaves in the way that the requirements it was built from say that it should.

Q : What is test plan ? A : Test plan is documented main work product of a test planning, which is carried out to establish the list of tasks that, if performed, will identify all of the requirements that have not been met in the software. The test plan documents the overall approach to the test. In many ways, the test plan serves as a summary of the test activities that will be performed. It shows how the tests will be organized, and outlines all of the testers needs which must be met in order to properly carry out the test. The test plan should be inspected by members of the engineering team and senior managers. Table 1 shows the outline of a typical test plan.

Table 1. Typical test plan

Q : What is test case ? A : Test case is a description of a specific interaction that a tester will have, in order to test a single behavior of the software. Test cases are very similar to use cases, in that they are step-by-step narratives that define a specific interaction between the user and the software. However, unlike use cases, they contain references to specific features of the user interface. The test case contains actual data that must be

entered into the software and the expected result that the software must generate. Table 2 shows an example of a test case.

Table 2. Example of test case

Q : How should the software test be executed ? A : The software testers begin executing the test plan after the programmers deliver the (high quality) alpha build , or a build that they feel is feature complete. There are typically several iterations of test execution : # The first iteration focuses on new functionality that has been added since the last round of testing. # A regression test is a test designed to make sure that a change to one area of the software has not caused any other part of the software which had previously passed its tests to stop working. # Regression testing usually involves executing all test cases which have previously been executed. # There are typically at least two regression tests for any software project.

Q : What is defect tracking system ? A : Defect tracking system is a program that testers use to record and track defects. It routes each defect between testers, developers, the project manager and others, following a workflow designed to ensure that the defect is verified and repaired. Every defect encountered in the test run is recorded and entered into a defect tracking system so that it can be prioritized. The defect workflow should track the interaction between the testers who find the defect and the programmers who fix it. It should ensure that every defect can be properly prioritized and reviewed by all of the stakeholders to determine whether or not it should be repaired. This process of review and prioritization referred to as triage.

Q : What is smoke test ? A : Smoke test is a subset of the test cases that is typically representative of the overall test plan. Smoke tests are good for verifying proper deployment or other non invasive changes. They are also useful for verifying a build is ready to send to test. Smoke tests are not substitute for actual functional testing.

Q : What is test automation ? A : Test automation is a practice in which testers employ a software tool to reduce or eliminate repetitive

tasks. Testers either write scripts or use record-and-playback to capture user interactions with the software being tested. This can save the testers a lot of time if many iterations of testing will be required. It costs a lot to develop and maintain automated test suites, so it is generally not worth developing them for tests that will executed only a few times.

Q : What is postmortem report ? A : Postmortem report is an overall account of the teams experience in building the software, and of the experience of the users and stakeholders in working with the team. The report should contain an honest assessment of how the team members, users, and stakeholders perceived the end product, and assessed the decisions made throughout the project. The purpose of the post-mortem report is to highlight the teams successes and identify any problems which should be fixed in future releases.

Week 9 Why Software Projects Fail ? Fixing Various Problems In Software Project Phases

< AVAILABLE ONLY IN THE SLIDES, NOT IN THE TEXTBOOK > Q : What are typical problems in planning ? A : # Lack of leadership : It takes more than a talented and motivated team to make a successful project; # Mid-course correction : A change in project priorities throws the team into disarray. This usually comes from a lack of understanding of the scope of the project.; # Detached Engineering Team : There is an artificial wall between the people who build the software and those who need it.

Q : How to fix problems in planning ? A : # Use a vision and scope document to define the needs of the users and stakeholders; # Use a project plan to keep every informed about how those needs will be met; # Use risk planning to keep the plan realistic.

Q : What are typical problems in estimation ? A : # Padded Estimates Generate Distrust : For some unreasonable reasons, programmers add extra time to their estimates, while project managers and senior managers quickly figure this out, start to question individual estimates and dont have good answers from programmers; # Self-Fulfilling Prophecy : A project manager under pressure simply imposes a deadline, and creates unrealistic estimates that meet it.

Q : How to fix problems in estimation ? A : # Adopting a repeatable estimation process like Wideband Delphi can help fix them; # By writing down assumptions, the team can handle risks without padding their time and even avoid the risks altogether # It reduces padding and increases honesty through transparency, by letting the team correct each other in an open meeting

Q : What are typical problems in scheduling ?

A : # Working Backwards From a Deadline : Project managers approach a non-negotiable deadline for a project by working backwards; # Misunderstood Predecessors : The project manger does not take the time to understand how tasks depend on each other.

Q : How to fix problems in scheduling ? A : # They can be avoided by adopting good planning and estimation practices and creating a project Schedule; # Schedule techniques like critical path analysis can help spot problems early on.

Q : What are typical problems in review ? A : # Problems Are Found Too Late : There are preventable defects in the software that arent caught until late in the project; # Big, Useless Meetings : A project manager, who want to avoid the same fails previously happened, calls a big meeting with everyone who could possibly have input, though the meeting drags on for hours, without making any real progress: # The Indispensable Hero : One critical person is seen as the clear top programmer, and all important work is sent through him.

Q : How to fix problems in review ? A : # Reviews can catch defects early, when they are cheaper to fix; # A review meeting only includes the people necessary for the work to be done; # Reviews especially code reviews can help the hero spread his expertise and knowledge

Q : What are typical problems in requirement ? A : # Iteration Abuse : Iteration can be a useful tool, but it is often abused; # Scope Creep : After the programming has started, users and stakeholders make small changes and the programmers agree to it. Eventually, the project slows to a crawl.

Q : How to fix problems in requirement ? A : # The team can adopt software requirements engineering practices to write down most of the changes before the work begins; # A change control process gives them a handle on the few changes that remain.

Q : What are typical problems in programming ? A : # Haunted by Ghosts of Old Problems : Programmers find that old bugs suddenly reappear without warning; # Broken Builds : The programmers deliver a build which does not work and the testers cant even begin to test it; # Spaghetti Code : Maintaining old code is the least desirable programming job in many organizations.

Q : How to fix problems in programming ? A : # Get control of the source code with version control software like Subversion; # Use unit tests and test-driven development to increase the quality of the build; # Use refactoring to keep the code readable.

Q : What are typical problems in testing ?

A : # Requirements Havent Been Implemented : The team delivers software missing behavior or even entire features; # Obvious Bugs Slipped Through : Inexperienced testers are expected to just bang on the software; #But It Worked For Us! : When a product is not tested in all environments in which it will be used, the tests will be thrown off.

Q : How to fix problems in testing ? A : # Software testers must be involved in every stage of development; # Test planning must be given adequate time on the schedule; # Sufficient budget must be provided for a testing environment.

Week 10 Understanding Changes

Q : Explain why changes fail ! A : # Short answer : Politics ! You have to make changes to the way people in your organization work (changes is not enough in the scope of your project). # Change is Uncomfortable : Nobody likes to think that they make mistakes. Making changes means talking about past mistakes and admitting that they are mistakes ! # Common Excuses - We Already Build Software Well : This is just the way software projects always go (People know that there are problems with the schedule and quality, but think that nobody ever does any better). # Common Excuses - Not Invented Here Syndrome : People intentionally avoid research or innovations that were not developed within the organization (Yes, NIH syndrome really happens !). # Common Excuses - Its Too Theoretical : When ideas dont make intuitive sense, they are dismissed as merely academic. # Common Excuses - It Just Adds More Bureaucracy : Any work other than programming is wasteful busywork that keeps the real work from getting done. # Common Excuses - You Cant Give Me More Work ! : Asking someone to review a document or make an estimate is asking them to do more work. # Common Excuses - Its Too Risky : Its safer to let a project fail in a way its failed before than to make a change that might not work.

Q : How to make change succeed ? A : Progress comes from making smart changes by : # Prepare Your Organization : ~ Weve always done it like this., ~ Take credit for the changes ~ Build support from the team ~ Work around stragglers # Plan for Change : ~ Create a vision and scope document ~ Inspect the vision and scope document ~ Be positive about the work thats already being done ~ Make the changes seem straightforward ~ Show that the changes will save time and effort ~ Stick to the facts

~ Schedule the changes # Push for Consensus # Use a Pilot Project to Build a Track Record # Measure Your Progress ~ Measuring cost # Bring In an Expert ~ Measurin quality

Week 11 Management and Leadership

Q : Explain responsibility, authority and accountability ! A : A person has responsibility for a task if he is given sufficient authority to perform it, and he is also accountable for its completion. A person has authority to perform a task only if he is has adequate control over the resources necessary to complete the task. A person is accountable for a task if failure to adequately perform that task carries professional consequences.

Q : What is delegation ? A : Delegation is assigning responsibility of a task to a team member. When delegating a task, the project manager must ensure that the team member has the authority to perform it and is accountable for the results

Q : Explain transparency in managing project ? A : # When the project manager creates a document, holds a meeting of interest to others, or makes an important project decision, all of the information produced should be shared and used with everyone involved in the project # All work products should be public : All team members, senior managers and stakeholders should have access to every work product produced for the project # Decisions should be made based on known guidelines : ~ Published standards documents help others understand the way certain roles must be filled ~ Documents should be based on templates when possible ~ Process documents ensure that each project is done using a repeatable process ~ Use performance plans to set expectations for individual team members

Q : Explain how organization should be managed in software project concern ! A : # Prevent senior managers from seeing software projects as a cost burden : A project is successful if its costs are justified by its benefits. # Show senior managers the impact of their decisions : Show senior managers that improving project management practices will help them meet their goals # Dont confuse flexibility with always saying yes : ~ Dont agree to an unrealistic schedule. ~ Change your approach when necessary.

~ Dont confuse easy to describe with easy to implement.

Q : Explain how software project team should be managed ! A : # Avoid common management pitfalls : ~ Dont manage from your gut. ~ Dont expect consensus all of the time. ~ Accept criticism # Avoid micromanagement : ~ Dont expect to review everything ~ Use transparency to your advantage # Address performance problems early ~ Work with each team member to develop a performance plan. ~ Set standards that are fair and attainable. ~ Measure each team members progress against known and agreed-upon goals. ~ Correct performance problems as early as possible. ~ Dont fall into the hands-on manager trap ~ Dont be afraid to let your team members make mistakes ~ Dont second-guess estimates. ~ Make your mistakes public.

Week 12 Managing an Outsourced Project

Q : Explain how to prevent outsourced project failure ! A : # Get involved : Dont be a hands-off client # Constantly communicate project goals : Transparency is especially important in an outsourced project. # Make Sure the Project Is Estimated Well : Vendors often estimate the work as part of contract negotiation

Q : Describe several important management issues in outsourced projects, which are different from the project developed in-house ! A : # Actively Manage the Project : A hands-off project manager is usually surprised when the software is delivered (Its not enough to just have weekly status meetings with no follow-up). # Build a relationship with the vendors management : Make sure the management at the vendor recognizes and rewards good work # Build a relationship with the team (at vendor) : A project manager doesnt have the same kind of relationship with the team that he would with a team in his own organization Gain credibility by making good decisions.

Q : Explain how collaboration with the vendor should be carried out in outsourced project ! A : # Plan and manage the project scope : The outsourced project starts with a scope and a budget (As opposed to an in-house project, which starts with a set of known resources). # Dont depend on the vendor to maintain the project plan and project schedule : When a project manager is responsible for the project, he must keep track of its status himself.

# Hold reviews and inspections : Use a collaborative inspection process that has been optimized for outsourced projects. # Dont delegate the entire design and programming of the project to the vendor : ~ Establish design constraints early on, ~ If possible, design the software in-house, or in collaboration with the vendor. ~ Monitor the code base using code reviews and project automation. # Take responsibility for the quality of the software : Quality is not just another deliverable that can be bought and paid for. # Dont Blindly Trust the Vendor : ~ Even though individual team members may have certifications or degrees, it doesnt mean that they are competent. ~ Just because the vendors organization is certified, that doesnt guarantee that they know better than you do how to run your project. ~ Dont be intimidated by the vendors pedigree. If something on the project looks wrong, it probably is!

Week 13 Process Improvement

Q : Explain the argument around the necessity of formal process in the process improvement ! A : Many process improvement experts see the world as black and white. They often feel that there are bad software teams without a formal process, and good teams that have one in place. But the world is not that simple ! There are teams still surviving without formal process !

Q : Explain any kind of teams which can be effective without a formal software process ! A : # Teams can consist of jack-of-all-trades programmers who understand the business of the organization. # Skunk works programmers may often take initiative and build useful software without input. # A highly capable development manager may be willing to put in an enormous effort.

Q : Explain why the team without formal process can survive ! A : An organization that produces software always has a software process. So, its just not formal, or documented and repeatable.

Q : What is a main weak point of teams without formal process ? A : Team without a formal process does not scale up easily : # Programmers who used to produce lots of software find that their projects have started to feel bogged down. # This often happens when a small programming group with a good track record is faced with having to build a project on a larger scale. # It also happens when teams expand.

Q : It is known that formal process is useful for expanding team. Explain other condition where the formal process is useful ! A : Formal process can also help in an organization where experts, users, or stakeholders are no longer readily available to the programmers.

Q : What is software process improvement ? A : Software process improvement is the art and science of changing an organizations software process in order to build better software. Software process improvement always involves looking at the big picture. This means writing down the entire software process as a whole and making sure that it is followed on each project. It involves looking at entire projects and identifying areas that can be improved.

Q : Explain frameworks and methodologies which can be used for process improvement ! A : Models and certifications that help assess the state of the organizations process and serve as a framework for improving that process. CMM, ISO 9000 and Six Sigma are frameworks. There are also methodologies that an organization can adopt that describe the complete set of activities, roles, and work products needed to build software. Rational Unified Process and Extreme Programming are methodologies. Capability Maturity Model (CMM) defines the characteristics of a mature, capable process in a way that can be measured and compared to processes at other organizations ISO 9000 is a family of quality management standards defined by the International Standards Organization (ISO) Six Sigma is an approach to improving quality in manufacturing and business processes (DMAIC approach : Define opportunities, Measure performance, Analyze opportunity, Improve performance, Control performance) Extreme Programming (XP) consists of a set of rules and practices that govern all areas of software development: planning, designing, coding, and testing. The goal of XP is to lower the cost of change. To meet this goal, many XP practices are highly iterative. By making the stakeholders part of the project team, XP addresses the problem of the hands-off customer. Rational Unified Process (RUP) is a popular off-the-shelf process based on the idea of highly iterative development. RUP includes a disciplined approach to requirements management that is based on the idea of managing changes. RUP incorporates software design using the Unified Modeling Language (UML), a visual modeling system for graphically representing the use cases, class model, object interactions, and components of the software