Software Development Process

1. Introduction
Course Contents | Prev : Next

Computers are becoming a key element in our daily lives. Slowly and surely they are taking over many of the functions that affect our lives critically. They are now controlling all forms of monetary transactions, manufacturing, transportation, communication, defence systems, process control systems, and so on. In the near future, they will be found in our homes, controlling all forms of appliances. Left to themselves, they are harmless pieces of hardware. Load the right kind of software, they can take you to the moon, both literally and figuratively. It is the software that gives life to them.When they are going to play such a crucial role, one small flaw either in the hardware or the software can lead to catastrophic consequences. The sad part is, while there are well defined processes based on theoretical foundations to ensure the reliability of the hardware, same thing can not be said about software. There is no theory for software devlopment as yet. But at the same time, it is mandatory that software always behaves in a predictable manner, even in unforeseen circumstances. Hence there is a need to control its development through a well defined and systematic process. The old fashioned 'code & test' approach will not do any more. It may be good enough for 'toy' problems, but in real life, software is expected to solve enormously complex problems. Some of the aspects of real life software projects are: Team effort: Any large development effort requires the services of a team of specialists. For example the team could consist of domain experts, software design experts, coding specialists, testing experts, hardware specialists, etc. Each group could concentrate on a specific aspect of the problem and design suitable solution. However no group can work in isolation. There will be constant interaction among team members. Methodology: Broadly there are two types of methodologies, namely, 'procedure oriented methodolgies' and 'object oriented methodologies'. Though theoretically either of them could be used in any given problem situation, one of them should be chosen in advance. Documentation: Clear and unambiguous documentation of the artifacts of the development process are critical for the success of the software project. Oral communication and 'back of the envelop designs' are not sufficient. For example, documentation is necessary if client signoff is required at various stages of the process. Once developed, the software lives for a long time. During its life, it has to undergo lot of changes. Without clear design specifications and well documented code, it will be impossible to make changes. Planning: Since the development takes place against a client's requirements it is imperative that the whole effort is well planned to meet the schedule and cost constraints. Quality assurance: Clients expect value for money. In addition to meeting the client's requirements, the software should also meet additional quality constraints. They could be in terms of performance, security, etc. Lay user: Most of the time, these software packages will be used by non-computer savvy users. Hence the software has to be highly robust. Software tools: Documentation is important for the success of a software project, but it is a cumbersome task and many software practitioners balk at the prospect of documentation. There are tools known as Computer Aided Software Engineering (CASE) tools which simplify the process of documentation. Conformance to standards: We need to follow certain standards to ensure clear and unambiguous documentation. For example, IEEE standards for requirements specifications, design, etc. Sometimes, clients may specify the standards to be used. Reuse: The development effort can be optimised, by reusing well-tested components. For example, mathematical libraries, graphical user interface tool kits, EJBs, etc. Non-developer maintenance: Software lives for a long time. The development team, may not be available to maintain the package. Some other team will have to ensure that the software continues to provide services. Change management: Whenever a change has to be made, it is necessary to analyse its impact on various parts of the software. Imagine modifying the value of global variable. Every function that accesses the variable will

be effected. Unless care is taken to minimise the impact, the software may not behave as expected. Version control: Once changes are made to the software, it is important that the user gets the right version of the software. In case of failures, it should be possible to roll back to the previous versions. Subject to risks: Any large effort is subject to risks. The risks could be in terms of non-availability of skills, technology, inadequate resources, etc. It is necessary to constantly evaluate the risks, and put in place risk mitigation measures.

Software Development Process

2. Software Quality

Course Contents | Prev : Next

The goal of any software development process is to produce high quality software. What is software quality? It has been variously defined as: Fitness for purpose Zero defects Conformability & dependability The ability of the software to meet customer's stated and implied needs Some of the important attributes that can be used to measure software quality are: Correctness: Software should meet the customer's needs Robustness: Software must always behave in an expected manner, even when unexpected inputs are given Usability: Ease of use. A software with a graphical user interface is considered more user-friendly than one without it Portability: The ease with which software can be moved from one platform to another Efficiency: Optimal resource (memory & execution time) utilization Maintainability: Ease with which software can be modified Reliability: The probability of the software giving consistent results over a period of time Flexibility: Ease with which software can be adapted for use in different contexts Security: Prevention of unauthorised access Interoperabilty: The abilty of the software to integrate with existing systems Performance: The ability of the software to deliver the outputs with in the given constraints like time, accuracy, memory usage Correctness is the most important attribute. Every software must be correct. The other attributes may be present in varying degrees. For example, it is an expensive proposition to make a software 100% reliable and it is not required in all contexts. If the software is going to be used in life critical situations, then 100% reliability is mandatory. But, say, in a weather monitoring system, a little less reliability may be acceptable. However, the final decision lies with the client. One should keep in mind that some of the above attributes conflict with each other. For example, portability and efficiency could conflict with each other. To improve efficiency, one may resort to using system dependent features. but that will affect the portability. In the days, when DOS ruled the world, it was possible to access the internal architecture directly to improve performance. To port such a program to any other platform would require enormous changes. So in practice there will always be a tradeoff.

3. What is a Process

Course Contents | Prev : Next

3.2 Process: Definition and Phases

A Process is a series of definable, repeatable, and measurable tasks leading to a useful result. The benefits of a well defined process are numerous. It provides visibility into a project. Visibility in turn aids timely mid-course corrections It helps developers to weed out faults at the point of introduction. This avoids cascading of faults into later phases It helps to organize workflow and outputs to maximize resource utilization It defines everybody's roles and responsibilities clearly. Individual productivity increases due to specialization and at the same time the team's productivity increases due to coordination of activities A good software development process should: View software development as a value added business activity and not merely as a technical activity Ensure that every product is checked to see if value addition has indeed taken place Safeguard against loss of value once the product is complete Provide management information for in-situ control of the process To define such a process the following steps need to be followed: Identify the phases of development and the tasks to be carried out in each phase Model the intra and inter phase transitions Use techniques to carry out the tasks Verify and Validate each task and the results Exercise process and project management skills The words 'verify' and 'validate' need some clarification. Verify means to check if the task has been executed correctly, while validate means to check if the correct task has been executed. In the context of software, the process of checking if an algorithm has been implemented correctly, is verification, while the process of checking if the result of the algorithm execution is the solution to the desired problem, is validation. The generic phases that are normally used in a software development process are: Analysis: In this phase user needs are gathered and converted into software requirements. For example, if the user need is to generate the trajectory of a missile, the software requirement is to solve the governing equations. This phase should answer the question: what is to be done to meet user ne eds? Design: This phase answers the question: How to meet the user needs? With respect to the above example, design consists of deciding the algorithm to be used to solve the governing equations. The choice of the algorithm depends on design objectives like execution time, accuracy, etc. In this phase we determine organisation of various modules in the software system Construction: Coding is the main activity in this phase Testing: There are three categories of testing: unit testing, integration testing, and system testing. There are two types of testing: Black box testing and White box testing. Black box testing focuses on generating test cases based on requirements. White box testing focuses on generating test cases based on the internal logic of various modules Maintenance: Maintenance is the last stage of the software life cycle. After the product has been released, the maintenance phase keeps the software up to date with environment changes and changing user requirements. The earlier phases should be done so that the product is easily maintainable. The design phase should plan the structure in a way that can be easily altered. Similarly, the code should be in such a way that it is easily read, understood, and changed. Maintenance can only happen efficiently if the earlier phases are done properly.

. Software Life Cycle Models

Course Contents | Prev : Next

In practice, two types of software life cycle models are used: sequential model and iterative model.

4.1 Waterfall Model

Sequential model, also known as water fall model, is pictorially shown thus: ANALYSIS DESIGN CONSTRUCTION TESTING IMPLEMENTATION

It represents the development process as a sequence of steps (phases). It requires that a phase is complete before the next phase is started. Because of the explicit recognition of phases and sequencing, it helps in contract finalisation with reference to delivery and payment schedules. In practice it is difficult to use this model as it is, because of the uncertainity in the software requirements. It is often difficult to envisage all the requirements a priori. If a mistake in understanding the requirements gets detected during the coding phase, then the whole process has to be started all over again. A working version of the software will not be available until late in the project life cycle. So, iteration both within a phase and across phases is a necessity.

4. Software Life Cycle Models

Course Contents | Prev : Next

4.2 Prototyping
Prototyping is discussed in the literature as a separate approach to software development. Prototyping as the name suggests, requires that a working version of the software is built early in the project life. There are two types of prototyping models, namely: Throw away prototype and Evolutionary prototype The objective of the throw away prototyping model is to understand the requirements and solution methodologies better. The essence is speed. Hence, an ad-hoc and quick development approach with no thought to quality, is resorted to. It is akin to 'code and test'. However, once the objective is met, the code is discarded and fresh development is started, ensuring that quality standards are met. Since the requirements are now well understood, one could use the sequential approach. This model suffers from wastage of effort, in the sense that developed code is discarded, because it does not meet the quality standards. Evolutionary prototyping takes a different approach. The requirements are prioritised and the code is developed for the most important requirements first, always with an eye on quality. Software is continuously refined and expanded with feedback from the client. The chief advantage of prototyping is that the client gets a feel of the product early in the project life cycle. As can be seen, evolutionary prototyping is an iterative model. Such a model can be characterised by doing a little analysis, design, code, test and repeat the process till the product is complete.

4.3 Spiral Model

Barry Boehm has suggested another iteartive model called the spiral model. It is more in the nature of a framework, which needs to be adapted to specific projects. Pictorially it can be shown thus:

It allows best mix of other approaches and focusses on eliminating errors and unattractive alternatives early. An important feature of this model is the stress on risk analysis. Once the objectives, alternatives, and constraints for a phase are identified, the risks involved in carrying out the phase are evaluated, which is expected to result in a 'go, no go' decision. For evaluation purposes, one could use prototyping, simulations, etc. This model is best suited for projects, which involve new technology development. Risk analysis expertise is most critical for such projects.

Software Development Process

4. Software Life Cycle Models

Course Contents | Prev : Next

4.4 ETVX Model

IBM introduced the ETVX model during the 80's to document their processes. 'E' stands for the entry criteria which must be satisfied before a set of tasks can be performed, 'T' is the set of tasks to be performed, 'V' stands for the verification & validation process to ensure that the right tasks are performed, and 'X' stands for the exit criteria or the outputs of the tasks. If an activity fails in the validation check, either corrective action is taken or a rework is ordered. It can be used in any development process. Each phase in the process can be considered as an activity and structured using the ETVX model. If required, the tasks can be further subdivided and each subtask can be further structured using the ETVX model.

4. Software Life Cycle Models

Course Contents | Prev : Next

4.5 Rational Unified Process Model

Among the modern process models, Rational Unified Process (RUP) developed by Rational Corporation is noteworthy. It is an iterative model and captures many of the best practices of modern software development. RUP is explained more fully in the module OOAD with UML. More information on RUP can be obtained here. The Rational Unified Process (RUP) is an iterative software development process framework created by the Rational Software Corporation, a division of IBM since 2003. RUP is not a single concrete prescriptive process, but rather an adaptable process framework, intended to be tailored by the development organizations and software project teams that will select the elements of the process that are appropriate for their needs. RUP is a specific implementation of the Unified Process.

Four Project Life cycle Phases

The RUP has determined a project life cycle consisting of four phases. These phases allow the process to be presented at a high level in a similar way to how a 'waterfall'-styled project might be presented, although in essence the key to the process lies in the iterations of development that lie within all of the phases. Also, each phase has one key objective and milestone at the end that denotes the objective being accomplished. The visualization of RUP phases and disciplines over time is referred to as the RUP hump chart. I. Inception Phase The primary objective is to scope the system adequately as a basis for validating initial costing and budgets. In this phase the business case which includes business context, success factors (expected revenue, market recognition, etc.), and financial forecast is established. To complement the business case, a basic use case model, project plan, initial risk assessment and project description (the core project requirements, constraints and key features) are generated. After these are completed, the project is checked against the following criteria: 1. Stakeholder concurrence on scope definition and cost/schedule estimates. 2. Requirements understanding as evidenced by the fidelity of the primary use cases. 3. Credibility of the cost/schedule estimates, priorities, risks, and development process. 4. Depth and breadth of any architectural prototype that was developed. 5. Establishing a baseline by which to compare actual expenditures versus planned expenditures. If the project does not pass this milestone, called the Lifecycle Objective Milestone, it either can be cancelled or repeated after being redesigned to better meet the criteria. II. Elaboration Phase The primary objective is to mitigate the key risk items identified by analysis up to the end of this phase. The elaboration phase is where the project starts to take shape. In this phase the problem domain analysis is made and the architecture of the project gets its basic form. The outcome of the elaboration phase is: 1. A use-case model in which the use-cases and the actors have been identified and most of the use-case descriptions are developed. The use-case model should be 80% complete. 2. A description of the software architecture in a software system development process. 3. An executable architecture that realizes architecturally significant use cases. 4. Business case and risk list which are revised. A development plan for the overall project. 5. Prototypes that demonstrably mitigate each identified technical risk. 6. A preliminary user manual (optional) This phase must pass the Lifecycle Architecture Milestone criteria answering the following questions: * Is the vision of the product stable? * Is the architecture stable? * Does the executable demonstration indicate that major risk elements are addressed and resolved? * Is the construction phase plan sufficiently detailed and accurate? * Do all stakeholders agree that the current vision can be achieved using current plan in the context of the current architecture? * Is the actual vs. planned resource expenditure acceptable? If the project cannot pass this milestone, there is still time for it to be cancelled or redesigned. However, after leaving this phase, the project transitions into a high-risk operation where changes are much more difficult and detrimental when made. The key domain analysis for the elaboration is the system architecture. III. Construction Phase The primary objective is to build the software system. In this phase, the main focus is on the development of components and other features of the system. This is the phase when the bulk of the coding takes place. In larger projects, several construction iterations may be developed in an effort to divide the use cases into manageable segments that produce demonstrable prototypes. This phase produces the first external release of the software. Its conclusion is marked by the Initial Operational Capability Milestone. IV. Transition Phase The primary objective is to 'transit' the system from development into production, making it available to and understood by the end user. The activities of this phase include training the end users and maintainers and beta testing the system to validate it against the end users' expectations. The product is also checked against the quality level set in the Inception phase. If all objectives are met, the Product Release Milestone is reached and the development cycle is finished.

Six Best Practices

Six Best Practices as described in the Rational Unified Process is a paradigm in software engineering, that lists six ideas to follow when designing any software project to minimize faults and increase productivity. I. Develop iteratively It is best to know all requirements in advance; however, often this is not the case. Several software development processes exist that deal with providing solution on how to minimize cost in terms of development phases. II. Manage requirements Always keep in mind the requirements set by users. III. Use components Breaking down an advanced project is not only suggested but in fact unavoidable. This promotes ability to test individual components before they are integrated into a larger system. Also, code reuse is a big plus and can be accomplished more easily through the use of object oriented programming. IV. Model visually Use diagrams to represent all major components, users, and their interaction. "UML", short for Unified Modeling Language, is one tool that can be used to make this task more feasible. V. Verify quality Always make testing a major part of the project at any point of time. Testing becomes heavier as the project progresses but should be a constant factor in any software product creation. VI. Control changes Many projects are created by many teams, sometimes in various locations, different platforms may be used, etc. As a result it is essential to make sure that changes made to a system are synchronized and verified constantly.

4. Software Life Cycle Models

Course Contents | Prev : Next

4.6 Agile Methodologies

All the methodologies described before are based on the premise that any software development process should be predictable and repeatable. One of the criticisms against these methodologies is that there is more emphasis on following procedures and preparing documentation. They are considered to be heavyweight or rigorous. They are also criticised for their excessive emphasis on structure. There is a movement called Agile Software Movement, questioning this premise. The proponents argue that software development being essentially a human activity, there will always have variations in processes and inputs and the model should be flexible enough to handle the variations. For example: the entire set of software requirements cannot be known at the begining of the project nor do they remain static. If the model cannot handle this dynamism, then there can be lot of wastage of effort or the final product may not meet the customer's needs. Hence the agile methodolgies advocate the principle "build short, build often". That is the given project is broken up in to subprojects and each subproject is developed and integrated in to the already delivered system. This way the customer gets continuous delivery of useful and usable systems. The subprojects are chosen so that they have short delivery cycles, usually of the order of 3 to 4 weeks. The development team also gets continuous feedback. A number of agile methodologies have been proposed. The more popular among them are SCRUM, DYNAMIC SYSTEMS DEVELOPMENT METHOD (DSDM), CRYSTAL METHODS, FEATURE-DRIVEN DEVELOPMENT, LEAN DEVELOPMENT (LD), EXTREME PROGRAMMING (XP). A short description of each of these methods follows: SCRUM: It is a project management framework. It divides the development in to short cycles called sprint cycles in which a specified set of features are delivered. It advocates daily team meetings for coordination and integration. More information on SCRUM can be obtained here DYNAMIC SYSTEMS DEVELOPMENT METHOD (DSDM): It is characterised by nine principles: 1. 2. 3. 4. 5. 6. 7. 8. 9. Active user involvement Team empowerment Frequent delivery of products Fitness for business purpose Iterative and incremental development All changes during development are reversible Baselining of requirements at a high level Integrated testing Collaboration and cooperation between stakeholders More information on DSDM can be obtained here CRYSTAL METHODOLOGIES: They are a set of configurable methodologies. They focus on the people aspects of development. The configuration is carried out based on project size, criticality and objectives. Some of the names used for the methodologies are Clear, Yellow, Orange, Orange web, , Red , etc. More information can be obtained from here. FEATURE DRIVEN DEVELOPMENT (FDD): It is a short iteration framework for software development. It focuses on building an object model, build feature list, plan by feature, design by feature, and build by feature. More information can be obtained from here. LEAN DEVELOPMENT (LD): This methodology is derived from the principles of lean production, the restructuring of the Japanese automobile manufacturing industry that occurred in the 1980s. It is based on the following principles of lean thinking: Eliminate waste, Amplify learning, Decide as late as possible, Deliver as fast as possible, Empower the team, Build the integrity in, See the whole. More information can be obtained from here. EXTREME PROGRAMMING (XP): This methodology is probably the most popular among the agile methodologies. It is based on three important principles, viz., test first, continuous refactoring, and pair programming. More information can be obtained from here. One of the important concepts popularised by XP is pair programming. Code is always developed in pairs. While one person is keying in the code, the other person would be reviewing. This site is dedicated to pair programming. The paper by Laurie Williams et al., demonstrates the efficacy of pair programming.

5. How to choose a process

Course Contents | Prev : Next

Among the plethora of available processes, how can we choose one? There is no single answer to this question. Probably the best way to attack this problem is to look at the software requirements. If they are stable and well understood, then waterfall model may be sufficient. If they are stable, but not clear, then throw away prototyping can be used. Where the requirements are changing, evolutionary prototyping is better. If the requirements are coupled with the underlying business processes, which are going through a process of change, then a model based on Boehm's spiral model, like the Rational Unified Process should be used. In these days of dynamic business environment, where 'time to market' is critical, and project size is relatively small, an agile process should be chosen. These are but guidelines only. Many organisations choose a model and adapt it to their business requirements. For example, some organisations use waterfall model, modified to include iterations within phases.