Professional Documents
Culture Documents
Notes Shared by Sir
Notes Shared by Sir
Planning involves selection of appropriate strategies to meet the project objectives, identification of the activities and their inter-
relationships, estimation of the effort required to execute these activities and preparing a schedule showing when and by whom each
activity will be carried out in the project. Planning is also concerned with identifying all the resource requirements and estimating the
cost of the project.
Planning should be done before the project activities begin. It should cover general project planning for the entire project and detailed
project planning for the immediately following period.
Planning is not a one-time activity, but a continuing process. A plan is subject to change when the environment or the project
requirements change. The plan document should be updated to reflect the changes.
▪ Scheduling of activities.
These activities may be carried out before finalising the Project Plan.
Project Plan is the controlling document for managing the software project. It defines the technical and managerial processes
necessary to meet the project requirements.
Project Plan should be documented for all projects giving the following details:
▪ Process definition
▪ Milestones
▪ Schedules
▪ Project Organization
▪ Other areas such as training & induction of the project team, tools to be used and client involvement.
Project Plan should be prepared as per the documentation standards prescribed in the manual. It should be reviewed and approved
by persons designated by the Delivery Assurance Manager (DAM) and be subjected to document control and change control
procedures.
▪ Assigning responsibilities.
Guidelines:
▪ Each activity should have clearly defined start and end points.
▪ Each activity should be of manageable size and may not exceed one week's duration. As far as possible, one activity should be
assigned to one person.
For maintenance phase, the detailed schedule should be prepared for every Problem Report or Change Request. Where
necessary activities specific to the Problem Report or Change Request may be identified and scheduled.
▪ The Detailed Project Plan should normally be confined to the current and next phase and should span a period of
not more than three months. Schedules beyond this period may not be accurate.
▪ The detailed schedule should be consistent with the overall project schedule.
▪ The Detailed Project Plan should provide sufficient details to evaluate whether the detailed plan will meet the
requirements of the overall plan. This may include information such as:
- Effort estimated
- Effort spent.
▪ The Detailed Project Plan should be prepared for every phase before the commencement of the phase.
It need not be documented for a phase if the required information is already available in the Project
Plan itself.
▪ The Detailed Project Plan need not be part of the Project Plan document. It may be prepared as a
separate document.
▪ The Detailed Project Plan should be drawn up after considering the dependencies among the activities.
▪ PERT or CPM techniques may be used for preparing the Detailed Project Plan.
Class notes on DevOps
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
You have 2 free stories left this month. Sign up and get an extra one for free.
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 1/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
Introduction
The Phoenix Project was written by Gene Kim, Kevin Behr, and George
Spafford, three of the thought leaders of the DevOps movement that’s
gained so much popularity in the past few years. It’s written in a novel-
format, which makes it a much more engaging and enjoyable read than
most professional books.
also a fun way of learning about business process improvement and cultural
transformation in IT.
The novelty isn’t in the originality of these ideas. These are proven,
decades-old concepts that have changed the course of manufacturing,
leadership, safety culture, security, IT operations, and software
engineering. However, despite being known in their fields, they have never
been tied together so neatly.
The reason for the popularity of the book is that the concepts within can
help any organization leverage IT to accelerate their pace by orders of
magnitude while maintaining quality. Achieving this is somewhat of a Holy
Grail in IT, and the business advantages are tremendous. It also creates the
sort of fulfilling culture that most of us can only dream of.
Synopsis
The story revolves around Bill Palmer, an IT director who works for the
fictitious aging automotive parts manufacturer, Parts Unlimited. Bill is
suddenly thrust into the role of VP of IT after yet another catastrophic IT
failure causes his former superiors to be unceremoniously fired. Bill has a
reputation for driving excellence through the practical application of
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 3/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
process and running a tight ship. Parts Unlimited CEO, Steve Masters, calls
upon Bill to bring order to the chaos in IT. The situation seems grim as the
company stock is in decline, company failures make news headlines, and
competitors demonstrate that they can innovate faster and provide more
customer value and are coming after the company’s market share. Bill
reluctantly accepts the challenge to help save the company.
Even more familiar are the problems the characters find themselves in
throughout the book. Just like in the real world, poor organizational
structure, technical debt, and tightly-coupled systems cause many of these
issues. And just like in the real world, getting out of these situations is
challenging because it requires high levels of innovation, determination,
and collaboration.
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 4/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
One of the pivotal characters is Dr. Erik Reid, an eccentric prospective Parts
Unlimited board member. Erik is gradually revealed to have in-depth
knowledge of IT and process engineering as he mentors Bill and his
colleagues. At crucial points, Erik introduces new concepts that Bill uses in
his attempts to stave off the company’s seemingly inevitable decline into
obscurity.
Concepts
The Phoenix Project is a wonderful introduction to several key concepts. To
understand the full context, it’s best to read the book and especially the
additional material. Consider this more of a quick reference.
The first type of work is Business Projects, which help drive business
processes and initiatives. These generally involve new features aimed at
providing new capabilities or efficiencies for customers. These are driven by
business leaders and can be managed as projects, and are usually not
tracked centrally, but instead managed by individual budget owners.
The second type of work is Internal Projects. These are generally smaller bits
of regular work that accumulates over time: maintenance and upgrades of
existing systems, retiring old technology, security patches, and so on. This is
the work that keeps the company functioning. These can also be planned
and run in a queue of work.
Changes are the third type of work and encompass bug fixes, version
changes, and any improvements that are generated by the first two types of
work. These are the routine updates and support that arise from the simple
existence of more applications and infrastructure.
The fourth and most damaging type, Unplanned Work, is work that is
necessary because other work needs to be redone, and it throws different
types of work into chaos. This is the insidious, invisible work that makes
other planned commitments impossible to meet because it slows or blocks
other kinds of work. Due to its detrimental effect on the other three types,
Unplanned Work needs to be avoided at all costs.
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 6/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
The point of this journey is to make it clear to Bill that making work visible
is the first step in starting to make sense of it all. If existing commitments
across the organization are not understood, it can’t be coordinated to
ensure that the priorities of the broader company are being met.
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 7/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
performed the first time must be discarded. Lean is organized around the
idea that organizations should continuously reduce waste so that the
resources saved from that can be redirected to other endeavours, such as
making the process more efficient, improve tools and procedures, and
conduct experiments to create new features or products.
Kanban Boards
Bill and his colleagues eventually counteract the effects of excessive WIP by
creating kanbans around their teams, thereby making work visible and
allowing them to prioritize the most valuable work. In particular, Brent’s
time is carefully guarded so that he can work on the right task.
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 8/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
The idea is that making work visible allows the team to see the big picture
around the work being done. Doing so makes the process very easy to
follow, fosters better collaboration, and makes WIP very obvious.
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 9/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
One nuance is that this doesn’t necessarily mean actual deployments, but
rather the ability to. Continuously deploying may be rare with new features,
but bug fixes or critical patches are kinds of updates that need to flow
through the system quickly and safely.
By applying the TOC, they identify the actual bottlenecks in the system and
introduce the practices and processes to continually improve the flow of
work and shift to a more strategic approach.
The TOC can be broken down as the continual identification and removal of
bottlenecks in any given system. To be able to apply this, an understanding
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 10/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
of the system itself must be acquired, which means understanding each step
in the process. This is deceptively hard because these steps are often hidden
in peoples’ heads. Until those steps are listed, understood, and mapped out
in a sequence, it’s tough to establish a repeatable process that results in
consistency and quality.
A practice called value-stream mapping is often the first step in applying the
TOC. This is precisely what Bill and his colleagues use to understand the
flow of the work. From there, they are able to unblock the system enough to
accelerate the flow of the work.
…the principles that all of the DevOps patterns can be derived from, which
we’re using in both the “DevOps Handbook” and “The Phoenix Project: A Novel
About IT, DevOps, and Helping Your Business Win.” We assert that the Three
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 11/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
Ways describe the values and philosophies that frame the processes,
procedures, practices of DevOps, as well as the prescriptive steps.
2. The Second Way — Get fast feedback and work from strong failure
signals to ever-weaker failure signals to get advance warning of quality
issues.
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 12/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
3. The Third Way — Use the efficiencies gained in The First Way and the
safety enforced by The Second Way to introduce rapid experiments that
help create qualitative gains.
The idea is that by following The Three Ways, your company will outpace
competitors by orders of magnitude and will win in the marketplace by
creating value for customers superior to its competitors.
Conclusion
It wasn’t until The Phoenix Project that these ideas were woven together and
packaged in a simple mental model that neatly establishes how they all fit
together, and clearly articulates the goal behind each part. Although the
ideas were out there, they weren’t packaged in such an engaging way.
The book was also written so that the characters, situations, and choices
would be familiar to readers. This can create a profound connection to the
material because the journey is not just a story, but follows an emotional arc
as the characters work together to pull themselves back from the precipice.
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 13/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
Closing Thoughts
For a long time, it felt futile to try to fight for quality. Despite the application
of software engineering principles, Agile methodologies, carefully-
considered architecture, and years of experience, my work still felt rushed.
There was a certain inevitability that the accumulation of insurmountable
technical debt would lead to the need to rewrite or retire. Management was
ever-fearful that engineers were squandering resources by blindly pursuing
the shiniest new tech, or maybe perfection, and continuously trying to
reduce delivery timelines. The vicious cycle would never end.
The Phoenix Project has had a profound effect on how I approach work. It
showed me that simultaneously building quality as well as the desired
functionality into a product as it’s developed drastically reduces rework and
saves time in the long-run. Furthermore, those savings can be re-invested
into the product to improve it even more. It’s a virtuous cycle where
everybody in the company wins.
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 14/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
It showed me that there is a better way to build and deliver software that is
fulfilling, where work can be collaborative. Where quality does matter as
much as delivery. Where being part of a learning culture and embracing
experimentation and ingenuity can help any company win. And where
making things better is the shared goal of the entire company. Even for
skeptics, that should be enough to warrant an investigation.
Disclaimer
I’m an employee of IBM. The views expressed in this blog are mine and
don’t necessarily reflect the positions, strategies, or opinions of the
company.
DevOps Book Reports Gene Kim The Phoenix Project Lean Software Development
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 15/16
8/30/2020 Book Summary: The Phoenix Project | by Stephen Kawaguchi | Level Up Coding
https://levelup.gitconnected.com/book-summary-the-phoenix-project-42ebf68addbd 16/16
INTRODUCTION
• Project management technique for measuring project progress in an
objective manner.
• Combine measurements of scope, schedule, and cost in a single integrated
system.
• Provides an early warning of performance problems.
• Improve
• the definition of project scope,
• prevent scope creep,
• communicate objective progress to stakeholders,
• keep the project team focused on achieving progress.
Elements of Project Control
• Measurement of Performance
• Effective CorrectiveActions
Project Monitoring
Project
Monitoring
• Planning and scheduling play an important role in project management
• But without monitoring the project, chances of completing the project
on schedule is difficult
• Two parameters are monitored
• Cost
• Time
• The following graph shows the Time-cost relationship for a project
Project
Monitoring
Cost over-run with schedule
Time over-run
Cost
Ideal
Curve
Time
It is important to note that Earned Value is based on the Actual quantities installed orActual
quantity of work done
• It compares what was planned to be spent on the work completed vs. what was
actually spent
• It helps to identify whether the project is ahead or behind schedule
• Negative value indicates that the project is over budget
Components of Earned value Analysis
http://www.iceincusa.com/16CSP/content/4_ev/gifs/image1.gif (17.01.2007)
Kanban vs Scrum vs Agile
by Matthias Marschall on July 27, 2015
raaphorst
When inflexible and wasteful software development processes are making your organization
inefficient, it’s time to introduce an agile methodology. Kanban vs Scrum then becomes an
essential question: Which agile software development methodology is better suited for my own
situation? And is Kanban agile? What about Scrum vs agile? Confusion is spreading… Let’s
have a look how to sort out all those questions.
Scrum is a well-defined process framework for structuring your work. Introducing Scrum is quite
a change for a team not used to agile software development: They have to start working in
iterations, build cross-functional teams, appoint a product owner and a Scrum master, as well as
introducing regular meetings for iteration planning, daily status updates and sprint reviews. The
benefits of the Scrum methodology are well understood: Less superfluous specifications and
fewer handovers due to cross-functional teams and more flexibility in roadmap planning due to
short sprints. Switching your organization to use Scrum is a fundamental shift which will shake
up old habits and transform them into more effective ones.
The initial introduction of Scrum is not an end in itself. Working with Scrum you want to change
your teams’ habits: Take more responsibility, raise code quality, increase speed. As your teams
commit to sprint goals, they are intrinsically motiviated to get better and faster in order to deliver
what they promised. Scrum leverages team commitment as change agent. It’s amazing to see
how much teams demand from themselves – often way more you as a manager ever dared ask
for.
The Kanban methodology is way less structured than Scrum. It’s no process framework at all,
but a model for introducing change through incremental improvements. You can apply Kanban
principles to any process you are already running (even to Scrum ). In Kanban, you
organize your work on a Kanban board. The board has states as columns, which every work item
passes through – from left to right. You pull your work items along through the in
progress, testing, ready for release, and released columns. And you may have various swim
lanes– horizontal “pipelines” for different types of work. The only management criteria
introduced by Kanban is the so called “Work In Progress (WIP)”. By managing WIP you can
optimize flow of work items. Besides visualizing work on a Kanban board and monitoring WIP,
nothing else needs to be changed to get started with Kanban.
For every column (state) on your Kanban board you should define a “Work In Progress”-Limit
(WIP Limit). The WIP limit tells you how much work items are allowed to be in a certain state at
any given point in time. If a state reaches its pre-defined WIP limit, no new work can enter that
state. The whole team has to help clear the filled up state first. Work items trapped in a state will
build highly visible clusters on the Kanban board. These clusters make bottlenecks in the
progress visible – you can simply look at the Kanban Board to see where your process needs
improvements. Making the need for improvement visible challenges your team to change the
way they work to avoid such bottlenecks in the future. That’s how WIP limit act as change agent
in Kanban.
Kanban vs Scrum
Looking at both agile software development methodologies it should be more clear what to
introduce when: If your organization is really stuck and needs a fundamental shift towards a
more efficient process, Scrum seems to be more appropiate. If you already have working
processes, which you want to improve over time without shaking up the whole system, Kanban
should be your tool of choice.
And Scrum vs Agile?
Asking for the differences between Scrum vs Agile or Agile vs Scrum is like asking for the
differences between “Water” and “Ice”. Ice is water in a specific physical state. The same could
be said about Scrum vs Agile. Scrum is agile in a specific shaping. It is an agile process
framework. Scrum and Kanban in software development are both specific shapings of an agile
software methodology. While Scrum vs Kanban or Kanban vs Scrum is comparing two agile
methodologies, Scrum vs Agile is comparing a concrete example with its fundamental principles.
The Scrum Guide
The Definitive Guide to Scrum:
The Rules of the Game
July 2011
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 2
Purpose of the Scrum Guide
Scrum is a framework for developing and sustaining complex products. This Guide contains the
definition of Scrum. This definition consists of Scrum’s roles, events, artifacts, and the rules that
bind them together. Ken Schwaber and Jeff Sutherland developed Scrum; the Scrum Guide is
written and provided by them. Together, they stand behind the Scrum Guide.
Scrum Overview
Scrum (n): A framework within which people can address complex adaptive problems, while
productively and creatively delivering products of the highest possible value. Scrum is:
Lightweight
Simple to understand
Extremely difficult to master
Scrum is a process framework that has been used to manage complex product development
since the early 1990s. Scrum is not a process or a technique for building products; rather, it is a
framework within which you can employ various processes and techniques. Scrum makes clear
the relative efficacy of your product management and development practices so that you can
improve.
Scrum Framework
The Scrum framework consists of Scrum Teams and their associated roles, events, artifacts, and
rules. Each component within the framework serves a specific purpose and is essential to
Scrum’s success and usage.
Specific strategies for using the Scrum framework vary and are described elsewhere.
The rules of Scrum bind together the events, roles, and artifacts, governing the relationships and
interaction between them. The rules of Scrum are described throughout the body of this
document.
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 3
Scrum Theory
Scrum is founded on empirical process control theory, or empiricism. Empiricism asserts that
knowledge comes from experience and making decisions based on what is known. Scrum
employs an iterative, incremental approach to optimize predictability and control risk.
Transparency
Significant aspects of the process must be visible to those responsible for the outcome.
Transparency requires those aspects be defined by a common standard so observers share a
common understanding of what is being seen.
For example:
A common language referring to the process must be shared by all participants; and,
A common definition of “Done”1 must be shared by those performing the work and those
accepting the work product.
Inspection
Scrum users must frequently inspect Scrum artifacts and progress toward a goal to detect
undesirable variances. Their inspection should not be so frequent that inspection gets in the way
of the work. Inspections are most beneficial when diligently performed by skilled inspectors at
the point of work.
Adaptation
If an inspector determines that one or more aspects of a process deviate outside acceptable
limits, and that the resulting product will be unacceptable, the process or the material being
processed must be adjusted. An adjustment must be made as soon as possible to minimize
further deviation.
Scrum prescribes four formal opportunities for inspection and adaptation, as described in the
Scrum Events section of this document.
1
See “Definition of “Done”, p. 15.
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 4
Scrum
Scrum is a framework structured to support complex product development. Scrum consists of
Scrum Teams and their associated roles, events, artifacts, and rules. Each component within the
framework serves a specific purpose and is essential to Scrum’s success and usage.
The rules of Scrum bind together the events, roles, and artifacts, governing the relationships and
interaction between them. The rules of Scrum are described throughout the body of this
document.
Scrum Teams deliver products iteratively and incrementally, maximizing opportunities for
feedback. Incremental deliveries of “Done” product ensure a potentially useful version of
working product is always available.
The Product Owner is the sole person responsible for managing the Product Backlog. Product
Backlog management includes:
The Product Owner may do the above work, or have the Development Team do it. However, the
Product Owner remains accountable.
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 5
The Product Owner is one person, not a committee. The Product Owner may represent the
desires of a committee in the Product Backlog, but those wanting to change a backlog item’s
priority must convince the Product Owner.
For the Product Owner to succeed, the entire organization must respect his or her decisions. The
Product Owner’s decisions are visible in the content and ordering of the Product Backlog. No
one is allowed to tell the Development Team to work from a different set of requirements, and
the Development Team isn’t allowed to act on what anyone else says.
Development Teams are structured and empowered by the organization to organize and
manage their own work. The resulting synergy optimizes the Development Team’s overall
efficiency and effectiveness. Development Teams have the following characteristics:
They are self-organizing. No one (not even the Scrum Master) tells the Development Team
how to turn Product Backlog into Increments of potentially releasable functionality;
Development Teams are cross-functional, with all of the skills as a team necessary to create
a product Increment;
Scrum recognizes no titles for Development Team members other than Developer,
regardless of the work being performed by the person; there are no exceptions to this rule;
Individual Development Team members may have specialized skills and areas of focus, but
accountability belongs to the Development Team as a whole; and,
Development Teams do not contain sub-teams dedicated to particular domains like testing
or business analysis.
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 6
The Scrum Master helps those outside the Scrum Team understand which of their interactions
with the Scrum Team are helpful and which aren’t. The Scrum Master helps everyone change
these interactions to maximize the value created by the Scrum Team.
Scrum Events
Prescribed events are used in Scrum to create regularity and to minimize the need for meetings
not defined in Scrum. Scrum uses time-boxed events, such that every event has a maximum
duration. This ensures an appropriate amount of time is spent planning without allowing waste
in the planning process.
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 7
Other than the Sprint itself, which is a container for all other events, each event in Scrum is an
opportunity to inspect and adapt something. These events are specifically designed to enable
critical transparency and inspection. Failure to include any of these events results in reduced
transparency and is a lost opportunity to inspect and adapt.
The Sprint
The heart of Scrum is a Sprint, a time-box of one month or less during which a “Done”, useable,
and potentially releasable product Increment is created. Sprints have consistent durations
throughout a development effort. A new Sprint starts immediately after the conclusion of the
previous Sprint.
Sprints contain and consist of the Sprint Planning Meeting, Daily Scrums, the development work,
the Sprint Review Meeting, and the Sprint Retrospective.
Each Sprint may be considered a project with no more than a one-month horizon. Like projects,
Sprints are used to accomplish something. Each Sprint has a definition of what is to be built, a
design and flexible plan that will guide building it, the work, and the resultant product.
Sprints are limited to one calendar month. When a Sprint’s horizon is too long the definition of
what is being built may change, complexity may rise, and risk may increase. Sprints enable
predictability by ensuring inspection and adaptation of progress toward a goal at least every
calendar month. Sprints also limit risk to one calendar month of cost.
Cancelling a Sprint
A Sprint can be cancelled before the Sprint time-box is over. Only the Product Owner has the
authority to cancel the Sprint, although he or she may do so under influence from the
stakeholders, the Development Team, or the Scrum Master.
A Sprint would be cancelled if the Sprint Goal becomes obsolete. This might occur if the
company changes direction or if market or technology conditions change. In general, a Sprint
should be cancelled if it no longer makes sense given the circumstances. But, due to the short
duration of Sprints, cancellation rarely makes sense.
When a Sprint is cancelled, any completed and “Done” Product Backlog Items are reviewed. If
part of the work is potentially shippable, the Product Owner typically accepts it. All incomplete
Product Backlog Items are re-estimated and put back on the Product Backlog. The work done on
them depreciates quickly and must be frequently re-estimated.
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 8
Sprint cancellations consume resources, since everyone has to regroup in another Sprint
Planning Meeting to start another Sprint. Sprint cancellations are often traumatic to the Scrum
Team, and are very uncommon.
The Sprint Planning Meeting is time-boxed to eight hours for a one-month Sprint. For shorter
Sprints, the event is proportionately shorter. For example, two-week Sprints have four-hour
Sprint Planning Meetings.
The Sprint Planning Meeting consists of two parts, each one being a time-box of one half of the
Sprint Planning Meeting duration. The two parts of the Sprint Planning Meeting answer the
following questions, respectively:
What will be delivered in the Increment resulting from the upcoming Sprint?
How will the work needed to deliver the Increment be achieved?
The input to this meeting is the Product Backlog, the latest product Increment, projected
capacity of the Development Team during the Sprint, and past performance of the Development
Team. The number of items selected from the Product Backlog for the Sprint is solely up to the
Development Team. Only the Development Team can assess what it can accomplish over the
upcoming Sprint.
After the Development Team forecasts the Product Backlog items it will deliver in the Sprint, the
Scrum Team crafts a Sprint Goal. The Sprint Goal is an objective that will be met within the
Sprint through the implementation of the Product Backlog, and it provides guidance to the
Development Team on why it is building the Increment.
The Development Team usually starts by designing the system and the work needed to convert
the Product Backlog into a working product increment. Work may be of varying size, or
estimated effort. However, enough work is planned during the Sprint Planning meeting for the
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 9
Development Team to forecast what it believes it can do in the upcoming Sprint. Work planned
for the first days of the Sprint by the Development Team is decomposed to units of one day or
less by the end of this meeting. The Development Team self-organizes to undertake the work in
the Sprint Backlog, both during the Sprint Planning Meeting and as needed throughout the
Sprint.
The Product Owner may be present during the second part of the Sprint Planning Meeting to
clarify the selected Product Backlog items and to help make trade-offs. If the Development
Team determines it has too much or too little work, it may renegotiate the Sprint Backlog items
with the Product Owner. The Development Team may also invite other people to attend in order
to provide technical or domain advice.
By the end of the Sprint Planning meeting, the Development Team should be able to explain to
the Product Owner and Scrum Master how it intends to work as a self-organizing team to
accomplish the Sprint Goal and create the anticipated Increment.
Sprint Goal
The Sprint Goal gives the Development Team some flexibility regarding the functionality
implemented within the Sprint.
As the Development Team works, it keeps this goal in mind. In order to satisfy the Sprint Goal, it
implements the functionality and technology. If the work turns out to be different than the
Development Team expected, then they collaborate with the Product Owner to negotiate the
scope of Sprint Backlog within the Sprint.
The Sprint Goal may be a milestone in the larger purpose of the product roadmap.
Daily Scrum
The Daily Scrum meeting is a 15-minute time-boxed event for the Development Team to
synchronize activities and create a plan for the next 24 hours. This is done by inspecting the
work since the last Daily Scrum and forecasting the work that could be done before the next
one.
The Daily Scrum is held at the same time and place each day to reduce complexity. During the
meeting, each Development Team member explains:
The Development Team uses the Daily Scrum to assess progress toward the Sprint Goal and to
assess how progress is trending toward completing the work in the Sprint Backlog. The Daily
Scrum optimizes the probability that the Development Team will meet the Sprint Goal. The
Development Team often meets immediately after the Daily Scrum to re-plan the rest of the
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 10
Sprint’s work. Every day, the Development Team should be able to explain to the Product Owner
and Scrum Master how it intends to work together as a self-organizing team to accomplish the
goal and create the anticipated increment in the remainder of the Sprint.
The Scrum Master ensures that the Development Team has the meeting, but the Development
Team is responsible for conducting the Daily Scrum. The Scrum Master teaches the
Development Team to keep the Daily Scrum within the 15-minute time-box.
The Scrum Master enforces the rule that only Development Team members participate in the
Daily Scrum. The Daily Scrum is not a status meeting, and is for the people transforming the
Product Backlog items into an Increment.
Daily Scrums improve communications, eliminate other meetings, identify and remove
impediments to development, highlight and promote quick decision-making, and improve the
Development Team’s level of project knowledge. This is a key inspect and adapt meeting.
Sprint Review
A Sprint Review Meeting is held at the end of the Sprint to inspect the Increment and adapt the
Product Backlog if needed. During the Sprint Review, the Scrum Team and stakeholders
collaborate about what was done in the Sprint. Based on that and any changes to the Product
Backlog during the Sprint, attendees collaborate on the next things that could be done. This is
an informal meeting, and the presentation of the Increment is intended to elicit feedback and
foster collaboration.
This is a four-hour time-boxed meeting for one-month Sprints. Proportionately less time is
allocated for shorter Sprints. For example, two week Sprints have two-hour Sprint Reviews.
The Product Owner identifies what has been “Done” and what has not been “Done”;
The Development Team discusses what went well during the Sprint, what problems it ran
into, and how those problems were solved;
The Development Team demonstrates the work that it has “Done” and answers questions
about the Increment;
The Product Owner discusses the Product Backlog as it stands. He or she projects likely
completion dates based on progress to date; and,
The entire group collaborates on what to do next, so that the Sprint Review provides
valuable input to subsequent Sprint Planning Meetings.
The result of the Sprint Review is a revised Product Backlog that defines the probable Product
Backlog items for the next Sprint. The Product Backlog may also be adjusted overall to meet new
opportunities.
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 11
Sprint Retrospective
The Sprint Retrospective is an opportunity for the Scrum Team to inspect itself and create a plan
for improvements to be enacted during the next Sprint.
The Sprint Retrospective occurs after the Sprint Review and prior to the next Sprint Planning
Meeting. This is a three-hour time-boxed meeting for one-month Sprints. Proportionately less
time is allocated for shorter Sprints.
Inspect how the last Sprint went with regards to people, relationships, process, and tools;
Identify and order the major items that went well and potential improvements; and,
Create a plan for implementing improvements to the way the Scrum Team does its work.
The Scrum Master encourages the Scrum Team to improve, within the Scrum process
framework, its development process and practices to make it more effective and enjoyable for
the next Sprint. During each Sprint Retrospective, the Scrum Team plans ways to increase
product quality by adapting the Definition of “Done” as appropriate.
By the end of the Sprint Retrospective, the Scrum Team should have identified improvements
that it will implement in the next Sprint. Implementing these improvements in the next Sprint is
the adaptation to the inspection of the Scrum Team itself. Although improvements may be
implemented at any time, the Sprint Retrospective provides a dedicated event focused on
inspection and adaptation.
Scrum Artifacts
Scrum’s artifacts represent work or value in various ways that are useful in providing
transparency and opportunities for inspection and adaptation. Artifacts defined by Scrum are
specifically designed to maximize transparency of key information needed to ensure Scrum
Teams are successful in delivering a “Done” Increment.
Product Backlog
The Product Backlog is an ordered list of everything that might be needed in the product and is
the single source of requirements for any changes to be made to the product. The Product
Owner is responsible for the Product Backlog, including its content, availability, and ordering.
A Product Backlog is never complete. The earliest development of it only lays out the initially
known and best-understood requirements. The Product Backlog evolves as the product and the
environment in which it will be used evolves. The Product Backlog is dynamic; it constantly
changes to identify what the product needs to be appropriate, competitive, and useful. As long
as a product exists, a Product Backlog also exists.
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 12
The Product Backlog lists all features, functions, requirements, enhancements, and fixes that
constitute the changes to be made to the product in future releases. Product Backlog items have
the attributes of a description, order, and estimate.
The Product Backlog is often ordered by value, risk, priority, and necessity. Top-ordered Product
Backlog items drive immediate development activities. The higher the order, the more a Product
Backlog item has been considered, and the more consensus exists regarding it and its value.
Higher ordered Product Backlog items are clearer and more detailed than lower ordered ones.
More precise estimates are made based on the greater clarity and increased detail; the lower
the order, the less detail. Product Backlog items that will occupy the Development Team for the
upcoming Sprint are fine-grained, having been decomposed so that any one item can be “Done”
within the Sprint time-box. Product Backlog items that can be “Done” by the Development Team
within one Sprint are deemed “ready” or “actionable” for selection in a Sprint Planning meeting.
As a product is used and gains value, and the marketplace provides feedback, the Product
Backlog becomes a larger and more exhaustive list. Requirements never stop changing, so a
Product Backlog is a living artifact. Changes in business requirements, market conditions, or
technology may cause changes in the Product Backlog.
Multiple Scrum Teams often work together on the same product. One Product Backlog is used
to describe the upcoming work on the product. A Product Backlog attribute that groups items is
then employed.
Product backlog grooming is the act of adding detail, estimates, and order to items in the
Product Backlog. This is an ongoing process in which the Product Owner and the Development
Team collaborate on the details of Product Backlog items. During Product Backlog grooming,
items are reviewed and revised. However, they can be updated at any time by the Product
Owner or at the Product Owner’s discretion.
Grooming is a part-time activity during a Sprint between the Product Owner and the
Development Team. Often the Development Team has the domain knowledge to perform
grooming itself. How and when grooming is done is decided by the Scrum Team. Grooming
usually consumes no more than 10% of the capacity of the Development Team.
The Development Team is responsible for all estimates. The Product Owner may influence the
Team by helping understand and select trade-offs, but the people who will perform the work
make the final estimate.
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 13
toward completing projected work by the desired time for the goal. This information is made
transparent to all stakeholders.
Scrum does not consider the time spent working on Product Backlog Items. The work remaining
and date are the only variables of interest.
Various trend burndown, burnup and other projective practices have been used to forecast
progress. These have proven useful. However, these do not replace the importance of
empiricism. In complex environments, what will happen is unknown. Only what has happened
may be used for forward-looking decision-making.
Sprint Backlog
The Sprint Backlog is the set of Product Backlog items selected for the Sprint plus a plan for
delivering the product Increment and realizing the Sprint Goal. The Sprint Backlog is a forecast
by the Development Team about what functionality will be in the next Increment and the work
needed to deliver that functionality.
The Sprint Backlog defines the work the Development Team will perform to turn Product
Backlog items into a “Done” Increment. The Sprint Backlog makes visible all of the work that the
Development Team identifies as necessary to meet the Sprint Goal.
The Sprint Backlog is a plan with enough detail that changes in progress can be understood in
the Daily Scrum. The Development Team modifies Sprint Backlog throughout the Sprint, and the
Sprint Backlog emerges during the Sprint. This emergence occurs as the Development Team
works through the plan and learns more about the work needed to achieve the Sprint Goal.
As new work is required, the Development Team adds it to the Sprint Backlog. As work is
performed or completed, the estimated remaining work is updated. When elements of the plan
are deemed unnecessary, they are removed. Only the Development Team can change its Sprint
Backlog during a Sprint. The Sprint Backlog is a highly visible, real-time picture of the work that
the Development Team plans to accomplish during the Sprint, and it belongs solely to the
Development Team.
Scrum does not consider the time spent working on Sprint Backlog Items. The work remaining
and date are the only variables of interest.
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 14
Increment
The Increment is the sum of all the Product Backlog items completed during a Sprint and all
previous Sprints. At the end of a Sprint, the new Increment must be “Done,” which means it
must be in useable condition and meet the Scrum Team’s Definition of “Done.” It must be in
useable condition regardless of whether the Product Owner decides to actually release it.
Definition of “Done”
When the Product Backlog item or an Increment is described as “Done”, everyone must
understand what “Done” means. Although this varies significantly per Scrum Team, members
must have a shared understanding of what it means for work to be complete, to ensure
transparency. This is the “Definition of Done” for the Scrum Team and is used to assess when
work is complete on the product Increment.
The same definition guides the Development Team in knowing how many Product Backlog items
it can select during a Sprint Planning Meeting. The purpose of each Sprint is to deliver
Increments of potentially shippable functionality that adhere to the Scrum Team’s current
Definition of “Done.”
Development Teams deliver an Increment of product functionality every Sprint. This Increment
is useable, so a Product Owner may choose to immediately release it. Each Increment is additive
to all prior Increments and thoroughly tested, ensuring that all Increments work together.
As Scrum Teams mature, it is expected that their Definition of “Done” will expand to include
more stringent criteria for higher quality.
Conclusion
Scrum is free and offered in this guide. Scrum’s roles, artifacts, events, and rules are immutable
and although implementing only parts of Scrum is possible, the result is not Scrum. Scrum exists
only in its entirety and functions well as a container for other techniques, methodologies, and
practices.
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 15
Acknowledgements
People
Of the thousands of people who have contributed to Scrum, we should single out those who
were instrumental in its first ten years. First there was Jeff Sutherland, working with Jeff
McKenna, and Ken Schwaber, working with Mike Smith and Chris Martin. Many others
contributed in the ensuing years and without their help Scrum would not be refined as it is
today. David Starr provided key insights and editorial skills in formulating this version of the
Scrum Guide.
History
Ken Schwaber and Jeff Sutherland first co-presented Scrum at the OOPSLA conference in 1995.
This presentation essentially documented the learning that Ken and Jeff had over the previous
few years applying Scrum.
The history of Scrum is already considered long. To honor the first places where it was tried and
refined, we recognize Individual, Inc., Fidelity Investments, and IDX (now GE Medical).
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 16
Revisions
This July 2011 Scrum Guide is different from its predecessor, the February 2010 Scrum Guide. In
particular, we have attempted to remove techniques or best practices from the core of Scrum.
These will vary based on circumstance. We will be starting a “Best Practices” compendium to
offer some of our experiences later.
The Scrum Guide documents Scrum as developed and sustained for twenty + years by Jeff
Sutherland and Ken Schwaber. Other sources provide you with patterns, processes, and insights
about how the practices, facilitations, and tools that complement the Scrum framework. These
optimize productivity, value, creativity, and pride.
Release notes covering the following differences between this and the February 2010 version
will be published elsewhere, including discussions on:
1. Release Planning
2. Release Burndown
3. Sprint Backlog
4. Product and Sprint Backlog Burndown
5. Commit is now forecast
6. Team (to Development Team)
7. Pigs and Chickens … the lore of Scrum
8. Ordered instead of prioritized
© 1991-2011 Ken Schwaber and Jeff Sutherland, All Rights Reserved Page | 17
INTRODUCING
CMMI DEVELOPMENT V2.0
TO PAN-INDIA SPIN
Kieran Doyle
Subject Matter Expert
CMMI Institute
This presentation contains proprietary information and may not be distributed without the express written permission of the CMMI Institute. ©2018 CMMI Institute. CMMIINSTITUTE.COM 5
DOING CAPABILITY AREAS AND PRACTICE AREAS
Governance
Building and Sustaining Capability
Implementation Infrastructure
Improving
Process Management
14
© 2018 CMMI Institute. All Rights Reserved.
BUILDING AND SUSTAINING CAPABILITY
• In the CMMI V2.0 model, the practice areas in the Building and Sustaining
Capability (BSC) capability area ensure that an organization's culture and
environment support processes and improvement that are persistent and
habitual.
• BSC practices address organizational persistence and habit from two different
perspectives:
o Governance (GOV)
o Implementation Infrastructure (II)
• BSC practices apply to the processes that the organization develops and uses and
NOT to the practice areas of the CMMI.
17
CMMI V2.0 TRANSFORMATION TO
A BUSINESS PERFORMANCE MODEL
Audience
• Anyone facing business
challenges and wants to address
them using the Capability
Maturity Model Integration
(CMMI) V2.0.
• Anyone who would like to
systematically, efficiently, and
effectively improve the
performance of their business by
adopting the CMMI.
Why Use The Guidance?
• Put an organization on the right
track to effectively apply CMMI
practices.
Introduction
A software project is made up of a series of phases. Broadly, most software projects comprise the
following phases.
During requirements gathering, the specific requirements of the software to be built are
gathered and documented. The requirements get documented in the form of System Requirement
Specifications (SRS) document. This document acts as a bridge between the customer and the
designers chartered to build the product.
Planning
The purpose of the planning phase is to come up with a schedule, the scope, and resource
requirements for a release. The planning phase is applicable to both development and testing
activities. At the end of this phase, both project plan and test plan documents are delivered.
Design
The purpose of the design phase is to figure out how to satisfy the requirements enumerated
in the System Requirement Specifications document.
Development or Coding
Design acts as a blueprint for the actual coding to proceed. It comprises of coding the
programs in the chosen programming language. In addition to programming, this phase involves
the creation of product documentation.
Testing
As the programs are coded, they are also tested. In addition, after the coding is complete,
the product is subjected to testing. Testing is the process of exercising the software product in
predefined ways to check if the behavior is the same as expected behavior. By testing the product,
an organization identifies and removes as many defects as possible before shipping it out.
Once a product is tested, it is given to the customers who deploy it in their environments.
As the users start the product in their environments, they may observe discrepancies between the
actual behavior of the product and what they were given to expect. Such discrepancies could end
up as product defects, which need to be corrected.
Testing, Verification, and Validation
Verification and Validation (V&V) is the process of checking that a software system
meets specifications and that it fulfils its intended purpose. Verification is the checking of or testing
of items, including software, for conformance and consistency with an associated specification.
Software testing is just one kind of verification, which also uses techniques such as reviews,
inspections, and walkthroughs. Validation is the process of checking what has been specified is
what the user actually wanted.
The purpose of testing is to uncover defects in the system. Testing is done by a set of people
within a software product organization whose goal and charter is to uncover the defects in the
product before it reaches the customer. When a product is tested with appropriate and realistic tests
that reflect typical usage patterns by the intended users, the chances of the product satisfying the
customer’s requirements is much higher. While testing does not guarantee zero defects, effective
testing certainly increases the chances of customer acceptance of the software.
Integration testing
Integartion testing exposes defects in the interfaces and interaction between integrated
components (modules). The individual subsystems should be integrated and tested together before
a full blown system test can be done.
System testing
System testing tests an integrated system to verify that it meets its requirements. Before
the product is deployed in the customer’s environment, the product vendor should test it as an
entire unit to make sure that all the software requirements are satisfied by the product that is
developed. System testing can be sub-divided into:
o Functional testing
o Non-Functional testing
System integration
System integration testing verifies that a system is integrated to any external or third party
systems defined in the system requirements.
Acceptance testing
Acceptance testing can be conducted by the end-user, customer, or client to validate
whether or not to accept the product. Acceptance testing may be performed after the testing and
before the implementation phase.
Functional Testing
Functional testing involves testing a product’s functionality and features. It helps in verifying what
the system is supposed to do. It aids in testing the product’s features and functionality. It has only
two results as far as requirements fulfillment is concerned-met or not met. If requirements are not
properly enumerated, functional requirements may be understood in many ways. Hence functional
testing should have very clear expected results documented in terms of the behavior of the product.
Functional testing comprises simple methods and steps to execute the test cases. It requires in-
depth customer and product knowledge as well as domain knowledge so as to develop different
test cases and find critical defects, as the focus of the testing is to find defects. Failures in functional
testing normally result in fixes in the code to arrive at the right behavior.
Functional testing is performed an all phases of testing such as unit testing, component
testing, integration testing and system testing. Functional testing requires design and architecture
knowledge for better understanding. The main focus in functional testing is on the product features.
As functional testing is performed at various testing phases, there are two obvious problems. One
is duplication and the other one is gray area. Duplication refers to the same tests being performed
multiple times and gray area refers to certain tests being missed out in all the phases. A small
percentage of duplication is advisable, as different people from different teams test the features
with different perspectives, yielding new defects. Gray areas in testing happen due to lack of
product knowledge, lack of knowledge of customer usage, and lack of co-ordination across test
teams.
There are multiple ways system functional testing is performed. Some of the common techniques
are given below,
Design/architecture verification
Business vertical testing
Deployment testing
Beta testing
Certification, standards, and testing for compliance.
Design/Architecture Verification
In this method, the test cases are developed and checked against the design and architecture
to see whether they are actual product-level test cases. System level test cases are created first and
verified with design and architecture to check whether they are product-level or component-level
test cases. Functional system test focuses on the behavior of the complete product. Functional
testing is performed at various test phases, it is important to reject the test cases and move them to
an earlier phase to catch defects early and avoid any major surprise at a later phase.
Deployment Testing
System testing is the final phase before product delivery. Testing that happens in a product
development company to ensure that customer deployment requirements are met is called offsite
deployment. Deployment testing after the release of the product by utilizing the resources and setup
available in customers’ locations is called onsite deployment.
Beta Testing
There are many reasons for a product not meeting the customer requirements. To reduce
the risk of rejection, which is the objective of system testing, periodic feedback is obtained on the
product. Sending the product that is under test to the customers and receiving feedback is called
beta testing. The activities that are planned and executed according to a specific schedule are called
beta program. Defect fixes are sent to customers as soon as problems are reported and all necessary
care has to be taken to ensure the fixes meet the requirements of the customer.
The following examples are different types of testing that should be considered during Functional
System testing:
User interface testing involves testing how the application and the user interact. This includes
how the application handles keyboard and mouse input and how it displays screen text, images,
buttons, menus, dialog boxes, icons, toolbars and more. Functional testing is commonly done by
human testers, but is made much easier and more reliable by tools.
Performance testing is a testing that is performed, from one perspective, to determine how fast
some aspect of a system performs under a particular workload. It can also serve to validate and
verify other quality attributes of the system, such as scalability and reliability.
Sanity test is a basic test to quickly evaluate the validity of a claim or calculation. It is a very
brief run-through of the functionality of a computer program, system, calculation, or other analysis,
to assure that the system or methodology works as expected, often prior to a more exhaustive round
of testing.
Smoke testing is a preliminary to further testing, which should reveal simple failures severe
enough to reject a prospective software release. Sometimes the tests are performed by the
automated system that builds the final software. In software testing, a smoke test is a collection of
written tests that are performed on a system prior to being accepted for further testing. This is also
known as a build verification test.
Exploratory testing seeks to find out how the software actually works, and to ask questions
about how it will handle difficult and easy cases. The testing is dependent on the tester’s skill of
inventing test cases and finding defects. The more the tester knows about the product and different
test methods, the better the testing will be.
Regression testing is any type of software testing which seeks to uncover regression bugs.
Regression bugs occur whenever software functionality that previously worked as desired stops
working or no longer works in the same way that was previously planned. Typically regression
bugs occur as an unintended consequence of program changes. Common methods of regression
testing include re-running previously run tests and checking whether previously fixed faults have
re-emerged.
Test Automation
Developing software to test the software is called test automation. Test automation is the use of
software to control the execution of tests, the comparison of actual outcomes to predicted
outcomes, the setting up of test preconditions, and other test control and test reporting functions.
Commonly, test automation involves automating a manual process already in place that uses a
formalized testing process. The software that is used to carry out the task of testing is generally
referred to as Testing Tools.
There are tools available that help the testers in the automation of the GUI which reduce
the test time as well as the cost, other test automation tools support execution of performance tests.
Testing tools can help automate tasks such as product installation, test data creation, GUI
interaction, problem detection (consider parsing or polling agents equipped with oracles), defect
logging, etc., without necessarily automating tests in an end-to-end fashion.
Many tools are available in the market and IBM Rational Functional Tester (RFT) is one
of them. RFT is a GUI test automation tool. It is an automated, functional and regression testing
tool. The tool provides capabilities to record a user’s interactions with an application and create a
script representing the same. The user can modify the script to enhance or make specific changes
if required. The user can playback this script to repeat the same sequence of actions that was
performed during recording. The user can create a script to represent a test case. This test case can
be considered to have passed if the script executes without any failure, else it is a failed test case.
In an iterative development cycle, multiple builds of the application are delivered in short
time frames. Manually testing the existing and new functionality in each of the new builds can be
a challenge. RFT provides the ability to create a test harness to replay test cases on multiple builds
to ensure that both existing and new functionality work as designed. RFT supports both java and
visual studio .net scripting. Let us consider a simple example of testing a .net application using
RFT,
The visual studio .net application has Record button which records all the processes that is
manually done in testing the application. The scripting in VB.net stores all the processes
executed. The option of creating separate test projects makes it very easy for the tester to
use this tool on a range of applications.
There is a recording monitor available which gives various controls to the user while
manually testing the application like pause, stop, validate, etc...
As the tester performs manual testing, all the steps are recorded and coded as VB .net
scripts on the recording monitor
The ease of data driven testing allows the tester to use the wizard to validate data and store
it in tables.
Data verification and action wizard helps one to store a specified data and check it
repeatedly when further testing is performed in future.
When the recording is done the editor page shows the list of scripts, the test code, test data,
and all the objects that were created during interaction.
The tester can peform a number of operation in the editor, then running the test scripts
automatically validate the application for new test data
A Log page generated gives the test results of the test run performed with the details in a
specified format.