SCRUM vs.

Waterfall: Point and Counterpoint

Note: CMM is a service mark of the Software Engineering Institute at Carnegie Mellon
University.

Shawn Presson, Director of Organizational Practice, ITS Services, Inc. says:

Why, why, why does everything think the waterfall life cycle is exclusively linear? It was
designed to be extremely iterative, it allowed for the fact that there may be parallel, non-
sequential sets of requirements under development (hence multiple, simultaneous
"waterfalls"), it in no way assumed that requirements will be static during a project. It was
only after the DoD got its hands on it that this interpretation of the model came about, but
why are we listening to the DoD on something like this? ...

Why, why, why do we assume that "repeatable" and "defined" models preclude emergence?
These terms are obvious reference to the CMMs, but the CMMs don't lock you into waterfall,
spiral, or any other life cycle or methodology. With reference to requirements, the CMM
says, "don't let your junior developer bet the farm on meeting a requirement that the
project doesn't know about." Does SCRUM cover this point? Great, then SCRUM is the
METHOD whereby this important principle (NOT "process") contained in the model is carried
out...SCRUM (or JAD, or whatever) is the METHOD whereby your company avoids
inadvertently lying about its capabilities (which is what the CMM planning is largely about,
anyway.)... (See full text)

Ken Schwaber of the Agile Alliance responds:

Reading Shawn Presson’s comments regarding CMM, waterfall, and agile reminded me of
Barry Boehm’s comments at XP/Agile Universe’02 and the comments of several CMM
authors at INCOSE’02. I was reminded of the good intentions and sound footings of CMM
and Waterfall, both initiated to address problems in systems development and systemic
project failures rates that have only marginally improved over the decades...
...CMM, waterfall spring from a different theoretical basis than agile processes. Increased
precision and definition are at their core, which is one of the alternatives for controlling a
process. However, this approach only works when the definition and the problem have a
mapping approaching 1:1 and the problem domain (technical, requirements, and people) is
relatively simple: “It is typical to adopt the defined (theoretical) modeling approach when
the underlying mechanisms by which a process operates are reasonably well understood.
When the process is too complicated for the defined approach, the empirical approach is the
appropriate choice.” Process Dynamics, Modeling, and Control, Ogunnaike and Ray, Oxford
University Press, 1992.

The agile process is based on the empirical approach, accepting the complexity of the
problem and addressing it through frequent inspection and constant adaptation...Empiricism
is seen even in the Agile Manifesto, where the signatories pose agility as an empirical
counterpoint to a defined approach, rather than as a new set of defined absolutes. The
various agile processes implement empiricism with the underlying practices of iterative
development, frequent inspection of status and increments of work, self-organizing teams,
emerging architecture and requirements, and solid collaboration... (See full text)
From: Ken Schwaber [ken.schwaber@verizon.net]
Sent: Sunday, September 15, 2002 4:53 PM
To: Jeff Sutherland
Subject: RE: Waterfall methodology

Jeff,

Reading Shawn Presson’s comments regarding CMM, waterfall, and agile reminded me of Barry
Boehm’s comments at XP/Agile Universe’02 and the comments of several CMM authors at
INCOSE’02. I was reminded of the good intentions and sound footings of CMM and Waterfall,
both initiated to address problems in systems development and systemic project failures
rates that have only marginally improved over the decades.

I was vastly amused by Presson’s suspicion that we’re only contrasting agile and previous
approaches to sell books. If I could even afford to drink Starbuck’s coffee with book royalties, I
would be pleased. The contrast is painted from two perspectives: the current state of CMM and
waterfall, and the theoretical contrast between CMM/waterfall and agile processes.

I was challenged at INCOSE’02; “how do you intend to prevent agile from being turned into a
pile of junk, as has happened to CMM.” CMM was transformed from its principles and vision
into its practices, KPA’s, certification, and institutionalization by DoD implementations,
consultants, carpetbaggers and everyone else trying to make a buck. The same thing happened
with structured methods, information engineering, and CASE tools that would generate code
from design and design from code. All reduced to overhead and rubble. Of course the kernels of
modeling, precision of expression, thinking before coding, and having a process persist, but for
many the weight of the commercial implementations have done their damage.

So, some of the contrast and reaction of “agilists” to CMM and waterfall are to current
implementations, not the vision nor the intentions. All of you have to is look at CMM as
implemented at most major organizations or waterfall as implemented in such methodologies as
Navigator to understand the damage. I was brought into one CMM Level 3 shop because it was
so badly implemented that development had literarally stopped. Problems couldn't be reduced to
an adequate level of agreement and definition to proceed with coding. However, the rest of the
contracst occurs because CMM, waterfall spring from a different theoretical basis than agile
processes. Increased precision and definition are at their core, which is one of the alternatives for
controlling a process. However, this approach only works when the definition and the problem
have a mapping approaching 1:1 and the problem domain (technical, requirements, and people)
is relatively simple: “It is typical to adopt the defined (theoretical) modeling approach when the
underlying mechanisms by which a process operates are reasonably well understood. When the
process is too complicated for the defined approach, the empirical approach is the appropriate
choice.” Process Dynamics, Modeling, and Control, Ogunnaike and Ray, Oxford University
Press, 1992.
The agile process is based on the empirical approach, accepting the complexity of the problem
and addressing it through frequent inspection and constant adaptation. Empiricism is seen even
in the Agile Manifesto, where the signatories pose agility as an empirical counterpoint to a
defined approach, rather than as a new set of defined absolutes. The various agile processes
implement empiricism with the underlying practices of iterative development, frequent
inspection of status and increments of work, self-organizing teams, emerging architecture and
requirements, and solid collaboration. You will not find many detailed task definitions, pert
charts, or assignments in agile processes; by contrast, CMM and waterfall progressively
implement themselves by increasing the amount of detail.

Agile is not a silver bullet. It is simply a correct approach to complex process control. It reflects
a lean manufacturing approach to software development, and – as manufacturing started doing
twenty years ago – eschews the Taylor model of defined process control.

I expect the success of agile processes to be challenged by the same degenerative, buck-making
processes that hurt CMM and waterfall. I expect adopting agile processes to be quite difficult in
many organizations because of the degree of change required; indeed, our agile processes have
primarily succeeded when organizations absolutely need the projects to succeed and can’t risk
failure. I expect agile to also be challenged by the abysmal state of engineering practices in our
industry, such as code management, release control, testing, and builds. Agile implementations
highlight poor engineering practices immediately, whereas model and definition-based
approaches let this failure slide until near the end of the project.

Shawn Presson is a kindred spirit, not a foe. We all are trying to improve the software
development process.

Ken Schwaber
Agile vs. XP: The Differences and Similarities

by Robert C. Martin

In common usage the terms XP and Agile are sometimes used to mean the same thing, and sometimes to mean
very different things. What are their origins, and what are their true definitions. Is Agile the same as XP? Is XP
Agile? We’ll answer those questions here.

XP

The term XP predates the term Agile by several years. XP stands for Extreme Programming, and is a suite of
practices, principles, and values invented by Kent Beck in the late ‘90s. Nowadays the principles and values are not
as well known, but the practices survive. Those practices are:

The Planning Game

Development proceeds in very short iterations, typically 1-2 weeks in duration. Prior to each iteration features are
broken down into very small stories. Stories are estimated by developers and then chosen by stakeholders based
on their estimated cost and business value. The sum of story estimates planned for the current iteration cannot
exceed the sum of estimates completed in the previous iteration.

Whole Team

The team consists of developers, business analysts, QA, project managers, etc. The team works together in a lab
space or open area where collaboration and communication are maximized.

Acceptance Tests

Stories and features are defined by automated tests written by the business analysts, and QA. No story or feature
can be said to be done until the suite of acceptance tests that define it are passing.

Small Releases

Systems are released to production, or pre-production very frequently. An interval of 2-3 months is the maximum.
The minimum can be once per iteration.

Continuous Integration
The whole system is built and tested end-to-end several times each day. While new tests are made to pass, no
previously passing tests are allowed to break. Developers must continuously keep the system in a deployable state.

Collective Ownership

Code, and other work artifacts, are not owned by individuals. Any member of the team may work on any artifact at
any time.

Coding Standard

Code, and other work artifacts, look as if they were written by the team. Each team member follows the team
standard for format and appearance of the artifacts.

Metaphor

Names within code and other work artifacts are chosen to be evocative of the system being created.

Sustainable Pace

Building software is a marathon, not a sprint. Team members must run at a rate they can sustain for the long haul.
Overtime must be carefully controlled and limited. Tired people do not win.

Pair Programming

Code and other work artifacts are produced by pairs of individuals working together. One member of the pair is
responsible for the task at hand, and the other helps out. Pairs change frequently (every two hours or so) but
responsibility stays with the owner.

Test Driven Development

Developers are not allowed to write production code until they have written a failing unit test. They may not write
more of a unit test than is sufficient to fail. They may not write more production code than is sufficient to pass the
failing test. The unit tests are maintained and executed as part of the build process. No previously passing unit test
is allowed to fail.

Refactoring

Code, and other work artifacts, are continuously reviewed and kept as clean as possible. It is not sufficient that
code works; it must also be clean.
Simple Design

The simplest design that suffices for the task at hand, is the right design. More complex and general designs may
become useful later, but not now. We do not wish to carry the weight of that complexity while it is not needed.
Sufficient for the day are the complexities therein.

Agile

XP was one of several so-called “lightweight” methods. Others in that category were SCRUM, DSDM, Crystal, FDD,
etc. Each of these methods was created as a reaction to the consistent and increasing failure of the heavier,
waterfall derived methods. Each of these lightweight methods had it’s own set of principles, values, and practices.
Each was championed by its own group of adherents. In February of 2000, Object Mentor called a meeting of these
adherents in Snowbird, Utah. The purpose of that meeting was to see whether or not we could merge the values
and principles of the various methods into a consistent whole.

Remarkably, the attendees of that meeting managed to set aside their differences and worth together to form a
manifesto, and then a set of principles that encompassed all of the lightweight methods. They also came up with a
name. They called it: Agile. The Agile Manifesto describes the values we chose:

Manifesto for Agile Software Development

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we
have come to value:

 Individuals and interactions over processes and tools

 Working software over comprehensive documentation
 Customer collaboration over contract negotiation
 Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning,
Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin, Steve Mellor, Ken Schwaber,
Jeff Sutherland, Dave Thomas

This set of values was accompanied by a set of principles, agreed to by all:

Principles behind the Agile Manifesto

We follow these principles:

  Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
 Welcome changing requirements, even late in development. Agile processes harness change for the
customer's competitive advantage.
 Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to
the shorter timescale.
 Business people and developers must work together daily throughout the project.
 Build projects around motivated individuals. Give them the environment and support they need, and trust
them to get the job done.
 The most efficient and effective method of conveying information to and within a development team is
face-to-face conversation.
 Working software is the primary measure of progress.
 Agile processes promote sustainable development. The sponsors, developers, and users should be able to
maintain a constant pace indefinitely.
 Continuous attention to technical excellence and good design enhances agility.
 Simplicity--the art of maximizing the amount of work not done--is essential.
 The best architectures, requirements, and designs emerge from self-organizing teams.
 At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its
behavior accordingly.

Conclusion

XP is a set of practices that conform to the values and principles of Agile. XP is a discrete method, whereas Agile is
a classification. There are many Agile methods, XP is just one of them.

Having said that, none of the other Agile methods are as well defined, or as broad in scope as XP. Scrum, for
example, is roughly equivalent to XP’s Planning game practice, with elements of Whole Team. While there are
differences in the details, it is fair to say that Scrum is a subset of XP. Indeed, many Scrum teams augment their
process by adding in many of the XP practices such as Acceptance Testing, Pair Programming, Continuous
Integration, and especially Test Driven Development.

Of all the Agile methods, XP is the only method that provides deep and profound disciplines for the way developers
do their daily work. Of those disciplines, Test Driven Development is the most revolutionary and impactful.
Agile vs. RUP

At our company we have these meetings, where people from the same competence in a region come
together to talk about all kinds of stuff, from business related to more fun related stuff. Because I went to
the Dutch Developer Days 2005 with Edwin Waegemakers, we were asked to do a little presentation. We
decided to give three small presentations, one about Team System, one about integration strategy and
one about Agile Development. I would do first and last.

I've written about the Agile presentation at DevDays before and planned to use it in my own presentation,
but a little less on the archetypes and a little more my personal experience. The plan was to bring it lightly
and at the same time trigger our RUP 'specialists' as an Agile hooligan, and I can say it worked pretty
well. I placed Agile strongly against RUP, although in practice this of course isn't the case. Or at least not
as hard as I had put it.

Agile vs. RUP

For a lot of developers documentation can be a drag to write. I have a personal experience on a project I
joined, where they over-used RUP. Normally you get a pile of documentation of about 2" high. On the
project, I got a pile of about 10" high. If I wanted to read that through, before actually doing something
useful. The project had just started, and there wasn't anything in it like functional specs. Just project
guidelines and rules and technical visions, etc, etc, etc. So I used that in the presentation, with the Agile
tenet to produce working software over extensive documentation. I highlighted extensive, as some people
got the idea Agile has no documentation at all. That triggered some reactions!

Fun thing was, that our director just had seen a presentation last week, where RUP was presented as thé
answer to all problems in projects, sort of speak. This week I presented it as if RUP was about just
documentation, and Agile development would really work. His conclusion was that now he knows he's
getting old, as he had never seen (technological) advancements update so fast. Last week RUP was
everything, now it's already outdated and replaced. ;-)

Project swing
Of course Agile isn't a replacement for RUP and you cannot appoint one as a clear winner. I don't think
that's necessary either. I think RUP is great, but you have to have experienced people doing RUP
projects as you don't want to overload everyone with documentation, but also have to carefully look at
what the customer wants. RUP also uses iterations, but for some reason I have the feeling Agile
development can better understand and react to what the customers needs, not what he thinks he wants.
That comes to show in this funny picture of a swing. It's funny to see how everyone within a project has
(or can have) a different view on the swing. I think it's hard to get the same swing in every picture. But my
believe is, it's much harder to retrieve from a customer to what he really wants and needs. To get the
same picture in how a customer explains it and what a customer really needed, you need good iterations
and a lot of communication. For some reason, I have the feeling communication in RUP is more through
documents where as in Agile it's face to face communcation between the customer and the entire team.
And I think that's an important difference.

Of course everything should still be managed, also documentation. ;-)

I'm particullary interested in how to manage the customer and/or final deadline. Once the customer gets
the hang of changing and adding functionality, how can we manage them and the project so that we can
deliver what we promised up front, and still meet some final deadline. Where is the line, between making
agreements with a customer, and kind of just see what happens?

For those who want to know more about Agile Development :

• Manifesto for Agile Software Development - Some big names are behind Agile.
• Martin Fowler's New Methodology
• Agile via SCRUM can be found on ControlChaos
• Agile via Extreme Programming
• Agile Methods
• Darrell Norton's weblog
• Steve Eichert's weblog
Agile vs. Lean

Deutsch Español Français ‫ العربيه‬中国

A recent discussion on the Scrum Development list (Start of Discussion) provides a

good follow up to my parting words in The Art of Obstacle Removal about agile
practices themselves becoming obstacles. I have excerpted a small amount of the
discussion and added my own comments here.

In the agile community, most of us have bought into and adopted the Agile mental
model. But that mental model has assumptions embedded into that may not be correct
under all circumstances. Part of that mental model includes the efficacy of the work
item backlog as a tool for tracking, prioritizing and communicating work to be done in
the future.

I will be the first to say that Agile is far better than waterfall in almost every situation
(the exception being the mythical project with stable requirements, business
environment, technology and team).

That said, let's look at backlogs from another perspective: if a backlog was the only
thing you delivered to the customer, would they pay for it? If you spent even just a few
hours coaching a business representative to build a backlog, but there was no team to
implment it, would that person really find value in the backlog itself? Would they be
able to deliver ROI just from a backlog? I think the answer is a clear "no".

That set of questions may seem silly, but from a lean perspective, they are among the
most important questions. Is an artifact/process value-added or non-value-added?

Since the backlog is clearly not a value added artifact/process (pause for effect)... it is
waste!

When one is doing agile effectively, the backlog may in fact be one of the larger sources
of waste! If the team has a stable velocity, there will come a point when the backlog
becomes the constraint on the overall cycle time going from idea to ROI. If the backlog
is large/long, and as Mary Poppendieck said if there is more than ...maybe two or three
sprints of work.... in there, then it may be time to find ways of constraining the size of
the backlog. I have two suggestions:

Capacity of Team(s):

Find a way to increase the capacity of the team so that the backlog size reduces and
then goes into a steady state. This may mean augmenting the staff.

Gate Backlog Items by ROI

(This is just theory to me at this point.) Make it a condition that all backlog items must
have a positive ROI. In other words, the cost of building the backlog item needs to be
less than the value delivered, taking into account time value of money. Don't let items
onto the backlog unless this positive ROI can be demonstrated.

I believe the lack of this second discipline (assigning ROI to work items) is one of the
main reasons that most agile methods such as Scrum allow an unlimited backlog size:
there is no realistic way to gate the work that gets onto the backlog!

Until teams get to Agile nirvana (stable velocity, continuous delivery of business value,
high-performance cohesive teams), having an unlimited backlog seems like a very
reasonable thing: it's not the constraint or the primary source of waste. However,
eventually the backlog will become a primary source of waste and it needs to be treated
in a disciplined manner.

With a stable and well-functioning team, what other ways might there be to reduce the
size of the backlog so that cycle time is substantially reduced?

Related articles can be found in the following categories: