Professional Documents
Culture Documents
Delivery Case Study
Delivery Case Study
Levitt
Chris Fry
Steve Greene
Colleen Kaftan
Professor Raymond E. Levitt, Chris Fry and Steve Greene of Salesforce.com, and Colleen Kaftan prepared this case under the
auspices of the Stanford Collaboratory for Research on Global Projects. CRGP cases are developed solely as the basis for classroom
discussion, and are not intended to serve as endorsements, primary data sources, or illustrations of either effective or ineffective
management practices.
Copyright 2009 Stanford Collaboratory for Research on Global Projects, Stanford, California.
Clayton M. Christensen articulated the concept of disruptive innovation in The Innovators Dilemma:
When New Technologies Cause Great Firms to Fail (HPSP, 1997), and elaborated it along with his
colleagues in many subsequent publications, courses, and analytical tools.
This section draws on Raymond Levitt, Lecture Notes from Converting Strategy Into Action, Stanford
Advanced Project Management executive program. See http://apm.stanford.edu.
The tools and processes for doing so were encoded in the Project Management Body of Knowledge
(PMBOK), an internationally recognized standard for professional certification maintained by the Project
Management Institute.
See, for example, Michael Hammer and James Champy, Reengineering the Corporation: A Manifesto for
Business Revolution (New York: HarperBusiness, 2001), and Thomas H. Davenport, Process Innovation:
Reengineering Work Through Information Technology (Boston: Harvard Business School Press, 1993).
each customers specific needs. Others focused on developing a capacity for continuous
product innovation. A few brave competitors opted to offer both innovative products and
integrated solutions, with all the organizational complexities such a dual strategy implied.
The new array of project challenges ranged from large, complex, initiatives to smaller
concurrent daily operational improvements. The single common denominator was the
pace of competition. The most dynamic firms in the fastest growing sectors of the global
economysemiconductors, computers, financial services, IT, and non-profits, for
exampleneeded a discipline for managing large numbers of big and small projects in
rapidly changing markets and technologies. Product, project, and program managers
needed a more flexible, process-light framework in which to apply their organizational
methods and tools.
The big CRM solutions companies adopted a waterfall approach to software
development, whichwhile more nimble than the traditional aircraft, construction, and
pharmaceutical industry methodologiesstill relied on the sequential performance of
largely separate functional activities, based on a predefined plan and budget. Project
priorities were established centrally, usually by customer-facing product managers who
determined user needs and then worked across functional lines to push the project
forward. Exhibit 2 shows the typical sequence of functional activities in waterfall
development.
all these other things we had to do, it grew into this more waterfall
process, where everything went down the line.
The R&D group eventually organized around feature teamsproject groups who were
assigned to develop centrally-designated user capabilities. Whenever the product
managers identified a desirable feature to develop, they would assign people from across
the functions to the team charged with building it. People were often assigned to several
teams at once, and many had trouble prioritizing their work across teams.
By the time Steve Greene and Chris Fry came to Salesforce.com in early 2005, the R&D
headcount had grown to well over 150. Fry described one drawback he observed in the
feature team model:
Thered be a weekly meeting for each team. Everyone would show up and
discuss all the problems for an hour, and then just go back to their desks.
There was no accountability. Youd go to all the feature team meetings
you were assigned to, and the next week youd go back and say, What
was it I promised to do? And nobody was tracking if you were on 20
teams or just one.
Steve Greene elaborated:
So basically, youre pushing prioritization down to the individual
contributors, because theyre on five teams and they would decideon a
daily basiswhats the most important thing, and thered be confusion
across the organization because everybody had a different idea of what the
highest priority was. And management didnt have visibility into this,
because they couldnt attend the 20 meetings every week.
Parker Harris began to notice other problems with the feature team / waterfall
approach:
Wed set formal user expectations and then product management would
build a prototype and write a functional spec. Development would then
write a technical spec, and on down the line. The timelines were fixed, and
everybody would end up padding their estimates and still being late, and
then blaming the delay on other people upstream in the process.
At the end, everyone blamed QA [quality assurance], because QA is at the
end of the waterfall. Then everyone started pointing fingers at each other,
and it also caused really weird behavior where people would work outside
the process: Maybe I shouldnt pay attention to this process, and just be
late with my stuffthe release wont go out without it, and if my areas
late, maybe well get more attention later onand so on.
By summer of 2006, the R&D headcount reached nearly 300. Many new hires came from
bigger companies such as SAP, with centralized approaches to software development.
They brought with them a big company culture, according to Steve Greene, so that by
mid-2006, even though Salesforce.com was considerably smaller than most of its
competitors, We were having just as much difficulty delivering releases as the big
5
companies were. Case in point: the schedule for Release 144 had already slipped several
times, and some of the features were still not ready for shipping.
Greene described the internal effect of the delay:
We had huge morale problems. Because if youre a technologist, you want
to build something useful, and you want to get it out there to your
customers to use as quickly as possible. You dont want to just talk about
it. And we had people who had been at salesforce more than a year
people we hired from larger organizations, with the promise that they
could accomplish things quickly heresome of them had never actually
released a feature to production.
Fry agreed:
The institutional knowledge of how to do releases wasnt as strong as it
should be. And our development people like to create things that people
use, but we werent getting that positive feedback into the team, so there
was low morale across the board.
On the customers side, the service outages of late 2005 and early 2006 exacerbated the
development issues. Customers depended on dial-tone reliability for on-demand
services, and when they were unable to access their data, they began to question the new
business paradigm. If Salesforce.com couldnt assure customers of reliable service, and
couldnt even deliver new features on a regular basis, why should any customer base its
operations on the hosted services model?
The Shinkansen Project
In summer of 2006, Parker Harris asked Steve Greene and Chris Fry to explore the
release train development model that many considered to be the driving force behind
the rapid development capabilities at eBay. To underscore the ugency of speeding up
Salesforce.coms development process, the three named their initiative Shinkansen, after
the famed Japanese bullet trainsarguably, along with Frances TGV and Germanys
ICE, the fastest in the world.
An eBay-commissioned joint benchmarking study in 2006-2006 described the release
train system as follows:
As the term implies, this process is like a train that has a fixed number of
seats for passengers and a pre-set schedule. Companies decide in advance
the number of releases theyd like to issue each year, as well as the size of
each release. The release size is usually based on the required level of
effort as defined by person days, or developer days. Teams of
developers work furiously to complete their new products in time for a
certain release train. If they miss the train, they must wait for the next
release. 6
Release trains departed every two weeks on schedule at eBay. When a releasable feature
made it onto the train, it would then integrate with other new features and the existing
platform much as rail cars couple together, and undergo extensive quality testing. This
late integration model proved less attractive for Salesforce.com, in part because the
company had recently developed an automated testing capability for continuous
integration of most new code. The developers were responsible for testing and integrating
whatever fell outside the automated process. Many considered this capability to be an
important competitive weapon for Salesforce.com.
Fry and Greene also encountered widespread resistance to the idea of an externallydeveloped process imposed from on high. After three months of trying to plan how to
introduce the release train methodology, the project group decided to abandon the cause.
On the heels of this less-than-promising endeavor, Greene and Fry decided to propose an
agile development approach in August 2006. Both were familiar with the method, but
they did further research to prepare a detailed description for Harris, and set off to
persuade him to try it. If Harris agreed, they hoped to turn the Shinkansen team into an
agile development team.
http://www.prtm.com/uploadedFiles/Strategic_Viewpoint/Articles/Article_Content/PRTM_eBay_benchm
arking.pdf
7
Information in this section comes from many sources, including K. Beck, Extreme Programming
Explained (Addison Wesley, 2000), M. Poppendieck and T. Poppendieck, Lean Software Development
(Addison Wesley, 2003), and M. Cohn and D. Ford, Introducing an Agile Process to an Organization,
Computer, June 2003, pp. 74-78. See also http://www.mountaingoatsofware.com/daily_scrum
cycles, each of which followed a specific rhythm. (Exhibit 4 charts the activities in a
typical sprint-to-release calendar. Exhibit 5 distills the scrum lifecycle.)
Every thirty-day sprint began with a planning meeting, in which a product owner
identified desirable features, or a product backlog of user stories, in order of priority.
(A typical user story might read, I can predict customer purchasing patterns, or I can
compile hourly sales by distribution channel.) The scrum team evaluated the backlog
list, agreed on the number of top priority items they could deliver during the sprint, and
committed to doing so. Once that commitment was in place, no further requirements or
changes could be introduced during the month-long sprint. The scrum team of no more
than 12 peopleincluding product managers, developers, quality engineers, user
experience designers, usability testers and technical writersworked together every day
to accomplish their sprint commitments. They used highly visible shared tools, such as
wall-mounted task boards and Excel spreadsheets, to keep track of their progress in
completing the tasks in a release burndown. Team members who had completed their
tasks would step in to assist other team members who needed help completing their tasks.
Each 30-day sprint ended with a sprint review, in which team members demonstrated
their accomplishments to the team and management during the sprint. Sprint reviews
were meant to be simple, natural discussions of new capabilities, rather than highly
prepared formal presentations. This reinforced the Agile principle of only including code
that is truly donei.e., coded, usability tested, QA tested and documented, and thus
ready to be integrated into the next release.
The objective was to deliver fast and deliver early, while avoiding the typical scope
creep and roadblocks that tended to plague extended waterfall development projects.
Every sprint could produce shippable capabilities, whether the company was ready to
release them or not. One intended side effect was to eliminate the crisis management and
panic associated with more ambitious, sequential, waterfall-based feature releases with
lengthy (and often unpredictable) development calendars.
Roles and Rituals
Agile processes required designating a product owner, a scrum master, and scrum team
members (often for multiple teams coordinated by a scrum of scrums) for each sprint.
People tended to stay in these roles over time, so that working relationships and team
procedures could evolve to fit the team members preferences and the requirements of
their specific tasks.
Perhaps the most critical daily event for a scrum team was a short morning stand-up
meetingtypically lasting no more than about 15 minuteswhich set the context for the
coming days work. These meetings were held at the same time and the same location
every day, and were open to anyone who wanted to attend. There was a strict distinction
between those who were involved and those who were committed in terms of their
rights to participate in the meetings.
Scrum experts used a chicken and pig metaphor to describe the difference:
There is an old joke in which a chicken and pig are talking and the chicken
says, Lets start a restaurant. The pig relies, Good idea, but what should
we call it? How about Ham and Eggs? says the chicken. No thanks,
says the pig, Id be committed, youd only be involved.
Only committed scrum team members were allowed to talk at daily meetings. Each one
had to answer just three questions, designed to reinforce team commitments and remove
roadblocks:
The scrum masters job was to remove the impediments every day, so that the team could
meet its sprint commitments. Allowing chickens to listen in on daily meetings was
meant to promote visibility throughout the organization without impeding the teams
work.
Was Harris correct in suggesting they could do it without outside help from consultants?
And would any developers agree to devote their time and energy to a changeover, with
Release 144 still stuck in project management limbo? As they thought about how to
proceed, Fry and Greene recalled Harriss comment at their August meeting:
Salesforce.com hasnt made a big investment in processes because we havent known
which processes to implant. If Harris was so sure that the Agile was best, how much of
the companys time, energy, and money could they expect him to invest into making it
work?
10
Scrum doesnt account for the fact or the reality of the waterfall. You cannot
deny this by superimposing scrum over it.
It seems like we spend more time talking about scrumthan we spend time
talking about and working on salesforce.com.
Stop trying to implement scrum, and look at how many releases we can really do
in a year.
To make matters worse, by the middle of the third sprint in December 2006, it was
obvious that the teams would not be able to complete all the prioritized features for
Release 144. This created a dilemma for Greene and Fry: should they finish the sprint on
time, and deploy what was done and releasable, or should they ask Parker Harris and
the executive team for a delay?
What Next?
On the one hand, to push the release out yet again could undermine the message that
Salesforce.com really intended to implement ADM throughout the R&D organization and
beyond. But to demo an incomplete set of features might introduce ADM in less than
favorable light. Would ADM then go the way of Shinkansen and other failed initiatives?
If so, what would be the consequences, internally and in the marketplace?
Most people involved in the rollout believed that they could use the ADM process to
teach, adapt, and reinforce ADM itself. This would take timeprobably measured in
multiple sprints. But where should they go from here? How should they identify the
priorities for each successive sprint? For example, should they focus on bringing the most
resistant, poor-performing groups up to speed, or start with the ones that were doing
somewhat better and eager to learn more? Would it be better to have most groups
performing adequately, or should they work on creating a few stars to serve as
examples for the rest of the organization? And would Salesforce.com be able to keep
adapting ADM to meet the needs of the organization as it continued to grow?
Finally, and more immediately, how should they handle Release 144not to mention
146, 148, and others still to come? Fry and Greene thought the companys continued
success could depend on making sure they were asking the right questions, and then
getting the answers right.
11
EVP Marketing
Senior Director UI
Documentation
Manager
Chris Fry
VP Platform
VP Applications
Senior Director
Core
SVP QE
VP Quality
Assurance
Steve Greene
Senior Director,
Tools & Agile
Process
Usability
Manager
SFA
Content
Platform SoS
Applications SoS
CRM
SVP Development
Apex
Sandbox
API
Core SoS
Core
Automation
Systest
12
13
Our highest priority is to satisfy the customer through early and continuous delivery
of valuable software.
Welcome changing requirements, even late in development. Agile processes harness change for
the customer's competitive advantage.
Deliver working software frequently, from a couple of weeks to a couple of months, with a
preference to the shorter timescale.
Business people and developers must work together daily throughout the project.
Build projects around motivated individuals. Give them the environment and support they need,
and trust them to get the job done.
The most efficient and effective method of conveying information to and within a development
team is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be
able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.
Simplicity--the art of maximizing the amount of work not done--is essential.
The best architectures, requirements, and designs emerge from self-organizing teams.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts
its behavior accordingly.
14
15
Scrum Lifecycle
Daily Scrum
Meeting
Product
Backlog
Sprint
Backlog
Sprint Review:
Demo Potentially
Releasable New
Functionality
Retrospective
16