Download as pdf or txt
Download as pdf or txt
You are on page 1of 69

Continuous Delivery 2.

0:
Business-leading DevOps Essentials
1st Edition Qiao Liang
Visit to download the full and correct content document:
https://ebookmeta.com/product/continuous-delivery-2-0-business-leading-devops-ess
entials-1st-edition-qiao-liang/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Grokking Continuous Delivery MEAP V06 Christie Wilson

https://ebookmeta.com/product/grokking-continuous-delivery-
meap-v06-christie-wilson/

Grokking Continuous Delivery (MEAP v06) Christle Wilson

https://ebookmeta.com/product/grokking-continuous-delivery-
meap-v06-christle-wilson/

Continuous Delivery for Kubernetes (MEAP V06) Mauricio


Salatino

https://ebookmeta.com/product/continuous-delivery-for-kubernetes-
meap-v06-mauricio-salatino/

Continuous Delivery with Docker and Jenkins, 3rd


Edition Rafa■ Leszko

https://ebookmeta.com/product/continuous-delivery-with-docker-
and-jenkins-3rd-edition-rafal-leszko/
Continuous Integration (CI) and Continuous Delivery
(CD): A Practical Guide to Designing and Developing
Pipelines 1st Edition Henry Van Merode

https://ebookmeta.com/product/continuous-integration-ci-and-
continuous-delivery-cd-a-practical-guide-to-designing-and-
developing-pipelines-1st-edition-henry-van-merode/

Pipeline as Code Continuous Delivery with Jenkins


Kubernetes and Terraform 1st Edition Labouardy

https://ebookmeta.com/product/pipeline-as-code-continuous-
delivery-with-jenkins-kubernetes-and-terraform-1st-edition-
labouardy/

Primary Mathematics 3A Hoerst

https://ebookmeta.com/product/primary-mathematics-3a-hoerst/

Leading a Business School 1st Edition Julie Davies

https://ebookmeta.com/product/leading-a-business-school-1st-
edition-julie-davies/

Business Essentials 1st Edition David Vance

https://ebookmeta.com/product/business-essentials-1st-edition-
david-vance/
Continuous Delivery 2.0
Continuous Delivery 2.0
Business-leading DevOps Essentials
Qiao Liang
First edition published 2022
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742

and by CRC Press


2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN.

Published with arrangement with the original publisher, Posts & Telecom Press.
© 2022 Taylor & Francis Group, LLC

CRC Press is an imprint of Taylor & Francis Group, LLC

Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace
the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in
this form has not been obtained. If any copyright material has not been acknowledged, please write and let us know so we may
rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any
form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and
recording, or in any information storage or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, access www.copyright.com or contact the Copyright
Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are not available on CCC,
please contact mpkbookspermissions@tandf.co.uk

Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for identification
and explanation without intent to infringe.

ISBN: 9780367490478 (hbk)


ISBN: 9781032117997 (pbk)
ISBN: 9781003221579 (ebk)

DOI: 10.1201/9781003221579

Typeset in Times LT Std


by KnowledgeWorks Global Ltd.
Contents

Foreword xv
Target Readers xvii
Content xix
How to Read This book? xxi
Preface 1 xxiii
Preface 2 xxv
Acknowledgement xxvii
Author’s Note xxix

1 Continuous Delivery 2.0 1


1.1 Overview of Software Engineering Development 1
1.1.1 Waterfall Model 1
1.1.2 Agile Method 2
1.1.3 The DevOps Movement 3
1.1.4 Continuous Delivery 1.0 4
1.2 Continuous Delivery 2.0 7
1.2.1 Lean Thinking 7
1.2.2 Double-Flywheels Model 9
1.2.3 Four Core Principles 10
1.2.3.1 Doing less 11
1.2.3.2 Continuous decomposition of problems 11
1.2.3.3 Insistence on fast feedback (FFB) 11
1.2.3.4 Continuous improvement and measurement 11
1.2.4 Tangram of Continuous Delivery 12
1.3 Summary 12

2 Value Discovery Loop 15


2.1 Significance of Discovery Loop 15
2.2 Four Key Steps in Discovery Loop 16
2.2.1 Questioning 17
2.2.2 Targeting 18
2.2.3 Co-creation 20
2.2.3.1 Measurable Impact Mapping 20
2.2.3.2 User Journey Mapping 22
2.2.4 Refining 23
2.3 Working Principles 24
2.3.1 Decomposition and Quick Trial and Error 25
2.3.2 Verify Only One Point at a Time 25
2.3.3 Allow Failure 26
2.4 Common Methods for Co-Creation and Refining 27
2.4.1 Decorative Window 27
2.4.2 Minimum Viable Feature 29
2.4.3 Special Zone 30
2.4.4 Directional Explorer 30
v
vi Contents

2.4.5 Corn Dolly 31


2.4.6 Minimum Viable Product 32
2.5 Matters Needing Attention 32
2.5.1 Multi-Role Participation 32
2.5.2 A Cyclic Process 32
2.5.3 Risk Is Not Equivalent 33
2.5.4 Be Aware of God’s Perspective 33
2.5.5 All Is Number 34
2.5.6 Snake Crawler Effect 34
2.6 Summary 35

3 Fast Verification Loop 37


3.1 Goal of Verification Loop 37
3.2 Four Key Steps in Verification Loop 37
3.2.1 Building 38
3.2.1.1 Timeboxing 38
3.2.1.2 Task decomposition 39
3.2.1.3 Continuous Verification 39
3.2.2 Running 39
3.2.3 Monitoring 40
3.2.4 Decision-Making 40
3.3 Working Principles 40
3.3.1 Building Quality in 40
3.3.2 Eliminating Waiting 41
3.3.2.1 Make value flow through “pull” 41
3.3.2.2 Self-service tasks 42
3.3.3 Automating Routine Work 44
3.3.4 Monitoring Everything 44
3.4 Summary 44

4 A Suitable Organizational Culture for Continuous Delivery 2.0 45


4.1 Security, Mutual Trust, and Continuous Improvement 45
4.1.1 Don’t Be Afraid of Failure 45
4.1.2 Mutual Trust 46
4.1.3 Continuous Improvement 46
4.2 Four Steps in Culture Shaping 46
4.2.1 Behavior Determines Culture 47
4.2.2 Google’s Engineer Culture 48
4.2.3 Etsy’s Continuous Testing Culture 49
4.3 Action Guide 50
4.3.1 Value Orientation 50
4.3.2 Fast Verification 51
4.3.3 Continuous Learning 51
4.3.3.1 Regular retrospective 51
4.3.3.2 Event-replay mechanism 51
4.4 Importance of Measurement 55
4.4.1 Four Attributes of Measurement Indicators 55
4.4.1.1 Leading and lagging indicators 55
4.4.1.2 Observable and actionable indicators 55
4.4.2 The Goal of Measurement Is to Improve 56
Contents vii

4.5 The “Improvement Kata” for Continuous Improvement 57


4.6 Summary 58

5 Software Architecture for Continuous Delivery 59


5.1 Dividing The System into Modules 60
5.1.1 Requirements for a Continuous Delivery Architecture 60
5.1.2 Principles for System Decomposition 60
5.2 Common Architectural Patterns 61
5.2.1 Microcore Architecture 61
5.2.2 Microservice Architecture 62
5.2.3 Monolithic Architecture 63
5.3 Architecture Transformation Patterns 65
5.3.1 Demolisher Pattern 66
5.3.2 Strangler Pattern 67
5.3.3 Decorator Pattern 68
5.3.4 Database Sharding 69
5.4 Summary 70

6 Collaborative Management around Business Requirements 71


6.1 Overview of Software Release Life Cycle 71
6.1.1 Inception Period 71
6.1.2 Delivery Period 73
6.2 Pros and Cons of Requirements Decomposition 74
6.2.1 Benefits of Requirements Decomposition 76
6.2.1.1 Build consensus and coordinate work 76
6.2.1.2 Small-batch delivery to speed up the flow of value 77
6.2.1.3 Embrace change at a low cost 77
6.2.1.4 Multiple integrations and timely feedback on quality 77
6.2.1.5 Boost the morale of team members 78
6.2.2 Costs of Requirements Decomposition 78
6.2.2.1 Explicit cost arising from requirements decomposition 78
6.2.2.2 Iterative cost arising from small-batch development, testing,
and deployment 79
6.3 Way of Requirements Decomposition 79
6.3.1 Sources of Requirements 79
6.3.2 Technical Debts Are Another Requirement 80
6.3.3 Roles Involved in Requirements Decomposition 81
6.3.4 The Unequal “INVEST” Principle 81
6.3.5 Five Ways to Split User Stories 82
6.3.5.1 Splitting by path 82
6.3.5.2 Splitting by contact point 83
6.3.5.3 Splitting by data type or format 83
6.3.5.4 Splitting by rule 83
6.3.5.5 Splitting by exploration 84
6.3.6 Seven Components of Each User Story 84
6.4 Requirements Analysis and Management Tools 84
6.4.1 User Story Mapping 85
6.4.2 User Story Tree 85
6.4.3 Dependency Graph 86
6.4.4 Digital Management Platform 86
viii Contents

6.5 Tools for Team Collaboration 87


6.5.1 Shared Calendar 87
6.5.2 Retrospective Meeting 87
6.5.3 Visualized Story Wall 89
6.5.4 Definition of “Done” 90
6.5.5 Continuous Integration 90
6.5.6 Story Verification 90
6.6 Summary 91

7 Principles for Deployment Pipeline and Tool Design 93


7.1 Simple Deployment Pipeline 93
7.1.1 Simple Development Process 93
7.1.2 Initial Deployment Pipeline 94
7.1.3 Execution of the Deployment Pipeline 96
7.2 Design and Use of Deployment Pipeline 96
7.2.1 Principles for Designing a Deployment Pipeline 96
7.2.1.1 Build once, run anywhere 96
7.2.1.2 Loosely coupled with business logic 97
7.2.1.3 Parallelization 97
7.2.1.4 Quick feedback to take precedence 97
7.2.1.5 Important feedback to take precedence 97
7.2.2 Principles for Teamwork 98
7.2.2.1 Stop the line 98
7.2.2.2 Security audit 98
7.3 Composition of A Deployment Pipeline Platform 98
7.3.1 Overall Architecture of the Platform 98
7.3.1.1 True north source 99
7.3.1.2 Deployment pipeline platform 99
7.3.1.3 Infrastructure service layer 100
7.3.2 Basic Capabilities of the Platform 100
7.3.3 Strategy for Tool Chain Construction 101
7.4 Cloudification of Infrastructure Services 101
7.4.1 Interpretation of the Collaboration Process of Infrastructure Services 102
7.4.2 Build System 104
7.4.3 Automation Test System 105
7.4.4 Deployment System 106
7.4.5 Environment Management System 106
7.5 Management of Artifact Repository 107
7.5.1 Classification of Artifact Repository 107
7.5.1.1 Internal artifact repository A (temporary) 108
7.5.1.2 Internal artifact repository B (formal) 108
7.5.1.3 Artifact repository X (external) 108
7.5.1.4 Temporary mirror C, formal mirror D, and external mirror Y 108
7.5.2 Principles for Management of Artifact Repository 108
7.6 A Variety of Deployment Pipelines 109
7.6.1 Multi-Component Deployment Pipeline 109
7.6.2 Individual Deployment Pipeline 109
7.6.3 Evolution of Deployment Pipeline 110
7.7 Build Self-Service Tools for Engineers 111
7.8 Summary 113
Contents ix

8 Branch Strategy Conducive to Integration 115


8.1 Purpose of The Version Control System 115
8.1.1 CVCS 115
8.1.2 DVCS 116
8.1.3 Basic Concepts in VCS 118
8.2 Common Branching Modes 118
8.2.1 Trunk-Based Development and Release 119
8.2.2 Trunk-Based Development & Branch-Based Release 120
8.2.3 Branch-Based Development & Trunk-Based Release 122
8.2.3.1 Feature-based branch 124
8.2.3.2 Team branch 126
8.3 Evolution of Branch Modes 126
8.3.1 Troika 126
8.3.2 Git Flow 127
8.3.3 GitHub Flow 128
8.4 How to Choose a Suitable Branching Strategy? 128
8.4.1 Release Pattern 129
8.4.1.1 Project release pattern 129
8.4.1.2 Release train pattern 129
8.4.1.3 Intercity express pattern 130
8.4.2 Relationship Between Branch Strategy and Release Cycle 132
8.5 Summary 133

9 Continuous Integration 135


9.1 Origin and Definition 135
9.1.1 Original Definition 136
9.1.2 Integration Process 136
9.2 Six-Step Check-In Dance 137
9.2.1 Four Key Points of Check-In Dance 138
9.2.1.1 Effect of the three times of verification 138
9.2.1.2 Twice personal verifications are a necessity 139
9.2.1.3 Ensure the performance of a personal build
before code submission 139
9.2.1.4 The contents of quality verification to be included
in each build 139
9.2.2 Synchronous and Asynchronous Modes 140
9.2.3 Self-Check Sheet 140
9.2.3.1 Trunk-based Development and frequent submission 141
9.2.3.2 Content of each submission is a complete task 141
9.2.3.3 Finish check-in build within 10 minutes 141
9.2.3.4 After a failed check-in build, do not submit new code or
check out the problematic code 141
9.2.3.5 Fix the check-in build or roll back within ten minutes 141
9.2.3.6 Passing the automation build will boost up the
confidence in software quality 142
9.3 Trade-off Between Speed and Quality 142
9.3.1 Staging Build 142
9.3.2 Builds Submitted by Multiple People at the Same Time 143
9.3.3 The Power of Cloud Platform 144
9.4 Implement CI Practices in the Team 146
x Contents

9.4.1 Quickly Establish a CI Practice for the Team 146


9.4.1.1 Scripted build and setup CI tool 147
9.4.1.2 Add an existing automation verification script
to the build 147
9.4.1.3 Choose a branch strategy that is conducive to CI 147
9.4.1.4 Adopt the six-step check-in dance 147
9.4.1.5 Continuous optimization 147
9.4.1.6 Engineers change habits and improve skills 148
9.4.2 Branch Strategy and Deployment Pipeline 148
9.4.2.1 Trunk-based Development & Release 148
9.4.2.2 Trunk-based Development & Branch-based Release 149
9.4.2.3 Branch-based Development & Trunk-based Release 149
9.4.2.4 Multi-component integration 150
9.5 Common Problems in Implementation 151
9.5.1 Work Habits of Engineers 151
9.5.2 Code Scanning is Often Ignored 151
9.5.3 Lack of Automation Testcases 152
9.6 Summary 152

10 Automation Test Strategies and Methods 153


10.1 Self-Positioning of Automation Test 153
10.1.1 Advantages of Automation Test 154
10.1.2 Investment Required for Automation Test 155
10.2 Break Through the Dilemma of Traditional Automation Test 156
10.2.1 Characteristics of Traditional Automation Test 156
10.2.2 Layering of Automation Test 157
10.2.3 Different Types of Test Pyramid 160
10.2.3.1 Test Pyramid of Microcore Architecture 161
10.2.3.2 Test Pyramid of Microservice Architecture 161
10.3 Implementation Strategy of Automation Testing 163
10.3.1 Add Starting Points for Automation Testcases 163
10.3.1.1 Write tests for code hotspots 163
10.3.1.2 Write tests side by side with new features 163
10.3.1.3 Expand up and down from the middle layer
of the test pyramid 163
10.3.1.4 Quality of automation tests is more important
than quantity 164
10.3.2 Increase the Execution Frequency of Automation Testing 164
10.3.2.1 Share automation testcases 164
10.3.2.2 Developers are the first users of automation tests 164
10.3.3 Characteristics of Good Automation Testing 165
10.3.3.1 Testcases must be independent of each other 165
10.3.3.2 The testcases must be stable 165
10.3.3.3 The testcases must run fast 165
10.3.3.4 The testing environments should be unified 165
10.3.4 Share Responsibilities for Maintenance 165
10.3.5 Test Coverage 166
10.4 Key Points of User Acceptance of Automation Tests 168
10.4.1 Build a Layered Framework in the First Place 168
10.4.2 The Total Number of UAT should be as Small as Possible 170
Contents xi

10.4.3 Reserve API for Automation Testcases 170


10.4.4 Prepare for Debugging 171
10.4.5 Prepare Test Data 171
10.5 Other Quality Inspection Methods 172
10.5.1 Diff-Approval Testing 172
10.5.2 Code Style Check and Code Dynamic/Static Analysis 173
10.5.3 Application of Artificial Intelligence (AI) in the Field of Testing 174
10.6 Summary 174

11 Software Configuration Management 175


11.1 Put Everything Under SCM 175
11.1.1 Goals of SCM 175
11.1.2 Scope of SCM 176
11.1.3 Principles of SCM 176
11.1.3.1 Everything has a unique identifier 177
11.1.3.2 Sharing the True North source 179
11.1.3.3 Standardization and automation 180
11.2 Versioning of Software Artifacts 180
11.2.1 Anti-Pattern of Package Management 180
11.2.2 Centralized Package Management Service 181
11.2.3 Meta Information of Packages 182
11.2.3.1 Unique ID 182
11.2.3.2 Source 182
11.2.3.3 Dependency 183
11.3 Management of Package Dependency 184
11.3.1 Explicit Declaration of Dependencies 184
11.3.2 Automatic Management of Dependencies 186
11.3.3 Reduction of Complex Dependencies 186
11.3.3.1 Too many dependencies or over-long chains 187
11.3.3.2 Dependency conflict 188
11.3.3.3 Circular dependency 189
11.4 Management of Environmental Infrastructure Management 190
11.4.1 Four States of Environment Previsioning 190
11.4.1.1 “Wilderness” featured by “human brain + handwork” 191
11.4.1.2 “Normalization” characterized
by “document + private script” 191
11.4.1.3 “Standardization” characterized by “paperless officing” 192
11.4.1.4 “Automatic state” characterized
by “controlled automation script” 194
11.4.2 DSL Application 195
11.4.3 Infrastructure as Code 196
11.5 Management of Software Configuration Items 197
11.5.1 Separation of Binary and Configuration 197
11.5.2 Version Management of Configuration 198
11.5.3 Storage of Configuration Items 199
11.5.4 Configuration Drift and Governance 200
11.6 Immutable Infrastructure and Cloud Applications 201
11.6.1 Implement Immutable Infrastructure 201
11.6.1.1 Physical machine mirroring technology
and virtual machine mirroring technology 201
xii Contents

11.6.1.2 Docker container technology 203


11.6.2 Cloud Native Application 203
11.6.3 Advantages and Challenges 204
11.7 Data Version Management 205
11.7.1 Changes of Database Structure 205
11.7.2 Data File 206
11.8 Association Between Requirements and Source Code 206
11.9 Summary 207

12 Low-Risk Release 209


12.1 High-Frequency Release is a Trend 209
12.1.1 High-Frequency Release by Dot-Com Companies 209
12.1.2 Coexistence of Benefits and Costs 212
12.2 Ways to Mitigate Release Risk 213
12.2.1 Blue-Green Deployment 213
12.2.2 Rolling Deployment 214
12.2.3 Canary Release 214
12.2.4 Dark Launch 216
12.3 High-Frequency Release Support Technology 217
12.3.1 Feature Toggle 217
12.3.2 Data Migration 219
12.3.2.1 Fields only to be added instead of being deleted 219
12.3.2.2 Data migration 220
12.3.3 Branch by Abstraction 222
12.3.4 Fix-Forward Instead of Rollback 223
12.4 Factors Affecting Release Frequency 223
12.5 Summary 225

13 Monitoring and Decision-Making 227


13.1 Scope of Production Monitoring 227
13.1.1 Monitoring of Backend Services 228
13.1.2 Monitoring of Consumer-Host Software 228
13.2 Data Monitoring System 229
13.2.1 Collection and Processing 229
13.2.2 Standardization 230
13.2.3 Monitoring Data System and Its Ability Measurement 230
13.3 Issue Handling System 232
13.3.1 Alarm Storm and Intelligent Management 233
13.3.2 Problem Solving Is a Learning Process 234
13.4 Test in Production 235
13.4.1 Flattening Trend of Testing Activities 235
13.4.2 Testing in Production 236
13.4.3 Chaos Engineering 237
13.5 Eastward or Westward 237
13.6 Summary 238

14 Large Internet Teams to Become Feature Teams 241


14.1 Case Profile 241
14.1.1 State before Improvement 242
14.1.1.1 Team organization structure 242
14.1.1.2 Development process and rhythm 242
Contents xiii

14.1.2 State after Improvement 243


14.1.2.1 Team organization structure 244
14.1.2.2 Workflow and rhythm 244
14.2 Improvement Methodology 244
14.2.1 Guiding Ideology 244
14.2.2 Improvement Steps 244
14.3 The Journey 245
14.3.1 Architecture Decoupling 245
14.3.2 Organization Decoupling 247
14.3.3 R&D Workflow Reengineering 249
14.3.3.1 The purpose and principles of multi-version release 250
14.3.3.2 Hybrid branch of virtual trunk 251
14.3.3.3 Create a unified product library 253
14.3.3.4 Two-level release 254
14.3.3.5 Quality assurance mechanism 255
14.3.3.6 Multipartite collaborative release 256
14.3.3.7 A build cloud for compilation 257
14.3.4 Optimization for Automation Efficiency 257
14.4 Summary 258

15 How Can Small Teams Implement a “Counter Attack”? 261


15.1 Background 261
15.1.1 A “Death March” before Improvement 263
15.1.2 Zero-Defect Delivery after Improvement 263
15.2 Improvement Methodology 264
15.2.1 Guiding Ideology 264
15.2.2 Selection of the Pilot Team 264
15.3 Stage 1: Preparation 265
15.3.1 Feature Introduction and Requirements Decomposition 266
15.3.2 Architecture Design and Requirements Dependency Identification 267
15.3.3 Workload Estimation and Scheduling 267
15.4 Stage 2: Delivery 269
15.4.1 Improve Workflow with Kanban 269
15.4.1.1 Identify bad smells 269
15.4.1.2 Let the value flow 270
15.4.1.3 Standard definition of done in explicit declaration 271
15.4.1.4 Graffiti design, eliminate waste 272
15.4.1.5 Clarify state and care about rapid flow of requirements 273
15.4.1.6 Avoid unnecessary task switching 273
15.4.1.7 No feedback is a “risk” 274
15.4.1.8 Limit the quantity of WIP 275
15.4.2 Bug-Free Delivery 276
15.4.3 Trunk-Based Development and CI 276
15.4.4 Shift-Left Testing 277
15.4.5 Code Review 278
15.4.6 Care More about Process in Addition to Results 279
15.5 Summary 279

16 DevOps Driven by Developers 283


16.1 Background 283
16.2 Original Working Mode 284
xiv Contents

16.3 Key Points of Improvement 285


16.3.1 Improvement Methodology 285
16.3.2 Define Goals of Improvement 285
16.3.2.1 Expectations of department director 286
16.3.2.2 Delivery pressure of team manager 286
16.3.2.3 Annoyances of project leaders 286
16.4 Stage 1: Agile 101 287
16.4.1 Make a Trustable Plan 287
16.4.1.1 Requirements decomposition 288
16.4.1.2 Relative estimation 288
16.4.1.3 Initial plan 289
16.4.2 Start the Development Stage 291
16.4.2.1 Selection of iteration interval 291
16.4.2.2 Team collaboration process 292
16.4.3 Constraints on Process Quality 293
16.4.3.1 How to consciously abide by CI discipline? 293
16.4.3.2 How to shorten the build time? 294
16.4.3.3 Developers cannot run automated tests 295
16.4.3.4 Automation testing strategy 295
16.4.3.5 What to do in case of insufficient environments
for automation tests? 296
16.4.3.6 How to determine a requirement can be tested? 297
16.4.3.7 How to do performance and stress testing? 297
16.4.4 Summary of Stage One 300
16.5 Stage 2: DevOps Transformation 300
16.5.1 “Conflict” with Ops Engineers 301
16.5.2 Specific Obstacles in High-Frequency Deployment and Release 302
16.5.3 Overall Solution Design 302
16.5.3.1 Adjustment of automation testing strategy 302
16.5.3.2 Convenience of running tests 303
16.5.3.3 Put test code with product code 303
16.5.3.4 SCM optimization 303
16.5.3.5 Package management 304
16.5.3.6 Optimization of deployment and monitoring 305
16.5.4 Team Changes in the DevOps Stage 305
16.6 Summary 306

Appendix A: Three Evolutions of Software Engineering 309


Appendix B: User Story Estimation by Relative Size 323
Index 329
Foreword

Since the term “software engineering” was coined, “quality” and “efficiency” have been its goals. Most
information technology (IT) institutions have been making explorations along this path, they first came up
with the waterfall model and then Capability Maturity Model Integration (CMMI) which has been wor-
shiped as the “Bible” by many IT groups. When the “agile movement” arose, they decided to “do” Agile;
when they heard about “Continuous Delivery”, they changed their mind to “do” Continuous Delivery.
Now DevOps has come out and swept around the IT industry, becoming topic of salons and conferences
and even hit the headline of various media. Not surprisingly, many companies cast their eyes at DevOps…
Of course, there are successful “stories”, but they seem to have nothing to do with us. Even the people
by our side cannot agree on these ideas or practices. It is so perplexing, just like walking in a fog, we can
hear beautiful sound around us and lights flickering in the distance, but we cannot see clearly the road
beneath our feet.
Years of working experience has enabled me to have a new understanding of this field, and summa-
rize and reflect on it. “Continuous Delivery” sounds attractive and catches people’s imagination. Business
people seem to see a glimmer of hope that “all the requirements presented in the morning can be devel-
oped in the afternoon”. Yet too many companies underestimate the difficulties they face. Some of these
difficulties are explicit: no automation test or automation deployment, let alone Trunk-based Development.
Some difficulties are implicit, such as the long-standing “wall” between functional departments. Business
people are unhappy about developers’ slow delivery, while developers are complaining about the unreli-
able requirements presented by business people. This is probably due to their different modes of value
thinking.
The goal of this book is to help all roles in a company change their way of value thinking, improve
the end-to-end business value delivery in software services, raise the teamwork efficiency, so as to achieve
fast verification in a safe and reliable way, and shorten the time to realize genuine business value. In other
words, the book not only elaborates on the process “from product backlog to workable software”, but also
proposes a double closed loop as “value discovery-fast verification” (see Figure 0.1), which implies the
origin of “Continuous Delivery 2.0” – title of this book.
As it turns out, there is no one-size-fits-all business management solution that will be perfect for
every business problem. However, only when managers start from the overall perspective and resist the

FIGURE 0.1 Double-flywheels of Continuous Delivery.

xv
xvi Foreword

temptation of local optimization can they lead their companies to create greater value despite of limited
resources. The book provides an overall framework, showing the principles and practices involved in each
node of the framework, as well as their strengths and constraints.
If you apply the Double-flywheels of Continuous Delivery 2.0 across a company, it is an organiza-
tional transformation at the enterprise level. If you introduce it to a team, it will be path for it to improve
it work mode. Since Continuous Delivery 2.0 is a management framework, companies are bound to tailor
it to their own circumstances. For this reason, the book contains many practical examples showing how
other businesses or teams achieved their goals by implementing these practices. These cases also show
that solutions and implementation paths are unlikely to be replicated between companies, so they must
follow the principles in the book in combination with their specific situations (product forms, stage of
business competition, size of team, personnel skills, software system architecture, as well as corporate
management mechanism and culture), and explore their own path step by step.
Target Readers

The book is intended for front-line managers of IT corporations or key technicians who are about to
become front-line managers, as well as those who are dedicated to software project management and
software process improvement. It is also valuable for founders of startups or technology executives in
fast-growing companies. It is not available for the senior managers of large and sophisticated software
companies. Of course, I’d be happy if they could take an interest in this book.
For product owners, development managers, test managers, and operations managers, this book will
give them a more comprehensive perspective on how to work, discover more information beyond of their
scope of work, and learn how to collaborate with other roles.
For process improvers or change makers, they can learn from successes or failures from other teams
or companies in the real cases discussed in this book.
For founders of startups and executives of fast-growing companies, I hope they could find some effec-
tive ways to manage and expand their businesses from the beginning at the lowest possible cost.
The real cases listed in the book will help readers fully understand the double-flywheels of Continuous
Delivery 2.0 and all necessary practices at each stage.

xvii
Content

Before getting into the main content of this book, let’s talk about the structure of this book. It consists
of three parts: Part 1 introduces the double-flywheels of Continuous Delivery 2.0. Part 2 focuses on the
issues you may encounter while using the Continuous Delivery 2.0 framework and the principles you need
to follow for making improvement. Part 3 shares typical cases about certain companies and teams for
readers to learn about varied implementation priorities and solutions. The content of this book is briefly
introduced as follows:
Part 1 describes the double-flywheels of Continuous Delivery 2.0 and the four working principles
of Continuous Delivery 2.0. It also introduces the implementation steps and related principles of the two
closed loops as Value Discovery Loop and Fast Verification Loop.

• Chapter 1 explains the inevitability of Continuous Delivery, and introduces the “Double-
flywheels of Continuous Delivery 2.0” and its four basic principles.
• Chapter 2 lists the four key steps of the Value Discovery Loop (“Discovery Loop” for short),
as well as the guiding principles and practices at each step. Continuous Delivery will be more
powerful if business problems could be subject to “lean” thinking; otherwise, it is likely to
retreat into the single-loop Continuous Delivery 1.0 with only the “Fast Verification loop”.
• Chapter 3 briefly describes the main activities and working methods at each link in the Fast
Verification loop (“Verification Loop” for short).

Part 2 expounds the working principles and implementation methods of the three plates in Continuous
Delivery 2.0 Tangram. These three plates are made up of organizational mechanism, software architec-
ture, and infrastructure. Since organizational mechanism is a complex subject, this book only discusses
the culture needed for Continuous Delivery and the four steps in creating such culture. Such contents as
organizational structure, talent structure, and incentive mechanism are about to be analyzed in the sequel
of this book. Infrastructure is the most fundamental part in the product development process. Part 2 first
introduces the deployment pipeline and tool design principles in Continuous Delivery, and then highlights
the five areas for constructing and optimizing the deployment pipeline, that is, business requirements
management, branch and configuration management, build and environment management, automation
test management, and deployment and monitoring management.

• Chapter 4 argues that the improvement of Continuous Delivery capability requires an organiza-
tional culture which is safe, credible, and constantly improvable; and introduces the four steps
implemented by Toyota and Google to improve organizational culture.
• Chapter 5 talks about the importance of software architecture in achieving fast verification
of Continuous Delivery, the characteristics of software architecture favorable for Continuous
Delivery, and three ways of software architecture transformation, namely, “demolition”, “stran-
gling”, and “decoration”.
• Chapter 6 explains how to use constraint theory and lean thinking to find bottlenecks in the
process, make team members smoothly collaborate in handling business requirements, and
increase the flow of requirements.
• Chapter 7 introduces the design principles and toolchain construction drafts for the deployment
pipeline that Fast Verification Loop relies on.

xix
xx Content

• Chapter 8 discusses the significant impact of branch strategy of codebase on Continuous


Delivery and how to implement deployment pipelines in different branching approaches. And
finally, you will see how the number of code branches affects the release strategy.
• Chapter 9 reviews the history of continuous integration, explains how to determine if a team is
practicing continuous integration, and presents five steps for a company to implement continu-
ous integration.
• Chapter 10 discusses the factors to be considered for developing an automation test strategy
prior to software release, and describes the requirements for writing and running automation
testcases in continuous integration. Finally, it teaches the team how to write and add automa-
tion testcases for legacy systems in order to improve the return on investment (ROI) of automa-
tion test.
• Chapter 11 focuses on software configuration management, which is the foundation for the Fast
Verification Ring in Continuous Delivery. To do a good job in the configuration management of
code, configuration items, environment, and data, and finally realize one-click deployment and
one-click testing, so that all stages in the collaboration process are fully automation and free
from affecting each other. In this way, each role is able to more valuable things, and let machine
do what it is good at – constant repetition.
• Chapter 12 introduces the techniques and approaches for reducing the risks in production
deployment and release.
• Chapter 13 describes the importance of data collection and analysis while software is running,
as well as metrics to measure data monitoring, including correctness, completeness, and time-
liness of data. This chapter also introduces the test flattening trend, quality inspections, and
drills in production environment.

Part 3 introduces and analyzes typical cases to readers. They occurred in companies of different
types or teams of different sizes, with software products carrying varied features. All of these cases were
of my personal experience. Their background, difficulties, and solutions are introduced in detail to inspire
readers.

• Chapter 14 introduces a case happened in an Internet product team made up of 100 engineers.
This team spent a year in building the fast-running double-flywheels of Continuous Delivery 2.0,
and managed to have it operated continuously.
• Chapter 15 describes how a small team working on a large project transformed the teamwork
model from “Death March” to “zero-defect delivery” under the condition of impossible “shift-
right testing”.
• Chapter 16 shows how a microservice-oriented team transformed into a “DevOps team” in the
process of project implementation by gradually transforming infrastructure modules, improv-
ing quality and frequency of delivery, and motivating operations staff to make changes as well.
How to Read This book?

How to read this book? It depends on what do you want to get from this book. You can read it through
to learn all the knowledge applicable to my work, the reason why Continuous Delivery 2.0 is an organi-
zational capability, how to measure this capability, and how to apply principles in real scenarios to solve
specific problems.
Of course, you can read any chapter as you like to learn to improve your work. Chapters 14~16 enu-
merate some cases happened in small and large teams, as well as in a large corporation, introducing their
work methods and improvement tools in detail. You may get enlightened to find a way out of a stalemate.
Anyway, I appreciate for your feedback on this book. How do you like it? What else do you want
to know? Do you have any suggestions? You can scan the QR code below, subscribe the WeChat official
account of Continuous Delivery 2.0, or visit http://www.continuousdelivery20.com/ for extended reading.

xxi
Preface 1

I first met Qiao Liang in Beijing in 2007 when we worked together on go.cd, the first CI/CD tool to sup-
port deployment pipelines. What we learned from building that product was crucial to the development of
the material in the Continuous Delivery book.
Over the last ten years, these ideas have gone from being bleeding edge to accepted wisdom. Every
serious tech company aspires to the idea of being able to release at any time, on demand, without workers
having to spend evenings or weekends doing deployments. Being able to release software quickly, fre-
quently, and safely in turn enables delivering in small batches, which means we can get fast feedback on
our ideas. We can build prototypes and test them with real users, and avoid building features that deliver
no value to users. This, in turn, means better products, happier customers, and happier employees. These
capabilities are important for every organization that uses technology.
But it’s hard to get there. Organizations need to evolve an architecture that supports fast, effective
testing and rapid deployment, and deploy a culture of experimentation. The cultural element is essential
both for successful implementation, and to put in place the product management practices enabled by
Continuous Delivery.
Qiao Liang has worked with all kinds of organizations in China to help them implement Continuous
Delivery and realize its benefits. I can’t think of anybody better to write a book on how to implement
these ideas based on real-life experiences. I wish you every success as you experiment with getting bet-
ter at software delivery, and using that capability to build better products and services and happier, more
productive teams.

Jez Humble
Author of Continuous Delivery, co-founder and CTO of DORA

xxiii
Preface 2

We are in the second half of the mobile Internet era. This means that, for the mainstream population, the
basic needs of the head may have been met or “saturated”, and the future direction may be the subversion
of the existing model, or the full satisfaction of the long-tail personalized needs, or better serving the
needs of China’s ethnically diverse communities. In any case, I believe there is a good chance that the cost
of innovation and the likelihood of failed innovation will continue to rise in the future.
In this context, it becomes particularly important to find new opportunities quickly and reduce inno-
vation costs through extreme product development and operation efficiency. Mr. Qiao Liang is a master
in the field of software engineering. As an agile consultant of Tencent, he has worked with us for years
and experienced the whole process of the development of mobile Internet. Mr. Qiao has a thorough under-
standing of various problems in the product development and operation system, and he is tireless in the
pursuit of efficiency.
“In this rapidly changing Internet environment, how can software development teams balance quality
with speed? How to deliver software to users as quickly as possible so as to get their fastest feedback?”
Mr. Qiao and I had an in-depth discussion on these issues many years ago. “There is no balance between
quality and speed. Only product quality is ensured to certain extent, the pursuit of delivery speed is mean-
ingful”, said Mr. Qiao sharply. Unlike most consultants who like to say “It depends”, Mr. Qiao pointed
out an immutable law throughout the product life cycle. Although different product stages have differ-
ent requirements for product quality, such requirements are almost constant at the same stage. “How to
deliver product value quickly while meeting quality requirements?” The contents of this book are mainly
unfolded to answer this question.
In order to quickly identify new opportunities, the product iteration must be accelerated, but it may
increase the fixed costs in each iteration, such as the cost for product quality test and for product release.
This transactional cost greatly hinders the iteration rate of product. In this book, Mr. Qiao breaks downs
the steps for operating the Value Discovery Loop and Fast Verification Loop in Continuous Delivery. On
the one hand, it brings you new dimensions and perspectives to probe into business problems; and on the
other hand, it enlightens you to pay attention to the details in daily work to seek for optimization, which
helps to reduce the fixed costs in iteration and innovate products more efficiently. At the same time, you
will learn how to enhance the overall combat capability of your team.
This book Continuous Delivery 2.0 exhibits the theoretical framework summed up by Mr. Qiao
from a large number of practical projects; it is theoretical achievement based on solid realistic foundation.
Instead of piling up empty theories, this book introduces plenty of positive and negative examples to sup-
port the principles and methodologies in software industry. This book is a pragmatic guide, just like its
author who is a low-profile and down-to-earth person. It is especially recommended to the readers, either
researchers in software industry or practitioners dedicated to R&D of operation system with efficient soft-
ware applications. As I believe, in Internet industry, efficiency is the key to success in the future.

Zeng Yu
Vice President of Tencent

xxv
Acknowledgement

The publication of this book is attributed to many people’s hard work. First of all, I’d like to thank the
editorial team for their strong support, particularly Yang Hailing, Chief Editor of Posts & Telecom Press.
Thanks to her encouragement, I finally got up the courage to write down my consulting practices and
experiences in software development to share with more people. Though this book is not perfect, it’s a
retrospect of my years’ endeavor and a summary of industrial achievements, including theories, methods,
and practical tools. Second, I’ll thank my friend Ren Fake for his proofreading of many chapters of this
book and for his valuable suggestions. Thirdly, I’d like to thank CRC Press for publishing the English
version.
Of course, I’m grateful to my former colleagues at ThoughtWorks, especially the GoCD team and my
partner Jez Humble. I learned agile development and lean thinking from them. It was this work experi-
ence that made me proficient in Continuous Delivery practices, thereby guiding my clients solve practical
problems in recent years. Wang Pengchao, another friend of mine, ushered me into the wonderland of
Extreme Programming. He is now working in Facebook responsible for the development of Cassandra.
Finally, I must thank my wife and my son – the loves of my life. I once intended to giving up writing
this book, it was their encouragement and accompany that helped me regain confidence. My wife is the
first reader of this book, she helped me streamline and refine the language, and made the book in a more
reasonable layout.

xxvii
Author’s Note

In 2002, I happened to get the book Extreme Programming Explained: Embrace Change written by Kent
Beck, and couldn’t help reading it through. The software development methods introduced in the book
are utterly different from those in our actual work. The practices, such as test-driven development (TDD),
continuous integration, pair programming, and user story, seem novel and unrealistic. I kept wondering
how can a team do all of these things? So I tried some of them, but not thorough enough since I was
still a little “skeptical”. For example, I skipped over TDD, and only added some unit tests; I left out pair
programming, but insisted on code review; and I preferred daily build to continuous integration. Certain
effects started to show after a period of time, but not so significant.
Until 2005, a friend of mine showed me how did his team deliver a software project with the agile
development method and how did they write code. Each time the code was modified, a series of automa-
tion testcases were written and executed; each submission was accompanied with continuous integration.
It was a brand-new coding experience to me: developers rarely need to start the program, but detect the
problems in the code through single-step debugging. I finally came to believe agile teams at not fictitious,
they are real and not far away from me.
I joined ThoughtWorks in 2007, hoping to experience the agile development method. As a require-
ments analyst and delivery manager, I took part in the development of GoCD – an advanced CI and release
management system. My partner was Jez Humble, product owner of this project and later co-author of the
book Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation.
Many practices described in his book come from our GoCD development process. From bringing forth
the product idea to its official launch, I went through a complete product development process where I got
access to the agile development method and learned how to do Continuous Delivery.
After 2009, as an external consultant or internal coach, I started to provide consultations on transi-
tion to Agile Development and Lean Production to domestic and foreign companies. Among my clients,
there are leading PC Internet companies, traditional IT firms, powerful domestic corporations, and fast-
growing mobile Internet startups. In the process of working with my clients, I’ve gained an in-depth
understanding of “Continuous Delivery” and greater confidence in helping them realize the value of
Continuous Delivery.
Back to 2007, I was believing that many agile development practices including Extreme Programming
(XP) were a magic key for fast and high-quality software delivery. After 2010, I came to realize that
although practice itself is important, it must be supported by organizational management, and work ideas
and concepts. So I got a new pet phrase as “Don’t talk about ‘Agile’, just solve problems!”. After 2012,
more software development methods and agile genres began to prevail in China, but the core concepts
and main working principles behind them have not fundamentally changed. Regardless of the method, we
should start from “solving problems”, and an important prerequisite for doing so is to “correctly define
problems and reach a consensus”.
I’m certainly not a supporter of such absurd argument as “thought is useless”. On the contrary, I
believe that thought is essential to everyone’s cognition and understanding of things. But years’ consulting
experiences tell me that a correct understanding of things cannot ensure implementation of correct ideas
and concepts, nor can it provide direct and substantial assistance to companies. The goal of promoting
correct thoughts is to guide project teams or their customers to address difficult problems.
Just like business management, the development process of software engineering is also a process
where various methodologies emerge and develop endlessly. After the term “software engineering” was
coined in the 1960s and the waterfall software development model was proposed in the 1970s, the iterative

xxix
xxx Author’s Note

incremental development came out in 1985, Barry Boehm brought forth the Fountain Model in article A
Spiral Model of Software Development and Enhancement published in 1986, the Capability Maturity
Model Integration (CMMI) and various lightweight software development methods emerged in the 1990s,
the Agile Manifesto was officially released in the early 21st century, and then Lean software development
method, Kanban method, Continuous Delivery, and the DevOps movement sprang up successively in
recent years. All these changes not only reflect the rapid evolution in IT industry but also prove that no
single theory or method can solve all the problems in this industry on its own.
I do hope that this book could show a whole picture of “Continuous Delivery” to readers. When
encountering problems related to the effectiveness of IT groups, readers will be able to deal with them
reasonably based on background knowledge learned from this book. Or at least they can get inspired from
the real cases included in this book so that they can take fewer detours when facing similar problems.
Continuous
Delivery 2.0
The classic book Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment
1
Automation, since its publication almost a decade ago, is widely recognized by software technicians for
its brilliant exposition of the principles and practices for software development. It is even taken as a bible
by the practitioners in software industry in recent years when the business environment is increasingly
characterized by “VUCA”.
VUCA, an acronym of volatility, uncertainty, complexity and ambiguity, was initially a hot military
word in the 1990s for describing the post-Cold War multipolar world, which was volatile, uncertain,
complex, and ambiguous. This acronym had been little known before 2001, especially the 9/11 terrorist
attack. But nowadays, business leaders with strategic vision like to characterize the “new normal” busi-
ness environment, which is out of order and fast changing with VUCA.
However, when we are going to apply those principles and practices in our daily work to achieve
Continuous Delivery, we will encounter all sorts of obstacles. For example, the mounting business pres-
sure has drained us of energy for making further improvement, a tight schedule for development and test-
ing forces us to work in the old way, and the risk is too high to ensure high-quality delivery by now if we
work in a new way. These problems arise from the development inertia of software engineering, and now
it is time for revamping the model.

1.1 OVERVIEW OF SOFTWARE


ENGINEERING DEVELOPMENT
The discipline of “software engineering” was born in 1968 at the time of the first software crisis. The rea-
son for this crisis was that the backward mode of software production no longer met the rapidly increasing
demand for software applications, thereby giving rise to a series of grave problems in software develop-
ment and maintenance. So people have made attempts, such as adopting the engineering approach applied
in the field of construction engineering, to solve these problems and fulfill the expectations for “delivering
software applications with required features on time and within budget”.

1.1.1 Waterfall Model


The Waterfall Model (see Figure 1.1) was first proposed by Dr. Winston W. Royce in the article Managing
the Development of Large Software Systems published in 19701. It divides the software development
process into multiple stages, and each stage has strict input and output criteria. Project managers hope to
solve the software crisis with such development model that is heavy on plan, heavy on process, and heavy

1 Dr. Royce also mentioned in the article that he believes in this concept, but the implementation described above is risky and
invites failure and recommended to do it twice.

DOI: 10.1201/9781003221579-1 1
2 Continuous Delivery 2.0

FIGURE 1.1 Waterfall Model.

on document. All the software development methods that have the three “heavy” traits are collectively
referred to as the “Heavyweight Software Development Method”.
In the 20th century, each stage of the Waterfall Development Model took months. Before the first line
of product code was written out, Party A and Party B had already made significant efforts to determine
the scope of requirements, and review the software requirements specifications as long as hundreds of
pages. In spite of this, the two parties might still argue about “whether the scope of requirements has
changed”, “whether the software is delivered on time”, or “whether the delivered software meets the pre-
set business requirements”.

1.1.2 Agile Method


Since the 1980s, microcomputers started to popularize rapidly. Then in the 1990s, there had been a robust
demand for software applications. However, the Chaos Report 1994 produced by the Standish Group
shows that the proportion of failed software delivery or difficult delivery remained high (only 16.2% of
software projects were successfully delivered, 52.7% of them confronted challenges, and 31.1% came to
nothing). In this context, lots of ambitious software practitioners who were dissatisfied with the deliver-
ables of Waterfall Model came up with several new development methods based on their own experiences.
The well-known methods of Scrum and Extreme Programming (XP) were typical creations of that time.
At the February 2001 meeting at Snowbird Utah, 17 software masters, including Martin Fowler and
Jim Highsmith, discussed the lightweight software development methods that emerged in those years, and
coined the term “Agile” on this occasion. They came up with the Agile Manifesto and set out “Twelve
Principles” for guiding the agile software methods (see Appendix A). All of them agreed that any soft-
ware development method that conforms to the values advocated by the Manifesto and follows the Twelve
Principles can be classified as the “Agile Method”. It can be seen that “Agile” is in fact a synonym for a
cluster of software development methods since its inception, rather than any specific method.
The Agile Method gives weight to individual initiative, face-to-face communication, changes, and
early delivery of valuable software through iterative and incremental development. Because of it, many
project teams have realized that software development is actually a process of continuous iteration and
learning, that is, software engineers need to learn and understand domain knowledge rapidly, translate into
the expressions of the digital world, and then keep iterating this process after discussions with business
1 • Continuous Delivery 2.0 3

FIGURE 1.2 Iterative development, multi-deployment, and release.

domain specialists. A software delivery plan involves multiple iterations, with each iteration giving rise to
a runnable software in the end (see Figure 1.2).
Since the Waterfall Model cannot bring forth a runnable software until the later stage of the project,
the Agile Method seems to be more superior in this regard. “Continuous Integration” – an engineering
practice while employing the Agile Method – was first adopted by a wider information technology (IT)
sector, including the teams that had not tried any Agile Method, because it stresses frequent automation
builds and testing, which prevent quality assurance (QA) team from doing repetitive work and smoothen
its communication with development team.
The most requests for IT at that time were those enterprise-level customized software applications.
Although the iterative model was employed for Agile Method, the interval between two software releases
was often as long as months or even more than a year. Therefore, the contradiction between business
personnel and the delivery team regarding requirements changes and delivery efficiency remained as the
principal contradiction. The deployment and release of an application took less costs of time and money
in the entire delivery cycle, so the deployment activities did not have any prominent contradiction with
the work of operation and maintenance. Besides, the deployment was usually performed by a dedicated
technical operation team, and free from bothering the research and development (R&D) team.
In this period, the top concern of software industry was neither the Waterfall nor the Agile Method,
but the deliverable software assemblies themselves. To be specific, what it concerned about was how to
quickly turn business requirements into deliverable software assemblies.

1.1.3 The DevOps Movement


The term “DevOps” sprouted on an impromptu topic “Agile Infrastructure” proposed by Andrew Shafer
at the Agile Conference 2008 held in Toronto. The independent Belgian IT consultant Patrick Debois was
so excited to learn of a like-minded person that he hunted him down at the conference and had that talk
in the hallway. At his initiative, the “DevOpsDays” community conference was held in Ghent, Belgium in
2009 when the term “DevOps” was coined and officially made public.
In 2010, the website The Agile Admin published an article What is DevOps, pointing out that “DevOps is
a term for a group of concepts that, while not all new, have catalyzed into a movement and are rapidly spreading
throughout the technical community”. This article also gave a practical definition: “DevOps is the practice of
operations and development engineers participating together in the entire service lifecycle, from design through
the development process to production support”. It further proposed that “DevOps is also characterized by
operations staff making use many of the same techniques as developers for their systems work”.
The definition of DevOps on Wikipedia has kept been updated. As of 2017, it was defined as follows:

DevOps (a clipped compound of “development” and “operations”) is a software development process


that emphasizes communication and collaboration between product management, software development,
and operations professionals. DevOps also automates the process of software integration, testing, deploy-
ment, and infrastructure changes. It aims to establish a culture and environment where building, testing,
and releasing software can happen rapidly, frequently, and more reliably.

In fact, at the time of writing this book, a uniform definition of DevOps was not yet available. Just like
every practitioner or every firm has their own view of “Agile”, they may look at DevOps in different ways.
4 Continuous Delivery 2.0

Some people regard DevOps as a subset of Agile, some people argue that “if Agile was done correctly, it
would be DevOps”, while some people go so far as to say that DevOps is a set of practices centering on
automation and it is more or less associated with Agile.
DevOps originated from the application of Agile thinking and practice in development and opera-
tions, however the guidance on how to achieve the goal of DevOps was little at that time. Fortunately, the
book Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation,
which came out almost in the same period, has made DevOps more concrete and more actionable in daily
work. It sets out a series of principles, methodologies, and practices for participants in the DevOps move-
ment to follow, such as the strongly advocated principles of “everything is code, automating everything,
and early feedback by deployment pipeline”.
From my view, DevOps is not a criterion, a pattern or a set of fixed methods, but a trend for manage-
ment of IT organizations. That is to say, it is able to break the barriers between the functional departments
of an IT organization in various ways, transform their original cooperation model and make them better
integrated, so as to further expedite their business iteration. This trend is sure to change the existing roles
and division of labor within an IT organization, and even affect relevant business structure. In the case
of Internet companies, software plays an extremely critical role in their business development; in other
words, their business performance is closely bound up with IT efficiency, that’s why they are more moti-
vated and anxious to conform to this trend.
As a trend for management of IT organizations, DevOps is universally applicable in the domain of IT.
But for IT organizations in different industries and enterprises, their management should be customized
according to industry characteristics and specific situations of enterprises.

1.1.4 Continuous Delivery 1.0


In 2006, Jez Humble, Chris Read, and Dan North co-published a paper titled The Deployment Production
Line to discuss the production efficiency brought along by software deployment, and put forward the
“deployment production line” for the first time:

Testing and deployment can be a difficult and time-consuming process in complex environments com-
prising application servers, messaging infrastructure and interfaces to external systems. We have seen
deployments take several days, even in cases where teams have used automation builds to ensure their
code is fully tested.

In this paper we describe principles and practices which allow new environments to be created, config-
ured and deployed to at the click of a button. We show how to fully automate your testing and deployment
process using a multi-stage automation workflow. Using this “deployment production line”, it is possible
to deploy fully tested code into production environments quickly and with full confidence that you can fall
back to a previous version easily should a problem occur.

The above three authors were all working at ThoughtWorks – an initiator, practitioner, and promoter
of the Agile Method. The software development methods adopted by this company are also derived from
XP. These authors not only introduced the prototype of the “deployment production line” in combination
of their own practices, but also presented four basic principles to address the most common challenges
facing automation of the build and deployment process.
These principles are:

1. Each build stage should deliver working software – don’t have separate stages for intermediate
artifacts such as frameworks.
2. Deploy the same artifacts in every environment – manage your runtime configuration separately.
3. Automate testing and deployment – use several independent testing stages.
4. Evolve your production line along with the application it assembles.
1 • Continuous Delivery 2.0 5

FIGURE 1.3 The deployment pipeline defined by David Farley in 2007.

In 2007, David Farley, also from ThoughtWorks, published a paper titled The Deployment Pipeline –
Extending the Range of Continuous Integration, explaining that the “deployment pipeline”2 is to tie
together multiple quality verification levels and their validation content in an automation way, as shown
in Figure 1.3.
In December of the same year, ThoughtWorks formed a team in Beijing to build a tool for manage-
ment of software continuous release by adopting the concept of “deployment pipeline”. Jez Humble was
the product manager, and I was responsible for the product delivery and business analysis. The first
version of this product was released in July 2008 by the name of “Cruise” (now known as “GoCD”3).
Its most important feature is “deployment pipeline”, and the design of most features (including built-in
artifact repository and reference to multi-stage artifacts) embodies the “Build once, Run anywhere”
principle.
While being busy with the R&D of GoCD, Jez Humble and David Farley coauthored a book
Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, with
its English version published in 2010, and Chinese version coming out in 2011 and winning the Jolt Award
in the same year, marking an official debut of the term “Continuous Delivery”. According to Jez Humble,
“Continuous Delivery is the ability to get changes of all types – including new features, configuration
changes, bug fixes and experiments – into production, or into the hands of users, safely and quickly in a
sustainable way”. It should be noted that Continuous Delivery defined like this is written as “Continuous
Delivery 1.0” in this book for distinguishing from “Continuous Delivery 2.0” (see Section 1.2).
In their book, Jez Humble and David Farley have discussed the philosophies, principles, methods, and
practices of Continuous Delivery 1.0, and emphasized in the final chapter of the book that “Continuous
delivery is more than just a new delivery methodology. It is a whole new paradigm for running a business
that depends on software”4.
Lots of the principles and methods of Continuous Delivery 1.0 are concrete operational guidelines
for the DevOps movement, meaning that they can guide companies’ IT teams to carry out the spirit of the
DevOps movement (see Figure 1.4).

FIGURE 1.4 Continuous delivery 1.0 puts the release button back to the business side.

2 https://continuousdelivery.com/wp-content/uploads/2010/01/The-Deployment-Pipeline-by-Dave-Farley-2007.pdf
3 https://github.com/gocd
4 “Paradigm” is a philosophical and theoretical framework of a scientific school or discipline within which theories, laws, and
generalizations and the experiments performed in support of them are formulated. To put it simply, “paradigm = philosophies +
methods”.
6 Continuous Delivery 2.0

FIGURE 1.5 Different roles in agile development.

The collaborators involved in Agile method are mostly product managers, software developers and
testers, and DevOps is more related to delivery teams (including developers and testers) and DevOps engi-
neers, while Continuous Delivery 1.0 involves product demand side, software R&D teams, and DevOps
engineers (see Figure 1.5).

CONTINUOUS DEPLOYMENT AND CONTINUOUS DELIVERY


Many people regard Continuous Deployment as an advanced state of Continuous Delivery, in which
code change is automatically deployed into a production environment as soon as it has success-
fully passed all quality verification, and exempted from any approval. But in fact, the two words of
“Deployment” and “Delivery” have different meanings.
“Deployment” is a technical operation, that is, it first acquires a software artifact from some-
where and then installs it on a compute node by following the pre-designed scheme, and ensures the
normal start of the system; but it is not necessary to “release or deliver any part containing business
function”. In contrast, “Delivery”, which is often referred to as “release”, is a business decision,
meaning that newly-built features can play a role in creating business value after they are delivered
to them, for example, they are seen and used by customers (users).
The fact that “Deployment” and “Delivery” have become almost equivalent words is caused by
historical reasons. In the past, there had been a long software release cycle, thus making it released
immediately after new features were deployed. Then “Deployment” had gradually become syn-
onymous with “Release” over time. In order to ensure software quality, IT department generally
didn’t allow irrelevant code (which had nothing to do with the currently released new features, such
as undeveloped features and incomplete feature sets) to be deployed to a production environment.
Therefore, each deployment must be a release of major features.
With the advent of Internet services, it is common to see “Deployment” and “Delivery” have
different content and frequency. The code changes can be deployed to production for multiple times,
but not released to users unless they are about to address business needs (see what the arrows
indicate in Figure 1.4). For example, at Facebook’s 2011 tech talk, the company’s release engineer
Chuck Rossi pointed that “right now on Facebook.com there is already the code for every major
thing Facebook is going to launch in the next six months and beyond! It’s the Gatekeeper which
stops us from seeing it”.
1 • Continuous Delivery 2.0 7

1.2 CONTINUOUS DELIVERY 2.0


Continuous Delivery 1.0 focuses on the process “from code change submission to product release” (see
Figure 1.6), and provides a set of working principles and good practices that can raise the efficiency of
development activities.
But while doing my consultation work, I’ve found that some software features, after being completed,
didn’t have any influence on customers or users, and some features were not used at all, regardless of
their painstaking development. What a huge waste! And more features stand “idle”, more waste will be
produced. How can we make sure our hard work is fruitful? Or how can we verify the futile features at a
small cost?

1.2.1 Lean Thinking


The book The Lean Startup (2011) of Eric Ries, co-founder and CTO of IMUV, has given me some inspi-
rations. Its core idea is that when we develop a new product, we’d better first make a simple prototype – a
minimum viable product (MVP). It is not to prelude a perfect final product, but to verify our business
hypothesis. After receiving the real user feedback on each experiment, we should sum up the merits and
demerits of the product features, then iterate quickly and fix bugs continuously to find the way to success
before running out of our resources, so as to cater to the market demand in the end.
Eric Ries emphasized in his book that the lean startup is a “Build-Measure-Learn” process for veri-
fication and learning (see Figure 1.7), meaning that in this process we will rapidly convert our ideas into
products, measure user feedback, and then decide to pivot or stick to our original intention.
In his book, which focuses on the initial stage of start-up, the Lean Thinking runs through the process
where the product is “from zero to one”. In fact, this thinking also applies to the process where the product
is “from one to n”.

FIGURE 1.6 Concerns of Continuous Delivery 1.0.


8 Continuous Delivery 2.0

FIGURE 1.7 “Build-measure-learn” feedback loop.

In their coauthored book Lean Thinking (1996), James P. Womack and Daniel T. Jones pointed out
that the Lean Thinking is to guide enterprises to define their production value according to user require-
ments, organize all production activities based on the value stream, draw the value flow between produc-
tion activities and provide on-demand products, so as to identify the wastes inadvertently arise from the
productive process and then have them eliminated.
According to the theory of Lean Thinking, “wastes” refer to the production activities or management
processes that do not add value to quality products or services from the perspective of customers. This
theory has summed up two types of production activities: value-added and non-value-added activities,
with the latter identified as “wastes”. Those activities labelled as “wastes” are further divided into neces-
sary and unnecessary non-value-added activities, with the latter synonymous with pure wastes. From the
customers’ point of view, necessary non-value-added activities, although create no value, can help to avoid
(potentially) greater wastes or reduce systemic risks (see Figure 1.8).
For example, assembly on a production line is a value-added activity, and quality inspection is a nec-
essary non-value-added activity, but waiting for production due to scant material supply and rework due to
quality defects are regarded as unnecessary wastes. In the full life cycle of SaaS, short for “Software-
as-a-Service”, there are also multiple “wastes”, such as ineffective features, waiting in each production
link (see Figure 1.9), unattended documents, software defects, and mechanical duplication of work.
Although “eliminating all wastes” is almost impossible, we still need to fully implement the philoso-
phy of “identifying and eliminating all wastes”, and keep optimizing processes and working methods to
achieve high-quality, low-cost, risk-free, and fast delivery of customer value.

FIGURE 1.8 Example of wastes in software development.


1 • Continuous Delivery 2.0 9

FIGURE 1.9 Value-added activities and wastes from user perspective.

1.2.2 Double-Flywheels Model


Since 2009 when Flickr – one of the most popular photo sharing sites – claimed to have ten times of
deployment every day, “trunk-based development + CI + continuous release” has become a mainstream
software R&D and management model for renowned Dot-com companies in Silicon Valley to cope with
the VUCA environment. This change was not brought forth by the technical teams, but provoked by the
demands from both business side and product side. In order to better understand massive users in the
VUCA environment, and quickly validate business hypotheses and solutions, the business and product
sides have adopted a new business exploration model, which stimulated the transformation of the soft-
ware R&D and management model; then these two models began to promote each other and give rise to
a “Double-flywheels Model” for internet-based software R&D and management, namely, “Continuous
Delivery 2.0” (see Figure 1.10).
Continuous Delivery 2.0 is a thinking framework for product R&D. It combines lean startup with
Continuous Delivery 1.0, stresses rapid closed-loop control between business and IT, sticks to the guide-
line of “Lean Thinking”, and fully implements the philosophy of “identifying and eliminating all wastes”,
in an aim to help enterprises achieve high-quality, low-cost, risk-free, and fast delivery of customer value
in a sustainable manner.
For companies undertaking software delivery, the goal of developing software is to create customer
value. To this end, they should not only focus on rapid development of software features but also on
the business correctness of the deliverable software and the ways for quick verification and resolution
of business problems with limited resources. It means that they must constantly explore and discover
the real business problem to be solved, propose scientific goals, and then come up with a minimum
viable solution. After that, they should verify whether this problem could be addressed through fast
implementation of this solution and collection of data from real user feedback. This is an intact business

FIGURE 1.10 Double-flywheels Model of Continuous Delivery 2.0.


10 Continuous Delivery 2.0

closed-loop starting from raising problems to solving problems, which is known as the “double-fly-
wheels” of Continuous Delivery 2.0.
It consists of two connected loops: The first one is the “Discovery Loop”, mainly aiming at identi-
fying and defining the business problems and working out a minimum viable solution for entering the
second loop, namely, the “Verification Loop”. The main objective of the second loop is to deliver the
minimum viable solution at the top speed, collect real feedback in a reliable manner, analyze and verify
the effect of this solution so as to determine the next action, as shown in Figure 1.10.
The Discovery Loop consists of four steps in a sustainable cycle: questioning, targeting, co-creation,
and refining.

1. “Questioning” is to define problems. By raising targeted questions to customers, we can find out
their specific requirements, and the underlying reasons for these requirements, that is, the fun-
damental problems to be addressed after satisfying these requirements. In this way, the project
team can know well about what goals they will achieve or what problems they will solve. Any
problems in vague definition will lead to biased solutions. So we must clearly define problems
before searching for their solutions.
2. “Targeting” is to define the target designator. After defining the business problems, we should
collect and analyze the relevant data, remove interference information, identify assumptions,
obtain the appropriate measurable metrics, and use them to describe the current situation, and
at the same time discuss and define the desired outcome of our next action.
3. “Co-creation” is to jointly discover a variety of viable solutions to a problem, solve this problem
together in a creative manner, or have these solutions to be verified.
4. “Refining” is to select the minimum viable solution from all viable solutions. It may be a single
solution or a combination of multiple solutions.

The Verification Loop also consists of four steps in a sustainable cycle: building, running, monitor-
ing, and decision-making.

1. “Building” is to accurately convert the minimum viable solution into a software package that
meets the quality requirements according to the non-digital description.
2. “Running” is to deploy the software package that meets the quality requirements in a produc-
tion environment or pass it on to users and make it at their service.
3. “Monitoring” is to collect the data generated in the production system and monitor the system
to ensure its normal operation. At the same time, it presents the business data in an appropriate
form and in a timely manner.
4. “Decision-making” is to analyze the collected data and compare them with the corresponding
target obtained at the Discovery Loop, so as to decide the direction of the next step.

The Discovery Loop is like the front wheels of a car that decide its driving direction, while the
Verification Loop is like the rear wheels that drive the car to go fast and smoothly. The two loops are
supporting each other: the smaller scale of the viable solution generated at the Discovery Loop can speed
up the running of the Verification Loop; while the more rapidly running Verification Loop can help the
Discovery Loop to get real feedback as soon as possible, which is to the benefit of quick decision-making
and timely verification or adjustment of the direction of progress.

1.2.3 Four Core Principles


“Continuous Delivery 2.0” refers to a capability of enterprises, namely, while ensuring high quality and
low cost and mitigating risks, an enterprise can accelerate the double-flywheel in a sustainable way, and
maintain frequent interaction with external entities to obtain timely and real feedback, so as to create
more customer value in the end.
1 • Continuous Delivery 2.0 11

The four essential principles for accelerating the double-flywheels of Continuous Delivery 2.0 are
introduced as follows:

1.2.3.1 Doing less


“Our biggest problem is lack of human resource” – a common complaint from my customers when I
worked with them. No matter their firm is big or small sized, it seems that their ambition always exceeds
their delivery capacity and they have endless requirements to deal with. Is that really good to do more?
Definitely not. In fact, we should resist the temptation to “build the best features through a fair amount of
planning”, and insist on doing less and finding ways to verify new ideas as early as possible.
As Mike Moran has commented in his book Do It Wrong Quickly, Netflix believes that 90% of the
things they want to do are probably wrong. In a paper Online Experimentation at Microsoft, Ron Kohavi
and others5 pointed out that Microsoft has elaborately designed and developed a host of functional fea-
tures to improve key metrics, but only about a third of them can make it.
When asked how can five engineers support 40 million users? Mike Krieger, co-founder and CTO of
Instagram, succinctly put it: “Do less, and do the simple thing first”.

1.2.3.2 Continuous decomposition of problems


Complex business problems are bound to contain many uncertainties that affect the speed and effect
of problem-solving. By decomposing the problems layer by layer before implementing the solution,
the team can have a better understanding of the business and identify risks more rapidly. Enterprises
should believe that even a big problem or a wide range of changes can be broken down into a number
of small changes that are available for quick resolution and feedback, thereby eliminating risks as
early as possible. Compared with designing a bunch of features and planning a one-time release that
lasts for months, a far better approach is to keep trying out new ideas and delivering them to users
separately.

1.2.3.3 Insistence on fast feedback (FFB)


After breaking down the problems, we should not immerse ourselves in our work and ignore the feedback
on the results of each completed task, otherwise we may lose the other half of the benefits from problem
decomposition, that is, we may fail to confirm that the risks are mitigated or eliminated. Only in the way
of FFB can we get an early idea of the quality and effect of the accomplished work.

1.2.3.4 Continuous improvement and measurement


No matter what improvement is made, if we fail to measure its results in some way, how can we prove that
it is really done? Before we start to tackle a problem, we need to find an appropriate measurement and put
it on an equal footing with the corresponding functional requirements, and then fulfill these requirements
together with problem-solving.
For example, a data company once failed to put forward a correct direction for improvement due to
lack of metric data. It had developed a software platform for recruiting data labeling volunteers. The prod-
uct was divided into multiple iterations, and each iteration would have several features released, but no
features were available for data collection and statistical analysis, which were major concerns of product
staff. As a result, the team knew that this platform could help people complete their work, but they had no
idea whether they could work efficiently, nor could they determine the direction for next-step optimization
of the platform.

5 See https://ai.stanford.edu/~ronnyk/ExPThinkWeek2009Public.pdf
12 Continuous Delivery 2.0

FIGURE 1.11 Tangram of Continuous Delivery.

1.2.4 Tangram of Continuous Delivery


The above subsections have discussed the guiding ideology, work philosophy and core principles of Continuous
Delivery 2.0. It impresses people that it is fairly effective in adapting to the rapidly changing market environ-
ment and fierce market competition. So, how can companies make Continuous Delivery 2.0 an organizational
capability and their DNA, as well as their own competitive edge that gets ahead of their rivals.
The implementation and improvement of the Double-flywheels Model for Continuous Delivery
involve multiple departments and different roles within an enterprise, rather than accomplished indepen-
dently by a single department. The idea, philosophy and principles of Continuous Delivery 2.0 must be
implemented across the entire enterprise. Therefore, an enterprise has to take actions in three aspects:
organizational management mechanism, infrastructure, and software architecture, each of which contains
multiple elements, as shown in Figure 1.11.
All roads lead to Rome, and Rome wasn’t built in a day. In terms of Continuous Delivery, each
enterprise may have a different implementation path, different length of required period, and different
requirements for various aspects of capacity. Just like playing the “tangram”, a traditional Chinese intelli-
gence-exploit toy with which each kid can make his own designed shape, each enterprise should draw an
exclusive blueprint for continuous delivery practice according to its own will and demands.

1.3 SUMMARY
Continuous Delivery 2.0 is based on the two philosophies of Continuous Delivery 1.0: “quick software
release in a sustainable way” and “MVP” for lean startup. It stresses to be business-oriented, decompose
business problems from the beginning, and find the correct direction of progress quickly through continu-
ous scientific discovery and rapid verification while reducing waste, which is referred to as the “Double-
flywheels Model”. Therefore, Continuous Delivery 2.0 involves multiple teams in an organization, and
requires their close cooperation to shorten the cycle of the double-flywheels, as shown in Figure 1.12.
1 • Continuous Delivery 2.0 13

FIGURE 1.12 Different roles in Continuous Delivery 2.0.

Continuous Delivery 2.0 follows four essential working principles: doing less, continuous decomposi-
tion of problems, insistence on FBB, and continuous improvement and measurement. Only in this way can
we keep accelerating double-flywheels of continuous delivery and speed up user feedback, and thereby
improving business agility. This requires leadership to overcome the “fear of failure”, replace the original
thinking model of software delivery management with the one of “scientific discovery – fast delivery”,
attempt quick trial and error, improve continuous delivery capability, and then develop existing businesses
and look for new business opportunities.
Another random document with
no related content on Scribd:
THE SERVANT OF GOD
SR. THÉRÈSE OF THE CHILD JESUS
AND OF THE HOLY FACE
Carmelite of the Monastery of Lisieux 1873-1897
“Blessed Margaret Mary having had two whitlows used to say she
had only really suffered from the first one, because it had not been
possible for her to hide the second from her Sisters, and thus it
became the object of their compassion.
“This feeling is natural to us; yet to wish that all should know when
we suffer is a very commonplace manner of acting.”
counsels and reminiscences
During the first months of her illness it was on her hard palliasse that
Sister Thérèse passed the time of rest, and her nights were very
bad: when asked whether she did not need some assistance during
those hours of pain, she replied: “Oh, no on the contrary, I think
myself very fortunate to be in a cell distant enough for my Sisters not
to hear me. I rejoice to suffer alone; but from the moment I am pitied
and surrounded with delicate attentions I can no longer feel this joy.”
hist. d’une ame, ch. xii
The Sister infirmarian remarking, “It is said that you have never
suffered very much.” Thérèse smiled and pointing to a glass
containing a draught of medicine, bright red in colour, replied, “See
this little glass, one would imagine it full of some choice liqueur, but
in reality I take nothing that is more bitter. Well! it is an image of my
life; to the eyes of others it has ever appeared clothed in the most
radiant hues; to them it seemed as though I drank a delicious liqueur,
while in truth it was bitterness. I say bitterness, and yet my life has
not been bitter, for I have known how to make of all bitterness my
sweetness and my joy.”
“You are in great pain at this moment, are you not?” “Yes ... but I
have so much desired to suffer.”
hist. d’une ame, ch. xii
“How it grieves us to see you suffer, and to think you may perhaps
have still more to endure,” the novices were saying to her.
“Oh! do not be troubled about me, I have arrived at the stage of
being no longer able to suffer, because all suffering is sweet to me.”
hist. d’une ame, ch. xii
A sister, who doubted her patience, noticed, when visiting her one
day, an expression of heavenly joy on her countenance and wished
to know the reason. “It is because of the very acute pain I am
feeling,” replied Thérèse, “I have always striven to love suffering and
to give it a cordial welcome.”
hist. d’une ame, ch. xii
“Why are you so gay this morning?” she was asked; “It is because I
have had two little trials, nothing gives me little joys like little trials.”
hist. d’une ame, ch. xii
Another time: “You have had a great many trials today.”
“Yes, but ... seeing that I love them!... I love everything the good God
sends me.”
hist. d’une ame, ch. xii
Again, when some one said to her: “It is dreadful—all you are
suffering.”
“No, it is not dreadful; could a little Victim of Love find anything
dreadful that her Spouse sends her? He gives me at each moment
what I can bear; not more; and the minute He increases my
sufferings He also augments my fortitude.
“Yet I could never ask for greater sufferings, for I am too little; they
would be my own—my own choosing, then I should have to bear
them by myself, and I have never been able to do anything all
alone.”
hist. d’une ame, ch. xii
During her long and painful agony she exclaimed: “The chalice is full
to the brim. Never could I have believed it possible to suffer so
much.... I can only find the explanation in my extreme longing to
save souls.... Oh! I would not suffer less.”
hist. d’une ame, ch. xii

FOOTNOTES:
[88] No reader should be discouraged by this chapter on
Suffering. What Sœur Thérèse says is very consoling for those
who are nailed to the Cross; and others must remember that God
had given to His humble Servant a sensible attraction for
suffering, which is a rare grace and reserved to very few souls,
though many imagine they possess it, and mistake their road
choosing to follow this supposed attraction. Without the sensible
desire and even though experiencing an invincible repugnance to
suffer, souls can be sanctified. What pleases God is that the
suffering be borne with love.
[89] Imit., III, ch. xxvi, 3.
[90] Ps., xci, 5.
[91] Cf. Eccles., xxiv, 29.
[92] Matt., xxvi, 39.
[93] Isa., lxiii, 3.
[94] Isa., lxiii, 5.
THE DIRECTION OF SOULS
Our Lord allowed me the consolation of closely studying the souls of
children.
Considering these innocent souls I used to compare them to soft
wax, upon which any impression may be stamped, the bad, alas! like
the good; and I understood those words of Jesus: “But he that shall
scandalize one of these little ones ... it were better for him that a
millstone were hanged about his neck and that he were drowned in
the depth of the sea.”[95] Oh! how many souls might attain to a high
degree of holiness if wisely guided from the very first.
I well know that to accomplish His work of sanctification, God has
need of no one, but just as He enables a skilful gardener to rear
plants that are delicate and rare, granting him for this end all the
knowledge necessary, while reserving to Himself the care of giving
the increase, so, too, does He will to be aided in His divine culture of
souls.
hist. d’une ame, ch. v
As soon as I penetrated into the sanctuary of souls I judged at the
first glance that the task exceeded my powers, and very quickly
placing myself in the arms of the good God I imitated the little child,
who, seized with sudden fear, tries to hide its golden head on its
father’s shoulder, and I said: “Lord, Thou knowest it, I am myself too
little to be capable of nurturing these Thy children; if Thou dost will to
give to them, through me, what is suited to each one, fill Thou my
little hand, and without leaving Thine arms, without even turning my
head aside, I will distribute Thy treasures to the souls who come to
me to seek sustenance. When they find it to their liking I shall know
that it is not to me they owe it but to Thee; on the other hand if they
complain, and find bitter what I offer them, my peace shall remain
undisturbed, I will try to convince them that this nutriment comes
from Thee, and I will carefully refrain from seeking any other for
them.”
When I thus understood that it was impossible for me to do anything
by myself, the task appeared to me simplified. Interiorly, I occupied
myself solely in trying to become more and more united to God—
knowing that the rest would be added unto me.
hist. d’une ame, ch. x
From afar it seems easy to do good to souls, to make them love God
more, to mould them after our own views and opinions. But coming
closer one feels, on the contrary, that to do good without the divine
assistance, is as impossible a thing as to bring the sun back after it
has set. One feels that it is absolutely necessary to forget our own
inclinations, our personal notions, and to guide souls, not by our own
way—the way we ourselves go—but by the particular way that Jesus
wishes to lead them.
hist. d’une ame, ch. x
When very young, and staying with my aunt, a book was given to me
to read. In one of the stories I saw that the mistress of a school was
much praised because she knew how to get on cleverly in the world
without offending anyone. This phrase I remarked especially: “She
would say to the one, ‘You are not wrong,’ and to the other, ‘You are
right;’” and while I read I was thinking: “Oh!... I would not have acted
thus; we must always tell the truth.” And so I do, always. Far more
difficult it certainly is, for when told of some little vexatious
occurrence, it would be easy to lay blame on the absent, and she
who complains would at once be pacified. Yes, but ... I do quite the
reverse. If I am not liked, what matter! Let no one come to me who
does not want to hear the truth.
counsels and reminiscences
That a reprimand may be fruitful it must cost in the giving; and it
must be given without a shade of passion in the heart.
One must not let kindness degenerate into weakness. When we
have blamed justly we ought to leave it so, and not yield to feelings
of distress at having given pain. To run after the aggrieved one in
order to console her, is to do more harm than good. To leave her to
herself is to force her to expect nothing from creatures, to have
recourse to the good God, to see her failings and to humble herself.
Otherwise she would grow accustomed to being consoled after a
deserved rebuke and would behave as does a spoilt child, who
stamps and cries, well knowing that this will make its mother return
to wipe away the tears.
counsels and reminiscences
Let the sword of the Spirit, which is the Word of God, abound in your
lips and hearts.[96] If we have to do with a difficult soul, let us not be
discouraged, nor ever abandon her. Let us have always “the sword
of the Spirit” to reprehend her for her faults, and not allow things to
pass for the sake of leaving ourselves in repose; let us fight
unceasingly, even without hope of gaining the victory. What matter
about success! Let us fight on, whatever be the weariness of the
struggle. Let us not say: “I can make nothing of this soul, she does
not understand; I must give it up.” Oh! what cowardice that would be.
We must do our duty unto the end.
[Sœur Thérèse in these passages refers to her charge as
Mistress of Novices.]
counsels and reminiscences
The novices expressed their surprise at finding that she guessed
their most hidden thoughts.
“Here is my secret,” she said to them: “I never give you any advice
without invoking the Blessed Virgin; I ask her to inspire me to say
what will do you the most good, and I myself am often astonished at
the things that I teach you. I simply feel in saying them to you, that I
am not deceived and that Jesus speaks to you by my mouth.”
hist. d’une ame, ch. xii
“Give us some advice as to how we ought to act concerning our
spiritual direction,” they said to her.
“With great simplicity and without depending too much on
assistance, which may fail you at any moment. You would soon be
forced to say with the Spouse in the Canticles: ‘The keepers ... took
away my veil from me and wounded me,’ it was only ‘when I had a
little passed by them I found Him whom my soul loveth.’[97] If with
detachment you humbly inquire where is your Beloved, the keepers
will direct you. Nevertheless, most frequently, you will find Jesus only
after you have passed by all creatures. For my part, I have many a
time repeated this verse of the Spiritual Canticle of St. John of the
Cross:

Send me no more
A messenger
Who cannot tell me what I seek.
All they who serve
Relate a thousand graces of Thee;
And all wound me more and more,
And they leave me dying,
While they babble I know not what.”

Trans. D. Lewis, M. A.
counsels and reminiscences

FOOTNOTES:
[95] Matt., xviii, 6.
[96] Ephes., vi, 17.
[97] Cant., v, 7; iii, 4.
THE BLESSED VIRGIN
How I love the Blessed Virgin! Had I been a Priest, oh! how I should
have spoken of her. She is represented as unapproachable, rather
ought she to be shown as imitable. She is more Mother than Queen.
I have heard it said that all the Saints are eclipsed by her radiant
brightness as the sun at rising makes the stars disappear. How
strange that seems—a mother eclipsing the glory of her children! I
think quite the contrary. I believe that she will immensely increase
the splendour of the elect.... The Virgin Mary! how simple does her
life appear to me....
hist. d’une ame, ch. xii
Sometimes I find myself saying to the Holy Virgin: “Do you know, O
cherished Mother, that I think myself more fortunate than you? I have
you for Mother and you have not, like me, the Blessed Virgin to
love.... You are, it is true, the Mother of Jesus, but you have given
Him to me, and He, from the cross gave you to us as our Mother, so
we are richer than you. Of old it was your desire that you might be
the little handmaiden of the Mother of God; and I, poor little creature,
I am, not your servant, but your child: you are the Mother of Jesus
and you are my Mother.”
xiii letter to her sister céline
O Mary, if I were Queen of Heaven and thou wert Thérèse, I fain
would be Thérèse to see thee Queen of Heaven!
8 September, 1897.
Last words written by Sister Thérèse
of the Child Jesus.
VARIOUS SUBJECTS
Léonie, no doubt finding that she was growing too old to play with
dolls, came one day to Céline and me, with a basket full of dolls’
clothes, odds and ends of pretty materials, trimmings, etc., on which
she had laid her doll, saying to us: “There, little sisters—choose!”
Céline looked, and took a knot of edging. After reflecting a moment I,
in turn, put out my hand saying: “I choose all!” and I carried off
basket and doll without further ceremony.
This trait of my childhood is, as it were, a summary of my entire life.
Later on when the meaning of perfection began to unfold itself to me,
I understood that to become a saint it is necessary to suffer much,
ever to seek after that which is most perfect, and to forget self. I
understood that in sanctity the degrees are many, that each soul is
free to respond to the advances of our Lord, to do little or much for
His sake, in a word, to choose between the sacrifices that He asks.
Then, as in the days of my childhood, I exclaimed: “My God, I
choose all! I do not wish to be a saint by halves; the thought of
suffering for Thee does not frighten me, one thing only do I fear—my
own will; take Thou my will, for I choose all that Thou willest.”
hist. d’une ame, ch. i
My Father took me for a pleasant tour during which I began to know
a little of the world. Around me all was gaiety and delight; I was
made welcome, petted, admired, in short, for fifteen days the
pathway of my life was strewn with naught but flowers. Holy Wisdom
well says that the bewitching of trifles overturneth the innocent mind.
[98] At the age of ten the heart easily allows itself to be dazzled, and I
own that this sort of life had some charms for me. Alas! how well the
world contrives to reconcile the delights of earth with the service of
God. How seldom does it think of death.
And death, nevertheless, has come to a great many of the people
whom I then knew, young, rich and prosperous. I like to go back in
thought to their beautiful dwellings, to ask myself where are they,
and what benefit do they now draw from the castles and parks where
I saw them enjoying all the comforts of life.... And I reflect that “all is
vanity”[99] “but to love God and to serve Him alone.”[100]
hist. d’une ame, ch. iv
What compassion I have for souls who are going astray. It is so easy
to lose one’s way in the flowery paths of the world. Undoubtedly for a
soul who has risen a little above the things of earth, the sweetness
offered is intermingled with bitterness, and the immense void of its
desires cannot be filled by the praises of a moment.
hist. d’une ame, ch. iv
We have but life’s brief day to save souls and thus to give to Jesus
proof of our love. The morrow of this day will be Eternity and then He
will render to you a hundredfold for the joys which you have
sacrificed for Him. He knows the extent of your sacrifice, He knows
that the grief of those dear to you increases your own still more; but
to save our souls He has Himself suffered this martyrdom. He too,
left His Mother, He saw the Immaculate Virgin stand at the foot of the
Cross, her heart transpierced by the sword of sorrow.
Ah! if the Divine Master would but grant to those whom you are
going to leave for His sake, a foresight of the glory He reserves for
you, the multitude of souls who in heaven will form your train, they
would be already recompensed for their great sacrifice in parting with
you.
ii letter to her missionary “brothers”
The good God has promised a hundredfold to all who have left father
or mother or sister for love of Him. These words are, I know, usually
applied to those who have entered the religious state, but in my
heart I feel that they were also spoken for the generous parents who
make to God the sacrifice of children whom they cherish more than
self.
letter to her cousin jeanne guérin
How can a heart given up to human affection be united intimately to
God? That, I feel is not possible. I have seen so many souls deluded
by this treacherous light, dart into it like the poor moth and burn their
wings, then return wounded to Jesus, the Divine Fire which burns
without consuming.
hist. d’une ame, ch. iv
In giving oneself to God the heart does not lose its natural
tenderness; on the contrary, its love grows deeper by becoming
more pure and more Christlike.
hist. d’une ame, ch. ix
There are souls for whom God’s mercy wearies not of waiting, and to
whom He gives His light only by degrees.
hist. d’une ame, ch. x
When in the morning we feel no courage, no energy for the practice
of virtue, this is a grace, this is the moment to “lay the axe to the root
of the tree”[101] depending solely on Jesus. If we fall all is retrieved
by an act of love, and Jesus smiles. He helps us without appearing
to do so, and the tears which the wicked cause Him to shed are
dried by our poor feeble love.
ii letter to her sister céline
We must practice the little virtues. This is difficult sometimes, but the
good God never refuses the first grace, which gives courage to
conquer self: if the soul corresponds to it she will find that she
immediately receives light. I have ever been struck with those words
of praise to Judith: “Thou hast done manfully, and thy heart has been
strengthened.”[102] We must first act with courage, then the heart is
strengthened and we go from victory to victory.
counsels and reminiscences
My God, how varied are the ways by which Thou dost lead souls. In
reading the Lives of the Saints we find a great number of whom
nothing has remained to us after their death: not the smallest
souvenir, not a written line. Others there are, on the contrary, like our
Holy Mother Saint Teresa, who have enriched the Church with their
sublime doctrine, not fearing to reveal the secrets of the King,[103] in
the hope that souls might know Him better and love Him more.
Which of these two ways pleases our Lord best? It seems to me that
they are equally pleasing to Him.
All the well-beloved of God have followed the inspiration of the Holy
Spirit by whom the Prophet wrote: “Say to the just that all is
well.”[104] Yes, all is well when we seek only the Divine Will.
hist. d’une ame, ch. ix
How narrow are the thoughts of creatures! When they see that a soul
has lights which surpass their own, they conclude that the Divine
Master loves them less. Since when, then, has He lost the right to
make use of one of His creatures, in order to dispense to His
children the sustenance needful for them?
hist. d’une ame, ch. x
When we are grieved at our powerlessness to do good, our only
resource is to offer to God the works of others. In this you see the
benefit of the communion of Saints.
counsel and reminiscences
Though I do not undervalue beautiful thoughts that seem to unite us
to God, I have long understood that we must carefully guard against
leaning too much upon them. The most sublime inspirations are
nothing without deeds.
Other souls, it is true, may draw therefrom much profit if they testify
humble gratitude to God for being permitted to share the feast of one
of His privileged children. But if the privileged one were to grow vain
of her spiritual riches, if her prayer resembled that of the Pharisee,
she herself would become like to a person starving to death before a
well-served table while all her guests take from it abundant
nourishment, and cast perhaps a look of envy on the possessor of so
much wealth.
hist. d’une ame, ch. x
The Well-Beloved has no need of our glorious deeds nor of our fine
thoughts. If He desire sublime conceptions has He not His Angels,
whose knowledge surpasses infinitely that of the world’s greatest
geniuses? It is not then either intellect or talent that He looks for here
below.... He has called Himself the Flower of the Field[105] to show
us how much He cherishes simplicity.
xiv letter to her sister céline
To keep the word of Jesus, this is the sole condition of our
happiness, the proof of our love for Him; and this word—it seems to
me that it is Himself, since He is called the Uncreated Word of the
Father.
xviii letter to her sister céline
Jesus needs neither books nor Doctors of Divinity in order to instruct
souls; He, the Doctor of Doctors, He teaches without noise of words.
hist. d’une ame, ch. viii
As Text there is a verse which I utter unwillingly each day. It is this:
“Inclinavi cor meum ad faciendas justificationes tuas in æternum,
propter retributionem.” (I have inclined my heart to do Thy
justifications for ever, because of the reward.)[106] Interiorly, I hasten
to say: “O my Jesus, Thou knowest well that it is not for the reward I
serve Thee, but solely because I love Thee, and for the sake of
saving souls.”
counsels and reminiscences
In Heaven only shall we see the absolute truth concerning all things.
On earth, even in the Holy Scripture, there is a certain obscurity: it
grieves me to see differences in the translations; had I been a Priest
I would have learned Hebrew, so that I might be able to read the
Word of God in that human language in which He deigned to
express it.
counsels and reminiscences
We are not yet in our Fatherland, and temptation must purify us as
gold is purified by the action of fire.
xviii letter to her sister céline
It is best not to expose oneself to the combat when defeat is certain.
hist. d’une ame, ch. ix
God is often satisfied with our desire of labouring for His glory.
hist. d’une ame, ch. ix
A soul in the state of grace has nothing to fear from the demons,
who are cowards, capable of flight before the gaze of a child.
hist. d’une ame, ch. i
To the pure all is pure,[107] the simple and upright soul sees not evil
in anything, since evil exists in impure hearts only and not in material
objects.
hist. d’une ame, ch. vi
The good God has told us that in the Last Day He “will wipe away all
tears from our eyes,”[108] and without doubt, the more tears to be
dried, the greater will be the consolation.
iii letter to sr. marie du sacré-cœur
The Spouse in the Canticles, not having been able in repose to find
her Beloved, arose, she says and went about the city to seek Him,
but in vain ... she could not find Him save beyond the ramparts. It is
not the will of Jesus that we should find His adorable Presence
without effort. He hides Himself, He envelopes Himself with
darkness.... It was not thus He acted in regard to the multitudes, for
we read in the Gospels that the people were in admiration when He
spoke.
Weak souls Jesus charmed by His divine utterances. He was trying
to render them strong for the day of temptation and of trial; but small,
truly, was the number of His faithful friends when He was silent[109]
before His judges. Oh, what melody for my heart is that silence of
the Divine Master.
xv letter to her sister céline
I have read in the Holy Gospel that the Divine Shepherd leaves in
the desert all His faithful flock, to go in haste after the sheep that is
lost. How touching is this confidence. He is sure of them, they are
captives of love—how could they break away? Even so does the
well-beloved Shepherd of our souls rob us of the sense of His
presence in order to give to sinners His consolations; or else, if He
leads us to Mount Thabor it is for one moment ... the valleys are
nearly always the place of pasture, it is there He takes His repose at
mid-day.[110]
xvi letter to her sister céline
The sole crime with which Herod reproached our Lord was folly ...
and frankly, that charge was true. Yes, it was folly to come seeking
the poor shallow hearts of mortals, therein to make His throne. He,
the King of Glory Who sitteth above the Cherubim! Was not His
happiness complete in the company of His Father and the Spirit of
Love? Why come to earth to seek out sinners and to make of them
His friends, His chosen companions?
xx letter to her sister céline
Mary, breaking the fragile vase poured out upon the Head of her
Saviour a perfume of great price[111] and the whole house was filled
with the fragrance thereof.[112]
The Apostles murmured against Magdalene; and this it is which still
happens with regard to us: some, even of the most fervent Christians
think we (Carmelites) are exaggerated, that like Martha we ought to
serve Jesus, instead of consecrating to Him the vases of our lives
with the perfumes which are hidden within. And yet what matters it—
the breaking of these vases—since our Lord is consoled, and the
world in spite of itself is made sensible of the fragrance they exhale.
And oh! how necessary are these perfumes to purify the unhealthy
atmosphere that it breathes.
xx letter to her sister céline
“One day when I was in tears,” relates a novice, “Sister Thérèse of
the Child Jesus told me to acquire a habit of hiding my little troubles,
adding that nothing renders community life more difficult than
inequality of temperament.
“You are quite right,” I answered, “I have thought so myself, and in
future I shall never cry but when alone with the good God; to Him
only shall I confide my trials, He will always understand and console
me.”
“Shed tears before the good God!” she replied with vivacity, “take
care you do no such thing. Still less, by far, before Him than before
creatures ought you to exhibit signs of sadness. He has but our
monasteries, this dear Master, to rejoice His Heart; He comes
amongst us to find a little repose, to forget the continual lamentations
of His friends in the world who for the most part, instead of
recognizing the value of the Cross, meet it with repining and with
tears; and would you behave like the generality of people?... Frankly
that is not disinterested love—disinterested love is for us to console
Jesus, not for Him to console us.
“He is, I know, so kind of heart that if you weep He will dry your
tears; but afterwards He will go away quite sorrowful, not being able
to find in you the repose He sought. Jesus loves the joyous heart, He
loves the ever smiling soul. When will you learn to hide your troubles
from Him, or to tell Him in gladsome tones that you are happy to
suffer for His sake?”
counsels and reminiscences
The soul is reflected in the countenance: like to a little child always
content, your countenance should be invariably calm and serene.
When you are alone be still the same, because you are ever in the
Angels’ sight.
counsels and reminiscences
Our Divine Lord wishes to have His court here below as on High, He
desires angel-martyrs, angel-apostles.
xi letter to her sister céline
A sister, greatly grieved at seeing her so ill often exclaimed: “Oh,
how sad is life!” But Sœur Thérèse would at once correct her,

You might also like