80238ffirs.

qxd:WileyRed 7/11/07 7:22 AM Page iii

The Shellcoders
Handbook
Discovering and Exploiting Security Holes
Second Edition

Chris Anley
John Heasman
Felix FX Linder
Gerardo Richarte
The Shellcoders Handbook: Discovering and Exploiting Security Holes
(1st Edition) was written by Jack Koziol, David Litchfield, Dave Aitel,
Chris Anley, Sinan Eren, Neel Mehta, and Riley Hassell.

Wiley Publishing, Inc.

80238ffirs.qxd:WileyRed 7/11/07 7:22 AM Page ii
80238ffirs.qxd:WileyRed 7/11/07 7:22 AM Page i

The Shellcoders Handbook

Second Edition
80238ffirs.qxd:WileyRed 7/11/07 7:22 AM Page ii
80238ffirs.qxd:WileyRed 7/11/07 7:22 AM Page iii

The Shellcoders
Handbook
Discovering and Exploiting Security Holes
Second Edition

Chris Anley
John Heasman
Felix FX Linder
Gerardo Richarte
The Shellcoders Handbook: Discovering and Exploiting Security Holes
(1st Edition) was written by Jack Koziol, David Litchfield, Dave Aitel,
Chris Anley, Sinan Eren, Neel Mehta, and Riley Hassell.

Wiley Publishing, Inc.

80238ffirs.qxd:WileyRed 7/11/07 7:22 AM Page iv

The Shellcoders Handbook, Second Edition: Discovering and Exploiting Security Holes

Published by
Wiley Publishing, Inc.
10475 Crosspoint Boulevard
Indianapolis, IN 46256
www.wiley.com

Copyright 2007 by Chris Anley, John Heasman, Felix FX Linder, and Gerardo Richarte
Published by Wiley Publishing, Inc., Indianapolis, Indiana
Published simultaneously in Canada

ISBN: 978-0-470-08023-8

Manufactured in the United States of America

10 9 8 7 6 5 4 3 2 1

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form
or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as
permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior
written permission of the Publisher, or authorization through payment of the appropriate per-copy fee
to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978)
646-8600. Requests to the Publisher for permission should be addressed to the Legal Department, Wiley
Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4355, or
online at http://www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or
warranties with respect to the accuracy or completeness of the contents of this work and specifically
disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No
warranty may be created or extended by sales or promotional materials. The advice and strategies con-
tained herein may not be suitable for every situation. This work is sold with the understanding that the
publisher is not engaged in rendering legal, accounting, or other professional services. If professional
assistance is required, the services of a competent professional person should be sought. Neither the
publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or
Website is referred to in this work as a citation and/or a potential source of further information does not
mean that the author or the publisher endorses the information the organization or Website may provide
or recommendations it may make. Further, readers should be aware that Internet Websites listed in this
work may have changed or disappeared between when this work was written and when it is read.

For general information on our other products and services or to obtain technical support, please con-
tact our Customer Care Department within the U.S. at (800) 762-2974, outside the U.S. at (317) 572-3993
or fax (317) 572-4002.

Library of Congress Cataloging-in-Publication Data

The shellcoders handbook : discovering and exploiting security holes / Chris Anley ... [et al.].
2nd ed.
p. cm.
ISBN 978-0-470-08023-8 (paper/website)
1. Computer security. 2. Data protection. 3. Risk assessment. I. Anley, Chris.

QA76.9.A25S464 2007
005.8 dc22
2007021079

Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons,
Inc. and/or its affiliates, in the United States and other countries, and may not be used without written
permission. All other trademarks are the property of their respective owners. Wiley Publishing, Inc., is
not associated with any product or vendor mentioned in this book.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may
not be available in electronic books.
80238ffirs.qxd:WileyRed 7/11/07 7:22 AM Page v

This book is dedicated to anyone and everyone who understands that

hacking and learning is a way to live your life, not a day job or
semi-ordered list of instructions found in a thick book.
80238ffirs.qxd:WileyRed 7/11/07 7:22 AM Page vi
80238ffirs.qxd:WileyRed 7/11/07 7:22 AM Page vii

About the Authors

Chris Anley is a founder and director of NGSSoftware, a security software,

consultancy, and research company based in London, England. He is actively
involved in vulnerability research and has discovered security flaws in a wide
variety of platforms including Microsoft Windows, Oracle, SQL Server, IBM
DB2, Sybase ASE, MySQL, and PGP.
John Heasman is the Director of Research at NGSSoftware. He is a prolific
security researcher and has published many security advisories in enterprise
level software. He has a particular interest in rootkits and has authored papers
on malware persistence via device firmware and the BIOS. He is also a co-author
of The Database Hackers Handbook: Defending Database Servers (Wiley 2005).
Felix FX Linder leads SABRE Labs GmbH, a Berlin-based professional con-
sulting company specializing in security analysis, system design creation, and
verification work. Felix looks back at 18 years of programming and over a
decade of computer security consulting for enterprise, carrier, and software
vendor clients. This experience allows him to rapidly dive into complex sys-
tems and evaluate them from a security and robustness point of view, even in
atypical scenarios and on arcane platforms. In his spare time, FX works with
his friends from the Phenoelit hacking group on different topics, which have
included Cisco IOS, SAP, HP printers, and RIM BlackBerry in the past.
Gerardo Richarte has been doing reverse engineering and exploit develop-
ment for more than 15 years non-stop. In the past 10 years he helped build the
technical arm of Core Security Technologies, where he works today. His cur-
rent duties include developing exploits for Core IMPACT, researching new
exploitation techniques and other low-level subjects, helping other exploit
writers when things get hairy, and teaching internal and external classes on
assembly and exploit writing. As result of his research and as a humble thank

vii
80238ffirs.qxd:WileyRed 7/11/07 7:22 AM Page viii

viii About the Authors

you to the community, he has published some technical papers and open
source projects, presented in a few conferences, and released part of his train-
ing material. He really enjoys solving tough problems and reverse engineering
any piece of code that falls in his reach just for the fun of doing it.
80238ffirs.qxd:WileyRed 7/11/07 7:22 AM Page ix

Credits

Executive Editor Vice President and Executive

Carol Long Group Publisher
Richard Swadley
Senior Development Editor
Kevin Kent Vice President and Executive
Publisher
Production Editor
Joseph B. Wikert
Eric Charbonneau
Compositor
Project Coordinator, Cover
Craig Johnson,
Adrienne Martinez
Happenstance Type-O-Rama
Copy Editor
Proofreader
Kim Cofer
Jen Larsen
Editorial Manager
Indexer
Mary Beth Wakefield
Johnna VanHoose Dinse
Production Manager
Anniversary Logo Design
Tim Tate
Richard Pacifico

ix
80238ffirs.qxd:WileyRed 7/11/07 7:22 AM Page x
80238ffirs.qxd:WileyRed 7/11/07 7:22 AM Page xi

Acknowledgments

I would first like to thank all of the people that have made this book possi-
ble the (many) authors, of course: Gerardo Richarte, Felix FX Linder, John
Heasman, Jack Koziol, David Litchfield, Dave Aitel, Sinan Eren, Neel Mehta, and
Riley Hassell. Huge thanks are also due to the team at Wiley our excellent
Executive Editor Carol Long and our equally excellent Development Editor
Kevin Kent. On a personal note Id like to thank the team at NGS for a great
many hangovers, technical discussions, hangovers, ideas, and hangovers.
Finally, Id like to thank my wife Victoria for her enduring patience, love, and
gorgeousness.
Chris Anley

I would like to thank my friends and family for their unwavering support.
John Heasman

I would like to thank my friends from Phenoelit, who are still with me despite
the turns and detours life takes and despite the strange ideas I have, technical
and otherwise. Special thanks in this context go to Mumpi, who is a very good
friend and my invaluable support in all kinds of activities. Additional thanks
and kudos go to the SABRE Labs team as well as to Halvar Flake, who is
responsible for the existence of this team in the first place. Last but not least, I
thank Bine for enduring me on a daily basis.
Felix FX Linder

I want to thank those in the community who share what excites them, their
ideas and findings, especially the amazing people at Core, past and present,
and my pals in the exploit writing team with whom the sudden discovery

xi
80238ffirs.qxd:WileyRed 7/12/07 6:28 AM Page xii

xii Acknowledgments

never ends it is quite often simple and enlightening. I also want to thank
Chris and John (co-authors) and Kevin Kent from Wiley Publishing, who all
took the time to go through my entangled English, turning it more than just
readable. And I want to thank Chinchin, my love, whos always by my side,
asking me questions when I need them, listening when I talk or am quiet, and
supporting me, always.
Gerardo Richarte
80238ftoc.qxd:WileyRed 7/11/07 7:22 AM Page xiii

Contents

About the Authors vii

Acknowledgments xi
Introduction to the Second Edition xxiii

Part I Introduction to Exploitation: Linux on x86

Chapter 1 Before You Begin 3
Basic Concepts 3
Memory Management 4
Assembly 6
Recognizing C and C++ Code Constructs in Assembly 7
Conclusion 10
Chapter 2 Stack Overflows 11
Buffers 12
The Stack 13
Functions and the Stack 15
Overflowing Buffers on the Stack 18
Controlling EIP 22
An Interesting Diversion 23
Using an Exploit to Get Root Privileges 25
The Address Problem 27
The NOP Method 33
Defeating a Non-Executable Stack 35
Return to libc 35
Conclusion 39

xiii
80238ftoc.qxd:WileyRed 7/11/07 7:22 AM Page xiv

xiv Contents

Chapter 3 Shellcode 41
Understanding System Calls 42
Writing Shellcode for the exit() Syscall 44
Injectable Shellcode 48
Spawning a Shell 50
Conclusion 59
Chapter 4 Introduction to Format String Bugs 61
Prerequisites 61
What Is a Format String? 61
What Is a Format String Bug? 63
Format String Exploits 68
Crashing Services 69
Information Leakage 70
Controlling Execution for Exploitation 75
Why Did This Happen? 84
Format String Technique Roundup 85
Conclusion 88
Chapter 5 Introduction to Heap Overflows 89
What Is a Heap? 90
How a Heap Works 91
Finding Heap Overflows 91
Basic Heap Overflows 93
Intermediate Heap Overflows 98
Advanced Heap Overflow Exploitation 105
Conclusion 107

Part II Other PlatformsWindows, Solaris, OS/X, and Cisco

Chapter 6 The Wild World of Windows 111
How Does Windows Differ from Linux? 111
Win32 API and PE-COFF 112
Heaps 114
Threading 115
The Genius and Idiocy of the Distributed Common
Object Model and DCE-RPC 116
Recon 118
Exploitation 120
Tokens and Impersonation 120
Exception Handling under Win32 122
Debugging Windows 124
Bugs in Win32 124
Writing Windows Shellcode 125
A Hackers Guide to the Win32 API 126
A Windows Family Tree from the Hackers Perspective 126
Conclusion 127
80238ftoc.qxd:WileyRed 7/11/07 7:22 AM Page xv

Contents xv

Chapter 7 Windows Shellcode 129

Syntax and Filters 129
Setting Up 131
Parsing the PEB 132
Heapoverflow.c Analysis 132
Searching with Windows Exception Handling 148
Popping a Shell 153
Why You Should Never Pop a Shell on Windows 153
Conclusion 154
Chapter 8 Windows Overflows 155
Stack-Based Buffer Overflows 156
Frame-Based Exception Handlers 156
Abusing Frame-Based Exception Handling on
Windows 2003 Server 161
A Final Note about Frame-Based Handler Overwrites 166
Stack Protection and Windows 2003 Server 166
Heap-Based Buffer Overflows 173
The Process Heap 173
Dynamic Heaps 173
Working with the Heap 173
How the Heap Works 174
Exploiting Heap-Based Overflows 178
Overwrite Pointer to RtlEnterCriticalSection in the PEB 178
Overwrite Pointer to Unhandled Exception Filter 185
Repairing the Heap 191
Other Aspects of Heap-Based Overflows 193
Wrapping Up the Heap 194
Other Overflows 194
.data Section Overflows 194
TEB/PEB Overflows 196
Exploiting Buffer Overflows and Non-Executable Stacks 197
Conclusion 203
Chapter 9 Overcoming Filters 205
Writing Exploits for Use with an Alphanumeric Filter 205
Writing Exploits for Use with a Unicode Filter 209
What Is Unicode? 210
Converting from ASCII to Unicode 210
Exploiting Unicode-Based Vulnerabilities 211
The Available Instruction Set in Unicode Exploits 212
The Venetian Method 213
An ASCII Venetian Implementation 214
Decoder and Decoding 218
The Decoder Code 219
Getting a Fix on the Buffer Address 220
Conclusion 221
80238ftoc.qxd:WileyRed 7/11/07 7:22 AM Page xvi

xvi Contents

Chapter 10 Introduction to Solaris Exploitation 223

Introduction to the SPARC Architecture 224
Registers and Register Windows 224
The Delay Slot 227
Synthetic Instructions 228
Solaris/SPARC Shellcode Basics 228
Self-Location Determination and SPARC Shellcode 228
Simple SPARC exec Shellcode 229
Useful System Calls on Solaris 230
NOP and Padding Instructions 231
Solaris/SPARC Stack Frame Introduction 231
Stack-Based Overflow Methodologies 232
Arbitrary Size Overflow 232
Register Windows and Stack Overflow Complications 233
Other Complicating Factors 233
Possible Solutions 234
Off-By-One Stack Overflow Vulnerabilities 234
Shellcode Locations 235
Stack Overflow Exploitation In Action 236
The Vulnerable Program 236
The Exploit 238
Heap-Based Overflows on Solaris/SPARC 241
Solaris System V Heap Introduction 242
Heap Tree Structure 242
Basic Exploit Methodology (t_delete) 263
Standard Heap Overflow Limitations 266
Targets for Overwrite 267
Other Heap-Related Vulnerabilities 270
Off-by-One Overflows 270
Double Free Vulnerabilities 270
Arbitrary Free Vulnerabilities 271
Heap Overflow Example 271
The Vulnerable Program 272
Other Solaris Exploitation Techniques 276
Static Data Overflows 276
Bypassing the Non-Executable Stack Protection 276
Conclusion 277
Chapter 11 Advanced Solaris Exploitation 279
Single Stepping the Dynamic Linker 281
Various Style Tricks for Solaris SPARC Heap Overflows 296
Advanced Solaris/SPARC Shellcode 299
Conclusion 311
80238ftoc.qxd:WileyRed 7/11/07 7:22 AM Page xvii

Contents xvii

Chapter 12 OS X Shellcode 313

OS X Is Just BSD, Right? 314
Is OS X Open Source? 314
OS X for the Unix-aware 315
Password Cracking 316
OS X PowerPC Shellcode 316
OS X Intel Shellcode 324
Example Shellcode 326
ret2libc 327
ret2str(l)cpy 329
OS X Cross-Platform Shellcode 332
OS X Heap Exploitation 333
Bug Hunting on OS X 335
Some Interesting Bugs 335
Essential Reading for OS X Exploits 337
Conclusion 338
Chapter 13 Cisco IOS Exploitation 339
An Overview of Cisco IOS 339
Hardware Platforms 340
Software Packages 340
IOS System Architecture 343
Vulnerabilities in Cisco IOS 346
Protocol Parsing Code 347
Services on the Router 347
Security Features 348
The Command-Line Interface 348
Reverse Engineering IOS 349
Taking the Images Apart 349
Diffing IOS Images 350
Runtime Analysis 351
Exploiting Cisco IOS 357
Stack Overflows 357
Heap Overflows 359
Shellcodes 364
Conclusion 373
Chapter 14 Protection Mechanisms 375
Protections 375
Non-Executable Stack 376
W^X (Either Writable or Executable) Memory 381
Stack Data Protection 388
AAAS: ASCII Armored Address Space 394
ASLR: Address Space Layout Randomization 396
Heap Protections 399
Windows SEH Protections 407
Other Protections 411
80238ftoc.qxd:WileyRed 7/11/07 7:22 AM Page xviii

xviii Contents

Implementation Differences 413

Windows 413
Linux 417
OpenBSD 421
Mac OS X 422
Solaris 423
Conclusion 425

Part III Vulnerability Discovery

Chapter 15 Establishing a Working Environment 429
What You Need for Reference 430
What You Need for Code 430
gcc 430
gdb 430
NASM 431
WinDbg 431
OllyDbg 431
Visual C++ 431
Python 432
What You Need for Investigation 432
Useful Custom Scripts/Tools 432
All Platforms 434
Unix 434
Windows 435
What You Need to Know 436
Paper Archives 438
Optimizing Shellcode Development 439
Plan the Exploit 439
Write the Shellcode in Inline Assembler 439
Maintain a Shellcode Library 441
Make It Continue Nicely 441
Make the Exploit Stable 442
Make It Steal the Connection 443
Conclusion 443
Chapter 16 Fault Injection 445
Design Overview 447
Input Generation 447
Fault Injection 450
Modification Engines 450
Fault Delivery 455
Nagel Algorithm 455
Timing 455
Heuristics 456
Stateless versus State-Based Protocols 456
Fault Monitoring 456
Using a Debugger 457
FaultMon 457
80238ftoc.qxd:WileyRed 7/11/07 7:22 AM Page xix

Contents xix

Putting It Together 458

Conclusion 459
Chapter 17 The Art of Fuzzing 461
General Theory of Fuzzing 461
Static Analysis versus Fuzzing 466
Fuzzing Is Scalable 466
Weaknesses in Fuzzers 468
Modeling Arbitrary Network Protocols 469
Other Fuzzer Possibilities 469
Bit Flipping 469
Modifying Open Source Programs 470
Fuzzing with Dynamic Analysis 470
SPIKE 471
What Is a Spike? 471
Why Use the SPIKE Data Structure to Model Network Protocols? 472
Other Fuzzers 480
Conclusion 480
Chapter 18 Source Code Auditing:
Finding Vulnerabilities in C-Based Languages 481
Tools 482
Cscope 482
Ctags 483
Editors 483
Cbrowser 484
Automated Source Code Analysis Tools 484
Methodology 485
Top-Down (Specific) Approach 485
Bottom-Up Approach 485
Selective Approach 485
Vulnerability Classes 486
Generic Logic Errors 486
(Almost) Extinct Bug Classes 487
Format Strings 487
Generic Incorrect Bounds-Checking 489
Loop Constructs 490
Off-by-One Vulnerabilities 490
Non-Null Termination Issues 492
Skipping Null-Termination Issues 493
Signed Comparison Vulnerabilities 494
Integer-Related Vulnerabilities 495
Different-Sized Integer Conversions 497
Double Free Vulnerabilities 498
Out-of-Scope Memory Usage Vulnerabilities 499
Uninitialized Variable Usage 499
Use After Free Vulnerabilities 500
Multithreaded Issues and Re-Entrant Safe Code 500
80238ftoc.qxd:WileyRed 7/11/07 7:22 AM Page xx

xx Contents

Beyond Recognition: A Real Vulnerability versus a Bug 501

Conclusion 501
Chapter 19 Instrumented Investigation: A Manual Approach 503
Philosophy 503
Oracle extproc Overflow 504
Common Architectural Failures 508
Problems Happen at Boundaries 508
Problems Happen When Data Is Translated 509
Problems Cluster in Areas of Asymmetry 511
Problems Occur When Authentication and
Authorization Are Confused 512
Problems Occur in the Dumbest Places 512
Bypassing Input Validation and Attack Detection 513
Stripping Bad Data 513
Using Alternate Encodings 514
Using File-Handling Features 515
Evading Attack Signatures 517
Defeating Length Limitations 517
Windows 2000 SNMP DOS 520
Finding DOS Attacks 521
SQL-UDP 522
Conclusion 523
Chapter 20 Tracing for Vulnerabilities 525
Overview 526
A Vulnerable Program 527
Component Design 529
Building VulnTrace 538
Using VulnTrace 543
Advanced Techniques 546
Conclusion 548
Chapter 21 Binary Auditing: Hacking Closed Source Software 549
Binary versus Source-Code Auditing: The Obvious Differences 550
IDA ProThe Tool of the Trade 550
Features: A Quick Crash Course 551
Debugging Symbols 552
Binary Auditing Introduction 552
Stack Frames 552
Calling Conventions 554
Compiler-Generated Code 556
memcpy-Like Code Constructs 560
strlen-Like Code Constructs 560
C++ Code Constructs 561
The this Pointer 561
Reconstructing Class Definitions 562
vtables 562
Quick but Useful Tidbits 563
80238ftoc.qxd:WileyRed 7/11/07 7:22 AM Page xxi

Contents xxi

Manual Binary Analysis 563

Quick Examination of Library Calls 564
Suspicious Loops and Write Instructions 564
Higher-Level Understanding and Logic Bugs 565
Graphical Analysis of Binaries 566
Manual Decompilation 566
Binary Vulnerability Examples 566
Microsoft SQL Server Bugs 566
LSDs RPC-DCOM Vulnerability 567
IIS WebDAV Vulnerability 568
Conclusion 570

Part IV Advanced Materials

Chapter 22 Alternative Payload Strategies 573
Modifying the Program 574
The SQL Server 3-Byte Patch 575
The MySQL 1-Bit Patch 578
OpenSSH RSA Authentication Patch 580
Other Runtime Patching Ideas 581
GPG 1.2.2 Randomness Patch 583
Upload and Run (or Proglet Server) 584
Syscall Proxies 584
Problems with Syscall Proxies 587
Conclusion 596
Chapter 23 Writing Exploits that Work in the Wild 597
Factors in Unreliability 597
Magic Numbers 597
Versioning 598
Shellcode Problems 599
Countermeasures 601
Preparation 602
Brute Forcing 602
Local Exploits 603
OS/Application Fingerprinting 603
Information Leaks 605
Conclusion 606
Chapter 24 Attacking Database Software 607
Network Layer Attacks 608
Application Layer Attacks 618
Running Operating System Commands 619
Microsoft SQL Server 619
Oracle 620
IBM DB2 621
Exploiting Overruns at the SQL Level 623
SQL Functions 623
Conclusion 625
80238ftoc.qxd:WileyRed 7/11/07 7:22 AM Page xxii

xxii Contents

Chapter 25 Unix Kernel Overflows 627

Kernel Vulnerability Types 627
0day Kernel Vulnerabilities 636
OpenBSD exec_ibcs2_coff_prep_zmagic() Stack Overflow 636
The Vulnerability 638
Solaris vfs_getvfssw() Loadable Kernel Module
Traversal Vulnerability 642
The sysfs() System Call 644
The mount() System Call 645
Conclusion 646
Chapter 26 Exploiting Unix Kernel Vulnerabilities 647
The exec_ibcs2_coff_prep_zmagic() Vulnerability 647
Calculating Offsets and Breakpoints 652
Overwriting the Return Address and Redirecting Execution 654
Locating the Process Descriptor (or the Proc Structure) 655
Kernel Mode Payload Creation 658
Returning Back from Kernel Payload 659
Getting root (uid=0) 665
Solaris vfs_getvfssw() Loadable Kernel
Module Path Traversal Exploit 672
Crafting the Exploit 673
The Kernel Module to Load 674
Getting root (uid=0) 678
Conclusion 678
Chapter 27 Hacking the Windows Kernel 681
Windows Kernel Mode FlawsAn Increasingly Hunted Species 681
Introduction to the Windows Kernel 682
Common Kernel-Mode Programming Flaws 683
Stack Overflows 684
Heap Overflows 688
Insufficient Validation of User-Mode Addresses 688
Repurposing Attacks 689
Shared Object Attacks 689
Windows System Calls 690
Understanding System Calls 690
Attacking System Calls 692
Communicating with Device Drivers 693
I/O Control Code Components 693
Finding Flaws in IOCTL Handlers 694
Kernel-Mode Payloads 695
Elevating a User-Mode Process 696
Running an Arbitrary User-Mode Payload 699
Subverting Kernel Security 701
Installing a Rootkit 703
Essential Reading for Kernel Shellcoders 703
Conclusion 704
Index 705
THE CUCKOO'S EGG Page 4 of 254

ME, A WIZARD? UNTIL A WEEK AGO, I WAS AN ASTRONOMER, CONTENTEDLY DESIGNING

telescope optics. Looking back on it, I'd lived in an academic dreamland. All these
years, never planning for the future, right up to the day my grant money ran out.
Lucky for me that my laboratory recycled used astronomers. Instead of standing
in the unemployment line, I found myself transferred from the Keck Observatory at
the Lawrence Berkeley Lab, down to the computer center in the basement of the same
building.
Well, hell, I could fake enough computing to impress astronomers, and maybe
pick it up fast enough that my co-workers wouldn't catch on. Still, a computer
wizard? Not meI'm an astronomer.
Now what? As I apathetically stared at my computer terminal, I still thought of
planetary orbits and astrophysics. As new kid on the block, I had my choice of a
cubicle with a window facing the Golden Gate Bridge, or an unventilated office with
a wall of bookshelves. Swallowing my claustrophobia, I picked the office, hoping
that nobody would notice when I slept under the desk. On either side were offices
of two systems people, Wayne Graves and Dave Cleveland, the old hands of the
system. I soon got to know my neighbors through their bickering.
Viewing everyone as incompetent or lazy, Wayne was crossthreaded with the rest
of the staff. Yet he knew the system thoroughly, from the disk driver software up
to the microwave antennas. Wayne was weaned on Digital Equipment Corporation's Vax
computers and would tolerate nothing less: not IBM, not Unix, not Macintoshes.
Dave Cleveland, our serene Unix buddha, patiently listened to Wayne's running
stream of computer comparisons. A rare meeting didn't have Wayne's pitch, "Vaxes
are the choice of scientists everywhere and help build strong programs twelve
ways." Dave retorted, "Look, you keep your Vax addicts happy and I'll handle the
rest of the world."
Dave never gave him the satisfaction of getting riled, and Wayne's complaints
eventually trailed off to a mutter.
Great. First day on the job, sandwiched between two characters who were already
ruining my daydreams with their periodic disputes.
At least nobody could complain about my appearance. I wore the standard
Berkeley corporate uniform: grubby shirt, faded jeans, long hair, and cheap
sneakers. Managers occasionally wore ties, but productivity went down on the days
they did.
Together, Wayne, Dave, and I were to run the computers as a lab-wide utility.
We managed a dozen mainframe computersgiant workhorses for solving physics
problems, together worth around six million dollars. The scientists using the
computers were supposed to see a simple, powerful computing system, as reliable as
the electric company. This meant keeping the machines running full time, around the
clock. And just like the electric company, we charged for every cycle of computing
that was used.
Of four thousand laboratory employees, perhaps a quarter used the main
computers. Each of these one thousand accounts was tallied daily, and ledgers kept
inside the computer. With an hour of computing costing three hundred dollars, our
bookkeeping had to be accurate, so we kept track of every page printed, every block
THE CUCKOO'S EGG Page 5 of 254

of disk space, and every minute of processor time. A separate computer gathered
these statistics and sent monthly bills to laboratory departments.
And so it happened that on my second day at work, Dave wandered into my office,
mumbling about a hiccup in the Unix accounting system. Someone must have used a few
seconds of computing time without paying for it. The computer's books didn't quite
balance; last month's bills of $2,387 showed a 75-cent shortfall.
Now, an error of a few thousand dollars is obvious and isn't hard to find. But
errors in the pennies column arise from deeply buried problems, so finding these
bugs is a natural test for a budding software wizard. Dave said that I ought to
think about it.
"First-degree robbery, huh?" I responded.
"Figure it out, Cliff, and you'll amaze everyone," Dave said.Well, this seemed
like a fun toy, so I dug into the accounting program. I discovered our accounting
software to be a patchwork of programs written by long-departed summer students.
Somehow, the hodgepodge worked well enough to be ignored. Looking at the mixture of
programs, I found the software in Assembler, Fortran, and Cobol, the most ancient
of computer languages. Might as well have been classical Greek, Latin, and
Sanskrit.
As with most home-brew software, nobody had bothered to document our accounting
system. Only a fool would poke around such a labyrinth without a map.
Still, here was a plaything for the afternoon and a chance to explore the
system. Dave showed me how the system recorded each time someone connected to the
computer, logging the user's name, and terminal. It timestamped each connection,
recording which tasks the user executed, how many seconds of processor time he
used, and when he disconnected.
Dave explained that we had two independent accounting systems. The ordinary
Unix accounting software just stored the timestamped records into a file. But to
satisfy some bureaucrat, Dave had built a second accounting system which kept more
detailed records of who was using the computer.
Over the years, a succession of bored summer students had written programs to
analyze all this accounting information. One program collected the data and stashed
it into a file. A second program read that file and figured how much to charge for
that session. Yet a third program collected all these charges and printed out bills
to be mailed to each department. The last program added up all user charges and
compared that total to the result from the computer's internal accounting program.
Two accounting files, kept in parallel by different programs, ought to give the
same answer.
For a year, these programs had run without a glitch, but weren't quite perfect
this week. The obvious suspect was round-off error. Probably each accounting entry
was correct, but when added together, tenths of a penny differences built up until
an error of 75 cents accumulated. I ought to be able to prove this either by
analyzing how the programs worked, or by testing them with different data.
Rather than trying to understand the code for each program, I wrote a short
program to verify the data files. In a few minutes, I had checked the first
program: indeed, it properly collected the accounting data. No problem with the
first.
The second program took me longer to figure out. In an hour I had slapped
together enough makeshift code to prove that it actually worked. It just added up
time intervals, then multiplied by how much we charge for computer time. So the 75-
THE CUCKOO'S EGG Page 6 of 254

cent error didn't come from this program.

And the third program worked perfectly. It looked at a list of authorized
users, found their laboratory accounts, and then printed out a bill. Round-off
error? No, all of the programs kept track of money down to the hundredths of a
penny. Strange. Where's this 75-cent error coming from?
Well, I'd invested a couple hours in trying to understand a trivial problem. I
got stubborn: dammit, I'd stay there till midnight, if I had to.
Several test programs later, I began actually to have confidence in the
mishmash of locally built accounting programs. No question that the accounts didn't
balance, but the programs, though not bulletproof, weren't dropping pennies. By
now, I'd found the lists of authorized users, and figured out how the programs used
the data structures to bill different departments. Around 7 P.M. my eye caught one
user, Hunter. This guy didn't have a valid billing address.
Ha! Hunter used 75 cents of time in the past month, but nobody had paid for
him.
Here's the source of our imbalance. Someone had screwed up when adding a user
to our system. A trivial problem caused by a trivial error.
Time to celebrate. While writing this first small triumph into the beginning
pages of my notebook, Martha, my sweetheart, stopped by and we celebrated with
late-night cappuccinos at Berkeley's Cafe Roma.
A real wizard would have solved the problem in a few minutes. For me, it was
unknown territory, and finding my way around hadn't been easy. As a consolation,
I'd learned the accounting system and practiced a couple obsolete languages. Next
day, I sent an electronic mail message to Dave, preening my feathers by pointing
out the problem to him.
Around noon, Dave stopped by to drop off a pile of manuals, and casually
mentioned that he had never added a user named Hunterit must have been one of the
other system managers. Wayne's curt response: "It wasn't me. RTFM." Most of his
sentences ended with acronyms, this one meaning, "Read the fucking manual."
But I'd read the manuals. Operators weren't supposed to add a new user without
an account. At other computer centers, you just log into a privileged account and
tell the system to add a new user. Since we also had to make several bookkeeping
entries, we couldn't run such a vanilla system. Ours was complex enough that we had
special programs which automatically did the paperwork and the systems juggling.
Checking around, I found that everyone agreed the automatic system was so
superior that nobody would have manually added a new user. And the automatic system
wouldn't make this mistake.
Well, I couldn't figure out who had made this goof. Nobody knew Hunter, and
there wasn't an account set for him. So I erased the name from the systemwhen he
complained, we could set him up properly.
A day later, an obscure computer named Dockmaster sent us an electronic mail
message. Its system manager claimed that someone from our laboratory had tried to
break into his computer over the weekend.
Dockmaster's return address might have been anywhere, but signs pointed to
Maryland. The e-mail had passed through a dozen other computers, and each had left
a postmark.
Dave answered the message with a noncommittal "We'll look into it." Uh, sure.
We'd look when all our other problems disappeared.
Our laboratory's computers connect to thousands of other systems over a dozen
THE CUCKOO'S EGG Page 7 of 254

networks. Any of our scientists can log into our computer, and then connect to
a distant computer. Once connected, they can log into the distant computer by
entering an account name and password. In principle, the only thing protecting the
networked computer is the password, since account names are easy to figure out.
(How do you find account names? Just use a phone book most people use their names
on computers.)
Dockmaster's electronic mail message was a curiosity, and Dave passed it to
Wayne, attaching a question, "Who's Dockmaster?" Wayne forwarded it to me with his
guess "Probably some bank."
Eventually, Wayne bounced the message to me. I guessed Dockmaster was some Navy
shipyard. It wasn't important, but it seemed worth spending a few minutes looking
into.
The message gave the date and time when someone on our Unix computer tried to
log into Dockmaster's computer. So I scrabbled around the accounting files, looking
at Saturday morning's records. Again, the two accounting systems disagreed. The
stock Unix accounting file showed a user, Sventek, logging in at 8:25, doing
nothing for half an hour, and then disconnecting. No timestamped activity in
between. Our home-brew software also recorded Sventek's activity, but it showed him
using the networks from 8:31 until 9:01 A.M.
Jeez. Another accounting problem. The time stamps didn't agree. One showed
activity when the other account said everything was dormant.
Other things seemed more pressing, so I dropped the problem. After wasting an
afternoon chasing after some operator's mistake, I wasn't about to touch the
accounting system again.
Over lunch with Dave, I mentioned that Sventek was the only one connected when
Dockmaster reported the break-in. He stared and said, "Joe Sventek? He's in
Cambridge. Cambridge, England. What's he doing back?" Turned out that Joe Sventek
had been the laboratory's Unix guru, a software wizard who built a dozen major
programs over the past decade. Joe had left for England a year ago, leaving behind
a glowing reputation throughout the California computer community.
Dave couldn't believe Joe was back in town, since none of Joe's other friends
had heard from him. "He must have entered our computer from some network," Dave
said.
"So you think Joe's responsible for this problem?" I asked Dave.
"No way," Dave replied. "Joe's a hacker of the old school. A smart, quick,
capable programmer. Not one of those punks that have tarnished the word 'hacker.'
In any case, Sventek wouldn't try to break into some Maryland computer. And if he
did try, he'd succeed, without leaving any trace."
Curious: Joe Sventek's been in England a year, yet he shows up early Saturday
morning, tries to break into a Maryland computer, disconnects, and leaves behind an
unbalanced accounting system. In the hallway I mention this to Wayne, who's heard
that Joe's on vacation in England; he's hiding out in the backwoods, far away from
any computers. "Forget that message from Dockmaster. Sventek's due to visit
Berkeley RSN and he'll clear it up."
RSN? Real Soon Now. Wayne's way of saying, "I'm not sure when."
My worry wasn't Sventek. It was the unbalanced accounts. Why were the two
accounting systems keeping different times? And why was some activity logged in one
file without showing up in the other?
Back to the accounting system for an afternoon. I found that the five-minute
THE CUCKOO'S EGG Page 8 of 254

time difference between the time stamps came from our various computers' clocks
drifting over the months. One of our computer's clocks lost a few seconds every
day.
But all of Sventek's activities should have appeared in both tallies. Was this
related to last week's accounting problem? Had I screwed things up when I poked
around last week? Or was there some other explanation?
 Chapter 2

Hackers and Painters

When I nished grad school in computer science I went

to art school to study painting. A lot of people seemed surprised
that someone interested in computers would also be interested in
painting. They seemed to think that hacking and painting were
very different kinds of workthat hacking was cold, precise, and
methodical, and that painting was the frenzied expression of some
primal urge.
Both of these images are wrong. Hacking and painting have
a lot in common. In fact, of all the different types of people Ive
known, hackers and painters are among the most alike.
What hackers and painters have in common is that theyre
both makers. Along with composers, architects, and writers, what
hackers and painters are trying to do is make good things. Theyre
not doing research per se, though if in the course of trying to make
good things they discover some new technique, so much the better.

Ive never liked the term computer science. The main reason I
dont like it is that theres no such thing. Computer science is a grab
bag of tenuously related areas thrown together by an accident of
history, like Yugoslavia. At one end you have people who are really
mathematicians, but call what theyre doing computer science so
they can get DARPA grants. In the middle you have people work-
ing on something like the natural history of computersstudying
the behavior of algorithms for routing data through networks, for
example. And then at the other extreme you have the hackers, who
are trying to write interesting software, and for whom comput-
ers are just a medium of expression, as concrete is for architects

18
 hackers and painters

or paint for painters. Its as if mathematicians, physicists, and

architects all had to be in the same department.
Sometimes what the hackers do is called software engineer-
ing, but this term is just as misleading. Good software designers
are no more engineers than architects are. The border between
architecture and engineering is not sharply dened, but its there.
It falls between what and how: architects decide what to do, and
engineers gure out how to do it.
What and how should not be kept too separate. Youre asking
for trouble if you try to decide what to do without understanding
how to do it. But hacking can certainly be more than just deciding
how to implement some spec. At its best, its creating the spec
though it turns out the best way to do that is to implement it.

Perhaps one day computer science will, like Yugoslavia, get bro-
ken up into its component parts. That might be a good thing.
Especially if it meant independence for my native land, hacking.
Bundling all these different types of work together in one de-
partment may be convenient administratively, but its confusing
intellectually. Thats the other reason I dont like the name com-
puter science. Arguably the people in the middle are doing some-
thing like an experimental science. But the people at either end,
the hackers and the mathematicians, are not actually doing sci-
ence.
The mathematicians dont seem bothered by this. They happily
set to work proving theorems like the other mathematicians over
in the math department, and probably soon stop noticing that the
building they work in says computer science on the outside. But
for the hackers this label is a problem. If what theyre doing is
called science, it makes them feel they ought to be acting scientic.
So instead of doing what they really want to do, which is to design
beautiful software, hackers in universities and research labs feel
they ought to be writing research papers.
In the best case, the papers are just a formality. Hackers write
cool software, and then write a paper about it, and the paper be-

19
 hackers & painters

comes a proxy for the achievement represented by the software.

But often this mismatch causes problems. Its easy to drift away
from building beautiful things toward building ugly things that
make more suitable subjects for research papers.
Unfortunately, beautiful things dont always make the best sub-
jects for papers. Number one, research must be originaland as
anyone who has written a PhD dissertation knows, the way to
be sure youre exploring virgin territory is to to stake out a piece
of ground that no one wants. Number two, research must be
substantialand awkward systems yield meatier papers, because
you can write about the obstacles you have to overcome in order
to get things done. Nothing yields meaty problems like starting
with the wrong assumptions. Most of AI is an example of this
rule; if you assume that knowledge can be represented as a list of
predicate logic expressions whose arguments represent abstract
concepts, youll have a lot of papers to write about how to make
this work. As Ricky Ricardo used to say, Lucy, you got a lot of
explaining to do.
The way to create something beautiful is often to make subtle
tweaks to something that already exists, or to combine existing
ideas in a slightly new way. This kind of work is hard to convey
in a research paper.

So why do universities and research labs continue to judge hack-

ers by publications? For the same reason that scholastic aptitude
gets measured by simple-minded standardized tests, or the pro-
ductivity of programmers by lines of code. These tests are easy to
apply, and there is nothing so tempting as an easy test that kind
of works.
Measuring what hackers are actually trying to do, designing
beautiful software, would be much more difcult. You need a good
sense of design to judge good design. And there is no correlation,
except possibly a negative one, between peoples ability to recog-
nize good design and their condence that they can.

20
 hackers and painters

The only external test is time. Over time, beautiful things tend
to thrive, and ugly things tend to get discarded. Unfortunately, the
amounts of time involved can be longer than human lifetimes.
Samuel Johnson said it took a hundred years for a writers rep-
utation to converge.1 You have to wait for the writers inuential
friends to die, and then for all their followers to die.
I think hackers just have to resign themselves to having a large
random component in their reputations. In this they are no dif-
ferent from other makers. In fact, theyre lucky by comparison.
The inuence of fashion is not nearly so great in hacking as it is
in painting.

There are worse things than having people misunderstand your

work. A worse danger is that you will yourself misunderstand
your work. Related elds are where you go looking for ideas. If
you nd yourself in the computer science department, there is
a natural temptation to believe, for example, that hacking is the
applied version of what theoretical computer science is the theory
of. All the time I was in graduate school I had an uncomfortable
feeling in the back of my mind that I ought to know more theory,
and that it was very remiss of me to have forgotten all that stuff
within three weeks of the nal exam.
Now I realize I was mistaken. Hackers need to understand the
theory of computation about as much as painters need to under-
stand paint chemistry. You need to know how to calculate time and
space complexity, and perhaps also the concept of a state machine,
in case you want to write a parser. Painters have to remember a
good deal more about paint chemistry than that.
Ive found that the best sources of ideas are not the other elds
that have the word computer in their names, but the other elds
inhabited by makers. Painting has been a much richer source of
ideas than the theory of computation.
For example, I was taught in college that one ought to g-
ure out a program completely on paper before even going near
a computer. I found that I did not program this way. I found

21
 hackers & painters

that I liked to program sitting in front of a computer, not a piece

of paper. Worse still, instead of patiently writing out a complete
program and assuring myself it was correct, I tended to just spew
out code that was hopelessly broken, and gradually beat it into
shape. Debugging, I was taught, was a kind of nal pass where
you caught typos and oversights. The way I worked, it seemed
like programming consisted of debugging.
For a long time I felt bad about this, just as I once felt bad that
I didnt hold my pencil the way they taught me to in elementary
school. If I had only looked over at the other makers, the painters
or the architects, I would have realized that there was a name for
what I was doing: sketching. As far as I can tell, the way they
taught me to program in college was all wrong. You should gure
out programs as youre writing them, just as writers and painters
and architects do.
Realizing this has real implications for software design. It
means that a programming language should, above all, be mal-
leable. A programming language is for thinking of programs, not
for expressing programs youve already thought of. It should be
a pencil, not a pen. Static typing would be a ne idea if people
actually did write programs the way they taught me to in college.
But thats not how any of the hackers I know write programs. We
need a language that lets us scribble and smudge and smear, not
a language where you have to sit with a teacup of types balanced
on your knee and make polite conversation with a strict old aunt
of a compiler.

While were on the subject of static typing, identifying with the

makers will save us from another problem that aficts the sciences:
math envy. Everyone in the sciences secretly believes that mathe-
maticians are smarter than they are. I think mathematicians also
believe this. At any rate, the result is that scientists tend to make
their work look as mathematical as possible. In a eld like physics
this probably doesnt do much harm, but the further you get from
the natural sciences, the more of a problem it becomes.

22
 hackers and painters

A page of formulas just looks so impressive. (Tip: for extra

impressiveness, use Greek variables.) And so there is a great temp-
tation to work on problems you can treat formally, rather than
problems that are, say, important.
If hackers identied with other makers, like writers and paint-
ers, they wouldnt feel tempted to do this. Writers and painters
dont suffer from math envy. They feel as if theyre doing some-
thing completely unrelated. So are hackers, I think.

If universities and research labs keep hackers from doing the kind
of work they want to do, perhaps the place for them is in compa-
nies. Unfortunately, most companies wont let hackers do what
they want either. Universities and research labs force hackers to
be scientists, and companies force them to be engineers.
I only discovered this myself quite recently. When Yahoo
bought Viaweb, they asked me what I wanted to do. I had never
liked business much, and said that I just wanted to hack. When I
got to Yahoo, I found that what hacking meant to them was im-
plementing software, not designing it. Programmers were seen
as technicians who translated the visions (if that is the word) of
product managers into code.
This seems to be the default plan in big companies. They do it
because it decreases the standard deviation of the outcome. Only
a small percentage of hackers can actually design software, and
its hard for the people running a company to pick these out. So
instead of entrusting the future of the software to one brilliant
hacker, most companies set things up so that it is designed by
committee, and the hackers merely implement the design.
If you want to make money at some point, remember this, be-
cause this is one of the reasons startups win. Big companies want
to decrease the standard deviation of design outcomes because
they want to avoid disasters. But when you damp oscillations, you
lose the high points as well as the low. This is not a problem for
big companies, because they dont win by making great products.
Big companies win by sucking less than other big companies.

23
 hackers & painters

So if you can gure out a way to get in a design war with a

company big enough that its software is designed by product man-
agers, theyll never be able to keep up with you. These opportuni-
ties are not easy to nd, though. Its hard to engage a big company
in a design war, just as its hard to engage an opponent inside a
castle in hand-to-hand combat. It would be pretty easy to write
a better word processor than Microsoft Word, for example, but
Microsoft, within the castle of their operating system monopoly,
probably wouldnt even notice if you did.
The place to ght design wars is in new markets, where no
one has yet managed to establish any fortications. Thats where
you can win big by taking the bold approach to design, and hav-
ing the same people both design and implement the product. Mi-
crosoft themselves did this at the start. So did Apple. And Hewlett-
Packard. I suspect almost every successful startup has.

So one way to build great software is to start your own startup.

There are two problems with this, though. One is that in a startup
you have to do so much besides write software. At Viaweb I con-
sidered myself lucky if I got to hack a quarter of the time. And
the things I had to do the other three quarters of the time ranged
from tedious to terrifying. I have a benchmark for this, because I
once had to leave a board meeting to have some cavities lled. I
remember sitting back in the dentists chair, waiting for the drill,
and feeling like I was on vacation.
The other problem with startups is that there is not much over-
lap between the kind of software that makes money and the kind
thats interesting to write. Programming languages are interest-
ing to write, and Microsofts rst product was one, in fact, but
no one will pay for programming languages now. If you want to
make money, you tend to be forced to work on problems that are
too nasty for anyone to solve for free.
All makers face this problem. Prices are determined by supply
and demand, and there is just not as much demand for things that
are fun to work on as there is for things that solve the mundane

24
 hackers and painters

problems of individual customers. Acting in off-Broadway plays

doesnt pay as well as wearing a gorilla suit in someones booth
at a trade show. Writing novels doesnt pay as well as writing ad
copy for garbage disposals. And hacking programming languages
doesnt pay as well as guring out how to connect some companys
legacy database to their web server.

I think the answer to this problem, in the case of software, is a

concept known to nearly all makers: the day job. This phrase
began with musicians, who perform at night. More generally, it
means you have one kind of work you do for money, and another
for love.
Nearly all makers have day jobs early in their careers. Painters
and writers notoriously do. If youre lucky you can get a day job
closely related to your real work. Musicians often seem to work in
record stores. A hacker working on some programming language
or operating system might likewise be able to get a day job using
it.2
When I say that the answer is for hackers to have day jobs, and
work on beautiful software on the side, Im not proposing this as
a new idea. This is what open source hacking is all about. What
Im saying is that open source is probably the right model, because
it has been independently conrmed by all the other makers.
It seems surprising to me that any employer would be reluc-
tant to let hackers work on open source projects. At Viaweb, we
would have been reluctant to hire anyone who didnt. When we in-
terviewed programmers, the main thing we cared about was what
kind of software they wrote in their spare time. You cant do any-
thing really well unless you love it, and if you love to hack youll
inevitably be working on projects of your own.3

Because hackers are makers rather than scientists, the right place
to look for metaphors is not in the sciences, but among other kinds
of makers. What else can painting teach us about hacking?

25
 hackers & painters

One thing we can learn, or at least conrm, from the example

of painting is how to learn to hack. You learn to paint mostly
by doing it. Ditto for hacking. Most hackers dont learn to hack
by taking college courses in programming. They learn by writing
programs of their own at age thirteen. Even in college classes, you
learn to hack mostly by hacking.4
Because painters leave a trail of work behind them, you can
watch them learn by doing. If you look at the work of a painter in
chronological order, youll nd that each painting builds on things
learned in previous ones. When theres something in a painting
that works especially well, you can usually nd version 1 of it in a
smaller form in some earlier painting.
I think most makers work this way. Writers and architects seem
to as well. Maybe it would be good for hackers to act more like
painters, and regularly start over from scratch, instead of contin-
uing to work for years on one project, and trying to incorporate
all their later ideas as revisions.
The fact that hackers learn to hack by doing it is another sign of
how different hacking is from the sciences. Scientists dont learn
science by doing it, but by doing labs and problem sets. Scientists
start out doing work thats perfect, in the sense that theyre just
trying to reproduce work someone else has already done for them.
Eventually, they get to the point where they can do original work.
Whereas hackers, from the start, are doing original work; its just
very bad. So hackers start original, and get good, and scientists
start good, and get original.

The other way makers learn is from examples. To a painter, a

museum is a reference library of techniques. For hundreds of
years it has been part of the traditional education of painters to
copy the works of the great masters, because copying forces you
to look closely at the way a painting is made.
Writers do this too. Benjamin Franklin learned to write by
summarizing the points in the essays of Addison and Steele and

26
 hackers and painters

then trying to reproduce them. Raymond Chandler did the same

thing with detective stories.
Hackers, likewise, can learn to program by looking at good
programsnot just at what they do, but at the source code. One
of the less publicized benets of the open source movement is
that it has made it easier to learn to program. When I learned to
program, we had to rely mostly on examples in books. The one
big chunk of code available then was Unix, but even this was not
open source. Most of the people who read the source read it in
illicit photocopies of John Lions book, which though written in
1977 was not allowed to be published until 1996.

Another example we can take from painting is the way that paint-
ings are created by gradual renement. Paintings usually begin
with a sketch. Gradually the details get lled in. But it is not
merely a process of lling in. Sometimes the original plans turn
out to be mistaken. Countless paintings, when you look at them
in x-rays, turn out to have limbs that have been moved or facial
features that have been readjusted.
Heres a case where we can learn from painting. I think hack-
ing should work this way too. Its unrealistic to expect that the
specications for a program will be perfect. Youre better off if
you admit this up front, and write programs in a way that allows
specications to change on the y.
(The structure of large companies makes this hard for them to
do, so here is another place where startups have an advantage.)
Everyone by now presumably knows about the danger of pre-
mature optimization. I think we should be just as worried about
premature designdeciding too early what a program should do.
The right tools can help us avoid this danger. A good pro-
gramming language should, like oil paint, make it easy to change
your mind. Dynamic typing is a win here because you dont have
to commit to specic data representations up front. But the key
to exibility, I think, is to make the language very abstract. The
easiest program to change is one thats short.

27
 hackers & painters

Leonardos Ginevra de Benci, 1474.

This sounds like a paradox, but a great painting has to be better

than it has to be. For example, when Leonardo painted the portrait
of Ginevra de Benci in the National Gallery, he put a juniper bush
behind her head. In it he carefully painted each individual leaf.
Many painters might have thought, this is just something to put in
the background to frame her head. No one will look that closely
at it.
Not Leonardo. How hard he worked on part of a painting
didnt depend at all on how closely he expected anyone to look at
it. He was like Michael Jordan. Relentless.
Relentlessness wins because, in the aggregate, unseen details
become visible. When people walk by the portrait of Ginevra de

28
 hackers and painters

Benci, their attention is often immediately arrested by it, even be-

fore they look at the label and notice that it says Leonardo da Vinci.
All those unseen details combine to produce something thats just
stunning, like a thousand barely audible voices all singing in tune.
Great software, likewise, requires a fanatical devotion to beau-
ty. If you look inside good software, you nd that parts no one is
ever supposed to see are beautiful too. When it comes to code I
behave in a way that would make me eligible for prescription drugs
if I approached everyday life the same way. It drives me crazy to
see code thats badly indented, or that uses ugly variable names.

If a hacker were a mere implementor, turning a spec into code,

then he could just work his way through it from one end to the
other like someone digging a ditch. But if the hacker is a creator,
we have to take inspiration into account.
In hacking, like painting, work comes in cycles. Sometimes
you get excited about a new project and you want to work sixteen
hours a day on it. Other times nothing seems interesting.
To do good work you have to take these cycles into account,
because theyre affected by how you react to them. When youre
driving a car with a manual transmission on a hill, you have to back
off the clutch sometimes to avoid stalling. Backing off can likewise
prevent ambition from stalling. In both painting and hacking there
are some tasks that are terrifyingly ambitious, and others that are
comfortingly routine. Its a good idea to save some easy tasks for
moments when you would otherwise stall.
In hacking, this can literally mean saving up bugs. I like de-
bugging: its the one time that hacking is as straightforward as
people think it is. You have a totally constrained problem, and
all you have to do is solve it. Your program is supposed to do x.
Instead it does y. Where does it go wrong? You know youre going
to win in the end. Its as relaxing as painting a wall.

29
 hackers & painters

The example of painting can teach us not only how to manage

our own work, but how to work together. A lot of the great art of
the past is the work of multiple hands, though there may only be
one name on the wall next to it in the museum. Leonardo was an
apprentice in the workshop of Verrocchio and painted one of the
angels in his Baptism of Christ. This sort of thing was the rule, not
the exception. Michelangelo was considered especially dedicated
for insisting on painting all the gures on the ceiling of the Sistine
Chapel himself.
As far as I know, when painters worked together on a painting,
they never worked on the same parts. It was common for the
master to paint the principal gures and for assistants to paint the
others and the background. But you never had one guy painting
over the work of another.
I think this is the right model for collaboration in software too.
Dont push it too far. When a piece of code is being hacked by
three or four different people, no one of whom really owns it, it will
end up being like a common-room. It will tend to feel bleak and
abandoned, and accumulate cruft. The right way to collaborate,
I think, is to divide projects into sharply dened modules, each
with a denite owner, and with interfaces between them that are as
carefully designed and, if possible, as articulated as programming
languages.

Like painting, most software is intended for a human audience.

And so hackers, like painters, must have empathy to do really great
work. You have to be able to see things from the users point of
view.
When I was a kid I was constantly being told to look at things
from someone elses point of view. What this always meant in
practice was to do what someone else wanted, instead of what I
wanted. This of course gave empathy a bad name, and I made a
point of not cultivating it.
Boy, was I wrong. It turns out that looking at things from
other peoples point of view is practically the secret of success.

30
 hackers and painters

Empathy doesnt necessarily mean being self-sacricing. Far from

it. Understanding how someone else sees things doesnt imply that
youll act in his interest; in some situationsin war, for example
you want to do exactly the opposite.5
Most makers make things for a human audience. And to en-
gage an audience you have to understand what they need. Nearly
all the greatest paintings are paintings of people, for example,
because people are what people are interested in.
Empathy is probably the single most important difference be-
tween a good hacker and a great one. Some hackers are quite
smart, but practically solipsists when it comes to empathy. Its
hard for such people to design great software, because they cant
see things from the users point of view.6
One way to tell how good people are at empathy is to watch
them explain a technical matter to someone without a technical
background. We probably all know people who, though otherwise
smart, are just comically bad at this. If someone asks them at a
dinner party what a programming language is, theyll say some-
thing like Oh, a high-level language is what the compiler uses as
input to generate object code. High-level language? Compiler?
Object code? Someone who doesnt know what a programming
language is obviously doesnt know what these things are, either.
Part of what software has to do is explain itself. So to write
good software you have to understand how little users understand.
Theyre going to walk up to the software with no preparation, and
it had better do what they guess it will, because theyre not going
to read the manual. The best system Ive ever seen in this respect
was the original Macintosh, in 1984. It did what software almost
never does: it just worked.7
Source code, too, should explain itself. If I could get people to
remember just one quote about programming, it would be the one
at the beginning of Structure and Interpretation of Computer Programs.8

Programs should be written for people to read, and only

incidentally for machines to execute.

31
 hackers & painters

Piero della Francescas Federico da Montefeltro, 1465-66 (detail).

You need to have empathy not just for your users, but for your
readers. Its in your interest, because youll be one of them. Many
a hacker has written a program only to nd on returning to it six
months later that he has no idea how it works. I know several
people whove sworn off Perl after such experiences.9
Lack of empathy is associated with intelligence, to the point
that there is even something of a fashion for it in some places. But
I dont think theres any correlation. You can do well in math and
the natural sciences without having to learn empathy, and people
in these elds tend to be smart, so the two qualities have come to
be associated. But there are plenty of dumb people who are bad
at empathy too.

32
 hackers and painters

So, if hacking works like painting and writing, is it as cool? After

all, you only get one life. You might as well spend it working on
something great.
Unfortunately, the question is hard to answer. There is always
a big time lag in prestige. Its like light from a distant star. Painting
has prestige now because of great work people did ve hundred
years ago. At the time, no one thought these paintings were as
important as we do today. It would have seemed very odd to people
in 1465 that Federico da Montefeltro, the Duke of Urbino, would
one day be known mostly as the guy with the strange nose in a
painting by Piero della Francesca.
So while I admit that hacking doesnt seem as cool as painting
now, we should remember that painting itself didnt seem as cool
in its glory days as it does now.
What we can say with some condence is that these are the
glory days of hacking. In most elds the great work is done early
on. The paintings made between 1430 and 1500 are still unsur-
passed. Shakespeare appeared just as professional theater was be-
ing born, and pushed the medium so far that every playwright
since has had to live in his shadow. Albrecht Durer did the same
thing with engraving, and Jane Austen with the novel.
Over and over we see the same pattern. A new medium ap-
pears, and people are so excited about it that they explore most
of its possibilities in the rst couple generations. Hacking seems
to be in this phase now.
Painting was not, in Leonardos time, as cool as his work helped
make it. How cool hacking turns out to be will depend on what
we can do with this new medium.

33
Feature: Crop circles physicsworld.com

Coming soon to
a field near you
Serious studies of crop circles have long been hampered
by conspiracy theories and the secretive nature of
circle-makers plus scientists reluctance to engage with
a fringe topic. But, as Richard Taylor argues,
discovering how circle artists create their most complex
patterns could have implications for biophysics
Richard Taylor is One evening in July 1996 I was staying above a country
director of the pub near Avebury in Wiltshire, enjoying a weeks holi-
Materials Science day drive around the prehistoric sites of southern
Institute at the England. In the middle of the night, I awoke to the
University of Oregon, hushed sounds of three men talking in the car park
US, e-mail rpt@
below. They were huddled around a large sheet of
uoregon.edu
paper, and after 15 minutes of furtive discussion, they
sped off down a country lane. That same evening, 194
crop circles spanning a total of 115 m appeared in a
nearby field at Windmill Hill. Their pattern, which was
derived from an equation developed by Gaston Julia
in 1918, consisted of circles that defined three inter-
twined fractals (figure 1). This Triple Julia pattern is
mathematically complex: as late as the 1980s, even the
best computers lacked the processing power needed to
generate it on screen. Had those three men managed
to physically imprint the same pattern into a wheat field
during the short hours of that midsummer night? And
if so, how did they do it?
Some 15 years on, scientists still do not know the
answer. With more than 10 000 patterns documented
over the years, crop formations remain a major scien-
tific mystery, one that plays out in our fields and thus
in our food supply at the rate of one event worldwide
every summer evening. Physicists who have conducted
serious research on the techniques of crop-circle artists
have come away with fascinating insights, including
some that have led to practical advances, such as a
patented technique for accelerating crop growth. With
recent announcements that climate change has sup-
pressed crop growth by 3%, such advances offer clear
potential rewards for society. Yet crop-circle research
is not for the faint of heart, because physicists who enter
it must deal with media manipulation, hate-mail, con-
spiracy theories, supposed alien collaborations and
new-age nonsense not to mention the risk of being
viewed as less than serious by their colleagues.

Devils, aliens, whirlwinds and hoaxers

Speculation over the origin of crop circles has raged
since they were first reported in England in the 1600s,
with rolling hedgehogs, urinating cattle, romping

2 Physics World August 2011

physicsworld.com Feature: Crop circles

Photolibrary
At a Glance: Crop circles
Crop circles are patterns formed within crop fields and represent the work of the
most science-oriented art movement in history
First reported in the 1600s, these patterns appear around the world at a rate of
one every evening
The patterns, which can feature up to 2000 individual shapes, are often built using
hidden mathematical relationships such as diatonic ratios
Biophysicists have interpreted swollen stalks as evidence that the crops were
exposed to microwaves during formation of the circles, leading to patented
techniques for accelerated crop growth

romantic couples and the actions of a mowing devil

all offered as early explanations. In 1678 a series of cir-
cles in Hertfordshire was attributed to the devil
because the manufacture appeared to be beyond
human capabilities. According to a report in a 1678
issue of News Out of Hartfordshire, the devil placed
every straw with an exactness that would have taken up
above an age for any man to perform what he [the devil]
did that one night. More prosaically, the woodcut
print that accompanies the report also indicates that
the stalks within the circle were flattened rather than
broken a practice that continues today.
The first scientific explanations of crop circles focused
on cyclonic winds. In 1686 the British scientist Robert
Plot discussed crop-circle formation in terms of airflows
from the sky. Similarly, observations of the night sky by
another scientist, John Capron, in 1880 revealed a wind-
induced auroral beam above the circular spots of
flattened crop (Nature 22 290). However, as the phe-
nomenon gathered momentum, and more elaborate,
multi-circle shapes appeared in crop fields, most
observers concluded that these symbols of mathemati-
cal precision had to be the work of intelligent beings.
In the waning decades of the 20th century, this con-
clusion ignited a heated aliens-versus-humans debate,
with UFOlogists looking to outer space for the cir-
cles artistic creators, while cereologists concentrated
on hunting for terrestrial hoaxers. This debate was
complicated by the fact that the creators (whoever they
were) were clearly science-savvy. In particular, one for-
mation that appeared next to Chilbolton Observatory
in Hampshire appeared to be a reply to a search for
extraterrestrial intelligence signal beamed into space
30 years earlier.
As the debate raged, some scientists continued to
seek alternative natural explanations. One of the most
prominent was Terence Meaden, then a meteorologist
and physicist at Dalhousie University in Canada. In
1980 Meaden refined Caprons theory, proposing that
the curvature of hillsides in southern England affected
the local airflow, allowing whirlwinds to stabilize their
positions long enough to define circles in the crop fields.
Such scientific speculations received a severe blow
in 1991 when, to the glee of the British media, two unas-
suming men in their sixties declared that they had been
creating crop circles for more than 25 years. Their
hobby had begun one summer evening in the mid-
1970s, when artist Douglas Bower recounted a story to
his friend David Chorley about an Australian farmer
who had reported a UFO rising into the sky and leaving
behind a circular saucer nest. As Bower and Chorley
strolled home from the pub through the English coun-

Physics World August 2011 3

 Feature: Crop circles
1 Field of fractals
Steve Alexander
On 29 July 1996 this crop circle appeared on Windmill Hill near Avebury, UK. Its design is
based on an equation formulated by Gaston Julia in 1918. A similar Triple Julia fractal design
was also used in a crop circle in Switzerland last year.
tryside, they created their first imitation nest.
In the process, the pair unintentionally triggered a 15-
year duel between art and physics. Bower and Chorley
were trying to start a UFO hoax, so when Meadens
meteorological theories of crop-circle formation
showed signs of catching on, the pair increased the num-
ber of circles in their formations, hoping to demonstrate
that they were not weather-related. Meaden, however,
proved an inventive (albeit unwitting) opponent. By the
time Bower and Chorley went public, Meaden had
moved on from mere weather patterns to an electro-
magneto-hydrodynamic plasma vortex, which pur-
ported to explain not only the elaborate multi-circle
designs, but also the flat farm tractor batteries and eerie
lights that coincided with their formation!
Today, with the benefit of hindsight, such explana-
tions sound rather contrived. At the height of the
debate, though, no less a physicist than Stephen
Hawking was prepared to accept some version of
Meadens theory. When a spate of circles appeared in
the countryside near his Cambridge home in 1991,
Hawking told a local newspaper that crop circles are
either hoaxes or formed by vortex movement of air.
Frustrated, the artists countered by producing a pat-
tern that included two circles and five rectangles (fig-
ure 2). At this point, even Meaden conceded that these
straight-line designs, labelled pictographs by
researchers, were man-made, although he stressed that
simple circles could still be a consequence of atmo-
spheric phenomena. After all, even after Bower and
Chorley confessed to making 250 formations, that still
left more than 1000 other formations unaccounted for.
But the addition of straight lines did more than just rule
out natural causes for their designs. It also signalled a
turning point in the 400-year history of crop formations.
4 Physics World August 2011
physicsworld.com
Creating mathematical patterns
After Bower and Chorley announced their hoax, the
pictographs they created inspired a second wave of
crop artists. Far from fizzling out, crop circles have
evolved into an international phenomenon, with hun-
dreds of sophisticated pictographs now appearing
annually around the globe. Although up to half of each
years crop circles are in England, formations also
occur elsewhere in Europe, as well as in North and
South America, Russia, Australia, Japan and India.
Artists who readily admit to having made crop circles
in the past say they do not know who is responsible for
all of todays masterworks. This is partly because many
crop-circle artists have followed the conventions estab-
lished by their predecessors: creating their pictographs
anonymously, under cover of darkness, and leaving the
scene free of human traces. But although the new artists
are traditionalists in this sense, in other respects their
craft has moved on considerably. Todays artists, for
example, have access to computers, GPS equipment
and lasers to help map out their patterns, whereas
Bower had to create his straight lines using a sight
that consisted of a circular wire attached to his cap.
Scientists who are curious about the mathematics of
crop circles and how they are planned have two options:
they can stake out the car parks of rural pubs late at
night in the hope of catching artists in action; or they
can apply pattern-analysis techniques to the results.
History has shown that the stake-out option is risky.
Attempts to capture mapping techniques on film have
fuelled a cat-and-mouse game between artist and
researcher, in which the stealth of the former has usu-
ally resulted in public embarrassment for the latter. In
1990, for example, a prominent circles researcher and
engineer, Colin Andrews, co-ordinated the infamous
Operation Blackbird, in which a region near Westbury,
Wiltshire, was put under surveillance by the BBC and
patrolled by officials from the Ministry of Defence.
Despite such precautions, the dawn of the second day
revealed that artists had crept in under the cover of
night, performed their craft and left unhindered. Over-
enthusiastic researchers were dealt another humiliat-
ing blow in 1996, this time at the hands of the media,
when a sensational film clip called Olivers Castle Crop
Circle (available online) hoaxed a pattern materializ-
ing in a crop field.
Perhaps not surprisingly, most scientists have pre-
ferred to forgo stake-outs and instead analyse the pat-
terns left behind by these cunning artists. The
pioneering research published in 1996 in Science News
(150 239) by Gerard Hawkins (who was then an
astronomer at Boston University, US) examined crop
circles formed during 19781988. The 25 formations
he analysed consisted of single circles, multiple circles
and circles with concentric rings. Yet even for these
primitive patterns, Hawkins found a hidden artistic lan-
guage: he discovered that all of the formations were
built using hidden construction lines that were used
at the design stage but did not appear in the final pat-
tern. Examples are shown in blue in figure 3, along with
the yellow patterns of the resulting circles.
Hawkins used these construction lines to demon-
strate that crop circles are much more than arbitrarily
sized and randomly positioned patterns in fields.
physicsworld.com
2 Intelligently designed
The design of Bower and Chorleys original pictograph.
Instead, the construction lines dictate their relative
sizes and positions with precision and lead to some
highly exotic properties. In particular, ratios of various
diameters and areas within the designs were found to
cluster around the diatonic ratios for the white keys
on a piano. These ratios are the frequency ratios of
notes: middle D to C, for example, is 297/264 Hz =
9/8. The idea that crop formations possess a funda-
mental geometric harmony analogous to musical
chords has inspired musicians to use computer algo-
rithms to convert formations into melodies. The best-
known translator is Paul Vigay, and samples of his
music are available at http://bit.ly/lbUJQq.
Todays crop-circle designs are more complex than
ever, featuring up to 2000 individual shapes arranged
using intricate construction lines that are invisible to the
casual observer. The increase in available computing
power has also meant that iterative equations are now
frequently used to generate fractal shapes such as the
Triple Julia design, which reappeared in Switzerland
last year. Other famous fractal icons such as the
Mandelbrot set, the Julia set and the Koch snowflake
have also popped up regularly in crop fields since 1991.
Making a crop circle
Even the preliminary stage of crop-circle construction
mapping the proposed design is not an easy task.
The appearance of the first Triple Julia formation in
July 1996 was pre-empted by a single Julia formation
several weeks earlier. This warm-up design took a
team of 11 surveyors five hours just to measure out, and
a surveying company later estimated that one of its
engineers would have required at least five days to map
out each of the three intertwining patterns. But once
their maps are complete, crop-circle artists face a still
more difficult problem: how do you imprint patterns in
crops that are a challenge even to draw on paper?
Traditional circle-makers employed stompers
(wooden planks attached to two hand-held ropes), string
and garden rollers, plus bar stools to allow artists to vault
over undisturbed crops. Despite their primitive appear-
ance, stompers are a surprisingly efficient tool for flat-
tening crops, especially when driven by skilled hands.
However, modern designs have evolved beyond the tra-
ditional requirement that stalks be flattened rather than
broken: formations now feature stalks that are carefully
sculpted to create intricate textures within the geome-
tries. For example, the stalks in each of the circles of the
Triple Julia pattern formed a spiral. Multiple layers of
bent stalks can also be woven together, creating shad-
Physics World August 2011
Feature: Crop circles
3 Under construction
When the astronomer Gerard Hawkins analysed the designs of 25 crop circles, he found
evidence that circle artists were using construction lines (blue) as guides to determine where
rings and circles (yellow) should be imprinted in the field.
owy textures that evolve over days in the sunlight due to
the stalks phototropic responses.
Hence, to imprint their vast pictographs before sun-
rise, todays artists have to work in co-ordinated teams.
One such team is known as the Circlemakers, and when
in a rare breach of secrecy it allowed BBC film-
makers to document its construction of a 100-circle
roulette pattern in 1998, team members were observed
physically implanting circles at the remarkable rate of
one every minute. Circlemaker Will Russell summa-
rized their motivation: To push the boundaries of what
people think is humanly possible, while his colleague
Rod Dickinson stressed that this rate was sufficient to
imprint the Triple Julia pattern in one night.
Despite such claims, the larger scale and higher pre-
cision of the Triple Julia design would have made it sig-
nificantly more challenging to create than the
Circlemakers roulette. There are further signs that tra-
ditional physical imprinting techniques are reaching
their limits. One of 2009s pictographs required three
nights to complete, and its pattern progression is shown
in figure 4. If artists want to maintain the movements
secrecy and anonymity, it is clear that they will need to
exploit more efficient construction methods.
Biophysical speculations
Intriguingly, experiments carried out by biophysicists
raise the possibility that some circle-makers may
already be changing their methods. Independent stud-
ies published in 1999 and in 2001 reported evidence
consistent with what you would expect to see if the
crops had been exposed to radiation during the forma-
tion of patterns. The patterns studied date back to the
mid-1990s, and include the original Triple Julia.
Figure 5 shows the results of an investigation of pul-
vini, the visco-elastic joints that occur along wheat
stalks. Eltjo Haselhoff, a medical physicist, found that
pulvini on bent stalks within a 9 m-wide circle were If artists want
elongated compared with undamaged crops in the to maintain the
same field. Although several well-understood factors movements
can cause pulvini to swell, including gravitropism (the
directional growth of stalks in response to gravity) and secrecy and
lodging (bending of stalks caused by wind or rain anonymity, it is
damage), Haselhoff dismissed them based on the mag- clear that they
nitude of the increase, and its symmetric fall-off from will need to
the circles centre to its edge. exploit more
Haselhoffs findings built on the earlier research of
William Levengood, a biophysicist at a Michigan-based
efficient
crop-seed consultancy called Pinelandia Biophysics construction
Laboratory. Levengood, who found similar results on methods
5
Feature: Crop circles
4 Work in progress
This Woman Crop formation was created at Milk Hill, Wiltshire, over three nights in 2009. The first three photos from left to right were taken on 21 June (by Russell
Stannard), 23 June and 30 June (both by John Montgomery) respectively and show the patterns evolution. The unfinished-looking circles in the patterns tail (shown in
detail in the fourth photo) have prompted speculation that the artists were planning to continue the design on a fourth night.
95% of 250 crop formations in seven countries, pro-
posed that the elongated pulvini were a result of super-
heating from electromagnetic radiation. Such
radiation, he theorized, would cause stalks to fall over
and cool in a horizontal position. He found further evi-
dence for superheating in changes in the crops cellular
structure and in the numerous dead flies stuck to seed
heads in the formations.
Levengood and Haselhoff both followed up their
work by removing crop seeds from the field and plac-
ing them in growth chambers controlled for light,
humidity and temperature. They found that while seeds
taken from the surrounding crop grew at normal rates,
seeds from the formations grew up to four times slower
in 90% of the measured formations.
Although both researchers findings were published
in Physiologia Plantarum (W C Levengood 1994 92 356
and 1999 105 615; E H Haselhoff 2000 1 124), a peer-
reviewed journal dedicated to the science of plant
growth, their results failed to bring the crop-circle
debate to a close. The authors own speculations did
not help matters: Levengood interpreted his results as
evidence of Meadens plasma vortex theory, while
Haselhoff proposed that the sources of radiation were
the mysterious balls of light that some observers have
reported hovering over formation sites. Under the cir-
cumstances, scientists reluctance to explore such con-
troversial findings has outweighed their curiosity, and
neither Levengood nor Haselhoffs work has ever been
reconfirmed or disproved by subsequent studies.
Consequently, their research merely fuelled the long-
running discussions about human hoaxers, atmospheric
effects and, of course, extraterrestrial artists. Last June
I entered the debate by suggesting in Nature (465 693)
that terrestrial artists would not need to bend any laws,
but they would need mathematical skills to plot todays
epic designs and scientific awareness to exploit techno-
logical advances. This suggestion was met with anony-
mous hate-mail from UFOlogists and others accusing
me of spreading misinformation as part of a massive
6 Physics World August 2011
physicsworld.com
cover-up operation. I surfed the conspiracy websites to
find out who I was supposedly conspiring with and
found that the most likely culprit was a collaboration
between the UK, German and US secret services!
Although aliens and government conspiracies cannot
be excluded with 100% surety, Occams razor (which
states that explanations involving the fewest assump-
tions are the most likely) supports the human-artist sce-
nario. Might some artists therefore be supplementing
physical implantation techniques with microwaves?
Intriguingly, a group of crop-circle enthusiasts called
the BLT Research Team claims to be able to replicate
the observed changes to pulvini using 30 s exposures to
microwaves generated by magnetrons from readily
available microwave ovens. Todays magnetrons are
small and light, and some require only 12 V battery
power supplies. Haselhoff and Levengood used the
BeerLambert principle, which relates the absorption
of radiation to the properties of the material, to model
the radial dependence of the pulvini swelling. For a typ-
ical 9 m circle, Haselhoffs model indicated a radiation
point source placed 4 m above the circles centre. Once
superheated with this source, the stalk orientation
could be readily sculpted, speeding up circle creation.
Although this appealing hypothesis fits the published
facts, biophysicists will clearly need to expand on these
preliminary experiments if such speculations are to
become accepted.
Still seeking solutions
Determining the technology behind crop-circle mak-
ing has implications beyond mere curiosity and art
appreciation. Traces of some patterns (ghost forma-
tions) can still be seen in the subsequent years crop,
suggesting long-term damage to the crop field consis-
tent with Levengoods observations of stunted seed
growth. Crop formations are harvested every year, and
so these damaged crops are entering our food chain.
Intriguingly, Levengoods results showing stunted
growth came from crop circles that appeared early in
physicsworld.com Feature: Crop circles

5 Another curious pattern

5.0
length
standard deviation
4.0
average node length (mm)

3.0

2.0

1.0

0
b0 b1 b2 b3 b4 b5 b6 b7 b8

b8

b7

b6
b5
b4

b3
b2
b1

b0

Eltjo Haselhoffs studies of wheat stalks show that the length of stalk joints, or
pulvini, varies with their position relative to crop circles. (a) Average length of
pulvini (yellow bars) and their standard deviation (red bars) as measured at the
nine different locations b0b8 (b). At each location, 20 samples were measured.
Positions b8 and b0 lie outside the circular formation, and the average lengths of
pulvini there matched those measured in the remainder of the field.

the season in immature crops prior to anthesis (flow-

ering). However, he also reported that if the seeds were
instead removed from circles etched in mature crops,
then the growth rate was increased fivefold. This obser-
vation led Levengood to develop and patent Molecular
Impulse Response technology, which accelerates crop
growth by applying electrical pulses.
Crop-circle artists are not going to give up their
secrets easily. Researchers studying modern pic-
tographs have to take to the air to photograph the latest
patterns before they disappear forever under the har-
vesters blades. This summer, unknown artists will ven-
ture into the countryside close to your homes and carry
out their craft, safe in the knowledge that they are con-
tinuing the legacy of the most science-oriented art
movement in history. Can you unlock the secrets to
their success?

More about: Crop circles

S and K Alexander 2010 Crop Circle Year Book 2010
(Temporary Temple Press)
E H Haselhoff 2001 The Deepening Complexity of Crop Circles
(Frog Publishing, Berkeley)
R Irving and J Lundberg 2006 The Field Guide of Crop Circle
Making (Strange Attractor Press, London)

Physics World August 2011 7

 FOUR

Pizza Money

A
n early and apparent difference between the Two Johns internal
human engines was the way they processed time. It was the kind of
difference that made them perfect complements and the kind that
could cause irreparable conflict.
Carmack was of the moment. His ruling force was focus. Time existed for
him not in some promising future or sentimental past but in the present
condition, the intricate web ol problems and solutions, imagination and code.
He kept nothing from the pastno pictures, no records, no games, no compu-
ter disks. He didnt even save copies of his first games, Wraith and
Shadowforge. There was no yearbook to remind of his time at school, no
magazine copies of his early publications. He kept nothing but what he needed
at the time. His bedroom consisted of a lamp, a pillow, a blanket, and a stack
of books. There was no mattress. All he brought with him from home was a
cat named Mitzi (a gift from his stepfamily) with a mean streak and a reckless
bladder.
Romero, by contrast, was immersed in all moments: past, future, and
present. He was an equal opportunity enthusiast, as passionate about the
present as about the time gone and the time yet to come. He didnt just
dream, he pursued: hoarding everything from the past, immersing himself in
the dynamism of the moment, and chatting out the plans for what was to
come. He remembered every date, every name, every game. To preserve the
past, he kept letters, magazines, disks, Burger King pay stubs, pictures, games,
receipts. To inflate the present, he pumped up any opportunity for fun, tell-

45
ing a better joke, a funnier story, making a crazier face. Yet he wasnt manic,
he knew how to focus. When he was on, he was onloving everything, every-
body. But when he was off, he was offcold, distant, short. Tom Hall came up
with a nickname for the behavior. In computers, information is represented
in bits. A bit can be either on or off. Tom called Romeros mood swings the bit
flip.
That fateful morning of September 20, 1990, Romeros bit flipped right
on. It was a date he seared into his memory and Carmack would soon forget,
but it was equally important to both. Carmack had used his laser focus to
solve an immediate challenge: how to get a PC game to scroll. Romero used
Carmacks solution, Dangerous Dave in Copyright Infringement, to envision
what would come. Carmack had created a palette that Romero used to paint
the future. And the future, it became clear, had nothing to do with Softdisk.
After seeing Carmack, Romero couldnt contain his excitement. He darted
around the office, pulling others to come in and check out the game. Oh my
God, look at this, he said, as a couple of employees watched the demo play.
Is that the fucking coolest thing on the planet or what?
Oh, one of the guys replied lethargically, thats pretty neat.
Thats pretty neat? Romero responded. Wait a minute: this is like the
fucking coolest thing ever! Dont you understand?
The guys shrugged and said, Whatever, then returned to their offices.
Fucking idiots! Romero declared. By the time everyone else arrived, he
was on the verge of exploding. Tom, Jay, Lane, and Adrian were all in the
Gamers Edge office, watching amusedly as Romero held court, playing the
demo. Oh my God, Romero said, this is the fucking coolest thing ever! We
are fucking gone! We have to do this! We have got to start our own company
and get out of here with this, because Softdisk aint doing anything with it!
No ones going to see this! We need to do this on our own! This is too big to
waste on this company!
Jay was hanging on the doorway, his finger tips gripping the frame.
Eh, come on. he said, chortling. Hed seen Romeros giddiness before.
It was an enthusiasm that bordered on hyperbole. Romero got this excited
when he won a round of Pac-Man. He was a human exclamation point.
Romero froze, hands in the air. Dude, he said gravely, Im totally
serious.
Jay stepped in and shut the door behind him. Romero explained his ra-
tionale. First oft, this was a robust, sixteen-color game; Softdisk was interested
only in doing four-color games that appealed to the lowest common denomi-
nator of users. Second, this was essentially the Nintendo-style game made for
a PC, something on par with the bestselling console title in the world: Mario.
That meant the game was sure to sell, because everyone was getting a PC
and, naturally, everyone would want a fun video game to play. It was perfect.

46
 They already had the ideal team: Carmack, the graphics guru and resi-
dent Whiz Kid; Romero, the multitalented programmer and company cheer-
leader; Adrian, the artist and dark visionary; and Tom, game designer and
comic book surrealist. Although Romero was still displeased with Lane, he
was willing to give him one more chance to pull through. Regardless, the core
chemistry was potent. Carmacks steadfastness balanced Romeros elusive
passion. Adrians grisly tastes countered Toms cartoon comedy. All they
needed was someone to do the business sidehandle the finances, balance
the books, manage the team. Everyone looked to Jay. Dude, Romero told
him, youve got to be a part of this too.
Jay gave them his biggest bartender smile and agreed. Heres what I
think we should do, he said. This needs to be taken right to the top of
Nintendo. Now! If they could get a deal to do a PC port of Super Mario
Brothers 3, they could be in business, serious business. They decided to take
the weekend to make a complete demo of the game, with a few added levels
as well as the inclusion of the Mario character, and Jay would send it off.
There was only one problem, but it was sizable. It they were going to
moonlight this game, they didnt want Softdisk to know. This meant they
couldnt do it in the office. They would have to work on it at home. Thing
was, they didnt have the computers they needed to get the job done. The
five of them sat quietly in the Gamers Edge office, pondering the problem as
Dangerous Dave looped across the screen.
Carmack and Romero had both been without the computers they wanted
earlier in life. So this wouldnt be the first time they came up with a way to get
them.

The cars backed up to the Softdisk office, trunks open, waiting in the night.
It was late Friday night, long after all the other employees had returned home
to their families and television sets. No one would use the PCs from the office
on Saturday and Sunday, so they might as well make use of them, the gamers
figured. They werent stealing the computers, they were borrowing them.
After loading Softdisks computers in their cars, Romero, Jay, Carmack,
Tom, and Lane caravanned out of downtown. They drove away from the
run-down buildings, down the highway, until the scenery began to change to
low-hanging trees and swamps. Late-night fishermen lined a bridge with their
lines, in the purple-black murk. A bridge led them to South Lakeshore Drive,
the border of Shreveports main recreational front and main water supply,
Cross Lake.
Carmack, Lane, Jay, and an Apple II programmer at Softdisk named Jason
Blochowiak had scored a enviable coup not long before when they found a
four-bedroom house for rent right along these shores. Jay had bought a cheap
boat, which they docked there and used for frequent outings of knee board-

47
ing and skiing. In the large backyard was a swimming pool and a barbecue,
with which Jay, a cooking enthusiast, grilled up Flintstonean slabs of ribs.
The house itself had plenty of windows looking out on the scene, a large
living room, even a big tiled bathroom with a deep earth-tone-tiled Jacuzzi
tub. Jay had installed a beer keg in the fridge. It was a perfect place to make
games.
Over the weekend while making the Super Mario demo, the gamers put
the house to the ultimate test. They hooked two of the Softdisk computers up
on a large table that Carmack had been using to hold all-night Dungeons and
Dragons sessions with the guys. Romero and Carmack sat there program-
ming together. Tom did all the graphics and Lane animated the familiar little
turtle. Earlier they had videotaped the entire game play of Super Marie Broth-
ers 3. To capture all the elements, Tom kept running back and forth, pressing
pause on the VCR so he could copy the scenes.
Over those seventy-two hours, they fell into crunch mode. No one slept.
They consumed huge quantities of caffeinated soda. Pizza deliveries came
repeatedly. Jay worked the grill, churning out a stream of burgers and hot
dogs, which often went uneaten. They got the game down to a T: Marios
squat little walk, the way he bopped the animated tiles, sending out the coins,
the way he leapt on the turtles and kicked their shells, the clouds, the Venuss-
flytraps, the pipes, the smooth scrolling. By the time they finished, the game
was virtually identical to the bestselling hit in the world. The only noticeable
difference was the title screen, which, under the Nintendo copyright, credited
the makers, a company name the guys borrowed from Romero and Lane,
Ideas from the Deep.
With the game done, Jay put together a letter explaining who they were
and how they wanted Nintendo to take the unprecedented step of licensing
Super Mario for the PC. Hopes high, the boys taped up the box and sent it on
its way to Nintendo. When the response came back a few weeks later, it was
short and sweet. Nice work, the company said, but Nintendo had no interest
in pursuing the PC market. It was happy where it was as the world leader in
consoles. It was a disappointment for the group, especially following the ela-
tion of the lake house programming marathon. But it was not the end by any
means. There had to be someone out there who would appreciate their ac-
complishment. Romero knew just the guy.

Not long before, Romero had received his very first fan letter while working
at Softdisk. It was typewritten and cordial. Dear John, it read, Loved your
game. Just wanted to let you know it was a great game and I think you are
very talented. Have you played The Greatest Pyramid? It is almost the same
as your game. I was wondering if you made that game too? Or if you were
inspired by it? I can send you a copy if you want. Also whats your high score

48
for your game? Have you been programming long and what language did
you use? I am thinking about writing a game and any tips you have would be
helpful. Thanks from a big fan! Sincerely, Byron Muller.
Romero, the pack rat, had immediately taped the letter up on his wall and
showed it off to Carmack, Tom, Lane, and Adrian. A couple weeks later, he
got another fan letter, handwritten and a bit more urgent. Dear John, it
read, I loved your game (Pyramids of Egypt), it is better than another pyra-
mid type game that was in Big Blue Disk a few issues ago. I finished the
game alter staying up until 2:00 A.M. last night! Great fun! Whats your best
score on the game? Is there a secret key that advances to the next level auto-
matically? Do you know of any similar games? Please call me collect if you
want or please write. Thanks a million, Scott Mulliere. P.S. I think I found
a minor bug (undocumented feature?) in the game!
WowRomero beamedanother fan! He taped this letter up on the wall
next to the other one and, again, bragged to Carmack and Adrian, who rolled
their eyes. Soon after, Romero was flipping through PC Games Magazine
when he came to a brief article about Scott Miller, a twenty-nine-year-old
programmer who was having great success distributing his own games. In-
trigued, Romero read to the bottom of the article, where it listed Scotts ad-
dress: 4206 Mayflower Drive, Garland, Texas, 75043.
He paused. Garland, Texas. Garland, Garland, Texas? Who did he know
in Garland, Texas, on Mayflower Drive? He set down the magazine and looked
up on his wall. The fan letters! By now he had accumulated several of them
and, to his amazement, though they all were signed by different names, each
and every one had the same return address: Mayflower Drive, Garland.
Romero was pissed. Here he was showing off to the other guys about all
his supposed fans, when in fact it was just some loser fucking with his mind.
Who the fuck does Scott Miller think he is? Romero whipped around to his
keyboard and banged out a letter in fury: Scott: You, sir, have serious psy-
chological problems Whats the deal with the 16 million odd names youve
been writing under to reach me? Huh, Byron Muilliere, Brian Allen, Byron
Muller? How old are you, really? 15?
Romero fumed for a couple pages, then left the letter on his desk. The
next day, he came back cooled off and wrote another note.
Dear Mr. Miller, he typed, I have taken a considerable amount of time
to reply to your last letter. The reason is because I was infuriated when I
found out that you had written to me previously about 3-4 other times, all
under different names and I didnt know what was going on. My previous
reply is a real scorcher; thats why I didnt send it earlier. I am sending it
anyway just so you can see how pissed I was at the time. I am writing this
cover letter to soften the previous reply and to tell you that I am somewhat
intrigued by your numerous approaches. He sealed up both letters together
and sent them to Garland once and for all.

49
 A few days later, Romeros home phone rang. It was Scott Miller.
Romero laid into him about sending those fake fan letters, but Scott had
other things on his mind. Fuck those letters! Scott said breathlessly. The
only reason I did that was because I knew my only chance to get ahold of you
was to go through the back door.
Game companies at the time were extremely competitive and secretive,
especially when it came to their programming talent. When Romero had been
a young gamer, programmers like Richard Garriott or Ken and Roberta
Williams always got top billing, their names advertised in big letter on the
box. But by the early nineties, times had changed. Companies were not above
poaching. As a precaution, many game publishers would loom over their
staffs, monitoring calls to make sure that no one was trying to make a steal.
Scott, well aware of the sensitivity of his call, had chosen instead to try to lure
Romero into contacting him. It worked, though ironically not as originally
intended. He hadnt meant to piss Romero off. But now that he had his atten-
tion, he wasnt about to let it go.
We gotta talk, Scott continued eagerly. I saw your game Pyramids of
Egypt. It was so awesome! Can you do a few more levels of it? We can make a
ton of money.
What are you talking about?
I want to publish your game, Scott said, as shareware.
Shareware. Romero was familiar with the concept. It dated back to a guy
named Andrew Fluegelman, founding editor of PC World magazine. In 1980,
Fluegelman wrote a program called PC-Talk and released it online with a note
saying that anyone who liked the wares should feel free to send him some
appreciation money. Soon enough he had to hire a staff to count all the
checks. Fluegelman called the practice shareware, an experiment in eco-
nomics. Over the eighties other hackers picked up the ball, making their
programs for Apples, PCs, and other computers available in the same honor
code; Try it, if you like it, pay me. The payment would entitle the customer to
receive technical support and updates.
The Association of Shareware Professionals put the business, largely do-
mestic, between $10 and 520 million annuallyeven with only an estimated
10 percent of customers paying to register a shareware title. Forbes magazine
marveled at the trend, writing in 1988 that if this doesnt sound like a very
sound way to build a business, think again. Shareware, it argued, relied not
on expensive advertising but on word of mouth or, as one practitioner put it,
word of disk. Robert Wallace, a top programmer at Microsoft, turned a
shareware program of his called PC-Write into a multimillion-dollar empire.
Most authors, however, were happy to break six figures and often made little
more than $25,000 per year. Selling a thousand copies of a title in one year
was a great success. Shareware was still a radical conceit, one that, further-
more, had been used only for utility programs, like check-balancing programs

50
and word-processing wares. It had never been exploited for games. What was
Scott thinking?
As they talked, it became clear to Romero that Scott knew exactly what
he was doing. Scott, like Romero, was a lifelong gamer. The son of a NASA
executive, he was a conservative-looking guy with short, dark hair. He had
spent his high school days in Garland hanging out in the computer lab during
the day and at the arcade after school. He even wrote a strategy guide called
Shootout: Zap the Video Games, detailing all the ways, to beat the hot games of
1982, from Pac-Man to Missile Command. Scott soon took the inevitable path
and started making games of his own.
When it came time to distribute the games, Scott took a long, hard look at
the shareware market. He liked what he saw: the fact that he could run every-
thing himself without having to deal with retailers or publishers. So he fol-
lowed suit, putting out two text-based games in their entirety and waiting for
the cash to roll in. But the cash didnt roll; it didnt even trickle. Gamers, he
realized, might be a different breed from those consumers who actually paid
for utility shareware. They were more apt simply to take what they could get
for free. Scott did some research and realized he wasnt alone; other program-
mers who had released games in their entirety as shareware were broke too.
People may be honest, he thought, but theyre also generally lazy. They need
an incentive.
Then he got an idea. Instead of giving away the entire game, why not give
out only the first portion, then make the player buy the rest of the game directly
from him? No one had tried it before, but there was no reason it couldnt
work. The games Scott was making were perfectly suited to such a plan be-
cause they were broken up into short episodes or levels of play. He could
simply put out, say, fifteen levels of a game, then tell players that if they sent
him a check he would send them the remaining thirty.
In 1986, while working for a computer consulting company, Scott self-
published his first game, Kingdom of Krozan Indiana Jones-style adven-
tureas shareware, making the initial levels available through BBSs and
shareware catalogs. There was no advertising, no marketing, and virtually no
overheadexcept for the low cost of floppy disks and Ziploc bags. Because
there were no other people to pay off, Scott could price his games much lower
than most retail titles: $15 to $20 as opposed to $50 to $40. For every dollar
he brought in, Scott was pocketing ninety cents. By the time he contacted
Romero, he had earned $ 150,000 by word of mouth alone.
Business was so good, Scott explained, that hed quit his day job to start
his own shareware game publishing company, called Apogee. And he was
looking tor other games to publish. Romero was making perfect shareware
games and he didnt even know it, Scott said. An ideal shareware game had to
have a few ingredients: short action titles that were broken up in levels. Be-
cause the shareware games were being distributed over BBSs, they had to be

51
small enough for people to download them over modems. Large, graphically
intensive games, like those being published by Sierra On-Line, were simply
too big for BBS-based distribution. Games had to be small but fun and fast,
something adrenal and arcade-style enough to hook a player into buying
more. If Romero would give him Pyramids of Egypt, Scott would handle all
the marketing and order processing; the guys would receive some kind of
advance plus a 55 percent royalty rate, higher than theyd get from any major
publisher.
Romero was intrigued, but there was a problem. We cant do Pyramids
of Egypt, he explained, because Softdisk owns it. He could hear the disap-
pointment in Scotts sigh. Hey, he added, Screw that game! Its crap com-
pared to what were doing right now.
A few days later, Scott received a package with the Super Mario Brothers
3 demo from Ideas from the Deep. When he fired up the game, he was knocked
out. It looked just like the console versionsmooth scrolling and everything.
He grabbed the phone and talked to Carmack for hours. This guy is a genius,
Scott thought. Hes outthinking everybody. By the time they were through
talking, Scott was more than ready to make a deal. The gamers said they
would use this new technology to create a title specifically for Apogee to
release as shareware. Great, Scott said, Lets do it.
Now they just had to come up with a game.

After their initial conversation, Romero asked Scott to show them his seri-
ousness by sending them an advance. Scott responded with a check for two
thousand dollars, half his savings. There was only one thing he wanted in
return: A game by Christmas, two months away.
Romero, Carmack, Adrian, Lane, Tom, and Jay convened in the Gamers
Edge office to come up with the game. Tom was quick to point out that,
because they were using this console-style technology, they should make a
console-style game, something like Mario but different. Fueled by the en-
ergy, he was quick to volunteer himself with a fair amount of the bravado that
was becoming a requisite part of their clan.
Come on, what theme do you want? Tom said. Tell me, I can do
anything. How about science fiction?
They liked the idea. Why dont we do something, Carmack said, where
a little kid genius saves the world or something like that? Mmm.
Okay, yeah! Tom said. I have a great idea for something like that.
And in a blur he sped from the room and locked himself in his office in the
Apple II department. He could feel his head opening up, the ideas pouring
out in what sounded like the voice of Walter Winchell. Tom had long been a
huge fan of Warner Bros cartoons; Chuck Jones, the Looney Tunes animator,
was a god to him. Hed also watched Dan Aykroyds impression of The

52
Untouchables Eliot Ness as a kid. He thought about all these things, plus
Mario, plus, for flavor, a routine by the comedian George Carlin about people
who use bay leaves as underarm deodorant and go around smelling like bean
with bacon soup.
Tom typed until he had three paragraphs on his paper. Pulling it out of
the printer, he dashed back into the Gamers Edge office and read these words
in his best Winchell impression:

Billy Blaze, eight-year-old genius, working diligently in his

backyard clubhouse, has created an interstellar spaceship from old soup
cans, rubber cement, and plastic tubing. While his folk are out on the
town and the baby sitter is asleep, Billy sneaks out to his backyard
workshop, dons his brothers football helmet, and transforming into
Commander Keendefender of justice! In his ship, the Bean with Bacon
Megarocket, Keen dispenses justice with an iron hand!
In this episode, aliens from the planet Vorticon VI find out about the
eight-year-old genius and plan his destruction. While Keen is out ex-
ploring the mountain of Mars, the Vorticons steal his ship and leave
pieces of it around the galaxy! Can Keen recover all the pieces of his
ship and repel the Vorticon invasion? Will he make it back before his
parents get home? Stay tuned!

He looked around. Silence. Then, in a burst, everyone was laughing, even

the generally stoic John Carmack, who not only laughed but applauded. Com-
mander Keen was on board. Where he would take them, they hardly knew.

The gamers werent just Softdisk guys anymore, they were, as they called
themselves, the IFD guys, co-owners of Ideas from the Deep. Softdisk, as a
result, took on an even greater pallor. But it was a day job, a job they all
needed since there was no real money coming in yet and no guarantee that it
would come in at all. They decided, then, to continue working on titles for
Gamers Edge during the day while they churned out Commander Keen from
the lake house at night.
They became all the more efficient at borrowing the Softdisk comput-
ers. Every night after work theyd back their cars up to the office and load the
machines. The next morning theyd come in early enough to bring the com-
puters back. They even got a little cocky about it. Though the machines were
top of the line, they wanted some minor adjustments made. Jay began
moseying on down to the Softdisk administration office to request new parts.
Al Vekovius took notice of the requests but didnt think too much of them.
He was still gung ho about Gamers Edge and the potential to break into the
PC marketplace. So whatever the gamers wanted, the gamers would have.

53
 From October to December 1990, they worked virtually nonstop to get
Keen done for Scott by Christmas. And it wasnt just one Keen; it was a trilogy
called Invasion of the Vorticons. Trilogies were common in the games indus-
try for the same reason they were common in books and films; they were the
best way to build and expand a brand identity. Tom, who assumed the role of
creative director, mapped out the game plan.
Mario, this was not. As a hero, an eight-year-old misfit who steals his
dads Everclear for rocket fuel was more identifiable than a middle-aged Ital-
ian plumber. It was as if the gamers had followed that golden rule of writing
about what they knew. Tom, as a kid, used to walk around in a Green Bay
Packers helmet and red Converse sneakers, just like Billy Blaze. And. in a
sense, they were all Billy Blazes, oddball kids who modified technology to
create elaborate means of escape. Keen was a punk, a hacker. And he was
saving the galaxy, just as countless hackers like Carmack and Romero used
technology to save themselves.
The roles were set: Carmack and Romero were the programmers, and
Tom the lead designerthe person in charge of coming up with the game play
elements, from the story and setting to the characters and weapons. Carmack
and Romero were happy to leave Tom to the creative work; they were too
busy programming. Carmack was refining his engine, getting the smooth
scrolling down to the point where Keen could move as fluidly left or right as
he could up or down, Romero, meanwhile, was working the editor, the pro-
gram that allows the developers to put together the graphics of the game
characters, rooms, monsters. It was essentially a game designers construc-
tion kit. Carmack and Romero were in sync.
Not everyone else gelled quite as well. Lane was now officially kicked out
of the Keen development. Despite Romeros fondness for him as a friend, he
felt that Lanes energy was lacking. Adrian was having problems of his own.
Though he was recruited later to help them work on Keen, Adrian hated the
project. It was too cutesy. Tom had a target audience in mind: kids, he
said, or those who have kidlike mentalities like we do. Adrian hated kiddie
stuff. Even more, he hated cutesy. Worst of all was cutesy kiddie. And now
here he was having to sit all night drawing pizza slices, soda pop, and candy.
Tom came up with a little character called a Yorp with a big fat green body
and one periscopelike eye over his head. Even the monsters were cute. In
most games, when a character died, it would simply disappear, vanish. But
Tom had other notions. He was eager to incorporate some larger philosophi-
cal ideas, as he said. He loosely based character on ideas hed read in Freuds
Civilisation and Its Discontents; a guard was made to represent an id. He
wanted to teach kids that when people or even aliens die, they really die, they
leave corpses. So he wanted the dead creatures in the game to just remain:
not graphic or bloody corpses, just dead Yorps. Cute dead Yorps.

54
 The cuteness of the characters wasnt the only thing bugging Adrian, it
was the cuteness of their creator. Tom was getting on his nerves. He would
run around the house, craning his neck and making sounds to show Adrian
exactly what the alien creatures in the games were supposed to look like.
Romero would usually crack up at these displays. Adrian took a liking to
Romero, who shared his taste in heavy metal and his appreciation ol sick
humor; but Tom, in Adrians mind, was just plain annoying. To make mat-
ters worse, they had to share a desk, and Tom was so full of energy that he
kept bobbing his knee up and down, inadvertently hitting the table when
Adrian was trying to draw. But it was better than working at the last open
space in the house, next to the litter box used tor John Carmacks cat, Mitzi.
Tom had no idea how Adrian felt. He thought he was just quiet.
For the majority of the time, however, those late nights at the lake house
were a perpetual programming party. With Iggy Pop or Dokken playing on
the stereo, the guys all worked into the wee hours. Occasionally, theyd take a
break to play Super Mario on the Nintendo or maybe a round of Dungeons
and Dragons. Carmack had been building a large D&D campaign for the
guys, and on Saturday nights theyd gather around a table and play into the
early morning hours. With Carmack as Dungeon Master, the game took on
depth and complexity. It was quickly becoming the longest and deepest D&D
game hed ever created. And there were no signs of it letting up.
Other times, theyd cruise the lake on the boat. Jay quickly became the
designated driver; his impeccable focus gave him the ability to drive not only
fast but steady. A couple times they let Romero drive, but he was having too
much fun, steering the boat precipitously off course. Jay also fell comfortably
into the role of manager or, in a sense, frat house president. While the guys
worked, he would grill up ribs on the barbecue or restock the sodas. They
were under the gun and needed all the help they could get.
They didnt need any help getting motivated, however. Carmack, in par-
ticular, seemed almost inhumanly immune to distraction. One time, Jay tested
Carmacks resolve by popping a porno video into the VCR and cranking it to
full volume. Romero and the others immediately heard the oohs and aahs,
and, turned around cracking up. Carmack, though, stayed glued to his moni-
tor. Only after a minute or so did he acknowledge the increasingly active
groans. His sole response was Mmm. Then he returned to the work at
hand.

Back at Softdisk, Al Vekovius was beginning to grow suspicious of his star

gamers. Jay was continually requesting parts for the computers. And the other
guys were behaving more curtly and elusively. His first suspicion came shortly
after they were working on their new game for Softdisk, a ninja warrior title

55
called Shadow Knights. Al had never seen a side scrolling like this tor the PC.
Wow, he told Carmack, you should patent this technology.
Carmack turned red. If you ever ask me to patent anything, he snapped,
Ill quit. Al assumed Carmack was trying to protect his own financial inter-
ests, but in reality he had struck what was growing into an increasingly raw
nerve for the young, idealistic programmer. It was one of the few things that
could truly make him angry. It was ingrained in his bones since his first read-
ing of the Hacker Ethic.
All of science and technology and culture and learning and academics is
built upon using the work that others have done before, Carmack thought.
But to take a patenting approach and say its like, well, this idea is my idea,
you cannot extend this idea in anyway, because I own this ideait just seems
so fundamentally wrong. Patents were jeopardizing the very thing that was
central to his life: writing code to solve problems. If the world became a place
in which he couldnt solve a problem without infringing on someones pat-
ents, he would be very unhappy living there.
Carmack was becoming more blunt and insulting about other topics as
well, most notably the rest of the Softdisk staff. Youve got a lot of terrible
programmers here, he said. They just stink. It was as if Carmack simply
didnt care how he alienated himself from the rest of the employees.
Al began dropping by the Gamers Edge office more often, only to dis-
cover more strange behavior. He once walked in to find Carmack, Romero,
and Tom huddled around Romeros computers with their backs to the door.
When Al made his presence known, they quickly dispersed. He stepped over
and asked them what was going on. Nothing but dirty jokes, Al, Romero
replied, gingerly. When Al looked at the screen, it was suspiciously blank.
Later he commented to Carmack that Romero was acting strangely, which
struck Al as odd since Romero was always so nice. Carmack considered this
momentarily, then, as always, blurted out his unedited perception of the truth:
Romero was just being friendly, Carmack said. When you turn your back,
he hates your guts.
By Thanksgiving, the guys were immersed in the death schedule back at
the lake house. Sleep was not an issue. Neither was showering. Eating was
something they essentially had to remind themselves to do. To help keep
them fed while they crunched on Keen, Scott had begun sending the team
weekly hundred-dollar checks labeled pizza bonus, playing off the pepperoni
slice icon that appeared in Keen. Pizza was ids fuel. It was, as Carmack en-
joyed noting, the perfect invention: hot, quick, and containing a variety of
food groups. When Jay opened an envelope from Scott and waved the check
in the air, everyone would declare pizza money!
Scott was confident hed see a return on his investment. He had initiated a
full-on blitz. Because of his own success, he had built strong ties with the
heads of various BBSs and shareware magazines across the country. He called

56
every one of them, preparing each for a game that would revolutionize the
industry. Before long, whenever people logged on to a BBS, they would see a
title screen reading: Coming soon From Apogee: Commander Keen. Scott
was putting his reputation on the line. But there was never a doubt in the
gamers minds that Keen would deliver.
Torn was in overdrive on the design, bouncing ideas like Ping-Pong balls
off Romero. If Romero doubled over laughing, he knew he was on the right
track. Scott offered his own advice for the game. One of the reasons for
Mario Brothers popularity, he wrote them in a letter, is that you can con-
tinue playing the game in search of secret or hidden bonuses, et cetera. I
would really like to see something like this implemented in Keenit would
really add to the game I think,
Like duh! The guys responded. They loved finding secrets in games.
Already secrets were like a subculture among programmers. Sometimes there
were secret levels, or inside jokes, or tricks that had no real bearing on the
outcome of the game. These were called Easter eggs. The mother of all eggs
occurred in 1980, when intrepid Atari 2600 geeks stumbled on a secret room
in the geometric role-playing game Adventure, only to find the flashing words
Warren Robinett. Some players haplessly shot at the name. Others just
scratched their heads. Robinett was a disgruntled Adventure programmer
who wanted recognition following a corporate takeover.
Tom came up with some tricks for Keen. In episode one, players could
find a secret hidden city it they pulled a combination of moves, like throwing
themselves in the line of fire of an ice cannon. Around the game he inserted
cryptic signs written in what was supposed to be the Vorticon alphabet. If
players stumbled into a secret area, they could get the translations.
The guys were so enthusiastic that they decided to put in a preview of
their upcoming games, which, at the time, didnt exist. They described more
installments of Keen as well as a new game based on characters and elements
of Carmacks evolving Dungeons and Dragons world. The Fight for Justice,
they wrote, A completely new approach to fantasy gaming. You start not as
a weakling with no foodyou start as Quake, the strongest, most dangerous
person on the continent. You start off with the hammer of thunderbolts, the
ring of regeneration, and a trans-dimensional artifact all the people you
meet will have their own personalities, lives, and objectives The Fight for
Justice will be the finest PC- game yet.
The lake house was filled with the sense of unlimited possibilities. And
the bond between Carmack and Romero was becoming stronger by the day.
It was like two tennis players who, alter years of destroying their competition,
finally had a chance to play equals. Romero pushed Carmack to be a better
programmer. Carmack pushed Romero to be a better designer. What they
shared equally was their passion.

57
 This was most clear to Carmack one late weekend night. He was sitting in
the house working at his PC as lightning flashed outside. Mitzi curled lazily
on top of his monitor, her legs draping over the screen. The heat of her body
was causing Carmacks heat-sensitive display to ooze its colors. He pushed
Mitzi gently from the monitor, and she scurried away with a hiss.
A rainstorm had picked up, and it was mighty. Cross Lake spilled into the
backyard like the prelude to a horror movie. The lake was so high that it
pushed the ski boat to the top of the boathouse. Long black water moccasins
slithered toward the deck. The bridge leading to Lakeshore Drive was com-
pletely washed out. When Jay arrived after having been out for the day, there
was no way to get in. It was, as he described it, a turd floater of a storm,
bringing everything from the bottom of the lake to the surface. He turned
away to wait it out.
Romero arrived with a friend later to find the bridge even worse than
when Jay got there. There was simply no way he was going to get his car over
the flooded expanse. And there were probably alligators and moccasins now
making it their home.
Back in the house, Carmack resigned himself to working on his own that
night. After all these hours, he had come to appreciate Romeros diverse range
of talents, gleaned from years of making his own Apple II games. Romero
had been not only a coder but an artist, a designer, and a businessman. On
top of all that, he was fun. Romero didnt just love games; in a sense, he was a
game, a walking, talking, beeping, twitching human video game who never
seemed to let anything get him down. Like a game character, he could always
find an extra life.
Just then the door behind Carmack swung open. Mitzi dashed under his
feet. Carmack turned to see Romero standing there with his big thick glasses,
soaking wet up to his chest, lightning flashing behind him, a big smile on his
face. It was a real moment, a moment so impressive that Carmack actually
saved it in his thin tile of sentimental memories. This one he wanted for
future access: the night Romero waded through a stormy river to work.

On the afternoon of December 14, 1990, Scott Miller pressed a button on

his PC and uploaded the Commander Keen shareware episode Marooned on
Mars to the first BBS. For $30, players could purchase the other two episodes,
which Scott would ship on floppy disks in Ziploc bags. Before Keen, Scotts
total shareware sales were about $7,000 per month. By Christmas, Keen was
approaching $30,000.
The game was, as Scott told the numerous editors and BBS controllers
who were deluging him with calls, a little atom bomb. No one had seen
anything like it for the PCthe humor, the graphics, the side-scrolling Mario-
type action. Superlative alert! heralded one reviewer. Be prepared to hear

58
praise like we have rarely heaped on any program. Keen sets a new stand-
ard for shareware games, declared another. For stimulating, velvet-smooth
and cutting edge PC arcade action, wrote a third, there is nothing better
than Commander Keen from Apogee Software. Nothing. The game wasnt
just on par with Nintendo, it concluded, it was better.
Fans couldnt agree more. They were deluging Apogee with letters of
praise and letters inquiring about the next games in the Keen series. All the
main BBSs were ablaze with conversation about Keentricks, secrets, strate-
gies. Gamers were pleading tor information to decode the Vorticon alphabet.
Scott was so swamped that he recruited his mother and his first employee, a
teenage programmer named Shawn Green, to help with the demand. When
Shawn showed up for work the first morning, he was greeted by Scotts mother,
standing in her bathrobe holding two cordless phones. The second she handed
him one, it started to ring.
Romero, Carmack, and the rest of the group celebrated with a huge party
at the lake house on New Years Eve. The stereo cranked Prince. The grill
smoked. Revelers boated around the lake. Romero, who rarely drank, made
this night a special occasion. It had been a great year but a tough yearone
that had cost him his wife and kids. Faced with the choice, hed chosen the
game life over the family life. Though he spoke frequently with his boys and
saw them as often as he could, he was living with a new family now: the
gamers. And he wanted this night together to last.
He, Tom, and Jay were drunk on white wine and champagne in the
kitchen. Romero saw Carmack standing in the corner by himself, sober. Come
on, Carmack, he slurred, you gotta drink, dont be a baby! Its going to be
1991!
Normally in these situations Carmack wanted nothing less than to disap-
pear into the wallpaper. This kind of scenesocializing, cavortingwas never
his domain. He would rather be reading or programming. But contrary to
what the other guys might have thought, he wasnt inhuman. He was fun
loving too, just in his own way. He was thrilled to be working for himself,
making games, collaborating with people he admired and respected. It took
only a little coaxing from Romero to get Carmack to join them in downing
several glasses of champagne. The strongest thing theyd seen him drink be-
fore was Diet Coke. Some time later Romero found Carmack leaning quietly
against the kitchen wall. Hey, man, Romero said, You feeling buzzed yet?
You getting drunk, Carmack?
I am losing control of my faculties, Carmack replied. Mmm.
Then he stumbled away. Romero got a lot of mileage out of that response,
repeating it robotically to everyone throughout the night. It was good to see
Carmack loosen up.

59
Two weeks later, Jay walked out to the mailbox and came back brandishing
an envelope. It was the first residual check from Apogee. Pizza money!
they all said, as he opened it up. The check was for $10,500. With barely any
overhead expenses, it was gravy. At this rate theyd be making more than
$100,000 in their first year, more than enough for them to quit their day jobs
at Softdisk.
Al Vekovius still had no idea that they were moonlighting on the Keen
games, let alone doing it on the company computers. Gamers Edge was do-
ing quite well, and their latest games, Catacomb II and Shadow Knights, were
drawing raves. Softdisk had about three thousand subscriber who had paid
$69.96 per year to receive Gamers Edge every month. They knew he was
counting on them and werent sure how hed react to their mass departure.
Carmack and Romero made it clear they didnt care. This was their break,
after all. Tom, by contrast, was nervous about the move. He was worried
about getting sued by Softdisk, ruining their chances not only for making it
on their own but for enjoying the fruits of Keens success. Romero scoffed at
his worries. Dude, whats Al going to do if he sues you? You dont have
anything for him to get. All you have is a piece-of-shit couch, he said, point-
ing to the broken sofa in the living room. I mean, what the fuck? What are
you worried about losing?
Jay also expressed concern, urging the guys to handle this delicately with
their boss. Dont drop a bomb on him, he implored.
Dont worry, Romero said with his characteristic optimism. Every-
things going to be fine.
However, Als suspicions began to mount when an employee mentioned
something about the Gamers Edge guys moonlighting on their own games.
Al confronted Carmack, who he knew had a tough, it not impossible, time
telling lies. It was like feeding questions into a computer or adding numbers
on a calculatorthe answer always came out right. I admit it, Carmack said.
Weve been using your computers. Weve been writing our own games on
your time. Later he and Romero broke the news: They were going to leave,
and they were taking Adrian Carmack, their art intern, with them.
Al felt like hed walked into his house to find that someone had broken
his windows and stolen his television. But he didnt let himself get too far
down. Immediately he tried to turn the situation around.
Look, he said, lets try to salvage something out of this. Lets go into
business together! Lets form a new company! Ill support you. And you guys
write whatever games you want and Ill handle selling them. Well split eve-
rything fifty-fifty. And I wont take any legal action against you.
The offer caught them by surprise. They had assumed Big Al was going
to sue them, not finance their business. Now there was a new golden oppor-
tunity. All they wanted to do was have their own business, and they had no

60
interest in dealing with the hassles of taxes and distribution. If Al was going
to handle that stuff, what the hell? They agreed.
But when Al returned to the Softdisk office, he walked into a mutiny. The
entire company had gathered to demand an explanation. Carmack and
Romero came back from lunch and bragged about some big special deal they
were getting, one of the employees said. Whats the deal? Here these jokers
had cheated the company, used the company computers, and now youre
giving them hall of a new company? Why are you rewarding them?
Because its good business! AI said, because these guys are good!
Theyre going to make money for the company. Well all be successful. No
one was buying it. Either the gamers go, they said, or all thirty of them were
going to quit. Al sighed deeply and walked back to the Gamers Edge office.
You guys went and told everyone about this and created a nightmare, he
said. Do you realize what you have done?
Well, Carmack replied, we wanted to be truthful.
Yeah, but I could have positioned it a lot better, he said. I cant afford
to lose my staff. The deals off.

After several weeks of negotiations and threatened lawsuits, it was agreed

that they would contract with Softdisk to write one new game for Gamers
Edge every two months. It was demoralizing, not only for the Softdisk staff
but for Al. He saw that, despite their talent, the Gamers Edge guys really
were just boys living by their own rules, and cheating when necessary. Worst
of all, they had no sense of guilt. For them it was something to laugh about.
They never considered the people who worked at Softdisk. Before Carmack
left, Al pulled him aside and asked, Did you ever think about the people
who have worked so hard and supported you?
Carmack listened, but Als words didnt compute. He was looking into the
face of the past, of opportunities unrealized, of all the old authority figures
who had ever stood in his way. As always, he was blunt to a fault. I dont
care about them, Al would recall Carmack replying. Ill go back to making
pizzas before I stay at this crummy place.
On February 1, 1991, id Software was born.

61
 Chapter five Transport 109

proposals

5.1 Introduction a balanced such as the London Thames Gateway can be

best accommodated by bus, tram or light rail
and integrated approach such as the DLR, whereas the more dispersed
trips in Outer London tend to favour bus, bike
230 Chapter four outlines the strategic policies
(over a shorter distance) and car. Local short
that are required to meet the objectives of the
distance journeys present the best opportunity
MTS. There are a number of tools available
for walking and cycling.
to implement these policies such as investing
in the transport network to provide more
233 As a consequence, different areas of London
capacity and better connectivity, managing
require different policy interventions. Proposals
and influencing the demand for travel, and
for central London will inevitably focus on
the introduction of new technology to reduce
tackling congestion, increasing the capacity
emissions. As always, there are trade-offs which
of the rail network, encouraging walking and
means a balanced approach must be taken.
cycling, and managing demand. In Outer
London, proposals need to acknowledge the
231 Londons transport network is finite and there
role of the car, especially low emission cars.

Chapter five
is often competition for this limited space,
For Outer London town centres, measures to
whether it is for road space or rail paths. In
improve bus accessibility, public realm, walking
striking the right balance the proposals in this
and cycling will generally be prioritised. There
chapter recognise that all the varying needs
may be places where a number of options are
of London need to be met the needs for
possible. In these cases further work will be
international links (as a global city), for national
required to assess the most effective solution,
links (as a national capital), and more local links
bearing in mind the cost of the scheme in
(London is a place where people live, visit and
construction and during operation. The need
work). The proposals set out in this chapter
to recognise affordability and business case
cater for all of these needs.
constraints will be paramount given the current
financial environment, and this will inevitably
232 The specific transport proposals have to
preclude some schemes.
be considered in terms of effectiveness,
acceptability and cost. There may be an obvious
234 Due to the dispersed nature of trips in Outer
best mode for the task, for example, only
London, the role of the car is acknowledged as
rail-based modes can provide the sufficient
sometimes necessary, particularly for medium
capacity to cater for the very high volume
to longer distance trips. The use of cleaner,
same time, same place demand that occurs
low emission cars will be encouraged over others.
twice a day during the week to, and from,
central London. Similarly, regular high to
235 Most freight is moved by road. While a mode
medium volume demand over a short to
shift from road to rail and water is needed to
medium distance, particularly in growing areas
achieve the goals of the strategy, for some

Chapter five Transport proposals

110 Mayors Transport Strategy
types of freight, and for many servicing trips,
access by road will remain a requirement. The
strategy therefore needs to ensure that the
freight left on the roads (which will be the
majority) is moved as efficiently as possible
while contributing to goals of the strategy.
This will require cleaner, better driven vehicles,
better journey planning and the integration of
freight and land use planning.
236 Interventions can also be temporal as well as
spatial. The need to satisfy and cater for peak
demand means there is spare capacity off-peak.
Greater use of transport throughout the week
can help cover the fixed costs of provision and
make the best use of investment. The seven-
day week, 365-days-a-year, diverse economy
is something the London Plan seeks to
encourage through policy support for tourism,
retail, arts and entertainment and emerging
Chapter five Transport proposals
economic sectors, as well as more flexible
working practices. This highlights the need for a
transport strategy that is integrated with other
broader policy areas such as land use planning,
education and healthcare provision.
237 The proposals for each mode or policy area
contained in the following chapter are derived
from how they can best support the strategic
transport policies set out in chapter four, with
regard to the nature of the mode and policy
objectives. They have also been developed
using an integrated approach, taking account
of wider strategies within London such as the
London Plan, EDS, Air Quality and Housing
strategies, as well as looking beyond the GLA
boundary to the Greater South East region.

Proposals to manage and
enhance the transport system
5.2 National Rail, Crossrail,
Thameslink, London
Overground, DLR and Tramlink
5.2.1 Introduction
238 London is more dependent on rail than any
other city in the UK: 70 per cent of all rail
travel (including Tube journeys) in the UK is to,
from or within the Capital. Londons success is
bound up with the future of its rail network and
services. It is vitally important, therefore, that
Network Rail and the train operating companies
better serve the citys needs, and that the
Mayor has greater input and influence over
planning and delivery of their services.
5.2.2 International and National Rail
links and services
International rail passenger links
239 The strategy fully supports expansion of
international rail services that improve Londons
connectivity with Europe and provide a viable
alternative to air travel.
111
240 Eurostar services currently run non-stop or
make only one intermediate stop between
St Pancras International and the Channel
Tunnel, with services calling at either Ebbsfleet
International or Ashford International. Beyond
2010, EU policies will permit competition for
Chapter five
international rail services, which may result
in an increase in international high-speed rail
services to/from London. This will provide
an opportunity to encourage European
destinations and to use Stratford International
station to reduce congestion at St Pancras
International, and provide better international
connections to the Isle of Dogs and east London.
Proposal 1
The Mayor, through TfL, and working with
the DfT, Network Rail, the operators of
international rail services and other transport
stakeholders, will encourage the provision
of direct international rail services to a wider
range of European destinations, with some
of those new services serving Stratford
International station.
Chapter five Transport proposals
112 Mayors Transport Strategy
International and national rail freight
241 International and national movement of freight
plays an important role in the success of
Londons economy. The Mayor will work with
others to seek to deliver enhanced rail freight
capacity through supporting new terminals
to facilitate efficient movement of goods;
and encourage transfer of freight from road
to rail wherever possible.
242 The new 1.5bn container port, known as
London Gateway, near Tilbury, will provide
substantial additional port capacity in the
South East when it opens, generating new
rail freight flows through London.
243 A rail connected freight transhipment facility
at Howbury Park, near Slade Green, is being
developed, and is expected to open in 2010,
enabling the transfer of road freight to rail.
A new rail freight hub is also proposed at
Brent Cross/Cricklewood.
244 High Speed One (HS1) is an under-used facility
for rail freight and has the unique benefits of
the larger European gauge clearance for high-
cube containers and the possibility of express
freight services carrying high value goods.
To enable this, terminal facilities are required,
with the London riverside area of the
Thames Gateway being the identified site.
Chapter five Transport proposals
Proposal 2
The Mayor, through TfL, and working
with the DfT, Network Rail, train operating
companies, freight operating companies,
London boroughs and other transport
stakeholders, will support the development
of more rail freight terminals in or near
London, including connections to HS1 for
international freight, in line with the London
Plan policy to identify new sites for strategic
rail freight interchanges.
245 A number of rail routes in London are used by
freight services passing through the Capital,
for example, on their way to the Midlands from
Channel ports and from Europe via the Channel
Tunnel. While the use of rail for these kinds of
freight movements is supported, this through
traffic should have a minimal impact on the
capacity of the passenger rail network and on
local communities through noise.
Proposal 3
The Mayor, through TfL and working with the
DfT, Network Rail, train operating companies,
freight operating companies, London
boroughs and other transport stakeholders,
will support the development of National Rail
routes that relieve London of freight without
an origin or destination in the Capital.

Domestic high-speed rail
246 The introduction of the first domestic high-
speed services in the UK on HS1 in 2009, has
significantly reduced journey times from Kent
and the Thames Gateway to central London.
Elsewhere, domestic rail service journey times
from London to the regions can be longer than
passenger expectations, particularly when
compared to European competitors who have
invested in high-speed rail networks.
247 Capacity is critical on a number of main lines
out of London, in particular the East Coast
Main Line to Yorkshire, the northeast and
Scotland, and the Great Western Main Line
to the southwest and south Wales. The West
Coast Main Line to the west Midlands, the
northwest and Scotland has recently benefited
from a 9bn upgrade allowing faster and
more frequent services. However, according
to Network Rail, by 2020, the main line from
London to Birmingham and the northwest will
be full, given projected growth.
248 Proposals for a second high-speed line to link
the centre of London with Birmingham, in
the first instance, as part of a possible wider
domestic high-speed rail network, are currently
being considered by the DfT. This is based on
a detailed set of proposals developed by High
Speed Two, the company set up by the DfT
to investigate options for a new-high speed
line from London to the West Midlands and
potentially beyond. According to the DfTs
High Speed Rail Command Paper, published
in March 2010, such a new line could deliver
well over 2 of benefits for every 1 spent on
113
building the line, in addition to journey times of
49 minutes from central Birmingham to central
London. Were the line to be extended north
from Birmingham, to Manchester and Leeds
(in a Y-shaped network), it could offer journey
times of around 75 minutes between both cities
and London, as well as releasing significant
capacity on the existing West Coast Main Line
(and other routes) for more commuter and
freight services. The current plans allow access
to Heathrow via a connection with Crossrail
in west London, providing the potential for
improved connectivity between Heathrow
airport and other parts of the UK by high-
speed rail. However, further thorough research
Chapter five
is required to determine the optimum location
for such an interchange.
Proposal 4
The Mayor and TfL support the development
of a national high-speed rail network and
will work with the DfT, Network Rail, High
Speed Two and other transport stakeholders
to ensure that the main London terminal for
any new high-speed line is centrally located,
well-connected to the existing public
transport network, and widely accessible
to maximise access to jobs and Londons
population. It is currently considered that
Euston best meets these criteria. Further
evaluation will be made of this and other
potential termini, in particular, in relation
to links to Heathrow.
Chapter five Transport proposals
114 Mayors Transport Strategy
5.2.3 London and the South East
rail links and services
Accommodating growth on the
National Rail network
249 Rail will continue to be the dominant mode in
accessing central London, with three quarters
of all trips from Outer London to central
London made by National Rail, Tube and DLR.
As the number of trips in London increases,
inevitably the demand on the National Rail
network will increase, by approximately
35 per cent by 2031.
250 BAA/Heathrow Airport Limited have recently
applied to the Secretary of State for Transport
as part of the Transport and Works Act 1992
process, to authorise construction of Airtrack,
a new rail link connecting the existing rail
line from Waterloo to Reading with Heathrow
Terminal 5. More information about Airtrack is
contained in the airports section of the MTS.
251 The DfT has also announced its intention to
electrify the Great Western Main Line from
London Paddington to Oxford, Bristol and
Swansea. The Mayor welcomes this project
and is committed to working with the DfT to
maximise the benefits to London, including
the potential to extend Crossrail westwards to
Reading. The Mayor also remains committed to
seeing other rail electrification proposals taken
forward, such as Gospel Oak to Barking (which
will also contribute to improving air quality).
Chapter five Transport proposals
Crossrail
252 Crossrail is the biggest transport project in
Europe and a scheme of national importance
that will provide a rail spine across London
from east to west, as shown in Figure 30.
It is needed to underpin the most rapid
economic growth areas of London and will be
a significant capacity addition to the transport
network. Figure 29 illustrates the immense
scale of the scheme with the example of the
new station at Tottenham Court Road. This
will transform access to this area of the West
End from the current crowded and congested
conditions to a spacious, high quality and
attractive travel environment, capable of
supporting local economic intensification.
253 When completed in 2017, Crossrail will enable
the City and Canary Wharf to continue to
maintain their leading business status in the
world, support the continued development
of the West End, help to lock-in the legacy
benefits of the 2012 Games, and transform the
Thames Gateway through links to northeast
and southeast London and on into Essex.
254 Crossrail will facilitate easier, faster and
for the first time, direct journeys by public
transport from Heathrow to Canary Wharf,
as well as better links to Londons other
central business areas, helping to improve
Londons international links. Crossrail is the only
transport project that can deliver the capacity
needed to support more jobs in central London
and the Isle of Dogs, improve the east-west
employment corridor and support development
of new employment areas and new jobs in areas

Figure 29: Layout of the expanded Tottenham Court Road station
such as the Thames Gateway. Without Crossrail,
London will be prevented from continuing the
development of its second business centre at
Canary Wharf, and maximising its contribution
to the wider UK economy.
255 Crossrail provides the largest single increase
in public transport capacity exactly where
it is most needed. It will add 10 per cent to
the overall capacity of Londons rail network
through the provision of 24 high-capacity,
10-coach trains an hour in each direction in the
115
Chapter five
central section during peak periods. According
to forecasting work undertaken by TfL, an
additional 5.8 million passenger kilometres
are added to peak capacity and Crossrail will
deliver significant crowding relief on the Tube
(including the Central, Piccadilly, Metropolitan,
Hammersmith & City, Circle and Jubilee lines)
and the DLR.
256 As a project of national significance, Crossrail
will bring transport improvements that will be
felt across the country. The scheme will be a
Chapter five Transport proposals
116 Mayors Transport Strategy
Figure 30: Crossrail route connections and regional maps
catalyst for safeguarding a national economy
inextricably linked with that of London (see
spotlight on the wider economic benefits
of Crossrail). The provision of high quality
information to keep people informed about
the construction programme, especially to
minimise inconvenience to users of other
transport services, local residents and
businesses, will be a high priority for Crossrail.
Overall the benefits of Crossrail are estimated
to be worth at least 36bn in current prices
to the national GDP over the next 60 years.
Chapter five Transport proposals
Proposal 5:
The Mayor, through TfL, and working with
the DfT, Network Rail, train operating
companies, freight operating companies,
boroughs and other transport stakeholders,
will seek to ensure that Crossrail is delivered
by 2017, and that it is fully integrated
with the rest of Londons public transport
system; that the impacts of construction on
residents and businesses are minimised as
far as possible; and that the future benefits
Crossrail brings are monitored to ensure the
rail link achieves its objectives.

Spotlight
The wider economic benefits of Crossrail
Crossrail will deliver significant benefits to the
national economy, the wider southeast region as
well as to London. In addition to the wide ranging
transport benefits, such as reduced crowding,
additional capacity, new direct links across the
Capital and reduced journey times, Crossrail also
delivers a range of direct economic benefits.
It supports a development potential of more
than 260,000 jobs and 70,000 people within
the key Opportunity Areas alone (Isle of Dogs,
City fringe, Lower Lea Valley, Paddington) and
is expected to generate up to 30,000 extra jobs
through productivity improvements in central
London by 2027.
The Crossrail project will employ up to 14,000
people at the peak of construction between 2013
and 2015, and will contribute to an ongoing pool
of skilled workers through the new tunnelling
academies being established by the Mayor. In the
longer-term, Crossrail will generate an estimated
1,000 jobs when fully operational. Crossrail will
also require the services of regionally-based
manufacturers and other suppliers.
117
Crossrail will serve significant new residential
areas, for example, parts of the Thames
Gateway and beyond in the South East and
East of England regions, where employment
accessibility will be significantly enhanced,
bringing an additional 1.5 million more people
across London within an hours travel of the
West End, City and Docklands. However, it
is not just central London and Docklands
economies which benefit from Crossrail,
Outer London gains considerable benefits
too, as illustrated in Figure 30. Crossrail also
links key Outer London metropolitan town
centres, such as Romford, Ilford and Ealing,
delivering further regeneration benefits.
Chapter five
It will be important that other public investment
is coordinated to maximise these benefits.
To this end, the LDA is developing a Crossrail
Regeneration Investment Plan to identify and
prioritise potential interventions that would bring
further regeneration benefits to the areas around
key Crossrail stations.
Chapter five Transport proposals
118 Mayors Transport Strategy
257 London is the centre of the Greater South East
region of England, the fastest developing area
of the country, with a number of nationally
designated Growth Areas. Continued economic
growth is mutually beneficial much of
Londons workforce live beyond the GLA area.
It is therefore important to look across regional
boundaries. Future extensions of Crossrail, to
the east and west, could help reduce congestion
and improve connectivity and, together with
longer trains, ensure maximum benefit is
derived from the Crossrail infrastructure.
Proposal 6
The Mayor, through TfL, and working with
the DfT, Network Rail, train operating
companies and other stakeholders, will
consider future extensions of Crossrail that
reduce congestion and improve connectivity
on London commuter routes.
5.2.4 Londonwide rail links
and services
High Level Output Specification investment
in rail capacity
258 In July 2007, the DfT presented the HLOS
detailing the development of the National
Rail network to 2014 (control period four).
The HLOS was accompanied by a Statement
of Funds Available (SoFA), which detailed
proposals for funding arrangements for the
railways in the same time period to deliver
the proposals put forward in the HLOS. The
SoFA has now been reviewed by the Office
of Rail Regulation, agreed with the DfT and
approval given to Network Rails Delivery Plan
Chapter five Transport proposals
to implement HLOS. As such, HLOS is now a
committed programme of National Rail funding
from the DfT to 2014. The process is planned
to be repeated for control period five, from
2014 to 2019.
259 The improvements to Londons rail network
to be achieved by 2014 through HLOS, forms
a key element of the MTS and it is essential
that Thameslink and the remainder of the
committed HLOS1 programme are completed
to their original planned specification.
The improvements are shown in Figure 31,
and include:
Improvements to capacity on each of
Londons main radial rail corridors, including
the Thameslink programme and 900 new
carriages by 2014 (Thameslink by 2016)
Improved reliability
Acceptance of Oyster at all National Rail
stations in London from January 2010
Improving access at stations through
the Access for All obstacle-free stations
programme. Approximately 50 per cent of
stations in the first phase are located in
London or the South East
Enhancements to the freight gauge
and route capacity on the Felixstowe
Nuneaton line (providing a direct route
avoiding London for freight trains, which
releases capacity for more passenger trains
in London)
Gauge enhancements to the Gospel Oak
Barking line, which will provide a route
for rail freight services from the east to the
north of London

Spotlight
Thameslink
The Thameslink Programme is a 5.5bn project
to deliver a high capacity, north-south rail spine
through central London that complements
the east-west Crossrail route. It is planned for
completion in 2016 and will provide greater
capacity, higher frequencies, new services and
improved access to central London from a range
of destinations within the Capital and across
southeast England.
Major benefits from the works to expand the
capacity and operating network that will use
Thameslink, include a capacity increase in the
core section between Farringdon and Blackfriars
An artist's impression of Blackfriars station courtesy of Network Rail
119
of 127 per cent by 2016. Large-scale works
include the rebuilding of Blackfriars station,
which will become the first station to span the
Thames, providing direct access to both the north
and south banks; Farringdon, will have 12-car
platforms and become a major interchange
between Thameslink and Crossrail; London
Bridge will be significantly rebuilt to provide a
major increase in capacity and better facilities for
passengers. A new station will also be provided
at Brent Cross/Cricklewood as part of the
comprehensive redevelopment of this area.
Chapter five
Chapter five Transport proposals
120 Mayors Transport Strategy

Figure 31: Committed enhancements to Londons rail network1

London Overground Chiltern Great Northern
New routes from: Dalston to New Frequency improvements Frequency improvements
Cross, West Croydon and Crystal Palace
Connection between East London line
and North London line at Dalston West Coast West Anglia
Surrey Quays to Clapham Junction
extension, completion of new Trains lengthened and frequency 12-car capability Stansted and
orbital route increased Cambridge

Crossrail core scheme Great Western Great Eastern

New route from Maidenhead and Electrication Additional services
Heathrow to Sheneld and Abbey New Crossrail services New Crossrail service
Wood with 24 trains per hour
through central London Thameslink Essex Thameside
12-car capability 12-car capability on all routes
Up to 24 trains per hour through
central London

Wood Green

Harrow Romford
Ilford
Uxbridge

Ealing
Shepherds
Bush

Hounslow

Kingston
Bromley

Croydon
Sutton

Key South Western South Central

Opportunity or Intensication Area 10-car Windsor lines and inner 10-car inner capability
suburban capability
Outer London
Inner London South Eastern
Central Activities Zone 12-car inner suburban trains

1
Committed in HL0S control period four

Chapter five Transport proposals

 121

Figure 32: National Rail and Tube stress on radial corridors into central London

2006 2017 (with funded

K L investment) K L
J M J M
I N I N

H Central O H Central O
London London
G A G A
F B F B
E D C K L E D C K L
J M J M
I N I N

H Central O H Central O
London London
G A G A
F B 2031 (with full F B
2031 (with no further strategy, including
E D C E D C
investment beyond 2017) unfunded schemes)

Chapter five
Moderately stressed On average residents in the corridor experience moderate levels of crowding on rail public
corridor transport on their way to central London during morning peak for some part or all of their journey

Highly stressed On average residents in the corridor experience high levels of crowding on rail public transport on
corridor their way to central London during morning peak for some part or all of their journey

Severely stressed On average residents in the corridor experience severe levels of crowding on rail public transport on
corridor their way to central London during morning peak for some part or all of their journey

Note: Corridor letters correspond to corridors in Figure 8

Further rail investment beyond the

Proposal 7 current programme
The Mayor, through TfL, will seek to
ensure that Network Rail and the train 260 Despite the significant investment in the
operating companies deliver the committed National Rail network planned in the TfL
improvements to the rail network and Business Plan and HLOS, congestion will
services in London as set out by the DfTs still be a significant issue on some radial rail
High Level Output Specification for the corridors (Figure 32). Additional schemes
period 2009 to 2014. and interventions, as set out by this strategy,
will therefore be required to address specific
capacity and connectivity problems in the
longer-term. As most of these will need to

Chapter five Transport proposals

122 Mayors Transport Strategy
Figure 33: Further rail capacity needed in London
West Coast capacity Thameslink capacity enhancement
enhancement (serving Luton airport)
Croxley link
High Speed Two
Crossrail extensions
Airtrack and other orbital
links to Heathrow
Chelsea Hackney
line (Crossrail 2)
Longer trains on
South Western lines
Key
Opportunity Area
Rail termini
Route improvements
London-wide improvements
Chapter five Transport proposals
Great Northern capacity
enhancement
Enhanced orbital
rail links
Northern line Upgrade
2 and extension to
Battersea
Longer trains on South
Central and Thameslink
(serving Gatwick airport)
London terminals capacity
upgrades and strategic
interchanges
West Anglia
four-tracking, serving
Stansted airport
Chelsea Hackney line
(Crossrail 2)
Great Eastern capacity
enhancements
DLR extensions and
capacity enhancements
Longer trains on Essex
Thameside lines
Crossrail extensions
Rail/Tube improved
capacity and connectivity
Tramlink to southeast London,
enhancements including potential
and extensions Bakerloo line extension
Upgrade of all National Rail
stations and services to London
Overground standards and
integration with Oyster

be delivered by Network Rail and the train
operating companies through the franchising
system, it is essential that TfL works closely
with the DfT to influence the future HLOS and
franchising process to deliver the improvements
that are vital to Londons growth. TfL will
continue to press Network Rail to develop their
route utilisation strategies as part of an on-going
process to develop proposals for enhancing the
National Rail network. In addition, TfL will work
closely with Network Rail to inform franchise
specifications, and inform the HLOS process.
261 Figure 33 illustrates a number of options
to enhance the capacity provision on radial
rail routes to central London to address the
levels of crowding and congestion after the
measures in the TfL Business Plan and HLOS
have been implemented. The effectiveness
of these schemes at reducing rail and Tube
crowding, when combined with other capacity
improvements on the Underground network,
are shown in Figure 34.
262 In order to support Opportunity Areas set out
in the London Plan, it may be appropriate to
improve access to rail services by providing
additional stations. The case for such new stations
will be identified as part of the sub-regional
planning and Opportunity Areas Planning
Framework processes.
123
Proposal 8
The Mayor, through TfL, and working with
the DfT, Network Rail, train operating
companies, London boroughs and other
transport stakeholders, will seek further
rail capacity across Londons rail network,
beyond those schemes already committed.
The highest priorities in the medium term
are to further increase capacity on London
Overground; on southwest routes; on
West Anglia routes, including access to
Stratford; on Great Northern services; and
at congested stations.
In the longer term, further capacity solutions
Chapter five
may be required on a number of rail
corridors, such as the Brighton Main Line.
Chelsea Hackney line
263 The Chelsea Hackney line (or Crossrail 2) provides
significant new rail capacity on the northeast to
southwest corridor and major congestion relief
to existing rail and Tube lines. The route of the
line is safeguarded by Government and it is
essential that this safeguarding remains in place to
protect this important new line. Forecast demand
shows that crowding and congestion remains a
significant issue in this corridor (Figure 32), even
with new investments such as Crossrail and
Thameslink in place. This new line is needed in
the longer term to reduce crowding on existing
routes, but also to provide the capacity that is
required to meet Londons growth and provide
connections to the National Rail network,
including dispersal of people across London from
the main line termini. The introduction of HS2
will increase this need considerably.
Chapter five Transport proposals
124 Mayors Transport Strategy

Figure 34:The potential benefits from implementing measures beyond those in the TfL Business Plan
and HLOS to reduce crowding on Londons rail and Tube network
Increased crowding
in 2031 as a result of
growth in employment
and population Proportion crowded in 2031 without any new investment would be 67%
70
Proportion of passenger kilometres
crowded (weekday morning peak)

60 Proportion crowded in 2006 was 55%

50
Proportion crowded in 2031 would be 50%
40

30
Proportion crowded in 2031 with the full strategy package would be 32%1
20 Total impact
of the
strategy
10 on rail
crowding2
0
Crossrail, Tube Further Chelsea Further DLR Other investment
1. It would not be good value line upgrades, investment on Hackney line* extensions, including Airtrack,
for money or of benefit to the investment on the the National Rail Bakerloo line Crossrail
economy to attempt to National Rail network, north and south extensions and
eliminate all crowding network including extensions to extensions and other service
2. Rail crowding includes Thameslink and the DLR, Croxley link* improvments*
crowding on the Tube, DLR, other funded Northern line
National Rail and Tramlink measures and Underground
upgrades

Funded measures Unfunded measures

* Note: These schemes improve accessibility (connectivity more so than crowding)

Figure 35: National Rail punctuality (London and southeast operators total, including peak services)

National Rail punctuality (London and southeast operators total, including peak services)
Percent of arrivals within five

95
minutes of scheduled time

90

85

80

75

70
2002/03 2003/04 2004/05 2005/06 2006/07 2007/08 2008/09

Chapter five Transport proposals


264 It is important that the route of the Chelsea
Hackney line is reviewed to ensure it is providing
the maximum benefits and value for money.
Proposal 9
The Mayor will support new rail capacity in
the broad southwest to northeast corridor,
for example, new lines or services using the
Chelsea Hackney line safeguarded alignment.
TfL will undertake a review of the route to
ensure it is providing the maximum benefits,
including helping the onward dispersal of
passengers from central London termini and
value for money.
National Rail reliability
265 National Rail punctuality has improved
considerably since 2002/03 as a result of
the improved maintenance of infrastructure
by Network Rail, investment in new trains
and signalling, franchise reorganisation
to better reflect rail network geography,
franchise performance incentives and improved
timetabling. However, despite these substantial
achievements, there is still room for further
improvement (see Figure 35).
Proposal 10
The Mayor, through TfL, will seek to ensure
that the DfT, Network Rail and the train
operating companies achieve the HLOS
public performance measure for reliability,
as well as an overall reduction in significant
lateness and cancellations for London and
southeast services.
125
Station capacity enhancements
266 Many National Rail stations are congested,
especially at peak times and enhancements
to station capacity are required to improve
customer service and to enable Londons
growth in rail demand to be accommodated.
Schemes will be taken forward where they are
shown to be value for money and affordable.
Proposal 11
The Mayor, through TfL, and working
with the DfT, Network Rail, train operating
companies and London boroughs, will seek
to deliver capacity enhancements at some
Chapter five
of Londons most congested stations. The
highest priorities include:
a) Central London termini station
congestion relief and onward distribution
enhancements (the potential of all onward
modes will be considered)
b) Clapham Junction station capacity
enhancement (new improved links
between platforms, additional entrances
and more ticketing facilities)
c) Improved capacity at National Rail stations
with severe congestion, including Finsbury
Park, Bromley South, Wimbledon, Vauxhall
and Barking
d) Improved capacity at National Rail stations
with moderate congestion, including
Willesden Junction, Balham, West Croydon,
Putney, Norwood Junction and Surbiton
Chapter five Transport proposals
126 Mayors Transport Strategy
Improving customer standards across the
National Rail network in London
267 Despite improvements in National Rail services
being delivered by Network Rail, TfL and
others over the next 10 years, rail passengers
still do not always benefit from the level of
service or frequencies that those who travel
on the Tube enjoy, particularly outside peak
hours. The Mayor is keen that London benefits
from a full seven-day railway, supporting the
Capitals diverse economy with regular, frequent
services operating seven-days-a-week, without
disruptions due to engineering works, except
where absolutely necessary. The Mayor also
believes that train operating companies
should provide sufficient capacity in terms
of train lengths, to sufficiently meet demand
at all times. This includes weekends when
acute crowding can occur due to shorter
train formations.
268 The Mayor will promote a common service
standard across the London rail network and
believes that Overground service standards
provide an evidenced template to follow. This
has been adopted almost in full for the recent
South Central refranchise with more than 40
stations and extensive patterns of routes.
The specified minimum standards include:
Staffing over the traffic day
Oyster acceptance and retailing via ticket
machines within Travelcard Zones 16
Station facilities such as seating
and shelters
Multi-modal customer information at
stations, through posters and electronic
Chapter five Transport proposals
means, on-system and off, scheduled
and real time
Security features such as lighting,
CCTV and Help points
Graffiti removal, litter removal
and cleaning
Train frequency of at least four trains
per hour on each route where the
infrastructure allows, up to 23:00
First and last trains broadly aligned with
the Undergrounds operating hours
Cycle parking to basic standard and
monitoring regime at every station
within two years
269 This will require the influencing of the franchise
process, but could be better achieved by the
Mayor having greater powers over suburban
passenger services. It will also require a
reduction in disruption to passenger journeys
at weekends, acknowledging that sometimes,
the undertaking of maintenance works at
these times will remain inevitable. Network Rail
is already looking at how some engineering
works could be undertaken through shorter
overnight possessions instead, possibly through
undertaking more preparatory work off-site.

Spotlight
Central London rail termini
Increasing population and employment, plus
the additional capacity on much of Londons
rail network through the current rail investment
programme, means rail arrivals into central
London termini are forecast to increase by
about 25 per cent by 2031. This will result in
increased strain on National Rail termini, and
on the transport networks and services within
central London.
Limited concourse space means that station
crowding is a key issue. High levels of
connectivity and very large volumes of
interchange passengers add extra complexities
to this crowding and movement. The main
termini and interchanges, which are already at,
or near, capacity during peak periods need to be
reviewed to ensure they are safe and efficient
into the future.
127
Congestion relief schemes such as those at
Kings Cross St. Pancras and the proposed
scheme at Victoria can go a long way to
relieving the problems. However, these
schemes are expensive and disruptive, take
time to deliver and ideally need to be linked
with increases in onward dispersal capacity.
Encouraging more onward trips by walking and
cycling is a quick and cost-effective way to free-
up capacity on the Underground and buses,
and ensure that limited space on these services
is used more effectively. Better streets, better
information provision and better facilities for
cyclists, for example, cycle parking will help.
The strategic interchange concept, improving
Chapter five
and encouraging interchange at stations away
from the central London termini, would also
help onward dispersal (see proposal 46).
Chapter five Transport proposals
128 Mayors Transport Strategy
Proposal 12
The Mayor, through TfL, and working with
Network Rail, the train operating companies
and other transport stakeholders, will
encourage the achievement of a seven-day
railway by better planning and management
of necessary engineering and maintenance
work on the railway.
Proposal 13
The Mayor, through TfL, and working with
Network Rail, the train operating companies
and other transport stakeholders, will
encourage the provision of rail services
in London that meet common service
standards, including improved ambience,
amenities and wayfinding at all stations,
and staff availability at each station. It is
intended these improvements will be rolled-
out as franchises are renewed. However, they
would be better achieved if the Mayor had
more control over suburban rail services in
the London area.
5.2.5 Local and orbital rail links
and services
London Overground
270 Work on London Overground includes
reinstatement of disused National Rail routes
to link the East London and North London lines
at Dalston, modifying existing infrastructure
to reach West Croydon and Crystal Palace (the
East London line extension) and delivering
a rolling stock maintenance and control
Chapter five Transport proposals
centre at New Cross Gate. The North London
Railway, also part of London Overground,
is to be increased in capacity by up to 50
per cent and will also provide more frequent
services. A further phase now approved will
see the completion of the Inner London
orbital rail network between Surrey Quays
and Clapham Junction. Completion of the
London Overground network enhancements
in 2012 will provide an orbital rail network in
Inner London that will allow, either directly or
with a single change of trains, journeys such
as Clapham Junction to Dalston and Crystal
Palace to West Hampstead. This network of
Overground rail services is shown in Figure 36.
The DLR extension to Stratford International
in summer 2010 will also improve orbital
frequency and capacity in east London.
271 There are, however, significant further
opportunities to improve orbital travel, such
as by providing or improving interchange
opportunities where radial and orbital lines
cross. Forecasting shows that rapid demand
growth can be expected over the next
few years, which means that parts of the
Overground could justify investment in longer
five or six-car trains. TfL will also investigate
where there may be additional routes that
warrant extensions to the Overground,
if found to be feasible.
 129

Figure 36: London Overground route map (2012)

South Acton

Chapter five
2012 London Overground Service
All train services will be four carriages long by 2012
Step-free access from the platform to the street
(existing and planned)
Watford Junction Euston (3tph)
Richmond Stratford (4tph)
Camden Road Stratford (2tph, peak hours only)
Stratford Clapham Junction (2tph)
Willesden Junction Clapham Junction (2tph)
Dalston Junction Clapham Junction (4tph)
Highbury & Islington Crystal Palace (4tph)
Highbury & Islington New Cross (4tph)
Dalston Junction West Croydon (4tph)
Gospel Oak Barking (4tph)
tph trains per hour

5.2.6 Docklands Light Railway

Proposal 14
The Mayor, through TfL, working with
the DfT and Network Rail, will deliver the
committed investment in the Overground
network, investigate the feasibility of
providing further capacity to assist orbital
movement, and will review potential benefits
of extensions to the network of services.
272 Since opening in 1987, the DLR network has
grown to become a vital part of Londons
public transport system supporting growth
and regeneration in the Thames Gateway (see
London Docklands case study). Improvements
to the DLR include a network extension to
Stratford International and delivery of 55 new

Chapter five Transport proposals

130 Mayors Transport Strategy
carriages to enable three-car operation on most
of the system by 2010 (with further station
and infrastructure enhancements to follow).
Delivering new rolling stock to maximise the
benefits of full three-car operation across the
network, double tracking between Bow Church
and Stratford, installing a new signalling system
and renewing and replacing the existing rolling
stock will all contribute to ensuring that the
DLR is fully able to support continued growth
and regeneration in the Thames Gateway area.
273 An extension of the DLR to Dagenham Dock
has been identified as a key component of the
current housing plans for Barking Riverside.
There is also further scope to extend the DLR
network in the longer term to provide better
links to Stratford, improved orbital links and
connectivity in the Thames Gateway, and better
integration with the central London transport
system, for example, through a westwards
extension from Bank to Victoria.
Proposal 15
The Mayor, through TfL, will support
safeguarding the route of the DLR
Dagenham Dock extension as part of the
housing proposals for Barking Riverside,
and will investigate the feasibility of further
capacity and network expansion of the DLR
including options south of Lewisham, west
of Bank and north of Stratford International.
Chapter five Transport proposals
5.2.7 Tramlink
274 Tramlink has been a great success since
opening in 2000, providing important
orbital links into Croydon and connections to
neighbouring Outer London town centres
and encouraging a shift from car to public
transport. Improvements to Tramlink as part of
a 54m investment in maintenance, renewals,
upgrades and capacity enhancement, are taking
place between now and 2015.
275 Beyond these initial improvements,
consideration will be given to looking at
further extensions of Tramlink, with a strong
focus on a potential north-south axis, in
order to accommodate Croydons future
growth needs, and potentially to improve
east-west links to neighbouring Outer London
town centres to support improved orbital
connectivity. Both short extensions to provide
better access from key centres to Tramlink, and
longer extensions opening up completely new
routes are being considered.
Proposal 16
The Mayor, through TfL, and working with
the London boroughs and other transport
stakeholders, will investigate the feasibility
of providing extra capacity on the Tramlink
network and will review potential benefits
of extensions to the system.

5.3 London Underground
5.3.1 Introduction
276 Over the last decade, the Underground has
experienced unprecedented growth in demand,
with more than a billion passenger journeys
a year now made by Tube. The Underground
carries as many passengers as the entire
National Rail network with up to four million
journeys made each day, on 11 lines serving
270 stations. The Underground now provides a
higher volume of service than ever before while
achieving a record 79 out of 100 in terms of
customer satisfaction.
277 The strategy for the Tube is based on
understanding customer needs by combining
a reliable train service with the highest
standards of customer care. This means
renewing the networks infrastructure
(rolling stock, signalling, track, civil
structures and stations) to allow train
service capacity to keep up with rising
levels of demand, and creating a welcoming
and secure environment, offering personal
service to customers, providing accurate
and timely service information, and creating
ease of access.
278 LU has embarked on its largest investment
programme for 70 years, focusing on improving
reliability, delivering faster journey times, along
with increasing capacity across the network.
131
5.3.2 Renewal and repair of
the network
279 The Tube is the oldest metro system in the
world, with some infrastructure dating back to
the 1860s. After decades of under-investment,
the network is now undergoing a major
transformation programme comprising asset
renewal, rebuilding and refurbishment. The key
elements of the programme are:
Rolling stock replacing the majority of
train fleets, which if not renewed would
have an average age of 50 years in 2031,
15 years beyond their design life. The
new trains will allow quicker, more reliable
Chapter five
journeys and have higher capacity.
Where possible, innovative technologies
and design solutions will be used, for
example, regenerative braking and
walk-through carriages
Signalling existing trackside assets,
which are more than 40-years-old in many
cases, will be replaced with computer
based systems which will reduce delays
and increase network capacity by allowing
for a higher frequency of service. New
service control centres will enable better
information provision to customers and
allow staff to manage the train service
more effectively, delivering improved
journey time reliability and minimising
service disruptions
Track reduce the backlog of investment
to reduce safety risks, and remove speed
restrictions brought in to manage safety
risks to increase capacity
Chapter five Transport proposals
132 Mayors Transport Strategy
Civil structures renew assets such as
bridges, viaducts, embankments and
drainage systems to maintain a safe service,
reduce the risks of flooding, and the service
effects of speed and weight restrictions
Stations modernise stations by
replacing safety and service critical
systems such as fire, public address,
CCTV, lifts and escalators
280 Much of this work is brought together in a
programme of line upgrades which will provide
some of the capacity required to support the
Capitals economy and meet the demands of
the future. Without the line upgrades it would
not be possible to maintain the service that is
currently delivered. Work to upgrade each line is
therefore the cornerstone of the strategy for the
Tube. By the end of the current programme, the
Tube network will provide up to an additional
30 per cent capacity. New trains and signalling
systems will allow more trains to run, providing
quicker and more comfortable journeys. Beyond
this, the Tube will require investment to ensure
that its asset condition remains in a state of
good repair and does not fall to the levels seen
through earlier decades of under-investment.
281 The network faces the tremendous challenge
of keeping London moving on a daily basis
while simultaneously rebuilding the system.
This massive task cannot be achieved without
some disruption to services. In order to facilitate
these works and carry out regular essential
maintenance, some weekend closures are
necessary. The high profile campaign advising
customers to check before they travel has
proven highly successful and will continue,
Chapter five Transport proposals
employing many TfL Travel Tools to help
customers plan their journeys.
282 By 2012, this programme will deliver an upgrade
of the Jubilee, Victoria and Northern lines. By
2020, upgrades will have been completed on
the Piccadilly line, Sub-Surface Railway and
the Bakerloo line, with a further upgrade on
the Northern line. These are described in the
Spotlights on pages 133, 137 and 138. The
replacement of the Central line fleet will follow,
delivered into service from 2020. Considerable
benefits and efficiencies can be achieved
by specifying lighter, more energy efficient
and higher capacity trains for this and the
Bakerloo line.
Proposal 17
The Mayor, through TfL, will seek to deliver
upgrades to all Tube lines in a phased
programme to provide a significant increase
in network capacity. This will involve a
combination of new rolling stock and/
or signalling systems and other asset
replacement. As part of this, continued
investment to bring the network to a good
state of repair and maintain it at that level
will be supported.
5.3.3 Station refurbishments
and accessibility
283 Between 2003 and 2009, 124 stations
were refurbished and this programme will
continue subject to availability of funding.
This programme is delivering key system
improvements (CCTV, public address,
communications equipment and fire systems),

Spotlight
Transforming the Tube
Over the last decade, the number of journeys
made on the Tube has risen to record levels
of more than one billion journeys a year.
To support this growth and correct historic
under-funding, LU has embarked on its largest
investment programme for 70 years.
The line upgrades, which will include new
signalling and control systems, as well as the
introduction of new trains on some lines, will
focus on improving reliability. They will deliver
faster journey times and increase capacity by
up to 30 per cent across the network. By 2012,
this programme will deliver an upgrade of
the Jubilee, Victoria and Northern lines, each
providing between 20 and 33 per cent more
capacity into central London.
1
See Spotlight on Northern line Upgrades 1 and 2 on page 138
133
By 2020, upgrades will have been completed
on the District, Circle, Hammersmith & City and
Metropolitan lines (including air-conditioned
trains) increasing capacity across the sub-surface
network, while the Piccadilly line upgrade will
provide a 24 per cent increase in line capacity.
A second upgrade of the Northern line will ensure
further service enhancements1. The Bakerloo line
will also be upgraded.
Other enhancements include: major station
improvements at Victoria, Paddington
(Hammersmith & City), Tottenham Court
Road, Bond Street and Bank; cooling the Tube;
power upgrades to cater for expanded services;
and accessibility improvements.
Chapter five
Chapter five Transport proposals
134 Mayors Transport Strategy
as well as improving customer service features.
These include, passenger Help points, new
electronic information displays in ticket halls and
on platforms, improved seating and lighting, as
well as improvements to accessibility features
such as tactile strips and colour-contrasted
handrails for visually impaired people.
284 TfL is committed to improving accessibility
from street level to platform level on the Tube
network. Over time, TfL will seek to increase the
accessibility of the network, building upon the
foundation of step-free stations already in place
(see section 5.9).
Proposal 18
The Mayor, through TfL, will continue to
deliver an ongoing programme of Tube
station refurbishments and asset stabilisation
to ensure stations are operable and deliver
customer service requirements, and continue
to improve station accessibility over the life
of the strategy.
5.3.4 Station congestion relief
285 To relieve congestion experienced by Tube
customers at key locations across the network,
and to enable quicker journeys, some stations
require more extensive improvements to ensure
safe and efficient station operations while also
enhancing passengers journey experience. The
delivery of capacity enhancements to strategic
Underground stations and interchanges is critical
to the functioning of the Tube as an integrated
network to assist existing passenger flow, and
cater for future increases in demand.
Chapter five Transport proposals
286 Congestion relief is required at the key central
London interchanges of Victoria, Tottenham
Court Road, Bond Street, Paddington
(Hammersmith & City) and Bank. Capacity
increases at these stations will optimise the
benefits of investment from the line upgrades,
Crossrail and other developments. They will also
greatly improve central London step-free access.
287 Further station improvements at, for example,
Vauxhall, Finsbury Park and Highbury &
Islington, will improve strategic access to, or
interchange across, the network (see section
5.10.2, Strategic interchanges).
Proposal 19
The Mayor, through TfL, and working with
the London boroughs, private developers
and other transport stakeholders, will
develop and implement a prioritised
programme to deliver station capacity and
accessibility enhancements at Londons most
congested Underground stations, including:
a) Congestion relief schemes to complement
Tube line upgrades and/or integrate
with Crossrail at the key central London
interchanges of Victoria, Tottenham
Court Road, Bond Street, Paddington
(Hammersmith & City) and Bank
b) Schemes at further strategic Tube
interchanges that are critical to
Londons transport system (for example,
Vauxhall, Finsbury Park, Highbury
& Islington, Holborn, Camden Town,
Oxford Circus, Edgware Road and
Northern line City branch, in particular
Old Street and Moorgate)

c) Major strategic multi-modal/National
Rail interchanges on to the Underground
network to disperse onward demand arising
from National Rail proposals (HLOS2 and
HS2 proposals), for example, London Bridge,
Euston, Liverpool Street, Paddington,
Elephant & Castle and Waterloo
5.3.5 Cooling the Tube
288 High tunnel temperatures during the summer
months are one of the biggest challenges
facing the Tube, particularly for the deep
tunnelled sections of the Underground,
such as the Victoria line. These are generally
closed systems where the major proportion
of the energy that enters (for example, train
motors) is released as heat, which in turn raises
temperatures in the tunnels and on the trains.
As a result of increased train service capacity
(primarily through higher train frequencies)
and reduced journey times (mainly through
quicker acceleration and faster maximum
speeds), electricity use on the Underground
is anticipated to increase by 2020, resulting
in more heat being released in the tunnels.
This will be exacerbated by increasing
passenger numbers and possible increases in air
temperature due to climate change. Therefore
maintaining safe temperatures on deep
tunnelled sections of the Tube will present an
ever increasing challenge in future. Already the
Tube has implemented a programme to tackle
heat on the network, and this will continue over
the duration of the strategy.
135
Proposal 20
The Mayor, through TfL, will implement the
following measures to cool the Underground:
a) New air-conditioned rolling stock across
the sub-surface (Metropolitan, Circle,
Hammersmith & City and District) lines,
introduced progressively from 2010
b) Improved ventilation shafts and
replacement of out of service fans
5.3.6 Customer care
289 The core renewal programme is essential to
deliver a safe and reliable service, but TfL
Chapter five
aspires to the highest standards of customer
care. Customers most value getting from A
to B as quickly and reliably as possible, but
valued almost as much are the different aspects
of service experienced during the journey.
Customers value personal security, a welcoming
environment, accurate and timely information,
and when they need assistance, they want high
quality personal service from staff.
290 In 2008, for the third year running, the
Underground carried more customers than ever
before, in excess of one billion. At the same
time, customer satisfaction with the service also
reached a record high of 79 out of 100. Most
people scored the overall service provided at
nearly eight out of 10.
291 Customer information and personal service
from staff will be especially important as the
renewal programme, which can disrupt service,
is delivered over the next decade. TfL will
continue to invest in customer-focused training
Chapter five Transport proposals
136 Mayors Transport Strategy
for Tube staff including disability awareness
training, and in information systems to deliver
the right information to customers when they
want it, when planning or undertaking journeys
on the network.
292 Customers value the presence of staff especially
when travelling at night. The Underground
will continue to staff its stations and invest in
systems which enhance the management of
the station, such as CCTV, as well as ensuring
stations are well-lit and visibly managed, all
clean and graffiti free.
293 There is a rich heritage of art and design on
the Tube. High quality, value for money design
will continue to be a feature of the Investment
Programme. Art on the Underground, LUs art
programme, continues the Tubes long tradition
of working with artists. Licensed busking and
Poems on the Underground bring music
and poetry to the wide-ranging audience of
customers on the network. These initiatives
can delight customers and improve the
travelling environment.
Proposal 21
The Mayor, through TfL, will continue to
develop and implement measures to deliver
the highest standards of customer care on
the Underground, including the provision of
high quality information about engineering
works that affect regular Tube services, and
improved information on the accessibility of
the Tube network highlighting step-free and
mostly step-free routes.
Chapter five Transport proposals
5.3.7 Further improvements and
extensions to the network
294 Beyond the funded investment programme,
even with the introduction of Crossrail and
Thameslink, crowding will remain on the Tube
network. This crowding is shown in Figure 20
and contributes to the corridor stress shown
in Figure 32.
295 There are opportunities for improvements
to other aspects of the transport system,
particularly the (national) rail network to
relieve crowding on the Tube. Potential new
lines, such as Chelsea Hackney, would also
be designed in part to reduce Tube crowding
where possible. These potential enhancements
are described in section 5.2.
296 Over the lifetime of this strategy, there are also
opportunities for further enhancements and
extensions to the Tube network to improve
journey times and provide additional capacity
essential for the continued growth of London.
Any potential schemes will be subject to a
thorough value for money and feasibility
analysis, and will be considered in the light of
any future funding constraints. They will also
be integrated with enhancements to other
elements of the transport system, in particular
the rail network.
297 Further enhancements (beyond the funded
upgrades) to the Northern line are possible.
With private sector funding, there is the
potential to extend the line to Battersea to
support developer-led growth in the Vauxhall/
Nine Elms/Battersea Opportunity Area, an area

Spotlight
Sub-Surface Railway upgrade
The Sub-Surface Railway lines comprise the
Circle, District, Hammersmith & City and
Metropolitan lines, covering more than 300km
of track and representing around 40 per cent of
the Tube network. It is the part of the network
in greatest need of renewal. The upgrade
will be delivered progressively over the next
decade and will provide the greatest capacity
improvements across the entire network,
helping to reduce congestion and accommodate
predicted growth.
In July 2009, LU issued an invitation to tender
for the resignalling of the sub-surface lines.
This contract is the single Tube element of the
Investment Programme.
From December 2009, a new service pattern
was introduced on the Circle line, almost
137
doubling the frequency of trains on the
Hammersmith branch. It also improved the
reliability of the Circle line, with knock-on
benefits for reliability across the whole of the
Sub-Surface Railway. The new service pattern
will be followed by the introduction of a
new signalling system to allow higher train
service frequencies.
From 2010, 199 new, larger trains will enter
service. They will feature energy-saving
regenerative breaking, walk-through carriages,
better accessibility, improved customer
information (audio and visual) and enhanced
security. The Circle and Hammersmith & City line
trains will also have an additional carriage. Most
Chapter five
significantly, the new trains will be the first to
be air-conditioned on the Tube network.
Chapter five Transport proposals
138 Mayors Transport Strategy

Spotlight

Northern line Upgrades 1 and 2

The Northern line will be upgraded in two
phases, known as Upgrades 1 and 2. Work has
commenced on the first upgrade which will
introduce new computerised signalling and a High Barnet

new control centre to reduce journey times in Totteridge & Whetstone

T

Woodside Park
32
the morning peak by 18 per cent and increase
Edgware

Burnt Oak
Mill Hill East
West Finchley 24
22
Finchley Central

capacity by around 20 per cent.

Colindale
East Finchley
Hendon Central
Highgate
Brent Cross

Golders Green Archway

20
20
24
However, even with the delivery of Upgrade 1,
28

Hampstead Tufnell Park

K
Kentish T wn
To
24
forecast growth in demand will continue to place 28
28 24 20
Belsize Park

severe pressure on Londons north-south routes,

Chalk Farm

Camden To
T wn

particularly through the City, producing crowding Mornington

Crescent

levels similar to that experienced today. Hence Euston

Angel
32
24
Old Street

there is a need for a further upgrade.

Warren Street Kings Cross
St. Pancras

28 Goodge
22
24
Street Moorgate

Upgrade 2 will recast and partially separate Tottenham

T

20
20 Court Road Bank

the services on each of the two central London Leicester Square

24
branches of the line. This will reduce journey 20 Charing
Cross 28
times, deliver an additional 33 per cent capacity 24 Embankment London Bridge

on the City branch, a 17 per cent increase on 28 Waterloo

the Charing Cross branch and consequently Borough

reduce crowding. The upgrade will comprise River Thames

Elephant & Castle

additional rolling stock alongside the existing Oval

K
Kennington

fleet, additional train stabling facilities, power, Clapham High Street

Clapham North
Stockwell

improved signalling and tunnel cooling. Clapham Com

Common
28
32
32

Clapham Sou
outh

Balham
24 32

Together, the two upgrades will transform the

Tooting Bec
T

Tooting Broadway
T

Northern line. On the City branch, during the

Colliers Wood
28

South Wimbledon

busiest parts of the morning peak, service Morden

frequencies could rise from the current 20

trains per hour (tph) to 24tph following the
first upgrade, eventually reaching a potential
32tph in the northbound direction following
24
partial separation.
24

32

Chapter five Transport proposals


identified as capable of accommodating
20,000 to 25,000 new jobs and 16,000
homes by 2031.
298 The Bakerloo line has an important role in
Londons transport geography, serving the
strategic northwest-southeast corridor with
its important regeneration zones including
Harlesden, Paddington, Elephant & Castle and
inner southeast London. A Bakerloo southern
extension would allow the line to serve inner
and outer southeast London. This would create
a new southeast to northwest strategic route
through the Capital, serving areas with poor
transport accessibility and freeing up National
Rail capacity at London Bridge for other service
improvements.
299 The Croxley rail link, providing a new
connection between Croxley station on the
Watford branch of the Metropolitan line and
the now closed Croxley Green branch line
is a long standing proposal championed by
Hertfordshire County Council. Although outside
the GLA boundary, the link is supported
by the Mayor as it improves the regional
connectivity of northwest London by linking
the Tube network to the important National
Rail interchange at Watford Junction and the
employment, retail, leisure and healthcare
opportunities in Watford town centre.
139
Proposal 22
The Mayor, through TfL, and working with
the DfT, Network Rail, train operating
companies, London boroughs and other
stakeholders, will seek longer-term
enhancements and extensions to the
Underground network, including:
a) A privately funded extension of the
Northern line to Battersea to support
regeneration of the Vauxhall/Nine Elms/
Battersea area
b) A potential southern extension to the
Bakerloo line will be reviewed further. This
would utilise spare line capacity, improve
Chapter five
connectivity and journey times, while
providing relief to congested National
Rail approaches to central London from
the south/southeast, subject to resources
and the results of further study
c) A link at Croxley to join the Watford
branch of the Metropolitan line to
Watford Junction (funding to be secured
by Hertfordshire County Council in
conjunction with the DfT)
Chapter five Transport proposals
140 Mayors Transport Strategy
5.4 Londons bus network
5.4.1 Introduction
300 The bus has become one of Londons transport
success stories over the last decade. The
Capitals buses now carry 2.2 billion passengers
each year the highest level since 1962, with
service levels also at their highest since 1957.
Figure 37 shows the trend since 1971.
301 Key achievements include:
All buses are fitted with the iBus real time
audio and visual Next Stop signs
A highly accessible network, where more
than 90 per cent of London residents
are within 400 metres (approximately a
five minute walk) of a bus stop and all
buses (except heritage Routemasters)
are wheelchair accessible
Reliability is at the highest level ever
recorded. This has been achieved through
additional resources, incentivised contracts,
improved bus priority, enforcement and
central London Congestion Charging
Measures to make the system easier to
use, including straightforward service
patterns, better information, improved bus
stops and stations, simplified ticketing and
improvements to driver training based on
compulsory disability awareness training
Vehicle enhancements such as CCTV
coverage of the whole fleet, successful trials
of both diesel-electric hybrid and hydrogen
fuel cell engine technology
Chapter five Transport proposals
302 As a result, customer satisfaction levels
have increased substantially in recent years
(see Figure 38).
303 Buses play a key role in providing access to jobs
and services and are the most widely-used form
of public transport across London. Buses are the
predominant mode for public transport within
the suburbs and Inner London. Outer London,
in particular, relies on the bus network to provide
access to, and between town centres. Buses
also facilitate longer radial trips into London by
feeding into railway stations and by enabling
passengers to get to their final destinations in
central London. An effective bus network also
helps in reducing traffic volumes and overall
C02 emissions.
304 These roles will remain essential as London
develops. The bus network will need to
respond to changing demands. Improvements
in the quality of the experience offered to
passengers will need to be consolidated and
(where possible) enhanced to meet increasing
expectations.
5.4.2 Bus network development
305 Londons bus network is subject to a continuous
development process, enabling it to respond to
changing travel needs. This is passenger-led,
based on research and consultation.
306 Compared to other major world cities,
Londons bus network performs either equally
well, or better than average, on several key
performance indicators, including accessibility.
The cost efficiency of the bus network in
 141

London is 20 per cent ahead of most major to change, including new homes, workplaces,
international cities. London currently has a shopping centres and leisure attractions.
comprehensive orbital bus network, enabling
direct orbital journeys between neighbouring 308 Where major change or development is taking
centres in Outer London, shown in Figure 39. place in London, TfL will continue to work
closely with the boroughs and developers to
307 Continued development of the network will be ensure the needs and demands on the bus
necessary so that it can carry on responding network are fully understood, that plans for

Figure 37: London bus use since 1971

2,500
journey stages (millions)
Number of passenger

2,000

Figure 35
1,500

Chapter five
1,000

500 to follow from Real 451

0
1985/86

1987/88

1989/90

1991/92

1993/94

1995/96

1997/98

1999/00

2001/02

2003/04

2005/06

2007/08
1971

1973

1975

1977

1979

1981

1983

Figure 38: Improving bus journey experience

85
mean attribute score/100

80
Customer satisfaction

75

70

65

60
1998/09 1999/00 2000/01 2001/02 2002/03 2003/04 2004/05 2005/06 2006/07 2007/08

Bus service overall evaluation Information (bus stop and bus interior) Level of crowding

Personal safety and security average (at stop and on bus) Reliability (journey time x time waited)

Chapter five Transport proposals

142 Mayors Transport Strategy

Figure 39: Selected orbital and other bus services in Outer London

Wood Green
Brent Cross
Harrow Romford
Ilford
Uxbridge Stratford

Ealing
Shepherds
Bush Woolwich

Hounslow

Kingston
Bromley

Croydon
Sutton

Key
Selected orbital bus services Metropolitan town centre Major centre
Other bus services Potential metropolitan town centre District centre

any changes to the network are identified, and
opportunities for funding are fully explored.
309 It is essential that the bus network continues to
be developed in such a way to cater for the overall
shape and scale of growth across London. TfL
will undertake reviews of the strategic priorities
underlying the process approximately every five
years to ensure the bus network reflects the
pace of development in London, responds to
the challenges and opportunities of growth, and
aligns with possible revisions of the London Plan.
310 The development of the bus network will need
to support other transport investment, such
as Crossrail and other railway improvements.
In addition, there will be ongoing alterations
to the nature and distribution of services and
facilities provided by others, such as healthcare
and education, which the bus network will need
to reflect.
311 Keeping the bus network as accessible as it
is today will also be essential as the fleet will
continue to be the only city-wide accessible public

Chapter five Transport proposals


transport mode despite improved accessibility
of the rail networks. Similarly, buses can operate
24-hours-a-day, supporting Londons 24-hour
economy, whereas rail services often cannot due
to necessary maintenance constraints.
312 Network development will also consider the
improvement and efficiencies that can be
achieved at busy interchanges and major
transport hubs, both existing and emerging.
143
Chapter five
Proposal 23
The Mayor, through TfL, and working
with the London boroughs and other
stakeholders, will keep the development
of the bus network under regular review,
including reviews of the strategic priorities
underlying the process approximately
every five years, to ensure it caters for
growth in population and employment,
while maintaining ease of use, attractive
frequencies and adequate capacity,
reliable services, good coverage and good
interchange with other modes. All proposals
for change will be appraised to ensure that
they deliver good value for money and that
the funds available are being invested in
optimum service improvements.
Chapter five Transport proposals
144 Mayors Transport Strategy
Spotlight
A New Bus for London
The competition to design a New Bus
for London was launched in July 2008. This
attracted a large number of entrants, and in
December 2008, the Mayor announced
the winners.
The aim of the competition was to harvest
a range of creative ideas for a bus fit for the
21st century. These winning designs and
concepts have been passed on to bus
manufacturers, for development into final
design proposals.
Chapter five Transport proposals
The New Bus for London will meet London
Buses requirements for vehicles in public
service in London, including high standards of
accessibility, safety and emissions abatement.
In addition, it will be more durable, more fuel
efficient and better ventilated. The bus will
incorporate a double-deck and a platform at
the rear near-side corner, so passengers will be
able to board and alight easily.
The first prototype of the new bus will be on
Londons streets by 2011.

5.4.3 Bus service quality
313 As Londons economy grows, pressures on road
space will mean a continued requirement to
ensure that appropriate measures are taken to
maintain an attractive and reliable service for
bus users.
314 Bus priority measures such as bus-only roads,
bus lanes and selective vehicle detection at
traffic signals are essential tools in ensuring
that the limited people-carrying capacity
of the road network is being used most
effectively. These measures help reduce bus
journey times, improve bus reliability and
increase the efficiency of the bus network,
especially when they are considered as part
of a whole route approach. Bus priority
measures are systematically identified,
appraised and delivered at key locations,
including town centres and their approaches,
at new development sites, and links where bus
passengers represent a significant proportion
of all road users.
315 Quality incentive contracts have been very
successful in incentivising operators to
maintain a high level of reliability, and the iBus
system improves operators ability to control
services effectively.
145
316 Bus information has been transformed over
the last 10 years. Improved spider maps and
timetables are in place throughout the network.
iBus has provided Next Stop announcements
(audio and video) on all buses, and has
improved information on Countdown signs at
bus stops. Further development will include
delivery of real time information through
mobile phones and the internet, as well as
an expanded number of signs at stops.
Proposal 24
The Mayor, through TfL, and working
with the London boroughs and other
stakeholders, including developers, will
Chapter five
improve bus passengers journeys by
measures, including:
a) Incentivising bus operating contracts
and expanding staff training in order to
consolidate reliability improvements
b) Introducing measures such as bus priority
at critical locations
c) Ensuring that the appropriate enforcement
of bus priority is carried out
d) Implementing the Countdown 2 project
to deliver expanded access to real
time information and develop further
integration with digital communications
to provide real time bus information
Chapter five Transport proposals
146 Mayors Transport Strategy
5.4.4 Bus fleet development
317 The bus fleet has been progressively developed
with incremental enhancements to passenger
comfort and security, including upgraded
seat designs, CCTV throughout each bus, air-
cooling systems and outward-opening doors
to provide more space in the busiest part of
the vehicle. Emissions standards are ahead
of legal requirements. Diesel-electric hybrid
buses are in service on a trial basis, and all
new buses entering service from 2012 will
use hybrid technology.
318 TfL is also seeking to develop an iconic bus
design with its New Bus for London project.
Proposal 25
The Mayor, through TfL, will upgrade its bus
fleet to meet increased emissions standards
and will appoint bus manufacturers as part
of the New Bus for London project. It is
intended that the first prototype will enter
service during 2011.
Chapter five Transport proposals
5.5 Taxis, private hire, coaches
and community transport
5.5.1 Taxis
319 Londons 22,000 licensed taxis and 25,000
licensed taxi drivers provide about 200,000
trips a day to Londons visitors, residents
and businesses, with the majority of activity
concentrated in central London. In a 2007
survey for the London Chamber of Commerce
and Industry, 93 per cent of business leaders
regarded a good taxi service as being important
or very important to the London economy.
320 Although licensed taxi drivers can use most of
Londons bus lanes, they face much the same
challenges as other road users in terms of the
impacts of congestion. Measures outlined
elsewhere in the strategy to smooth traffic
flow will therefore be of significant benefit to
taxi drivers and their passengers. The essence
of an effective taxi service is the point-to-
point service offered, however, it is important
that measures to smooth traffic flow also take
account of the need for kerbside activity at
the start and end of journeys.
321 Londons taxi service is widely recognised as
the best in the world. The world-renowned
Knowledge of London that must be
demonstrated before a taxi driver is licensed to
ply for hire, means that these drivers have an
unparalleled understanding of Londons streets
and points of interest, as well as pride in their
profession. The Conditions of Fitness licence
requirement means that only vehicles which are

suitable for taxi work wheelchair accessible
and highly manoeuvrable can be used as
taxis in London.
322 Taxis are also recognised as a safe and quick
way of making door-to-door journeys, and are
particularly valuable for disabled people and
at times when other public transport is scarce.
TfL is working in partnership with the taxi
associations to reduce taxi vehicle emissions
and introduce a new low emission taxi.
This will enable them to play a role in
improving air quality in London and in
tackling climate change.
Proposal 26
The Mayor, through TfL, and working with
the London boroughs and other stakeholders
will support improvements to the taxi service
through a number of measures, including:
a) Continued highway priority for taxi
services, for example, access to bus lanes
b) Reduced taxi vehicle emissions and
development of low emission taxis
c) Provision of parking and waiting facilities,
including rest facilities
d) Provision of ranks and facilities
at interchanges
e) Taxi marshalling
f) Action against touting and illegal cabs
g) Improved driving behaviour, to be
encouraged through the licensing
procedure of taxi drivers
1
Except taxis
147
h) Ensuring regulated taxi fares changes
allow drivers and owners to continue to
recover the costs of providing the taxi
service and provide a sufficient incentive
for taxi provision to meet demand,
in particular at night
i) Continuous process improvements to
provide a modern and cost effective
licensing service
5.5.2 Private hire
323 There are about 50,000 PHVs in London,
operated by about 2,400 businesses, and
providing employment for about 54,000 people.
Chapter five
PHVs offer a range of valuable services across
the Capital, making a similar number of trips
in total to taxis. The PHV trade is very diverse,
covering all vehicles1 for up to eight passengers
offered for hire with the driver. As well as the
familiar minicabs and people-carriers, this
includes chauffeur and executive cars, some
patient transport and school support services
and a wide range of specialised operators.
Private hire also encompasses a number of
niche limousine and other bespoke services for
which little or no alternatives exist. PHV services
are spread more evenly across London than
taxi activity. Like taxis, PHVs are particularly
important at night when other public transport
is limited. For those travelling with mobility
impairments, heavy luggage or seeking to reach
a remote location, PHVs are a good quality,
value for money solution.
Chapter five Transport proposals
148 Mayors Transport Strategy

324 All PHV trips must be booked through a

licensed operator before the start of the
journey. This means that the customer has an
opportunity to ensure they get an appropriate
service at a fair price. The licensing standards
therefore allow a wider range of vehicles than
can be accepted as taxis, and drivers take a
less demanding topographical knowledge test.
Like taxis, all vehicles must be safe and suitable
for passenger carrying, and drivers must be
healthy, of good character, with an enhanced
criminal records check.
325 Licensing of private hire in London is relatively
new, with operators licensed in 2001, drivers
from 2003 and vehicles from 2004. PHVs
cannot drive in bus lanes, although like taxis
they are exempt from the Congestion Charge
and are allowed to stop to pick up or set down
passengers on red routes. The successful PHV
licensing process has seen an improvement in
standards across the industry, and along with
the Safer Travel at Night initiative, licensing has
reduced the levels of taxi touting and illegal
cab activity. As the licensed industry matures,
there may be opportunities to review the
contribution such services play and reappraise
the restrictions in force.
Proposal 27
The Mayor, through TfL, and working with the
London boroughs and other stakeholders, will
support improvements to private hire services
(especially minicabs) through the following:
a) Initiatives that deliver further the success
of the Safer Travel at Night scheme
b) Provision of facilities to pick up as well as
drop off passengers where appropriate
c) Action against plying for hire, touting,
un-roadworthy vehicles and illegal cabs
d) Continuous process improvements to
provide a modern and cost effective
licensing service
e) Lower emissions from PHVs
5.5.3 Coaches
326 Coaches play an important part in Londons
overall transport system, operating broadly five
types of service: UK short distance services,
including commuter coaches; UK long-
distance services; European services; private
hire or charter services; and airport services.
These services provide a range of benefits
to London, including supporting the central
London economy through commuter services,
and charter services to theatres and other
attractions. The majority of UK scheduled short
and long distance services, including European
scheduled coaches, serve Victoria Coach
Station, which is directly managed by TfL. The
organisation will work with coach operators to
maximise the use of the existing facilities to
increase capacity, given the anticipated growth
in demand for coach travel.
327 The provision of a conveniently located coach
terminal in London is important for both
operators and passengers. In the longer term,
the Mayor will work with all relevant partners to
investigate the feasibility of developing a series
of coach hubs or the potential for alternative

Chapter five Transport proposals


locations for coach station facilities to provide
easier access to the coach network, while
retaining good access to central London for
coach operators.
328 Another challenge for coach operators,
especially those who provide private hire or
charter services, is the location and availability
of pick-up and set down areas and, more
importantly, parking where drivers can take
149
their legally required break from driving.
Ensuring that adequate facilities are available
is a key function of the London Coach Forum,
which is coordinated by TfL, and provides a
valuable engagement mechanism with the
industry. TfL, working with the boroughs, will
continue to develop facilities for coaches,
balancing the needs of coaches with the needs
of other road users. To support this, the Mayor
will set out specific coach parking standards
Chapter five
Chapter five Transport proposals
150 Mayors Transport Strategy
to ensure parking at major visitor destinations
such as hotels, stadia and exhibition venues
is at appropriate levels to suit their individual
demand, and help reduce congestion and
improve visitor safety.
329 TfL also issues London Service Permits for
local services within London that are not part
of the London bus network. These include
some local bus services, sight-seeing tours,
and some commuter services. Such services
can also provide useful links in Outer London
with locations beyond the GLA boundary.
Proposal 28
The Mayor, through TfL, and working with
the London boroughs, coach operators and
other stakeholders, will seek to maximise
the use of the existing facilities to increase
capacity for coaches, given the anticipated
growth in demand for their use and to
develop parking standards for coaches. In
the longer term, the Mayor will work with all
relevant partners to investigate the feasibility
of developing a series of coach hubs or the
potential for alternative locations for coach
station facilities to provide easier access to
the coach network, while retaining good
access to central London for coach operators.
Chapter five Transport proposals
5.5.4 Community transport
330 Community transport refers to a broad range
of projects that provide an accessible transport
service, often aimed at particular sectors, such
as group travel and social car schemes.
331 The Mayor recognises the role that the
community transport sector plays in London
and will continue to engage with it through
the Community Transport Association. TfL will
continue to look at ways in which the transport
facilities provided by the community transport
sector can be closer coordinated with transport
facilities provided by the organisation, for
example, in terms of information provision.
Proposal 29
The Mayor, through TfL, and working
with the London boroughs and other
stakeholders, will encourage and support the
community transport sectors contribution to
the development and provision of transport
services in London.

5.6 Managing the road network
5.6.1 Introduction
332 Londons road network serves a variety of
purposes. It is, most obviously, the means by
which people travel from A to B by foot, cycle,
motorcycle, taxi, car, bus and by which the
vast majority of freight is moved, accounting
for over 80 per cent of all trips in London.
The Mayors focus on smoothing traffic flow
applies to all of these user groups. But the road
network also constitutes a very large proportion
of Londons public realm, where people can
relax, socialise and enjoy the atmosphere of
this world city (this role is discussed in detail
in the 'better streets' section). There are
conflicts between and within these two roles,
and the Mayor, through TfL, and the boroughs
will continue to seek to resolve these where
possible, taking into account the specific
function and circumstances of the part of the
road network involved. However, the overriding
objective is to maximise the efficient use of this
scarce resource, and this section outlines the
principal ways in which the Mayor proposes to
achieve this.
333 Responsibility for managing Londons road
network is shared between the Highways
Agency, TfL and the London boroughs:
The Highways Agency manages the M25,
M1, M11 and M4 motorways
TfL is responsible for the Transport for
London Road Network (TLRN), the busiest
and most economically important radial and
orbital arterial routes crossing the Capital,
accounting for around five per cent of total
151
road length, but carrying more than 30 per
cent of Londons traffic (see Figure 40).
The London boroughs are highways and
traffic authorities for the remaining strategic
and local roads in their individual borough
334 TfL manages traffic signals and traffic
control systems on all roads throughout
London and, under the Traffic Management
Act, has a Network Management Duty to
coordinate traffic management and other
interventions on the highway, and to facilitate
the overall movement of people and goods
across the Capital.
Chapter five
5.6.2 Road congestion
335 Road congestion manifesting itself in delay,
poor reliability and low network resilience
is a major issue for Londons transport system.
Congestion costs an estimated 2bn in lost
economic productivity, adversely affects
Londoners quality of life, causes frustration
to road users, contributes to a deterioration of
air quality and leads to higher CO2 emissions.
336 Levels of delay, reliability and resilience are
determined principally by the relationship
between the supply and demand for road
space. The supply (also known as the effective
road capacity) is determined by the amount
of physical road space available, junction
capacity, speed limits, the condition of highway
infrastructure including traffic signals, the
volume and duration of road maintenance
works, utilities works and the incidence of
vehicle collisions and breakdowns. The demand
placed upon the network capacity comprises
Chapter five Transport proposals
152 Mayors Transport Strategy

Figure 40: Transport for London Road Network

21

J 23
25
K L M
27
18
I

28

H 29
16 N

G
13
O 30

A
F 2

12
3

E 4

10 B
9
D
8
C
7 6

Key
Motorway Major orbital routes
8 Motorway junction A Major radial corridor*
Transport for London Road Network

* Note: Radial corridors are those identified in Figure 8

Chapter five Transport proposals


Figure 41:Causes of unusual congestion by proportion of duration
Note: Based on data gathered by the London Traffic Control Centre in 2008/09
moving vehicles, pedestrians, cyclists, parking
and loading.
337 Generally speaking, on a largely saturated road
network as in London, for a given capacity of
road, as traffic volumes rise, speeds will reduce
and congestion increases, leading to longer
journey times, less reliable journeys and lower
levels of customer satisfaction. In 2008, only
25 per cent of Londoners were satisfied with
the levels of traffic congestion.
338 As population and economic activity increases,
so will pressure on the road network, potentially
leading to significantly more delay, less reliable
journey times and reduced resilience of the
network to planned or unplanned interventions.
These congestion effects, in turn, are likely to
reduce the productivity and competitiveness of
the Capital as a whole, and particularly those
areas where local economies depend on reliable
road transport, for example, Outer London
town centres.
153
Collisions 31%
Vehicle breakdowns 8%
Local authority planned works 15%
Local authority unplanned works 2%
Utility planned works 11%
Utility unplanned works 8%
Special events 4%
Unplanned events 1%
Other (spillages, etc) 20%
339 Delay and journey time reliability vary according
Chapter five
to route, direction and time of travel, seasonal
factors, roadworks, traffic lights, planned and
unplanned events, and traffic volume. Figure 41
shows the causes of unusual severe congestion,
which affect journey time reliability, by duration
of delay, as recorded by the London Streets
Traffic Control Centre. The chart shows that
almost 40 per cent of unusual congestion is
caused by collisions and vehicle breakdowns,
and over a third is caused by planned and
unplanned road works.
340 Poor reliability and predictability of journey
times means those who use the road network
have to allow significantly longer for their
journeys to ensure they reach their destination
on time. Improving the reliability of journey
times on the road network (even if average
delays increase due to rising traffic volumes
or other factors) is of significant benefit to
motorists, freight operators and other users of
the road network. It enables them to predict
better how long a journey may take, allowing
Chapter five Transport proposals
154 Mayors Transport Strategy
for efficient and economic planning of social
and commercial activities and to reduce the
total amount of time they would otherwise
need to allow to make their journey.
5.6.3 Smoothing traffic flow
341 Smoothing traffic flow is the term used for
the Mayors broad approach to managing road
congestion and, in particular, improving traffic
journey time reliability and predictability. The
aim of the smoothing traffic flow approach
to managing the road network is to improve
conditions for cyclists and pedestrians as well
as vehicular traffic.
342 Smoothing traffic flow has six components:
Maximising the efficient and reliable
operation of the existing road network
Minimising the impact of planned
interventions on the road network that have
the potential to disrupt traffic flows
Minimising the disruption caused by
unplanned events (collisions, emergencies,
etc) as they occur and returning the
network to its planned steady state
operation as soon as possible
Maintaining road network assets in a
good state of repair in the interests of
safety and efficiency
Where a net benefit under proposal 35 can
be shown, developing the road network
Achieving targeted modal shift from
car journeys to more sustainable modes
(supported by the improvements in public
transport, walking and cycling conditions
Chapter five Transport proposals
and smarter travel measures and the
continued operation of the central London
Congestion Charging scheme in the original
area, provided for in chapter five)
343 As outlined in section 5.27, the Mayor
proposes to remove the Western Extension of
the Congestion Charging zone. The removal
of the zone is part of the Mayors broader
transport strategy. This includes the proposals
to better manage and smooth traffic flow,
as set out below, to mitigate the potential
effects of removal on congestion and emissions
on the road network formerly covered by the
charging zone.
5.6.4 Maximising the efficient and
reliable operation of the road network
344 The poor reliability of journey times means
those who need or wish to use the network
have to plan for the worst case scenario when
predicting how long their journey may take.
This makes for the inefficient and uneconomic
use of time and energy in respect of ordinary
day-to-day activities and can lead to journeys
being predicted to last for longer than the
actual average journey time. Increasing the
reliability of journey times on the road network
(even if average journey times are increased by
increased traffic volume) can result in motorists
having a better prediction of how long a
journey may take that is closer to the actual
average journey time.
345 The Mayor and TfL, working closely with the
boroughs, will therefore manage, as far as is
reasonably practical, the overall road network

and prioritise measures that improve the
reliability of journey times.
Proposal 30
The Mayor, through TfL, and working
with the London boroughs and other
stakeholders, will introduce measures to
smooth traffic flow to manage congestion
(delay, reliability and network resilience) for
all people and freight movements on the
road network, and maximise the efficiency of
the network. These measures will include:
a) Further investment in intelligent traffic
control systems (such as the urban
traffic control system, SCOOT) and the
infrastructure to support them
b) Allowing motorcycles and scooters to use
TLRN bus lanes subject to a trial period
and evaluating its impact
c) Upgrading, rationalising or removing
traffic management equipment and
optimising timings at signal controlled
junctions to keep traffic moving
d) Working with the DfT to pilot and
develop the concept of pedestrian
countdown at traffic signals to optimise
the amount of green time for both
pedestrians and road traffic
e) Planning and implementing a targeted
programme of improvements to the
existing road network, including junction
upgrades to improve traffic flow on the
most congested sections of the network,
and to improve conditions for all road users
155
346 A key component in the future management of
the road network is to increase the knowledge
of how the network operates and to employ
the most effective solutions and technological
developments to ensure its efficient operation.
To do this, TfL will continue to develop its
state-of-the-art dynamic traffic control system
ensuring that the system grows in capability
as the next generation of technological
advances allow it to do so cost-effectively.
This will bring together real time operational
data, historic analysis and predictive modelling
to more effectively respond to planned and
unplanned disruption, and to proactively
manage the available road capacity in real
Chapter five
time; for example, through increased on-street
control and by providing real time information
to drivers in a way that supports their journey
decisions. This capability will be deployed more
widely across the Capital, to better manage
the road traffic effects of expected economic
and population growth and to support the
development of Outer London.
347 In the period of the strategy, consideration
will be given to the increased use of real time
communications from vehicle to vehicle, and
between vehicles and on-street infrastructure
and a central traffic management control
system. The development of intelligent
transportation systems (ITS) technology will
allow the upgrading of Londons traffic signal
network. The aim is to create a state-of-the-
art traffic signal control system for the 21st
century capable of maximising the efficient
use of road capacity in London.
Chapter five Transport proposals
156 Mayors Transport Strategy
Proposal 31
The Mayor, through TfL, and working
with the London boroughs and other
stakeholders, will utilise advances in
ITS technology to better manage the
road network, improve real time traffic
management capability, lay the foundations
for communication with in-vehicle systems
and develop state-of-the-art traffic signal
control systems.
5.6.5 Minimising the impact of
planned interventions
348 Every year there are over half a million
roadworks on Londons road network that cause
traffic disruption. Many of these are unplanned
emergencies. A substantial proportion, however,
are planned (for example, scheduled utility
works, highway maintenance activities or special
events). Customer surveys show a low level of
satisfaction with how the works are managed
and the time taken for their completion.
Minimising disruption caused by these activities
is a particular priority for the Mayor.
349 Under its Network Management Duty, TfL
will facilitate cooperation between TfL, the
boroughs and utility companies to minimise
disruption to the existing road network for
all road users in London. The Mayors overall
approach is twofold:
To improve cooperation and communication
between highways authorities and works
promoters to improve the coordination
of roadworks
Chapter five Transport proposals
To develop incentives for works promoters
to reduce (or eliminate) the time they spend
digging up roads and disrupting traffic
350 As a first step towards more effective
coordination, TfL has developed an information
system known as LondonWorks that shows
the location of all works on its roads and the
surrounding network. This will enable action
to coordinate and effect the management
by contractors of these roadworks thereby
reducing their adverse impacts on traffic
flows and journey times. In addition, the
Mayors Code of Conduct for Roadworks
has brought together TfL and the main
companies responsible for utility works on
Londons major roads, to improve cooperation
and coordination. The Code of Conduct will
be extended to other utility companies and
boroughs with their agreement.
351 A new roadworks permit scheme has been
introduced that will ensure roadworks are
planned in an efficient and integrated manner
where possible, and are carried out quickly. The
permit sets out when works can take place,
the length of time allocated, days and times
on which road space will be made available to
contractors, as well as specifying penalties for
not keeping within the agreed restrictions.
352 The Mayor, through TfL, will work towards
implementing a lane rental scheme for
works promoters wanting to dig up the most
congested roads in the Capital. The rental
charge would reflect the cost to the economy
of taking temporary possession of road
capacity. It would aim to incentivise works
promoters to reduce the number and duration
of roadworks, and quicken the development of

techniques to minimise disruption. The scheme
would identify key junctions, times of the day
and network links, where roadworks can cause
significant traffic congestion. The lane rental
scheme, together with penalties for delay,
would help to ensure that any organisation
wanting to dig up the city's roads would
make every effort to cause as little disruption
as possible.
Proposal 32
The Mayor, through TfL, and working with
the London boroughs and utility companies,
will seek to minimise the adverse impact of
planned interventions on the road network
on the movement of people and goods, by:
a) Strengthening the Mayors Code of
Conduct for Roadworks to further improve
coordination between different highway
authorities and utilities across London
b) Utilising LondonWorks to improve
roadworks planning, coordination and
information availability
c) Encouraging collaboration between utility
companies and the use of innovative road
engineering techniques such as minimum
dig technology and temporary plating over
roadworks
d) Implementing the concept of lane rental
charges for utilities to reflect the value
of their temporary possession of road
capacity (in terms of cost of delay to the
road user) and to incentivise reductions
in the duration of roadworks
157
5.6.6 Minimising disruption from
unplanned events
353 There will always be unplanned events and
situations affecting the operation of the
road network which cannot be planned for in
advance, for example, emergencies, vehicle
collisions, breakdowns and burst water mains.
354 The adverse impacts of these events can be
minimised by highway authorities and other
agencies involved in the management of the
road network, by:
Developing physical capabilities and services
to respond to events
Chapter five
Reducing and mitigating the impacts through
effective real time traffic management
Taking appropriate remedial actions on
the ground
Providing effective real time information
Developing pre-arranged plans to deal with
events which can cause high risks to the
safety of the public when they occur
355 How these unplanned interventions that
adversely affect the normal operation of the
road network are managed has a direct impact
on the resulting levels of traffic disruption.
Highway and traffic authorities, the police and
utilities therefore have an important role in
identifying the potential causes of unplanned
events, minimising response and clear-up times,
and effectively managing traffic around such
incidents to minimise disruption.
Chapter five Transport proposals
158 Mayors Transport Strategy
356 Improving the accessibility and availability of
customer information about when and how to
travel, and how to avoid as far as practicable
the impact of incidents and interventions on the
network, can improve Londoners ability to move
around the city reliably. This can include better
information about incidents before journeys
begin (for example, through media available in
homes, offices and shopping centres), once they
are underway (for example, through improved
radio announcements), and in the immediate
vicinity of the event (for example, through the
use of variable message signs).
Proposal 33
The Mayor, through TfL, and working with
the London boroughs and other stakeholders,
will improve the real time management of
unplanned interventions and incidents on the
road network, and improve communications to
minimise the disruption and raise levels of public
satisfaction with road network management.
5.6.7 Maintaining road network assets
for safety and efficiency
357 Londons roads, pavements, bridges, tunnels
and traffic control systems represent billions
of pounds worth of public assets. Maintaining
them in a state of good repair is vital for the
safe and efficient operation of the network and
to achieve a good quality of life and economic
productivity.
358 Customer satisfaction with the physical condition
of roads and pavements in London is generally
higher than that for the management of traffic
Chapter five Transport proposals
and roadworks, and has remained stable at 50
per cent in recent years. However, roads are
vital public spaces which all of us use every
day. The condition of roads and pavements is
therefore fundamental to the quality of the
urban environment.
Proposal 34
The Mayor, through TfL, and working
with the London boroughs and other
stakeholders, will work in collaboration to
maintain cost-effectively Londons road
network assets in a good state of repair in
order to maximise their operational safety
and effectiveness, and to promote road user
satisfaction. This will include:
a) Conducting programmes for the
maintenance of roads, pavements,
bridges, tunnels and traffic systems so
that the TLRN and borough road network
is serviceable
b) Ensuring highway structures are
inspected regularly
c) Developing a Tunnels Safety Enhancement
Programme taking account of, among
other matters, fire risks, lighting,
communications and surveillance
5.6.8 Developing the road network
359 Due to limited space, the approach taken in
London is generally to get more from the
existing road network rather than conducting
a comprehensive road building or widening
programme. However, the strategy recognises
the potential need for local road capacity

enhancements in certain circumstances, such
as river crossings (see section 5.8), where there
can be demonstrated an overall net benefit
against the criteria set out proposal 35, below.
This approach is consistent with London Plan
policy 6.12.
Proposal 35
The Mayor, through TfL, and working with
the London boroughs and other stakeholders
will give consideration to new road schemes
where there is an overall net benefit when
judged against the following criteria:
a) The contribution to Londons sustainable
development/regeneration including
improved connectivity
b) The extent to which congestion (average
vehicle delay, unreliable journey times and
poor levels of network resilience) is reduced
c) How net benefit to Londons environment
can be provided
d) How conditions for pedestrians, cyclists,
public transport users, freight transport
and local residents can be improved
e) How safety for all is improved
All proposals will demonstrate how any
disbenefits will be mitigated.
5.6.9 Outcomes
360 The measures to smooth traffic flow described
previously will directly tackle the causes of
growth-induced congestion on Londons road
network. They are supported by measures
elsewhere in the strategy to improve and
159
promote mode shift towards public transport,
walking and cycling.
361 However, the precise effectiveness of this
complete package of measures over the 20 year
period of the strategy is uncertain. The level and
distribution of any such growth-induced delay
and deterioration of journey time reliability by
2031 will be dependent upon future investment
and travel patterns, neither of which are
precisely known. In addition, during this period
there is expected to be continuing technical
and social change that will affect the pattern of
demand for road space and the way in which the
road network can be managed.
Chapter five
362 Furthermore, peoples behavioural responses
and increased use of sustainable modes in the
future may differ from their responses today.
For example, as a result of reduced rate and/or
free public transport fares, todays children and
teenagers in London make greater use of public
transport than their predecessors and could
continue this higher level of use into adulthood,
inspired further by the behavioural legacy of the
2012 Games. As a result rates of car ownership
and use may be lower than todays levels.
363 Because of these uncertainties, the effectiveness of
the proposals in the strategy may be greater than
can be quantified using conventional techniques.
364 Figure 42 illustrates the potential effectiveness
of the transport strategy on reducing average
vehicle delays on the road network, showing
both the estimated level (in dark blue)
and possible greater levels of effectiveness
that might be achieved. Although a better
Chapter five Transport proposals
160 Mayors Transport Strategy

understanding of journey time reliability is
being gained rapidly through TfLs traffic
monitoring processes, still too little is known
to usefully forecast levels of future reliability.
Improvements to journey time reliability, while
expected, are therefore not shown. If the
measures in the strategy achieve their maximum
effectiveness then further interventions would
not be required, however, with lower levels of
effectiveness there would be a need for further
interventions in the longer term.

Figure 42:Mitigation of increased road congestion (average vehicle delay)

The policy measures shown in this figure will also improve journey time reliability which is a priority for Londoners and is the strategic outcome
measure for smoothing traffic flow. The relative effectiveness of these interventions will also vary spatially, for example between the CAZ, Inner and
Outer London

Chapter five Transport proposals


Spotlight
Managing roadworks
It is estimated there are around 500,000
holes dug in Londons roads each year. Until
recently there was limited ability to control the
roadworks taking place in London. This imposed
unacceptable costs in terms of congestion and
damage to the road surface.
The London Permit Scheme began in January
2010, with the immediate support of TfL and 18
London boroughs, and expressions of interest
from other boroughs have followed. The scheme
aims to ensure that roadworks are undertaken
in the least disruptive manner, are completed as
quickly as possible, and are coordinated so all
works required at a single location take place at
the same time, wherever practical. In its first three
months of operation, TfL approved around 12,000
applications to undertake works on the TLRN;
1,800 were refused. This alone will have reduced
the overall number of roadworks occurring and
161
level of disruption being experienced on Londons
busiest main roads.
The Mayor wishes to introduce a targeted lane
rental scheme. This scheme would apply charges to
those undertaking roadworks at the busiest time
of the day and on the most congested parts of the
network. These charges could be avoided by works
promoters if, for example, they undertook works at
non traffic sensitive times or employed innovative
working practices to allow the carriageway to
return to traffic use at peak times.
The objective of the scheme would be to focus
on the most congested roads in London and
the most important pinch points on the TLRN
Chapter five
and aim to reduce the volume and duration of
roadworks. The scheme would incentivise works
promoters by using charges that reflect the value
of their temporary possession of road capacity
(in terms of cost of delay to the road user).
Chapter five Transport proposals
162 Mayors Transport Strategy
5.7 The Blue Ribbon Network
5.7.1 Introduction
365 The Blue Ribbon Network encompasses the
Thames, the canals, tributary rivers, streams,
docks, reservoirs and lakes within London (as
shown in Figure 43), a considerable proportion
of which is navigable by passenger and freight
vessels. Building on the policies in the London
Plan, the strategy aims to maximise the
networks potential for passenger and freight
services, thereby relieving other congested
and crowded modes.
5.7.2 Making better use of the Thames
for passenger services
366 The Thames has been a strategic asset for
London throughout its history, providing a vital
link for people and goods. Currently, Londons
river traffic comprises a variety of freight and
passenger services for commuters and tourists.
At present, services operate from 22 piers
between Putney and Woolwich, nine of which
are under TfL management.
367 Riverbus services provide a comfortable,
accessible and occasionally faster alternative
to other modes for those within the catchment
zone of piers. In 2008/09 nearly 900,000
journeys were made by peak-time commuters
on the river. Demand has increased by over
600,000 journeys since 2003/04, driven by
riverside property development and commercial
developments in the Docklands, as well as
improvements to services part-funded by
TfL, boroughs and developers. In comparison
Chapter five Transport proposals
with land-based public transport, scheduled
commuter services may not be as cost-
effective to provide, and can produce higher
carbon emissions per passenger. However, if
loadings increase, the per capita economic
and environmental costs will fall. As further
residential, commercial and leisure facilities
develop in the Thames Gateway as well as
west London, river services may become more
popular, making better use of new and existing
capacity. Including leisure trips, a total of five
million passenger journeys are estimated to
have been made on the Thames in London
in 2009/10.
368 In order to maximise the potential of the
river in the build-up to the 2012 Games and
beyond, the Mayor has led the development of
a River Concordat between the Port of London
Authority (PLA), British Waterways, ODA, boat
operators, pier owners, riparian boroughs,
TfL, the LDA and other organisations with an
interest in improving passenger services on
the Thames. The River Concordat Action Plan
identifies six core workstreams:
Providing an integrated and enhanced
service for the 2012 Olympic Games
Increasing pier provision
Improving service quality
Integrating ticketing with land transport
Improving pier signage
Improving passenger information
 163

Figure 43: The London Blue Ribbon Network

4
8

3
5 10
9
1

7
6
11

12 17
16

14
15

13

Chapter five
Key 1. Grand Union Canal 8. Salmons Brook 15. River Wandle
Larger tributary rivers 2. River Brent 9. River Roding 16. Ravensbourne River
3. Silk Stream 10. River Rom 17. River Cray
Canals and river navigation 4. Pymmes Brook 11. Ingrebourne River
5. Moselle Brook 12. River Crane
Large lakes and docks 6. Regents Canal 13. Hogsmill River Note: Not all tributaries
7. River Lee Navigation 14. Beverley Brook are shown
River Thames

369 The following supporting areas are also being services. Work is ongoing to further integrate
given consideration: ticketing and passenger information with
Boat yard provision other modes of transport. Lack of customer
awareness of services on the river has been
Skills cited as one of the key barriers to greater use.
More reliable management information Better signage at public transport interchanges
and inclusion within the Legible London
Reducing the environmental impact of
wayfinding system may encourage more people
services
to use river services. Improved branding will
help customers choose the most appropriate
370 Early outcomes of the concordat include a
service for their needs.
new direct service between Canary Wharf and
London Bridge and the extension of Oyster
371 Feasibility work undertaken by TfL has
pay as you go ticketing on Thames Clippers
identified scope for additional passenger

Chapter five Transport proposals

164 Mayors Transport Strategy
services in east London, to complement
public transport networks. Potential exists
to connect new developments on either side
of the river with each other, the Docklands
and central London. These would improve
cross-river connectivity, capacity and boost
the local economy. In the short-term, this
could include new services between North
Greenwich (The O2) and East India pier, as
well as enhancements to the Woolwich Ferry.
There may also be demand for a new vehicle
ferry service serving Gallions Reach. Further
consideration to river crossings in east London
is given in section 5.8.
372 Options for increasing passenger use of the river
will be explored over the period of this strategy
through the sub-regional planning process,
development planning, as well as through the
on-going work of the concordat. Proposals
for new services from developers, operators,
local and central Government and other agencies,
will be welcomed by the Mayor. Sustained
commitment will be required from these partners
to ensure services remain viable over the course
of the MTS.
Proposal 36
The Mayor, through TfL, and working with
the Port of London Authority, ODA, boat
operators, pier owners, riparian boroughs
and other interested parties, will continue
by means of the River Concordat to work
to enable the development of Londons river
services to reach their full potential and to
better integrate river services into the land-
based public transport network.
Chapter five Transport proposals
373 Additionally, there may be scope for a new
passenger cruise terminal on the Thames
where there is capacity to accommodate large
vessels, which is supported by London Plan
policy 7.26. A new terminal could also support
Londons tourism and aid local regeneration
schemes, although the location would have
to be considered in light of other proposals,
such as new river crossings.
5.7.3 Pier capacity and supporting
infrastructure
374 A Pier Plan, commissioned by the LDA, has
reviewed the current status of piers from
Putney to the Thames Gateway region and
made recommendations for where additional
pier capacity should be located.
375 The most pressing need for more pier capacity
is in central London where demand is highest,
and competition for space between leisure
and commuter services is the greatest. TfL
will extend Tower Millennium Pier and the
concordats pier capacity workstream is
developing low-cost solutions to enable the
expansion of other central London piers.
376 Outside central London, TfL will undertake
feasibility work to identify if there is a need to
build new piers at North Greenwich and Canary
Wharf to relieve existing facilities. Further
opportunities for new piers will be explored by
the pier capacity workstream in line with the
Pier Plan and in conjunction with developers,
pier owners and other stakeholders; funding
will be sought from a range of sources.

377 The concordats pier capacity workstream is
also developing pier amenity standards to help
ensure more consistent levels of service across
the network.
378 New river traffic management facilities may
also help optimise pier use and reduce conflicts
between different passenger services. However,
this may entail alterations to the current
regulatory framework and would require the full
participation of partners in the River Concordat
and Government.
379 The Mayor also recognises the need for
sufficient supporting infrastructure for
waterborne transport both passenger and
freight. In particular the need for boat yards
to inspect, service and repair vessels. A report
conducted by the GLA in 2007 identified
the lack of suitable facilities in London and
recommended that at least one additional
facility be developed to cater for the larger
vessels operating on the Thames.
Proposal 37
The Mayor, through TfL, and working
with the London boroughs and other
stakeholders, will encourage the provision
of more pier capacity, particularly in central
London and will seek financial support for
new piers when considering development
proposals in the vicinity of the Thames. The
Mayor, through TfL, will also work with the
Port of London Authority, boroughs and
operators to identify and promote suitable
boat yard facilities in London.
165
5.7.4 Making better use of other rivers
and canals for passenger services
380 Current infrastructure investment in the
waterways surrounding the Olympic Park
site, together with increased marketing and
promotion is intended to leave a legacy of
increased recreational use. Opportunities to
provide leisure cruises around the park following
the 2012 Games are currently being explored.
381 Concordat partners including British Waterways,
the ODA and boroughs can help identify
opportunities to increase use of other waterways
in London such as the River Lee Navigation and
Grand Union Canal. Where passenger services
Chapter five
on these waterways are viable they will likely be
for leisure use, as journey times are generally
slow. New or expanded service provision will
be dependent on potential demand and local
environmental impacts.
382 There are also opportunities to improve
pedestrian and cycle access to the Blue Ribbon
Network and improve the quality of towpaths
for a range of leisure and non-leisure trips.
5.7.5 Making better use of rivers and
canals for waterborne freight
383 The Mayor recognises that transporting freight
(including waste) by water is a less damaging
option environmentally and can help ease the
impacts of congestion on the road network.
The shift from road freight to waterborne
freight and increasing goods transported (by
tonnage) from the current 1.84m tonnes
per year is an important element in reducing
Chapter five Transport proposals
166 Mayors Transport Strategy

Case study

Improving passenger services

on the Thames
The Thames is a great highway through leisure opportunities in central London, the Isle
central London but has been under-utilised of Dogs and North Greenwich. Since 2003/04,
for regular passenger travel for a number of river commuting has tripled from under
decades. The Mayor has instigated, through 300,000 trips to nearly 900,000 in 2008/09.
By the river1, a concerted effort by TfL, the
Through participation in the River Concordat,
boroughs, operators and encourages third
Thames Clippers accepted Oyster pay as
party investment to change this by: increasing
you go on its services in November 2009,
the number of piers and improving them,
allowing further integration with other public
operating new services at higher frequencies,
transport modes in London. The operator has
introducing new, faster boats and providing
recently launched new direct services to the
better information. The two main operators,
Docklands from London Bridge pier and service
Thames Clippers and TEC, now run a number
patterns are designed around the needs of rail
of regular services, from Woolwich in the east
passengers interchanging from the mainline
to Putney in the west.
station. Therefore fast ferry services now provide
As a result, more and more people are a feasible alternative to the Jubilee line for
discovering the river as a viable, attractive Docklands workers, operating high frequency
alternative to often congested Tube services, departures every 10 minutes in the peak.
buses and roads for travelling to work and for
1
By the river, launched by the Mayor in 2009, introduces the River Concordat and other riparian initiatives

Chapter five Transport proposals


vehicle emissions and improving the quality of
life in London. The Port of London within the
Greater London boundary remains a significant
port facility, handling a range of goods.
384 Water transport is particularly suited to bulk
movements of relatively low value cargoes for
which speed is less critical, aggregates and
waste/recyclates are prime examples. Within
London there are also other cargos such as
sugar, vehicle parts, metals, timber, foodstuffs,
fuel, oil and other bulk liquids. Water
transport is also well-suited to construction
and demolition activities connected with
building development. In the Olympic Park at
Stratford, waterways have been upgraded so
construction material can be transported by
water rather than road. The new Three Mills
Lock in Bromley by Bow can accommodate
barges weighing up to 350 tonnes (equivalent
to 17 average HGV loads).
385 Future potential for such movements include
major construction projects such as Crossrail
and the Thames Tideway Sewers. A range of
Government grants are available to operators
to offset both capital and operational costs.
386 Increasing waterborne freight will also depend
on the availability of wharf facilities to transfer
cargo between land and water. The Mayor has
safeguarded 50 such wharves on the Thames
and tidal tributaries and London Plan policy
will ensure that these sites are maintained
and used for waterborne freight (including
waste) transport.
167
387 This strategy supports the retention of these
wharves. Furthermore, the Mayor believes
that there is potential for additional transfer
from road to water, particularly for deliveries to
central and west London. This will necessitate
the reactivation of some of the safeguarded
wharves that are not currently in use, and
may require the addition of further wharves
in appropriate locations to serve this demand.
Road access will also be a consideration at
these sites.
388 TfL has researched the opportunities for
freight transport on Londons canal network
and, along with British Waterways will promote
these opportunities further. This will involve
Chapter five
the consideration of whether to safeguard a
number of wharves on the canal network.
Proposal 38
The Mayor, through TfL, and working with
the Port of London Authority, London
boroughs and operators, will seek to ensure
that existing safeguarded wharves are fully
utilised for waterborne freight (including
waste), and will examine the potential to
increase the use of the Thames and
Londons canal network for waterborne
freight transport.
Chapter five Transport proposals
168 Mayors Transport Strategy
5.8 River crossings
389 Historically, there have been fewer river crossings
in east London than in the west, due to the width
of the river, the types of existing land use and the
extent of shipping activity east of Tower Bridge.
This has resulted in limited interaction between
the residential population and businesses on
either side of the river, and is one of the factors
contributing to lower land values.
390 As the economy of east London has changed,
developments such as Canary Wharf, ExCel and
The O2 have increased the demand for travel
across the river significantly. Many of the large
new economic drivers for London are located
in east London, with the majority of these lying
north of the river, such as the Olympic Park and
adjacent Stratford City development, Canary
Wharf, ExCel and City airport. Access to these
growing destinations from southeast London
can be difficult due to the barrier effect of
the Thames.
391 Over the last 20 years progress has been made
on rail and passenger ferry crossings. The DLR
extensions to Lewisham and Woolwich Arsenal
and the Jubilee line extension have created new
river crossings and proved very successful in
improving cross-river connectivity. These will be
followed by the upgraded East London line in
2010 and Crossrail in 2017.
392 The Mayor supports improving the opportunities
for pedestrians and cyclists to cross the Thames in
east London, currently limited to the Greenwich
and Woolwich foot tunnels and the Woolwich
Ferry. In the shorter term, there is the potential
Chapter five Transport proposals
to make greater use of existing passenger ferries,
but the potential for new fixed links will be
explored whether for pedestrians and cycles only,
or in conjunction with other modes. Schemes
could include provision of new and enhanced
passenger/cycle ferries, new fixed links, or
innovative solutions such as cable cars, where
these could be appropriate. These crossings would
provide alternatives for local journeys on existing
crossings and free-up capacity for longer distance
travel on the Tube and DLR routes.
393 In addition, the Mayor is committed to ensuring
that where new or improved cross-river
pedestrian, cycle and public transport links exist,
car drivers, who could switch their journeys
to more sustainable modes, are encouraged
to do so to reduce the number of car trips at
congested crossings. Freight journeys can be
timed to avoid peak hours through improved
journey planning and supply chain interventions
such as DSPs. Other measures, such as
consolidation centres and modal shift to rail
and water, proposed elsewhere in this strategy
(see sections 5.2, 5.7 and 5.24) will also reduce
freight traffic in peak hours.
394 However, there will continue to be a need for
some journeys to be undertaken by vehicle, in
particular commercial traffic and the movement
of goods and the provision of services to support
a growing economy in east London. Drivers are
heavily dependent on the congested Blackwall
and Rotherhithe tunnels, each of which have
restrictions on the size of vehicle which can use
them, and the Woolwich Ferry. Beyond London,
the Dartford crossing, forming part of the M25
orbital motorway, also regularly operates at, or

close to, capacity. There is little resilience in the
event of an incident at one of these crossings,
and local businesses, particularly in southeast
London, suffer from this unreliability. The
projected increases in jobs and population in the
Thames Gateway will increase the problem of
highway congestion and road network resilience
at river crossings further. The Mayor is therefore
supportive of additional road-based river
crossings in east London as part of a package
of transport improvements.
395 The Woolwich Ferry has a lower capacity than
a fixed road crossing, but nevertheless provides
an essential cross-river link for some road
users, particularly HGVs and commercial traffic
crossing the river due to vehicle restrictions at
the Blackwall and Rotherhithe tunnels and at
Tower Bridge. The vessels and landing stages are
coming towards the end of their life, and there is
an opportunity to replace the existing equipment
with more efficient modern vessels.
396 The recent TfL review of potential river
crossing sites and options indentified several
areas where more investigation and work is
warranted. The package of river crossings will
have regard to the needs of all potential users,
including vehicles, freight, public transport,
walking and cycling. The package of new river
crossings in east London is shown in Figure 44.
397 A range of funding options are available for
crossings. For example, tolling on highway
crossings (section 5.27, see proposal 130 and
chapter eight) could both help to finance the
construction of schemes, as well as providing
a means of managing traffic demand.
169
Proposal 39
The Mayor, through TfL, and working
with the London boroughs and other
stakeholders, will take forward a package of
river crossings in east London, including:
a) A new fixed link at Silvertown to provide
congestion relief to the Blackwall Tunnel
and provide local links for vehicle traffic
b) An upgraded Woolwich Ferry and
consideration of a new vehicle ferry at
Gallions Reach to improve connectivity
c) Local links to improve connections for
pedestrians and cyclists
Chapter five
d) Consideration of a longer-term fixed link at
Gallions Reach to improve connectivity for
local traffic, buses, cyclists and to support
economic development in this area
e) The encouragement of modal shift from
private cars to public transport, using
new rail links including High Speed One
domestic services, Crossrail and the
DLR extension to Woolwich, reducing
road demand, and so road congestion
at river crossings, where possible
f) Support for Government proposals to
reduce congestion at the Dartford crossing
Chapter five Transport proposals
170 Mayors Transport Strategy

Figure 44: Options for new river crossings in east London

Local links to improve

pedestrian and cycle access

Support for maximising the Long-term fixed link

impact of new rail crossings at Gallions Reach

Upgraded New vehicle ferry

New fixed link at Silvertown Woolwich Ferry at Gallions Reach

Crossrail

Potential lower
Thames crossing at,
or downstream of,
Dartford (study
being led by the DfT)

Key
Potential shorter-term schemes
Potential longer-term schemes
Crossrail
Existing foot tunnels

Chapter five Transport proposals


5.9 A more accessible
transport system
5.9.1 Enhancing the physical
accessibility of the transport system
398 This section, together with section 7.2
(Accessibility Implementation Plan),
constitutes the Mayors proposals for the
provision of transport which is accessible to
persons with mobility problems as required
by the GLA Act 1999.
399 Londons transport system is one of the
oldest in the world. The legacy is a large,
comprehensive system, but also, in places,
notably the Tube and rail networks, one which
does not fully meet the accessibility needs of
Londoners, particularly mobility or sensory
impaired people.
400 Much is being done to address this issue (see
spotlight on transport accessibility). TfLs
Business Plan sets out transport projects that
are committed to be delivered before 2020,
many of which will improve the physical
accessibility of Londons transport system.
In particular, Crossrail will revolutionise the
accessibility of central London with step-free
interchanges at key stations such as Bond
Street and Tottenham Court Road. In addition,
accessibility improvements will be delivered
as part of the Tube upgrades (for example,
addressing platform-train interface and
installing platform humps at key locations) as
well as step-free access at key 2012 Games
Underground stations and through major
interchange schemes for example, Bank and
171
Victoria all building on the foundation of
step-free stations already in place. Major
redevelopment proposals such as Elephant
& Castle and Brent Cross/Cricklewood will
also provide an opportunity for further
improvements in station accessibility.
401 The DfTs Access for All programme,
augmented by Crossrail and other committed
investment, will increase coverage of step-
free access (from street to platform) to 46 per
cent of rail stations in London by 2017. This
equates to 160 step-free stations in 2017,
compared to 101 (31 per cent) in 2010. The
process allows for TfL to influence the DfTs
draft suggestions, which has successfully
Chapter five
been done for each tranche announced to
date. The East London line extension and
North London line enhancements will provide
a number of new and upgraded step-free
stations on the London Overground network.
402 As the Tube network is renewed, it will be
made more accessible. New trains to be
introduced on most Underground lines
will comply with Rail Vehicle Accessibility
Regulations (RVAR), and Underground
station refurbishments will provide or include,
for example:
Audible and visual information on all
platforms and in all ticket halls
Improved handrails to ensure appropriate
heights and designs and provide a visual
contrast with the wall
Improved steps and stairs to provide a visual
contrast at the leading edge of each riser
and tread
Chapter five Transport proposals
172 Mayors Transport Strategy

Spotlight

Transport accessibility for all

Progress has been achieved in recent years in
terms of the accessibility of transport services.
Currently, all buses in London are low-floor,
about 20 per cent of Tube stations and a third
of National Rail/London Overground stations
are step-free from street to platform, and all
DLR and Tramlink stations and vehicles are
fully accessible.
The DfTs Access for All programme, alongside
Crossrail and enhancements to the London
Overground network, will provide step-free
access coverage at 46 per cent of rail stations
in London, by 2017.
As the Tube network is renewed, it will be
made more accessible: new trains will comply
with RVAR and capacity enhancements at the
busiest stations on the network will provide
accessibility improvements. All Tube stations will
be refurbished with tactile markings and colour-
contrasted handrails.
However, there is even more to do to make
Londons transport system fully accessible for all.
The strategy will deliver further improvements, to:
Journey planning and information (for example,
real time bus information via mobile phone or
the internet)
Streets and town centre areas
(for example, balanced streets)
Better bus stop accessibility (for example,
removal of street clutter around bus stops)
Public transport staff and passenger helpfulness,
behaviour and attitude (for example, wider
availability of staff to assist passengers)
Further station and train accessibility
improvement (for example, new trains, Crossrail)
Door-to-door services (for example,
new vehicles for Dial-a-Ride)

Chapter five Transport proposals


Removing, modifying or highlighting
obstructions
Providing a visual contrast between
Help and Information points and the
surrounding walls
Installing induction loops at every Help
and Information point and providing
'listening points' for hearing aid users at
some bigger stations
Improving lighting and public address systems
Improving signs and wayfinding to help
people navigate around stations and trains,
including expanding use of pictograms
Installing tactile warning surfaces on every
platform and on all staircases
Increasing the amount of seating in ticket
halls, on platforms and in long corridors
and walkways
Providing more priority seating on trains, at
stations, in ticket halls and on platforms
Further improving the safety and security
of stations by increasing the coverage and
quality of CCTV, providing safer waiting
areas at specific stations with Help and
Information points in every ticket hall and
corridor and on every platform
Providing accessible unisex toilets at all
step-free stations where toilets already exist
403 In addition, trials of platform humps have been
successfully completed, and are being rolled-
out across the Tube system as new rolling
stock is introduced, as well as using other
infrastructure changes to provide level access
on to trains.
173
404 Improvements to the accessibility of the street
environment are important to complement
station access enhancements and are crucial
to the whole journey approach. With limited
resources available, a joined-up approach will
be required.
405 The increasing numbers of mobility scooters
used by mobility impaired people for trips
in London should continue to be supported
through an accessible street environment and
targeted enhancements with regard to the
safety of all road and pavement users.
Proposal 40
Chapter five
The Mayor, through TfL, and working with the
DfT, Network Rail, the London boroughs and
others will improve the physical accessibility
of the transport system by prioritising
step-free access at strategic interchanges,
improving street accessibility in town centres
and around accessible stations and maximising
the accessibility benefits of new transport
schemes, such as Crossrail. In doing so, the
Mayor will seek to maximise the benefits of
investment by ensuring that resources are
focused on improving accessibility for the
maximum number of people, while ensuring
an equitable balance across London.
406 In particular, it will be important to maximise
the benefits of the accessible bus fleet as this
is a relatively quick and cost effective way of
enhancing physical accessibility to the transport
system throughout London.
407 The Mayor recognises that the Blue Badge
parking scheme has contributed significantly
Chapter five Transport proposals
174 Mayors Transport Strategy
to expanding travel opportunities for those
with severe mobility difficulties. He supports
the provision of priority parking places for
Blue Badge holders, particularly in town centre
locations, at public services and stations, and
a 100 per cent discount from the Congestion
Charge scheme.
408 Further details of the approach to improving the
physical accessibility of Londons transport network
will be provided in TfLs Disability Equality Scheme
(DES). This is a statutory document, updated every
three years, which sets out in further detail what
TfL is going to do to ensure that the services it
offers are accessible to disabled people.
5.9.2 Enhancing information provision
409 Information is a critical enabler to making
the right choice about travel options and
needs to be timely and accessible. Disabled
people identify improvements in this area
as being a key factor in their ability to travel
independently and with a feeling of confidence
and personal safety. Enhancing pre-trip and
in-trip journey information and improving the
legibility of interchanges and facilities, will
bring benefits to all Londoners, and will go
some way to removing barriers to travel.
Proposal 41
The Mayor, through TfL, and working
with the London boroughs and other
stakeholders, will improve the availability,
quality, quantity and timeliness of
information about the transport system to
remove barriers to travel.
Chapter five Transport proposals
5.9.3 Improving staff service and the
attitudes of customers
410 It is recognised that the approach of some
staff and the attitude of some customers
needs to reflect a more considerate approach
to the needs of all users of public transport.
This can be achieved through raising customer
service standards, improved customer
relations programmes and disability awareness
campaigns, so that those passengers who
require additional assistance receive it as a
matter of course.
Proposal 42
The Mayor, through TfL, and working
with the London boroughs and other
stakeholders, will improve attitudes of
transport staff and travellers towards each
other to ensure excellence in customer
service and a courteous, safe and friendly
travelling environment that does not present
a barrier to travel.
411 Some people with mobility difficulties may
need to build up confidence before using the
public transport system independently. The
Mayor will support travel-mentoring initiatives
that help mobility impaired people to become
accustomed to using the accessibility features
on Londons public transport system.
412 Staff that are available throughout service
hours to provide assistance, information and
reassurance to all customers are particularly
valued by disabled people. All bus stations,
Tube and London Overground stations will

continue to be staffed from first to last service.
However, there remains the need for greater
consistency in availability and the training of
staff across the transport network.
Proposal 43
The Mayor, through TfL, will work to ensure
a greater staff availability to provide direct
assistance to customers and continue to
improve customer experience, by enhancing
staff training to ensure that the access needs
of disabled passengers are understood by all
frontline staff.
5.9.4 Door-to-door transport
413 Door-to-door transport takes people all the
way from the origin of their trip to their
final destination. Unlike mainstream public
transport, where people generally have to
access the system at predetermined stops,
door-to-door transport can, in most cases, pick
up and drop off passengers anywhere. The
general aim of schemes such as Dial-a-Ride,
Capital Call and Taxicard is to provide transport
options for people for whom the mainstream
transport network remains inaccessible.
414 Dial-a-Ride is a free door-to-door transport
service for disabled people who cant use buses,
trains or the Tube. It can be used for many
types of journeys, making it easier for disabled
people or people with lower levels of mobility
to go shopping and visit friends. Eligibility for
Dial-a-Ride membership includes those people
who have a permanent or long-term disability
which means they are unable, or virtually
175
unable, to use mainstream public transport
services some or all of the time.
415 Dial-a-Ride is a very successful and popular
service, currently catering for around 1.3
million trips a year. Customer satisfaction rating
is also running at more than 90 per cent.
416 Funding of door-to-door services has
significantly increased over the past few
years, delivering a number of enhancements.
Improvements to Dial-a-Ride have included
extending eligibility, scrapping fares and making
improvements to the call centre and booking
system. TfL continues to jointly fund Taxicard
with the London boroughs and fully fund Capital
Chapter five
Call, both of which provide subsidised transport
to people who have mobility impairment and
difficulty using public transport. Demand
continues to outstrip the supply for door-to-
door services and discussions between TfL and
London Councils are ongoing to explore the best
use of resources in maintaining and improving
these services.
Proposal 44
The Mayor, through TfL, will support door-to-
door services for people with mobility problems
who require this form of transport service.
5.9.5 Accessibility Implementation Plan
417 An accessibility implementation plan, as
required by the GLA Act, is set out in section
7.2 of this strategy.
Chapter five Transport proposals
176 Mayors Transport Strategy
5.10 Integrating Londons
transport system and services
5.10.1 Improving interchange
418 Interchanges, whether local or major transport
hubs, vastly expand the level of accessibility to
opportunities and services offered by Londons
transport system by enabling multi-modal
journeys, and those involving more than one
public transport service. The convenience,
comfort, information provision, safety and
reliability experienced at interchanges are
Figure 45: Strategic interchange concepts
Strategic interchanges will help to relieve passenger dispersal pressures at central London rail termini
through two primary means:
1) Enable interchange to orbital public
transport services to avoid the need
to enter central London
Origin
Destination
Rail termini Strategic interchange
Enhanced orbital trip opportunity
Chapter five Transport proposals
important determining factors in Londoners'
perceptions of the quality and attractiveness
of the transport system. Interchanges have a
crucial role to play in improving the efficiency
of Londons transport system, as well as the
relative attractiveness of public transport to
the car and tackling car dependency.
419 Interchanges not only enable travel choices,
but also provide opportunities to create better
places to live and work as well as support
population and employment growth in highly
accessible and sustainable locations. Improved
2) Enable interchange between National Rail and
Underground/bus services at a point prior to
the rail termini, thereby reducing pressure at
overcrowded rail termini interchanges
Origin
Destination
National Rail Underground/bus
Central area
 177

interchanges can support the alleviation of journey experiences, and help address key
crowding and congestion, maximise access environmental and quality of life concerns,
to business and employment markets (on a such as air quality, health and noise pollution.
London-wide, national and international scale), In collaboration with partners and stakeholders,
improve connectivity, improve passenger TfL has published Interchange Best Practice

Figure 46: Examples of strategic interchange locations outside central London

Tottenham
Hale
Seven Sisters/
South Tottenham Walthamstow Central/Queens Road
Finsbury Park
Hackney Downs/Central

Chapter five
West Hampstead
Highbury Stratford Barking
Willesden Junction & Islington
Queens
Park West Ham
Ealing Broadway
Woolwich
Vauxhall Elephant & Castle Arsenal
Peckham Rye
Richmond Lewisham
Clapham Junction Herne Hill
Tulse Hill Catford/Catford Bridge
Balham
Wimbledon

West Croydon Bromley South

East Croydon

Key Strategic interchange concept

Radial rail routes

London Overground orbital routes
Docklands Light Railway
Tramlink
Interchanges between radial rail routes and:
London Overground orbital routes
Other lines

Chapter five Transport proposals

178 Mayors Transport Strategy
Guidelines that provide advice and guidance
to those involved in planning, improving and
operating high quality effective interchanges.
Proposal 45
The Mayor, through TfL, and working with
Network Rail, the train operating companies,
London boroughs and other stakeholders,
will improve the customer experience and
physical accessibility at interchanges across
London through the application of the
principles set out by the TfL Interchange
Best Practice Guidelines of efficiency,
useability, understanding and quality
to all interchange schemes in London.
Such measures include:
a) Provision of consistent and enhanced
travel information
b) Improved walking and cycling facilities at,
and on routes to, public transport stations
and stops
c) Improved integration of public transport
services in London, both in terms of
service planning and physical location
d) Improved efficiency, effectiveness
and quality of interchanges across
London to further integrate Londons
transport system
e) Provision of consistent customer service
delivery standards
f) Assurance that interchange facilities have
sufficient capacity to meet travel demand
Chapter five Transport proposals
5.10.2 Strategic interchanges
420 Strategic interchanges are primarily radial
to orbital rail interchanges. They have the
potential to reduce travel times and relieve
crowding in central London, including
interchange capacity pressures at Londons
rail termini. Connectivity and central London
crowding relief benefits are offered by new
and enhanced orbital public transport services,
see Figure 45. Some also offer significant
development potential, due to their enhanced
public transport accessibility and connectivity.
Figure 46 shows potential key strategic
interchanges outside central London.
Proposal 46
The Mayor, through TfL, and working with
Network Rail, the train operating companies,
London boroughs and other stakeholders,
will prioritise improvements to strategic
interchanges, that will:
a) Provide opportunities for orbital public
transport services
b) Provide interchange opportunities before
arriving in central London, in order to
reduce interchange capacity pressure at
Londons rail termini
c) Provide opportunities to accommodate
population and employment growth,
with developer contributions towards
the interchange improvements sought in
appropriate circumstances

Spotlight
Woolwich Arsenal strategic interchange
The new DLR station at Woolwich Arsenal in
Greenwich has created a strategic interchange
in Outer London. The interchange enables
passengers travelling on radial National Rail
services into central London to interchange
at Woolwich and make orbital journeys
north, across the Thames, towards important
destinations such as Stratford home of the
Olympic Park London City airport and
Canary Wharf.
179
The interchange is heavily used by passengers
changing from orbital bus services travelling into
Woolwich from areas such as Plumstead. More
than 50 per cent of arrivals for DLR services
from Woolwich have interchanged from bus or
National Rail services. Cycle parking outside the
station is also heavily used.
The Woolwich Arsenal strategic interchange will
be further enhanced when the planned Crossrail
station opens in 2017.
Chapter five
Chapter five Transport proposals
180 Mayors Transport Strategy
5.11 London 2012 Olympic
and Paralympic Games
5.11.1 The 2012 Games legacy
421 s highlighted in chapter four, the Games
A the benefits of walking and cycling as a mode
provides London with a unique opportunity to
change peoples behaviour and encourage a
lower carbon, healthier life style by increasing
the awareness and use of walking, cycling and
public transport. This is particularly important
in Stratford, where the Olympic Park will be
fully accessible by public transport, as well
as walking and cycling routes. The level of
accessibility can be seen in Figure 47.
422 Transport can support the wider legacy benefits
of the 2012 Games in two key ways:
Firstly, new transport infrastructure being
delivered for the Games will improve
accessibility to jobs and services for local
people and offer a wider choice of travel
modes, for example, walking and cycling
routes and access to new DLR stations.
This will also contribute towards reducing
car dependency and road congestion, and
reduce carbon and other emissions
Secondly, promotion of more active modes
of travel following the Games, building on
the inspiration of the athletic performances
through targeted awareness programmes.
These include smarter travel to support
improvements in health and the local
environment
Chapter five Transport proposals
423 The London 2012 Active Travel Programme
will use the Games as a catalyst to encourage
spectators and the public to walk and cycle
more before, during and after the Games. The
programme will help to raise awareness about
of transport nationwide, and help increase
the number of such journeys across London
and the UK.
424 To promote healthy and environmentally
sustainable lifestyles beyond the Games,
initiatives are being developed to encourage
walking and cycling in the five Olympic boroughs
of Tower Hamlets, Hackney, Greenwich, Newham
and Waltham Forrest; and within the area of the
Olympic Park.
425 TfL is also providing extra funding for boroughs
in advance of the 2012 Games for public
realm improvement schemes at sites that will
attract significant visitor numbers during the
Games. These will transform the quality of
the environment and urban realm in line with
the principles of better streets and provide
a lasting legacy in the local area. Further
interventions may be appropriate.
426 Additional complementary transport
infrastructure investments may also be required
to realise the local benefits of the Olympic Park
that will be transformed through the Olympic
Legacy Masterplan.
 181

Figure 47:Olympic Park legacy international, national, regional and local connections

to north London and to northeast London

Stansted airport and Essex
Leyton

to north London
Stratford International

to Kent (and potentially

Stratford European destinations)
to St Pancras City
Stratford
Hackney Wick
to east London, Essex
and East Anglia

Chapter five
Improved bus network with a
n c rd
re

Olympic
ent
tow tratfo

major interchange at Stratford

Stadium
S

ay
to central London Greenw
Elevated n
B e c k to
to
to central London Pudding Mill to Essex
and Heathrow airport Lane
Bow Road West Ham
Bow Church

to Victoria and to east London

central London

to Canary Wharf, to Canary Wharf to London City airport,

Greenwich, Lewisham and Waterloo Woolwich Arsenal and
and southeast London southeast London

Highway network not shown

Subject to potential review by the Olympic Legacy Transport Plan process

Chapter five Transport proposals

182 Mayors Transport Strategy
427 The 2012 Paralympic Games will also provide
specific legacy benefits for Londoners that will
need to be embedded with further initiatives
after the Games. New infrastructure for the
Games, such as the DLR extension to Stratford
International, the upgrade of Stratford station
and step-free access at key Underground
stations such as Green Park, all contribute
towards more accessible public transport
services. The Paralympic Games will also do
much to promote the awareness of
disability among transport operators and
the general public.
428 In order to better understand the additional
interventions that may be required to ensure
maximum synergy with other intervention
programmes, to maximise benefits of transport
investment and ensure transport fully supports
the principle of convergence outlined in chapter
four, the Mayor proposes to develop a Transport
Legacy Action Plan with key partners. The plan
will be embedded in the sub-regional transport
plan process and delivered through the TfL
Business Plan and the boroughs LIPs.
Chapter five Transport proposals
Proposal 47
The Mayor in partnership with the London
boroughs, TfL and Olympic Park Legacy
Company, will develop a Transport Legacy
Action Plan and monitoring programme
to ensure the benefits of the legacy of
the 2012 Games are maximised and that
transport interventions support convergence
as set out in the five Olympic Boroughs
Strategic Regeneration Framework. The
plan will be monitored for 10 years after the
Games, and will define:
Partners and their responsibilities
The monitoring area within the five
Olympic boroughs
Key indicators and targets within the
monitoring area and London-wide
Actions and interventions required to
meet the targets
Annual review of targets

5.12 Londons airports
429 This section focuses on airport capacity
and surface access to airports. The Mayors
approach to broader aviation issues is outlined
in section 4.2.2.1 Supporting and developing
Londons international, national and inter-
regional transport connectivity and in draft
replacement London Plan policy 6.6. Measures
to improve the carbon efficiency of air transport
are set out in section 5.22 Reducing carbon
dioxide emissions of this strategy, including
proposal 101.
5.12.1 Airport capacity
430 Demand for air travel will continue to pose a
major challenge for London. The number of
passengers travelling through London area
airports amounted to almost 140 million in
2008, making this area the busiest in the world.
Unconstrained, demand is expected to rise to
290 million passengers each year by 2031.
Current airport capacity in the South East will,
however, limit trips to 180 million passengers
a year. This could have the effect of limiting
Londons economic growth and putting its
competitive position at risk.
431 Committed capacity enhancements, principally
Crossrail and the Piccadilly line upgrade, are
designed to accommodate demand based on
projected growth to existing permitted levels
of airport use.
432 A third runway at Heathrow would increase
capacity ultimately to 702,000 air traffic
movements per year. This would significantly
183
increase passenger demand for surface rail and
road access on already congested networks.
However, the DfT has stated that a detailed
surface access strategy is not a prerequisite
for a policy decision regarding a third runway.
Current forecasts suggest that by 2030, with a
third runway, non-transfer passenger numbers
could more than double, to 91 million per
year. This would reduce resilience and increase
crowding, congestion and delays elsewhere
along rail and road corridors serving Heathrow
and across west London as a whole. Public
transport would become less attractive for users,
especially at peak times. Airport expansion
would further threaten to reduce the quality
Chapter five
of life of many London residents.
433 Noise pollution currently affects a large number
of residents underneath the Heathrow flight
paths. DfT modelling estimated that in 2008
over 250,000 people were within the Leq (used
as the national airport noise exposure index)
noise contour of 57 decibels or above. Measures
to reduce noise pollution from aircraft are
outlined in proposal 89.
434 Air quality is also a serious issue at Heathrow.
The airport is at risk of failing to meet EU
NO2 maximum limit values in 2015. Surface
access trips further compromise air quality:
an extra three million car trips per year are
forecast between 2010 and 2015. Measures
to improve air quality are outlined in section
5.21 and policy 3 of the draft Mayor's Air
Quality Strategy.
435 Nevertheless, the Mayor recognises that
adequate airport capacity is critical to the
Chapter five Transport proposals
184 Mayors Transport Strategy
continued competitiveness of Londons
economy. For this reason, the Mayor will
consider whether optimum use is being made
of existing airport infrastructure (though mixed
mode operation is not favoured at Heathrow).
436 Stansted, Gatwick and Luton airports have
important roles serving London as well as the
South East. The Mayor will continue to work
with partners in neighbouring regions through
the Advisory Forum on Regional Planning for
London, the South East and East of England to
ensure that existing aviation infrastructure is
used to its fullest extent before other options
are considered for providing further airport
capacity (this is set out in the aviation section,
chapter six, London Plan).
437 Solutions are, however, not limited to additional
airport capacity. There is potential to increase
transfer from short-haul domestic and European
flights to rail journeys through existing and
possible future high-speed rail services, thus
freeing up take-off and landing slots for long-
haul capacity.
Proposal 48
The Mayor recognises that the provision of
adequate airport capacity serving the South
East is critical to the competitive position of
London in a global economy, but will oppose
any further increases in runway capacity
at Heathrow.
Chapter five Transport proposals
5.12.2 Surface access to airports
438 Londons four main airports will continue to
be the gateway to the city for the majority
of overseas visitors, so high quality, efficient
surface access is vital to promote London and
the UK as a place to visit and do business.
Completion of Crossrail and the Thameslink
upgrade will improve the public transport
capacity and connectivity of Heathrow and
Gatwick. The West Anglia National Rail
upgrades (including the four-tracking project
mentioned elsewhere in this section) will
also enhance the capacity of rail services to
Stansted. However, further improvements are
required, in particular to access Heathrow.
439 The Mayor supports the principle of Airtrack
(being promoted by BAA/Heathrow Airport
Limited), subject to clarification of its impact
on existing services to Waterloo, its impact
on level crossings, and the scheme having a
robust business case. If implemented, Airtrack
would deliver a new rail link connecting the
existing rail line from Waterloo to Reading
with Heathrow Terminal 5. This project would
cost around 650700m, and could be
delivered by 2014. Airtrack has the potential to
significantly improve connectivity to Heathrow
by enhancing public transport access from
southwest London, and support its role as
a major transport and employment hub.
440 Government has approved expansion at
Stansted, which will increase the airports
capacity from 25 to 35 million passengers
per year. A package of measures has been
developed to support this expansion. This

includes widening the M11 between junctions
6 and 8 (between the M25 and Stansted)
and the provision of further rail capacity
(including a new fleet of Stansted Express trains
ordered in 2009).
443 TfL has worked with airport operators through
441 BAA has submitted proposals to provide an
additional runway at Stansted, which could see
the number of passengers using the airport
rise from 35 million to 68 million by 2030.
The Mayor has concerns about the impact
this expansion will have on public transport
services into London and believes that essential
improvements to Tottenham Hale Underground
station and along the West Anglia main line
need to be funded before planning permission
is granted.
Proposal 49
The Mayor believes the aviation industry
should meet its full environmental and
external costs and supports the position of
The Future of Air Transport White Paper
published in 2003. This states that airport
operators should be responsible for paying
the costs of upgrading or enhancing road,
rail or other transport networks or services
where these are needed to accommodate
additional passengers travelling to, and from,
expanded or growing airports.
442 London City airport was granted planning
permission in 2009 to accommodate 120,000
flights per year. The opening of the DLR
extension to the airport means that, at more
than 50 per cent, it currently has the highest
public transport surface access mode share of
185
any airport serving London. DLR infrastructure
to support this growth is in place, though
further rail vehicles are required to provide the
required service capacity.
their airport transport forums to help improve
surface access to airports. Continued close
engagement with airport operators and
local boroughs will be essential to serve the
increasing numbers of air passengers and
encourage a shift from private car to reduce
congestion and improve surrounding air quality.
Proposal 50
Chapter five
The Mayor, through TfL, and working with
the London boroughs, DfT, airport operators,
Network Rail, train operating companies
and other stakeholders, will seek to improve
access to Londons airports for passengers
and staff by public transport, particularly
from those parts of London which do not
currently have good access by rail or bus; and
for goods through better management of the
road network, development of consolidation/
break-bulk centres and encouragement of
access by rail and waterway.
Chapter five Transport proposals
Chapter Six
The Annotated Bibliography Exercise
What is an Annotated Bibliography?
Why Write Annotated Bibliographies?
How many sources do I need?
Using Computers to Write Annotated Bibliographies
The Process of Writing the Annotated Bibliography
* A Sample Assignment
* The Annotated Bibliography and Collaboration
* Questions to Ask while Writing and Researching
* Review and Revision

What is an Annotated Bibliography?

As you develop a working thesis for your research project and begin to collect
different pieces of evidence, you will soon find yourself needing some sort of
system for keeping track of everything. The system discussed in this chapter is
an annotated bibliography, which is a list of sources on a particular topic that
includes a brief summary of what each source is about. This writing exercise is a
bit different from the others in this part of The Process of Research Writing in that
isnt an essay per se; rather it is an ongoing writing project that you will be
building as you discover new pieces of evidence for your research project.

Here is an example of an entry from an annotated bibliography in MLA style:

Parsons, Matt. Protecting Children on the Electronic

Frontier: A Law Enforcement Challenge. FBI Law

Enforcement Bulletin 69.10 (2000): 22-26.

This article is about an educational program used by the

U.S. Navy to educate people in the Navy and their families
about some of the things that are potentially dangerous to
children about the Internet. Parsons says that the
educational program has been effective.

Annotated bibliography entries have two parts. The top of the entry is the
citation. It is the part that starts Parsons, Matt and that lists information like
the name of the writer, where the evidence appeared, the date of publication, and
other publishing information.

Steven D. Krause | http://www.stevendkrause.com/tprw | Spring 2007

 The Process of Research Writing
Chapter Six, The Annotated Bibliography Exercise,2

Hyperlink: For guidelines on how to properly write citations for your

Annotated bibliographies, see Chapter 12, Citing Your Research Using MLA or
APA Style.

The second part of the entry is the summary of the evidence being cited. A good
annotated bibliography summary provides enough information in a sentence or
two to help you and others understand what the research is about in a neutral
and non-opinionated way.

The first two sentences of this annotation are an example of this sort of very brief,
just the facts sort of summary. In the brief summaries of entries in an
annotated bibliography, stay away from making evaluations about the source
I didnt like this article very much or I thought this article was great. The
most important goal of your brief summary is to help you, colleagues, and other
potential readers get an idea about the subject of the particular piece of evidence.

Summaries can be challenging to write, especially when you are trying to write
them about longer and more complicated sources of research. Keep these
guidelines in mind as you write your own summaries.

Keep your summary short. Good summaries for annotated

bibliographies are not complete summaries; rather, they provide the
highlights of the evidence in as brief and concise a manner as possible, no
more than a sentence or two.

Dont quote from what you are summarizing. Summaries will be more
useful to you and your colleagues if you write them in your own words.
Instead of quoting directly what you think is the point of the piece of
evidence, try to paraphrase it. (For more information on paraphrasing
your evidence, see Chapter 3, Quoting, Paraphrasing, and Avoiding
Plagiarism).

Dont cut and paste from database abstracts. Many of the periodical
indexes that are available as part of your librarys computer system
include abstracts of articles. Do no cut this abstract material and then
paste it into your own annotated bibliography. For one thing, this is
plagiarism. Second, cutting and pasting from the abstract defeats one of
the purposes of writing summaries and creating an annotated
bibliography in the first place, which is to help you understand and
explain your research.

Different writers will inevitably write slightly different summaries of the same
evidence. Some differences between different writers summaries of the same
piece of evidence result from different interpretations of what it important in the
research; theres nothing wrong with that.

Steven D. Krause | http://www.stevendkrause.com/tprw | Spring 2007

 The Process of Research Writing
Chapter Six, The Annotated Bibliography Exercise,3
However, two summaries from different writers should both provide a similar
summary. In other words, its not acceptable when the difference of
interpretation is the result of a lack of understanding of the evidence.

Why Write Annotated Bibliographies?

An annotated bibliography is an excellent way to keep track of the research you

gather for your project. Make no mistake about it it is extremely important
that you keep track of all of your evidence for your research project, and that
you keep track of it from the beginning of the process of research writing.

Theres nothing more frustrating than finding an excellent article or book chapter
you are excited about incorporating into your research project, only to realize
you have forgotten where you found the article or book chapter in the first place.
This is extremely frustrating, and its easily avoided by doing something like
writing an annotated bibliography.

You could use other methods for keeping track of your research. For example,
you could use note cards and write down the source information as a proper
citation, then write down the information about the source that is important. If
the material you know you want to use from a certain source is short enough,
you might even write a direct quote, which is where you write down word for
word what the source says exactly as it is written. At other times, you can write
a paraphrase, which is where you write down what the source means using your
own words.

While note cards and other methods have their advantages, annotated
bibliographies are an extremely useful tool for keeping track of your research.
An annotated bibliography:

Centralizes your research into one document that you can keep track of
both as a print-out of a word-processed file and as a file you save
electronically.

Allows you to copy and paste citation information into the works cited
part of your research project.

An annotated bibliography also gives you the space to start writing and thinking
a bit about how some of your research might fit into your project. Consider these
two sample entries from an annotated bibliography from a research project on
pharmaceutical advertising:

Siegel, Marc. Fighting the Drug (ad) Wars. The Nation 17

June 2002: 21.

Siegel, who is a doctor himself, writes about how drug

advertising has undermined the communication between

Steven D. Krause | http://www.stevendkrause.com/tprw | Spring 2007

 The Process of Research Writing
Chapter Six, The Annotated Bibliography Exercise,4
doctors and patients. He says that drug ads have driven up
the costs of prescription drugs, particularly big selling
drugs like those for cholesterol.

Wechsler, Jill. Minority Docs See DTC Ads as Way to

Address Race Gap. Pharmaceutical Executive

May 2002: 32, 34. WilsonSelect Database. Eastern

Michigan University Halle Library. 20 October

2002. <http://www.emich.edu/halle>.

This article is about a study that said that African-

American doctors saw advertising of prescription drugs as a
way of educating their patients. The ads are useful
because they talk about diseases that affect African-
Americans.

Even from the limited amount of information available in these entries, its clear
that a relationship between these articles exists. Both are similar articles about
how the doctor/patient relationship is affected by drug advertising. But both are
also different. The first article is from the newspaper The Nation, which is in
many ways similar to an academic journal and which is also known for its liberal
views. The second article is from a trade journal (also similar to academic
journals in many ways) that obviously is an advocate for the pharmaceutical
industry.

In other words, in the process of compiling an annotated bibliography, you are

doing more than keeping track of your research. You are starting to make some
comparisons and beginning to see some relationships between your evidence, a
process that will become increasingly important as you gather more research and
work your way through the different exercises that lead to the research project.

But remember: However you decide to keep track of your research as you
progress through your projectannotated bibliography, note cards, or another
methodthe important thing is that you need to keep track of your research as
you progress through your project!

How many sources do I need?

Inevitably, students in research writing classes always ask how many sources
they need to include in their research projects. In one sense, how many sources
do I need? is a utilitarian question, one usually attached to a students
exploration of what it will take to get a particular grade. Considered more
abstractly, this question is also an effort to explore the scope of a research project.

Steven D. Krause | http://www.stevendkrause.com/tprw | Spring 2007

 The Process of Research Writing
Chapter Six, The Annotated Bibliography Exercise,5
Like a certain page or word count requirement, the question how many sources
do I need? is an effort to get a handle on the scope of the research project
assignment. In that sense, asking about the number of sources is probably a
good idea, a little like asking how much something weighs before you attempt to
pick it up.

But ultimately, there is no right or wrong answer to this question. Longer

research projects tend to have evidence from more different sources than shorter
projects, but there is no cut-and-dry formula where X number of pages will
equal X number of sources.

However, an annotated bibliography should contain significantly more entries

than you intend or expect to include in your research project. For example, if
you think you will need or if your instructor requires you to have research from
about seven different sources, you should probably have about 15 different
entries on your annotated bibliography.

The reasons you need to find twice as many sources as you are likely to use is
that you want to find and use the best research you can reasonably find, not the
first pieces of research you can find. Usually, researchers have to look at a lot
more information than they would ever include in a research writing project to
begin making judgements about their research. And by far the worst thing you
can do in your research is to stop right after you have found the number of
sources required by the instructor for your project.

Using Computers to Write Annotated Bibliographies

Personal computers, word-processing software, and the Internet can make
putting together an annotated bibliography more useful and a lot easier. If you
use word-processing software to create your annotated bibliography, you can
dramatically simplify the process of creating a works cited or references
page, which is a list of the sources you quote in your research project. All you
will have to do is copy and paste the citation from the annotated bibliography
into your research projectthat is, using the functions of your computer and
word processing software, copy the full citation that you have completed on
your annotated bibliography page and paste it into the works cited page of
your research project.

This same sort of copy and paste function also comes in handy when doing
research on the web. For example, you can usually copy and paste the citation
information from your librarys online database for pieces of evidence you are
interested in reading. In most cases, you should be able to copy and paste
information you find in your librarys online database into a word processing
file. Many library databasesboth for books and for periodicalsalso have a
feature that will allow you to email yourself results from a search.

Keep two things in mind about using computers for your annotated
bibliographies:

Steven D. Krause | http://www.stevendkrause.com/tprw | Spring 2007

 The Process of Research Writing
Chapter Six, The Annotated Bibliography Exercise,6

You will have to reformat whatever information you get from the Internet
or your librarys databases in order to meet MLA or APA style.

Dont use the copy and paste feature to plagiarize! Simply copying
things like abstracts defeats one of the important purposes for writing an
annotated bibliography in the first place, and its cheating.

Assignment: Writing an Annotated Bibliography

As you conduct your research for your research writing project, compile an
annotated bibliography with 15-20 entries. Each entry in your annotated
bibliography should contain a citation, a brief summary of the cited
material. You will be completing the project in phases and a complete and
revised version of it will be due when you have completed your research.

You should think of your annotated bibliography as having roughly twice as

many sources as the number of sources you will need to include for the research
project, but your instructor might have a different requirement regarding the
number of sources required.

Also, you should work on this assignment in parts. Going to the library and
trying to complete this assignment in one sitting could turn this into a dreadful
writing experience. However, if you complete it in stages, you will have a much
better understanding of how your resources relate to each other.

You will probably need to discuss with your instructor the style of citation you
need to follow for your research project and your annotated bibliography.
Following a citation style isnt difficult to do, but you will want to be consistent
and aware of the rules from the beginning. In other words, if you start off
using MLA style, dont switch to APA style halfway through your annotated
bibliography or your research project.

Hyperlink: For an explanation of the differences of and the guidelines for

using both MLA and APA style, see Chapter 12, Citing Your Researching Using
MLA or APA Style.

Last, but not least, you will need to discuss with your instructor the sorts of
materials you need to include in your research and your annotated bibliography.
You may be required to include a balance of research from scholarly and non-
scholarly sources, and from traditional print resources (books, magazines,
journals, newspapers, and so forth) and the Internet.

Questions to ask while writing and researching

Would you classify the material as a primary or a secondary source? Does
the research seem to be difficult to categorize this way? (For more
information on primary and secondary sources, see Chapter 1, Thinking

Steven D. Krause | http://www.stevendkrause.com/tprw | Spring 2007

 The Process of Research Writing
Chapter Six, The Annotated Bibliography Exercise,7
Critically About Research and the section Primary versus Secondary
Research).

Is the research from a scholarly or a non-scholarly publication? Does the

research seem difficult to categorize this way?

Is the research from the Interneta web page, a newsgroup, an email

message, etc.? Remember: while Internet research is not necessarily
bad research, you do need to be more careful in evaluating the
credibility of Internet-based sources. (For more information on
evaluating Internet research, see Chapter 1, Thinking Critically About
Research, and the sections The Internet: The Researchers Challenge
and Evaluating the Quality and Credibility of Your Research.

Do you know who wrote the material you are including in your annotated
bibliography? What qualifications does your source say the writer has?

Why do you think the writer wrote it? Do they have a self-interest or a
political viewpoint that might make them overly biased?

Besides the differences between scholarly, non-scholarly, and Internet

sources, what else do you know about where your research was
published? Is it an academic book? An article in a respected journal? An
article in a news magazine or newspaper?

When was it published? Given your research topic, how important do

you think the date of publication is?

Are you keeping your summaries brief and to the point, focusing on the
point your research source is trying to make?

If its part of the assignment, are you including a sentence or two about
how you see this piece of research fitting into your overall research
project?

Revision and Review

Because of its ongoing nature, revising an annotated bibliography is a bit
different than the typical revision process. Take opportunities as you compile
your annotated bibliography to show your work in progress to your classmates,
your instructor, and other readers you trust. If you are working collaboratively
on your research projects, you will certainly want to share your annotated
bibliography with classmates who are working on a similar topic. Working
together like this can be a very useful way to get more ideas about where your
research is going.

It is best to approach the annotated bibliography in smaller stepsfive or six

entries at a time. If thats how youre approaching this project, then you will
always be in a process of revision and review with your classmates and your

Steven D. Krause | http://www.stevendkrause.com/tprw | Spring 2007

 The Process of Research Writing
Chapter Six, The Annotated Bibliography Exercise,8
instructor. You and your readers (your instructor and your classmates) should
think about these questions as you revise, review, and add entries:

Are the summaries you are including brief and to the point? Do your
readers understand what the cited articles are about?

Are you following a particular style guide consistently?

If you are including a sentence or two about each of your resources, how
do these sentences fit with your working thesis? Are they clarifying parts
of your working thesis that were previously unclear? Are they suggesting
changes to the approach you took when you began the research process?

Based on the research you have so far, what other types of research do
you think you need to find?

Steven D. Krause | http://www.stevendkrause.com/tprw | Spring 2007

 Chapter Seven
Understanding Self

I begin this chapter with an analysis of self-concept. I have reviewed how self-concept is
organized and the role the parents, teachers and peers play in the shaping the self-concept
of children. Additionally, I have identified the key factors that facilitate self-concept and
self-esteem. Facilitating self-understanding and enhancing self-esteem is the focus of this
chapter. The rationale for enhancing self-image and selfesteem is discussed in the section
Aims and rationale.
Section 1

Why we need a clear concept of self and how it can help.

Self-concept refers to the composite ideas, feelings, and attitudes people have
about themselves (Hilgard, Atkinson, and Atkinson, 1979: p.605). Self-concept is also
defined by Purkey (1988) as the sum of a complex, organized, and dynamic system of
learned beliefs, attitudes and opinions that each person holds to be true about his or her
personal existence. We could then regard self-concept as our attempt to explain ourselves
to ourselves, to build a scheme (in Piagets terms) that organises our impressions, feelings
and attitudes about ourselves (Woolfolk, 2001: p.73).
This model is not a permanent one nor is it unchanging, as our feelings, ideas,
impressions and attitudes changes from moment to moment. Purkey (1988) suggests that
self-concept is:
learned;
organized;
dynamic.
Self-concept is learnt in the sense that we are not born with it; self-concept is
gradually shaped and re-shaped by people who influence us. Essentially a childs self
picture is a learnt one; it depends on the description provided by their parents, family, peers
and teachers. However, it may not necessarily be a correct one, as it is formed mainly by
how people see the child.
When parents constantly tell their child that she/he is naughty, the child as a result
begin to see her/himself as naughty, or when they tell their offspring that she/he is lazy the
child begins to think she/he is lazy. However, it may not be a correct reading, as it is the

206
parents subjective opinion. Thus, two children with a similar behaviour pattern would have
different self-pictures according to the message relayed by their respective parents.
Teachers are the second most influential people in school-going childrens life
(Lawrence, 1996). Like parents, teachers also pass judgements on them. Sometimes the
labels the teachers give the child may be in sync with the label the parents have given.
Sometimes there may be a conflict. Fontana (1995) maintains that both reinforcement and
conflicts carry potential threats. In both cases the child is denied the opportunity for
developing the clear, rational notion of self.
Self-concept is organized. Most researchers (Damon and Hart, 1991) agree that
self-concept is generally characterized by orderliness and harmony. It is this organized
quality of self-concept that gives consistency to the personality. The idea behind this view
is that if it were not organized and changed readily we would not have individuality; we
would lack a consistent and dependable personality.
Success and failure have an impact on self-concept making it dynamic and ever
developing. In the healthy personality, there is constant assimilation of new ideas and
expulsion of old ideas throughout life, which make it dynamic.
Lawrence (1996) recommends that we look at self-concept as developing in three
areas:
self-image;
ideal self;
self-esteem.
Self-image is the individuals conceptualisation of his/her mental and physical
characteristics. The earliest image formed by a child is given by the parents. Going to
school and meeting other adults and children generates new experiences for the child.
She/he now learns whether she/he is good at learning or otherwise and whether or not
she/he is popular. Lawrence (1996) suggests that the more experiences one have the richer
the self-image.
Cooley (1902) developed the looking-glass theory of self. This metaphor illustrated
that the image an individual forms of his/herself was created by the perception of how
others perceived them. That is, the appraisals of others act as mirror reflections that provide
the information we use to define our own sense of self. Hence, we are in very great part

207
what we think other people think we are. This implies that parents, siblings, teachers, and
significant others have a great power in shaping the childs self-image.
The ideal self is the picture we form within us, of the person we would like to be.
Peer comparisons, values of the society we live in, media are significant factors that have
effect on our ideal self, as the image of the ideal self is formed by comparing him/herself to
others. The school child accepts the ideal images from the significant people around
him/her and strives to attain them.
Self-esteem is the individuals evaluation of discrepancy between self-image and
ideal self. It refers to an individual's sense of his/her value or worth, or the extent to which a
child values, approves of, appreciates and likes him/herself (Blascovich and Tomaka,
1991). The most broad and frequently cited definition of self-esteem within psychology is
that self-esteem is a favourable or unfavourable attitude toward the self (Rosenberg 1965:
p.15). James (1890) states that the intervening variable is personal expectation. His formula
is:
Self-esteem = Success
Pretensions
That is increasing self-esteem results when success is improved relative to expectations.
Lawrence (1996) argues that without this discrepancy, without levels of aspiration,
an individual can become poorly adjusted and indifferent to progress. He adds that just like
the nerve impulse is always active, even the psyche needs to be active and it would be an
error to imagine the ideal state as one of total relaxation. For a person to be striving is a
normal state, anything otherwise it would lead to neurotic behaviour.

My focus
The focus of my work is on facilitating self-understanding and enhancing self-
esteem. Kant, the German philosopher, proposed that self-knowledge is the beginning of all
wisdom, thus if we know ourselves well it would give a good foundation on which to build
our lives. This knowledge would enable us to know our likes and dislikes, strengths and
weakness, which would allow us to change our negative behaviour, maintain self-control
and become more responsible humans. Franken (1994) suggests that when people know
themselves they can maximize outcomes because they know what they can and cannot do
(p. 439).

208
Aims and rationale
Asking how self-understanding can be enhanced is a critical question. Here I would like to
explain my work in encouraging understanding of the self.

What I find interesting in the studies (e.g. Fontana, 1995; Lawrence, 1996; Mruk,
1999) concerning self-concept and self-esteem is the practical implication of the theories
suggested by the researchers in this field. To me the teacher, it is clear that I am in a
position to be able to influence a students self-esteem through my lessons and interactions
with them. My understanding of the principles of self-concept and self-esteem helped me to
have a better perception of the process. My relationship with the students benefited because
of the re-newed awareness the children had of themselves. It was a fourfold process, they
learnt about themselves and others as I learnt about them and myself.
I wanted to help the children learn about themselves, to enable them to develop into
competent, mature, self-motivated children. They would then feel more confident so they
would be able to cope with the stress that they faced in school and in life in general. Self-
esteem enhancement, as it has been observed, contributes positively towards both academic
achievements as well as towards personal and social development (Fontana, 1995). Schools
can aid students in these pursuits by helping them to develop the habit of excellence in
scholarship while at the same time nurturing the self-belief that is necessary to maintain that
excellence throughout their lives. Bandura (1986) maintained that educational practices
should be gauged not only by the skills and knowledge they impart for present use but also
by what they do to children's beliefs about their capabilities, which affects how they
approach the future. Students who develop a strong sense of self-esteem are well equipped
to educate themselves when they have to rely on their own initiative.
By the time I started teaching self awareness (23.10.2002), ten months into the
module, the children were competent in understanding emotions. The lessons in self-
understanding and self-esteem enhancement where planned keeping in mind the four
themes suggested by Lawrence (1996: p.29):
1. Trust activities
2. Expression of feelings
3. Positive feedback activities
4. Risk taking exercise

209
 These four topics were adapted to organize the lessons on self-understanding and
self-esteem enhancement, which would target:
Sharing feelings
Guessing the emotion
Remembering good times
Thinking positively
Positive feed back
Taking risk
Making friends
Trusting one another

Section 2- In the Classroom

The uniqueness of me

the process of image-making could help them discover a part of themselves that

mostly resides in their unconsciousness. Art was a way of displaying to the children,

I believe...the dimensions of themselves that I desperately wanted them to discover.

(Eisner, 1993: p. 5)

The design of the lessons on self was based on the celebration of the individuality. I used
puppets and poems to enable the students to witness, experience and understand something
unique. The curriculum on self continued by introducing the children to the concept of
personal metaphor. In the narrative Flicker of colour I have reflected on and analysed the
first few lessons on understanding self.

Finding an entry point to teach the children about the self was a daunting task. I
was perplexed as to how I would demonstrate or explain the uniqueness of each child when
I hit upon the idea of using the poems from Old Possum's Book of Practical Cats by Eliot
(1939). Moving away from the premise of performing a play to watching it, I planned
the finger-puppet show of Cats as I wished to explore a different artistic technique and
incorporate it into my class. I specially designed finger puppets depicting the different cats,

210
like Shimbleshanks, Mungojerry and Rumpleteazer, Old Deuteronomy and Rum-Tum-
Tugger.

Mungojerry and Rumpleteazer Old Deuteronomy

I used puppets (Slade, 2001) as:

Puppets encourage concentration.
Their toy like-likeness makes them beloved.
They stimulate imagination.
Puppets can become life-like in one sense, but are seldom photographic to look at,
and so nearer Child Art.
They can be used as a half-living visual aid.
After performing a puppet show, using the music from the Broadway musical Cats
I presented the children with identical cat masks to explain the concept about similarity.
The audience then became participants again as the children became feline creatures.
The children subsequently made masks for themselves. Although each child made
personal masks no work really displayed great individuality. The reason for this is everyone
copied his/her friends ideas. At this stage in the lesson on self-awareness, I was not unduly
worried. This was the just the second class and I had a long distance to cover and plenty of
time. The children had begun their journey of exploration carried out by a process of
imaging.
To help them find their personal metaphors I explored the possibility of working
with Native American names (Bassnett and Grundy, 1993). After explaining how natural
images are used to describe the character of a person I asked them questions to facilitate the
process of imaging. These questions guided the process:

211
 What do you like?
How do you see yourself?
What are your favourite animals and birds? Why do you like them?
How would you describe your friend?
What are the characteristics of your friend that appeal to you?
With help of the students input I compiled a list of descriptive words, which would help
them acquire a new name with an enriched image (see Table 14, Table 15 and Table 16,
Table 17).

Table 14: What they like.

What Samir Chandani Lali Manni Mukul
they
like:
To do Cricket Dodge -Ball See
Badminton Educational
places
To do Read comics

To eat Idli Finger rolls Fruits

To drink Pepsi Pepsi Limbu pani Pepsi

[lemonade]

Animals Leopard Dolphin Dinosaurs Deer Monkeys

Cheetah
Birds Parrot Owl Peacock Birds

Flowers Rose
Colours White Red Purple

Table 15: What they like.

What they Kanha Snehal Pratik Sarla Nihar
like:
To do Running Cricket Kho-kho Play Running
Jumping Running [an Indian Jumping
sport]
212
To do Listening to Talking Talk Talking
stories reading Sleep Jokes
Stories
To eat Idli Masala dosa Idli Pav Vada
Table 16: How their friends saw them.

Mukul Samir Chandani Lali Manni

All the time doing Thinks about herself Shy Not Shy
something
Shy Naughty Slim like a model Colourful

Talkative Kanjoos Talkative Talkative Talkative

[Miser]
Naughty out of Chakri[a fire cracker Proud
class which moves on
circles]
Monkey Dolphins Likes
Peacock Dinos
And birds
Plays well Ashanti Graceful Full of life Smiles
[Riotous and
disorderly]
Naughty Not bossy
Distracting Smiles Troublesome

Like her mother Long hair

Busy Bee Graceful Rose, Lively Dinosaur
Mr. Phool-jadi Talkative Rose, Little
Rabbit [sparkler] Proudy peacock, Colourful Birdie Princess
Dancing Dolphin

Note: the names chosen are boxed.

213
Table 17:How their friends saw them

Kanha Snehal Pratik Sarla Nihar

Bird Tiger monkey Eagle Likes birds and King/ royal
flowers
Slim Chatter box Shy Fat Shy

Fast Talks less Round cheeks Proud

Smiles a lot Smiles Clever Anxious
Shy Talks a lot in the Shy, kind Talks sensible Likes to sleep Mischievous
school bus. things Talks a lot
Out of class he is
talkative, naughty, full of
life
Giggles Proudy (sic) Smiles a lot
Laughs Simple
Is troublesome

Forgetful Draws well

Smiling bird Talking Moon Shy Painter
Swift Eagle Chatterbox Blue Eagle Princey Anxious lion
Naughty bird Monkey Royal Bull dog
Lively trouble Raat Rani
Running leopard [Lily of the Lion King
Night]

Note: the names chosen are boxed.

A flicker of colour

The material of the story is taken from the transcript of five classes (Class 20-24) held on
the 22nd, 23rd, 25th October and 16th and 23rd November 2002.The puppet show and the
mask making and using masks classes were held in October 2002, during the Diwali
vacations. Sarla narrates the following story. I choose Sarla as the narrator to re-present
my analysis, as it was she who used masks and the understanding of the self again, in
Class 48-18.10.2003. Nihar, Lali, Manni, Samir are the students and the Wise-one is me.
The lines sensible everyday names, names that never belong to more than one cat, a
name that's particular, a name that never belong to more than one are lines quoted
from the poem The Naming of Cats by Eliot (1939).
In the narrative A flicker of colour I have reflected on the lessons in which:
I performed the puppet show, with finger puppets (In the last meetings she had cats
dancing at her fingertips),

214
 the children performed with identical cat masks ( we begged her to makes us into
cats. but we all looked same and behaved in a similar manner.)
the children made individual masks (.. Then we gathered together the next day, and
tried to created new identities for ourselves)
the children played a American Indian naming game(The Wise-one suggested we
should begin by giving ourselves unique name)
The lines She would often carry music refer to the cassette player I carried to play
music during the sessions when the children did the walk and statue activities, the
manuscript- my journal, which I always carried to class and food- refers to the snacks we
partook during longer classes.

All the travellers sat on the floor crossed legged and wondered what we
would do next, when the Wise-one came in. She was older than us, not
grey haired and all, but older in years than the rest of us and therefore
we decided she was wiser than us. She would often carry music with her, a
manuscript...always, and sometimes food for us. She is kind and we like
to be with her ...we trust her.
The Wise-one is a storyteller and magician. Even when she talked
everyday, ordinary stuff she made them magical and we liked to listen to
her. Sometimes, Samir flew around like his namesake, the wind and created
chaos, but the Wise-one never got angry and was patient with him. She
could control the wind! The Wise-one made us realise how important he was
to our group. He was the first of us to do something well. So what, she
said if he does not do anything now besides ruffling us up. The wind-
Samir would settle down one day and do something creative again.
In the last meetings she had cats dancing at her fingertips. These cats
came from a far off place called England; they actually belonged to
someone called Eliot. With her husky voice she sang songs of cats that
were fat, were wicked, were thieves, were good and were proper. These
cats experienced affection, pain, warmth, and hatred, just like us. We
thought she was entertaining us but No!! She was helping us see
ourselves.
We liked her songs so much that we begged her to makes us into cats.
Which she did, but we all looked same and behaved in a similar manner.
That did not suit us. How could we be the same? We were different.
Chandani said, he is a boy and I am a girl!
We are wearing different clothes, said Nihar.

215
Our hair is different, said someone else.
The Wise-one agreed and said we could be as different from each other as
much as we know ourselves. We did not understand what that meant but we
were excited just to know we were different. We impulsively threw away
the masks which made us same. Then we gathered together the next day, and
tried to create new identities for ourselves...but we were still pale and
looked nearly like each other. How were we to become our real vibrant
selves?
The Wise-one suggested we should begin by giving ourselves unique name.
We all had sensible everyday names, but the Wise-one said Eliots cats
had names that never belong to more than one cat. She said even we
needed a name that's particular,...A name that's peculiar... a name
that never belong to more than one...
She then acquainted us with a custom of the natives of America. The Red
Indians had names like Running Brook and Flying Eagle. We had not heard
about these people and were interested to learn more. So the Wise-one
told us about a girl, who had blue-eyes, like the clear brook and she ran
fast...her name was Running Brook.
We all sat down and started talking about what we liked and did not
like...it may sound to you like it was a quiet and organised process. But
do you know what it sounds like when all eight (Mukul and Pratik rarely
travel with us) are talking? So much noise and excitement! I like this
and I like that...No, I said that first...but Wise-one I like such
and such too!
To help us remember what we said the Wise-one wrote for us, in her
manuscript. We found that all of us liked to run, and jump, play cricket,
eat chocolates and drink Pepsi. Did this mean all of us would have one
name?
The Wise-one, not giving up, went on asking us questions about ourselves
and we went on answering... We began with different animals and went on
to details about our character.
Kanha runs fast, smiles a lot, is shy, he laughs a lot, he is trouble
some, said Lali.
He is forgetful, said Nihar.
Putting all his characteristics together we thought of the name Naughty-
bird or Lively-trouble. However, Kanha did not like the names and we left
it at that and went on to Nihar.
Nihar is an enigma, he is so quiet, he always sits a little behind when
we sit in a circle. Even when we sit all huddled up he is always in the
back. He doesnt do things with us but sits back and watches. When we all

216
met for the first time he would not even tell us his name. Just
imagine!?? Telling your name is the simplest thing possible. We all do
know him now, after eleven months of travelling together, but he is still
very quiet and shy. He has a fascination for things that are grand. Good
cars, nice bungalow, lions and peacocks- they are our national animal and
bird, he said. His mother told the Wise-one Nihar ko proud se reheneka
mangta meaning Nihar wants to live with pride and dignity. He is good at
drawing.
Should we name him Shy Painter?
No, we all protested ...what a boring and drab name...
Royal Bulldog?
No, no. Nihar is not ferocious or ugly.
We went on concocting names...at last we reached a name for him...Lion
King.
We had begun an exciting journey of self-discovery...

Annual program
Most schools in India have an annual program, after the half yearly examinations (Chapter
5, p.145). It is an important function attended by the families of the student and usually
showcases the talents of the students studying in the school.
In this section I moved towards a product dance- drama (Chapter5, p.145) however
lessons followed a process-oriented mode. I have described and reflected on the
particulars of the process. The account includes a step-by-step description of the process of
making and performing and ultimately assessing the dance-drama. I have included my
reflections, in Times New Roman Bold, as they occur, as they are interrelated and
interactive with the process.

I sought to encourage the students to develop a strong sense of themselves as

capable persons and that each ones ideas are valuable. The school annual program, though
not timed by me, came at an opportune moment. I decided to take a calculated risk of
agreeing to execute a presentation for the annual program.
The children by now had:
Attended drama class for nearly twelve months. Showed an understanding in drama
skills and were at an advancedbeginners level.
Fairly strong competency in emotional understanding.

217
 Revealed a growth in their creativity.
Moved at a good pace towards self-understanding.
The lessons were geared towards using drama to teach pertinent current issues and
building on them in an on-going way. They were partially planned in that I had an idea of
what I wanted as the final shape. I had wanted to use the concept and design of street
theatre (see Glossary) for the play even though the play was to be performed on a
conventional or proscenium stage.
The children had witnessed a rehearsal of another group of students who were to
represent the school for a street theatre competition (Chapter 5, p.146) where they watched
me work with their older school friends. After the rehearsal my students and I had a
discussion about street theatre. Therefore when we started working for the annual program,
the children had a clear idea about street theatre.
The making of the annual program play had a feel of an open lesson, as the initiative
of the direction of the drama and the dialogues was negotiated with the children from the
outset. I had planned the lessons in such a manner that I would lead the children to the point
where the drama would need to be focussed and brought to a resolution. I had taken a
flexible route as I could not in the beginning anticipate with a degree of certainty what the
possible outcome of the play would be. I was prepared to bailout the children if need be and
bring the drama to a resolution. However, I planned to do that only at the very end.

Introducing the topic:

I invited the students to discuss the problems faced by the people of our country.
Through discussions and reflection I sought to assist them to gain awareness of their role as
citizens of India and decided which of the problems faced by Indians was most significant
to them.

Swaroop: Tell me what are the problems in our country?

Lali: Miss all are fighting, they are not moving (sic) in peace.

Lali was now beginning to express her views and after a year somebody besides Manni
answered first (e.g. Chapter 5, p.105, Appendix 5, p.30-1).

Nihar: Atankvadi [terrorist].

218
Sarla: Population [population explosion]
Swaroop: You said population yes, and you even said pollution?
Sarla: Yes.
Swaroop: Samir?
Samir: What?
I repeated my initial question, however he did not have an answer, he said, I no facing.
Meaning he did not have any problems, nor did he see any problems around him.
Not listening during reflection time was a problem I faced with Samir.

Opening the drama and enlarging the argument:

We reflected on the problems faced by the India at great length. The children were
concerned about the lack of cleanliness, poverty, the rising prices, computer viruses and
violence against animals. However, the problem upper most on the childrens mind in
January 2003 was terrorism. In 2002 we were faced with the burning of the train in Godra,
Gujarat (Vakil and Gohil, 2002) and the subsequent violence in that state which started in
February 2002 and went on till August 2002. That brought terrorism near our doorsteps.
The repercussions of the violence were felt strongly in Mumbai. In 2nd and 6th of December,
2002 there were two bomb blasts, one in a bus outside a crowded railway station and the
second at a McDonalds restaurant (Times of India, 2003). The fact that they should do
something to change their lives was a suggestion that came from them.

Lali: We should keep our surroundings...

Snehal: ...World clean and peaceful.
Swaroop: What is peace?
Manni: Peace is when we be friends with everyone we should even consider
them [people of different caste and religions] as a person
Swaroop: How can we as children make a difference?
Chandani: We should go to the temple [and pray for peace].
Sarla: give [donate] money.
Manni: We should not fight.
Snehal: We should share
Chandani: We should share our happiness with others.

219
All the while Samir went on distracting others, especially Nihar. However, Nihar in
spite of the distraction would answer questions directed to him, and would also give
his opinion to statements made by others. Manni at one point even commented that
Samir should do whatever he is doing during recess time. All the children were
extremely involved in the discussion and could pick up the links even though I had to
often stop to correct Samirs behaviour.

Transcript: Class 27-7.1.2003

The pledge:
How would I like my country to be? was something I suggested the children think about
and answer in the next class. I was pleased hear the childrens observations in the following
class.
Manni: First we have to bring peace and how do you say Shanti [spiritual peace]?
Peace and shanti in our country then violence will go away.
Chandani: Pollution is a problem and because of that hole[in the ozone layer] is
getting bigger we should do something to stop pollution.
Nihar: Miss, the computer virus.
Sarla: Miss, what about the HinduMuslim fighting we are all brothers and
sisters are we not?
Manni: Yes, Miss what about Gandhijis India?
Samir wanted to know what the name of his character would be.
Swaroop: Your name is Samir. Everybody in the play uses his or her own name. So you
have to say I am Samir. I dont like what is happening in my country. And I
will do this and that to make our country better. You are going to be the
scriptwriters.
Transcript: Class 28-8.1.2003

The play:

The play began with a bhajan [prayer/hymn] Raghupati raghava raja Ram, which
Gandhiji popularised during the Indian Independence movement. All the children
walk in chanting.
Swaroop: In this play there is no story but it has a very strong message.

220
 Manni: like Gandhijis three monkeys (see Glossary); see no evil, hear no evil,
speak no evil.
Transcript: Class 28-8.1.2003

Mannis idea of the Gandhian message of truth verses evil was the reason I
chose the bhajan, which introduced the theme of the play and the actors. The
lines Ishvara Allah thero naam, subko sanmati de bhagavan translated into
English mean; Ishvar and Allah are the names used by Your devotees, the
Hindus and Muslims, but You are only One, so it does not matter what name is
used to address You. God [bhagvan] grant all true knowledge and goodwill
[sanmati]. This hymn attempts to understand the mentality of truth or sense of
being, around which Gandhiji spun his life and his unique struggle.

The children represented the evils/problems that are damaging our environment and
country. They try to catch Manni, playing the part of the white dove, representing
peace, and persuade her to go with them.
Sarla and Chandani: I am menghaee [high prices and cost of living] I will not let
you survive.
Nihar and Samir: I am a computer virus; I will corrupt your systems.
Snehal: I am a thief come with me I will teach you to steal.
Kanha: I am a terrorist, join me and we will destroy the country.

Manni, the white dove, flies around on stage trying to dodge these tribulations, and
finally sits down to meditate. However, here too she finds no peace.
Lali: I am a great religion, come to me I will give you freedom.
Manni: No. I do not want to follow your ways. I do not need any one great religion. I
am a Hindustani [Indian] and humanity is my religion.
(Addressing the audience)
This is not our Hindustan [India]. Do you know how we want our Hindustan
to be?

The children then walk up one by one to the mike which is placed centre downstage,
introduce themselves to the audience and convey their messages to their friends.
I am Snehal; lets keep Mumbai clean and green.

221
 I am Lali; how we live is important, not how long we live.
I am Pratik; I think we are all equal in Gods eyes, we should not believe in castes and
races.
I am Nihar; I think we should not fight with each other we should live in peace.
I am Sarla, since my childhood I have been taught not to tell lies. I think honesty is
the best policy.
I am Samir; I wish I could remove poverty from our country.

Finally all the children join hands and dance to a song on national integration.

Assessment

The children performed extremely well in the annual program. However, that was
not the question upper most in my mind. What was important was the fact that they went on
stage and performed with abundant energy and enthusiasm in front of an audience that
consisted of over 1500 peers, teachers and parents. This act in itself showed they had made
great gains in their self-confidence (cf. Appendix 2, p.4). It was the first time all of them
performed in a proper theatre, as up till the fifth standard the children in this school, only
performed on a makeshift stage, in the school compound. Five of the ten had never
performed in front of an audience of any size. The only child who did not go on stage
during the grand rehearsal and the final show was Mukul. No amount of cajoling by his
classmates could persuade him to step on stage. On my part I did not compel him to
perform as I did not want to distress him by forcing him to do something he did not want.
At the same time I did not want him to feel utterly useless therefore I asked him to help
backstage with the music system.
The children filled in two assessment sheets after the annual program. The first
sheet: part 1(Appendix 11) was designed to encourage reflection-on-action, during the
making of the play and behaviour prior to the performance and during performance. In this
sheet the children assessed their classmates behaviour and their own. The students had to
indicate agreement or disagreement with each item on at a three-point scale: Agree (scored
as 3), Not Sure (scored as 2) and Disagree (scored as 1), higher scores indicating favourable
attitudes. The sheet was developed to provide an assessment of attitudinal and behavioural
changes in students. The items included reflections on the childrens achievement in:

222
 self expression;
self-acceptance;
self-awareness;
acceptance of others;
awareness of others.
I found that they showed significant understanding in completing this questionnaire
as compared to the questionnaire filled up by them at the end of Cycle 1. I believe this is
because:
The questions framed in this sheet (Appendix 11) were more relevant to their
immediate experience. Like:
Did your friend pay attention during rehearsals?
Gave ideas during the planning stage.
Listened to the teachers instructions.
This was the third self-assessment sheet the children were filling up and could now
understand the concept of self-assessment.
To make the assessment more effective I talked the children through the sheet,
explaining the questions one by one.
Swaroop: I had instructed you not to give instructions on stage, so if you gave
instructions on stage you did not follow my [the teachers] instructions.
Who did not follow my instructions on stage? Remember most of you
were nervous before going on stage? I was nervous.
Shenal: Yes, Miss? (With utter disbelief, the fact that I was nervous surprised most
of them).
Swaroop: Yes, I was very nervous who else was nervous? I remember Manni
saying lets not go on stage...----
---I want to explain the last question. Remember when we rehearsed our
play. We worked in narrow rooms like the geography room later we
worked in a normal classroom (see photograph a. and b. on p.148-9)
then we performed on stage, which was big. Who adjusted to working
on stage? Remember there were some children who stuck to each other in
spite of my saying spread out spread out.
Transcript: Class 34-1.2.2003

223
 Talking them through the questions did not to colour their views. All the children
expressed their opinion. This can be observed in the scoring sheet, all of which have a
distinct scoring pattern (Appendix 12). The only question that has an identical pattern of
answers was the group liked working with him or her, where eight children obtained an
ideal score of 27 points, and Mukul and Samir attained a virtually ideal score of 25 points.
This is because in actual fact everyone enjoyed working with each other.
The second sheet (Appendix 13) the children filled concerned, reactions of their
friends and significant others and how the children felt on receiving the feedback.
I met the childrens class teachers and music teachers after the program to assess the
change in the childrens behaviour after the program.
Music Teacher1: I cannot believe it was Nihar on stage ...and he spoke so well! Previously
Nihar always stayed in the background and tried to disappear into the
crowd. He showed total involvement for [sic] this play.
Music Teacher 2: Manni was superb! I liked the way she was so in charge of the play and
so confident.
Music Teacher 3: All the children performed so well their musical drama was the best in
the annual program.
Their class teachers filled in a short questionnaire (Appendix 14) in February 2003,
three weeks after the annual program. Their answers helped me to understand the change in
the students behaviour in school.
Eight of the children showed a marked improvement.
Kanha, who according to the teacher was not confident before and was an introvert
had become confident, started speaking in class and took active part in class
activities.
Samir was shy and wandered aimlessly and would not write unless he was
watched (teachers words). Now he loves working in a group, is friendly, more
confident, good at project work, got himself upgraded with latest happenings
[annual program]. (Samirs class teacher)
Chandani who was not confident initially became more confident and had also
showed remarkable change in her academic work.

224
 Manni had become more confident. She had now started making an effort to answer
in class; she was not worried about being incorrect. Her class friends/peer group
accept her as an achiever, they listen to her when she talks and respect her opinion.
Mukul was shy and moody and avoided contact, but now tries to answer in class and
is friendly. Has become more self-assured and asks the teacher to repeat or explain
something he has not understood.
Sarla is more confident and is happy in class. She was greatly motivated by the fact
that she played an active part in a highly successful venture in the annual program.
Lali always complained in class now she is more obedient and responsible.
However there is no sharp change, the change was gradual. Her class teacher
narrated an incident when Lali came to the staff room to give her a card, Lali
paused at the entrance, squared her shoulder, held the head up and walked in
confidently.
Nihar was disinterested in class work and fidgety; now he is a friendly, tries to
complete his work in time and is gaining in self-confidence.

Snehal and Pratik did not show any change in school. Pratik was very irregular in
his attendance, at the same time he was comparatively more confident to the other children
even when we started in January, 2002 as can be seen in the creativity assessment
(Appendix16). Snehal too, was a good and clever boy- as described by his parents in the
first interview in January 2002. He was regular in his attendance and from the beginning
was well behaved, he answered well and as one can see in the creativity scores (Table 23,
p.266) he started of on a medium level of some what present-average score n=3.1 and
progressed only to moderately present average score n= 4.1.
All the parents were extremely pleased by the way the children performed on stage
and changed the way they viewed their childrens abilities. The children on the other hand
experienced a boost in self-esteem with the complements received from their parents and
significant others. These are some of their reflections:
Chandani: I was playing passing the parcel at a party and as a forfeit auntie asked me to
perform the dance I did in the annual program because everybody said I did
dance well.
Sarla: My Grandmother said my play was soooo good, now my uncles want to see

225
 me act. My mother said she was so proud of me after we travelled home on the
school bus together, as all the children complimented her on my achievements.
Manni: My mother said I was excellent I feel so proud.
Samir: Everybody was clapping for our play. My friends say they dont know how I
could act and dance so well.
Lali: I am feeling proud of myself, because my teacher and parents said I was
excellent.
Transcript: Class 34 -1.2.2003

Making the play for the annual program and performing was a critical event in the
childrens learning (Chapter 5, p.145), it improved the childrens attitude in areas of self-
expression, self-acceptance, acceptance of others and self-awareness. It enabled the
students to become more accepting of others ideas and feelings and increases their
awareness of their own feelings and motivations. Importantly for the underachieving
students, it encouraged them to develop a strong sense of themselves as capable persons
and that their ideas were valuable. These skills are important not only for school
achievement but also personal development, improved relations with others, readiness to
learn and ultimate success.

Deepening understanding
The curriculum on self-understanding proceeded with trust walk exercises with the view
to help the children practise co-operation with each other, to help them empathise with
each other and to develop trust in each other

The focus of my research is not just concerned with children as they are at the
present, but as they will one day will become. The individuals notion of him/herself is
undergoing constant change, particularly in childhood. When the children in my drama
class experienced sudden success, their self-esteem was enhanced. However, I think it was
a gradual change which was initiated by emotional understanding and then further with
understanding of self. I perceive it as a movement towards self-maturity: that is, towards
self-concepts that are realistic and self-accepting (Fontana, 1995: p 244).
Psychologists like Erikson (1980), Rogers (1969), and Maslow (1970) suggest that a
well-balanced person is:

226
 aware of his/her worth;
able to empathise with others;
able to relate to others warmly and with an open mind, instead of using them for
personal reasons.
It would be incorrect to think that maturity only comes with the initiation of
adulthood and that mature children are old before their time. Mature children are those who
development is appropriate with their years and experience and show the qualities
suggested above (Fontana, 1995).
I used the trust walk or blind walk exercise (Appendix 17) with the view to help
the children practise co-operation with each other, to help them empathise with each other
and to develop trust. Reflections during these trust lessons played the most important role
in understanding self and others. Without the use of discussions during reflection time the
trust walk exercise waters down to an ordinary game.
I paired the children, one child blindfolded and the other acting as the guide. If the
child was uncomfortable with the idea of being blindfolded I asked them to merely close
their eyes. I did not want this experience to be frightening to the children and/or make them
feel like a victim (Gorchakov, 1954; Bolton, 1998). For the first exercise I did not allow
any child to take his/her partner outside the classroom for reasons of safety. However, to
make it challenging I arranged the small, nursery room chairs around the room to act as
obstacles.
I asked them to lead the blind partner by the hand in complete silence. After five
minutes, I asked the participants change roles. In the beginning, this exercise did not
work as everyone kept opening their eyes and peeking through the blindfold and
thereby defeating the cause.
To make the blind walk a challenge I asked them to take the blind partner round
the room and count the number of times the blind person banged into an object. This
not only made the game exciting but also made the guide treat the blind person more
carefully.
I then introduced oral communication into the game, where the guide could talk to the
blind person and ask him, for example, to stop, or be careful and walk slowly.
The most exciting blind walk game was played in the school compound, as it covered
a huge area of different levels and surfaces.

227
 The hall on the ground floor by itself was interesting as it had pillars, a snacks
counter and chairs.
There were different levels, like one step down, three steps down and a short
flight of steps, too.
A small childrens playground with swings and slides and monkey bars.
Just near by, in the safety of the compound was a parked school bus.

Kanha carefully leading Snehal.

The blind walk exercise performed on 15.2.2003 was the third time the children played
this game. By now they were getting comfortable at the thought of being lead by their
classmate.
Swaroop: How do you feel when you were blind?
Snehal: I was scared I would fall down.
Swaroop: Did feel scared the first time you did the stairs?
Sarla: Yes, Miss.
Chandani: I got scared ...I was nervous, because she took me in the garden. She showed
(sic) me all the toys ...and took me on the slide.
Swaroop: What about the guides? What did you think at that time?
Sarla: If I did not take care of her I will fall her [make her fall down].
so we should be careful.

228
 I help my Ba [grandmother]
Chandani: who is blind. So we have to care for him (sic). If she falls down we have to be
responsible. We have to catch them if they are falling down. We must try to
make the blind person feel as if they are seeing.
Snehal: I kept thinking that you must not fall down. (He acted as my guide)
Swaroop: Sarla was very careful. She made her partner touch the wall while coming down
the stairs.
Transcript: Class 36-15.2.2003

While we were playing in the school compound a girl (pseudonyms: Shruti) who
had come to school for some reason on a holiday asked for permission to join in. She was
so fascinated by the game that I allowed her to join in. Her reaction to Sarla being her guide
is worthy of note, as she merely identified Sarla as a student of the school. They had
however never met before as they were in different standards.

Swaroop: What did you feel when Sarla took you around the compound?
Shruti: I felt nice. I did not feel scared. Sarla was taking good care of me.
Swaroop: Yes, I noticed the way she held your hand in hers
Transcript: Class 36-15.2.2003

The fact that a stranger felt comfortable at being led in the blind walk by Sarla
speaks volumes for her ability to show care for a person.
In October, nearly twenty-one months after we stared the drama module, imbibing
Heathcote's idea of teaching-at-risk (in Bolton, 1986), I handed the power over to the
children. The class and I negotiated that each child would conduct a class of his/ her choice.
Sarla chose the mask making and the understanding of the self lessons for her class. She
took the students back to the mask making class and each of the children made their own
mask after discussing and reflecting on the class in which they chose their personal
metaphors (secret identity see p.230). The following story is in Sarlas voice as she
conducted the class (see Chapter 5, p.164). The story is based on the analysis of transcript
and data of Class 48-18.10 2003. I remember the old days... is a critique of the Sarlas
development from the days when she followed me around (see Appendix 3, p.12-3) to the

229
time when she confidently performed in the dance-drama, directed the same (Chapter 5,
p.150), and conducted a successful class in Understanding Self.

In which Sarla becomes the magician

I hope you like listening to stories... this is a story of a newly

initiated magician. I am the young magician and my name is Sarla. I have
an assistant, the Wise-one. I was going to perform magic for the first
time and help all the others to find themselves.
No... the Wise-one had already began the chanting of the spell a long
time back. What I was going to do is see if we could find ourselves
without the help of the Wise-one.
I had decided to do the magic of the masks. My assistant, the Wise-one
had got all the materials ...paper plates, paints, brushes, gum and
scissors....
We all knew the magic of the masks; we had done it before when we
became cats. But I remember that time we were not colourful. We were pale
reproductions of ourselves. We looked nearly like each other.
What an important job I had today! Nearly twenty-one months since we
started our journey. I was scared ...nervous, supposing no one listened
to me. The Wise-one could control the wind-Samir, bring Nihar-the fog out
into the sun, and help Lali-the little one to grow. And me, Sarla the
simple one, I was not plain any more. I am sure you can understand that
performing magic in front of the other ten travellers was a task I could
not have even imagined I would be able to do in the beginning of our
journey.
I remember the old days when I followed the Wise-one everywhere she went,
I was like her puppy dog. I did not allow anyone to hold her hand or sit
next to her. In the magic circle, when we had our meetings, I had to sit
next to her. Now I was slowly taking tiny steps and moving away on my
own, like a young bird ready to fly out of the safety of the nest.
The magic of masks helps convey us to another world, stirring up our
powers of imagination in the search of self-knowledge. At this point I
must make it clear, masks could be used as a disguise, like we used them
to become cats. Masked could be used to hide ones self. But I have used
the magic of masks to help my fellow travellers find themselves again.
The Wise-one suggested we all try to geometrical designs and colours we
liked. The Wise-one is growing old and forgetful ...she forgot we had
already decided our secret identity. We were Mr. Rabbit, Raat Rani

230
[Rani means queen] wild animals like Running Leopard and Chatterbox
Monkey.

Lotus. Mr. Rabbit.

Sunflower. Queen of Hearts.

Nihar insisted he was the national flower of India, the lotus, because he
did not want to become a cat again ... Shenal transformed into a
sunflower ...Mukul became a rabbit ...Samir and Kanha ...became wild and
exotic animals and I metamorphosed into a queen of hearts...
The magic worked! This time we had become brightly coloured and had very
definite identities. We were not like the pale sun on a cloudy monsoon
day. We were like the vibrant sun in the Indian summer.

231
 145

Chapter Eight
The Geek Squad

B rands dont get much bolder than The Geek Squad, the technology support
company. From a one-man start-up in 1994 the company has grown to become
an internationally admired brand offering technical support for home users and
small businesses when their computer hard drive crashes or they get a virus. But
that was just the start of their ambition. The Geek Squad, with just 100 locations,
merged with US retail consumer electronics brand, Best Buy, in 2002 and absorbed
Best Buys then 700 stores on its path to global domination. If there was ever a
case of a minnow swallowing a whale this was it. Today the company numbers
well over 1,000 stores and 150,000 employees, and has recently bought 50 per
cent of the UKs Carphone Warehouse to bring the Best Buy brand to the UK.
Despite the 40 per cent year-on-year growth rate, the company feels small,
with Robert Stephens, the founder, still actively involved in hiring and training
new people. One of Roberts favourite sayings is: We only have temporary custody
of talent, and the enormous lengths that he goes to in creating a bold, zany culture
where geeks can feel at home is testament to this belief.
The whole organization dramatizes the Geek Squad experience through language,
organizational structure, titles and even the uniforms the Special Agents wear. It
has created an enormous word-of-mouth effect. As Robert says, Marketing is a tax
that you pay for being unremarkable.

Bold practices

The Geek Squad

The Geek Squad brand promise is Well save your ass.
New hires are presented with a Mission Impossible task to test their
ability to navigate around the city in which they will be working.
Organizational titles, structure and roles are all designed to change the
way people think about themselves.
Above-the-line marketing until recently was a measure of last resort, with
word-of-mouth recommendation being the primary source of new business.
146 Chapter Eight The Geek Squad

Despite never being visited by customers, Agents working in the call centres
all wear the Geek Squad uniform to preserve their identities.
The words Geek Squad are moulded on the soles of the Agents shoes so that
in the event they are walking in snow they leave an imprint of the logo as free
advertising.

Robert Stephens
founder and chief inspector
Robert Stephens started the Geek Squad Robert Stephens
in 1994 with $200. His vision for the com- 2011 The Geek Squad
pany is global domination.

The thing that was uppermost in my mind

when I conceived The Geek Squad was
starvation.
I tell people that the best thing that
ever happened to me when I started my
company was that I had no money be-
cause otherwise I would have paid some-
one else to come up with a brand and a
name and a logo. The reason start-ups
are often so innovative is because of
what they dont have; what they dont
know; which allows them to see things
through fresh eyes and take chances.

The Geek Squad mission

When I started out I had never created a company before, but, you know, Id heard
some of the things you were supposed to do. You should have a mission state-
ment. Well, whats a mission statement? Okay, well, it really clarifies why the
company exists, what its ultimate end goal is. My problem is that most mission
statements are so corny, and yet, when you have no money and youre about to
drop out of college for the second time, you kind of figure, hey, this had better
work or I am going to have a pretty nasty reputation as a quitter or a person who
doesnt really finish what they start. So I said, okay, really, the goal of every com-
pany is global domination.
 Robert Stephens founder and chief inspector 147

I believe if Microsoft had just told everybody We would like to dominate the
operating system, and said so in that clear, unambiguous language, they would
have at least earned respect for clarity and consistency. So were very honest, our
goal is simple: Complete global domination of the technology services market. Of
course, this doesnt mean physical domination, rather domination by reputation.
You need to be clear about what your goal is and thats ours. Notice that our mission
says technology services. Our plan was always to support screens and networks
but when I first started in 1994, there werent many networks around so it was
mainly screens. Now there are screens in your pocket, on your mobile phone,
screens in your car and screens in your living room. Now you can kind of see
were just going to be doing the same thing for the next 2030 years as more
and more screens proliferate on more and more networks.
In the early 1990s, as normal people started buying computers, there was such
a rush to keep up with the demand that Dell and Microsoft and the other manufac-
turers just focused on moving boxes and product. Service was an afterthought.
But when commoditization occurs, as it does in every industry sooner or later, service
becomes a great differentiator. Now hardware commoditization has occurred and
it is increasingly happening with software. Commoditization and complexity are
creating too many choices for consumers, too many buttons to push, too many soft-
ware updates to keep track of, and customers like their lives made simple: one
number to call when they need help. So that was my big idea.

There is what you do, then theres how you do it

My intuition was my research but whenever I fixed computers to make extra
money in college, I did start noticing a pattern. First, there is what you do in busi-
ness and then secondly theres how you do it. What you do is your function but
how you do it becomes your reputation. I noticed that showing up on time is a
form of advertising and being polite is a precursor to word-of-mouth recommenda-
tions, and explaining what youre doing and not talking down become just as
important as fixing the technology. And it wasnt just that the manufacturers like
Dell Computers were not offering enough customer support, it was also that the
technology support companies who were supposed to provide service were rude,
showed up late, did not keep their promises or just didnt do the job right the first
time. This created the opportunity for one company to come in and become the
brand that represented the ideal of how it should be. And that was really the basis
of the business model.

The DNA
Then I said, Why dont I give it a name thats generic and that implies intelligent
people banding together in the service of whatever support you need and the
148 Chapter Eight The Geek Squad

name itself is the advertising. It tells you everything you need to know about the
brand, the organizational culture, what the people are like and what they do. Its a
kind of paradox between the individual smart person the geek but then the
power of the network the squad. No one individual in this company will know
everything but together they know everything. And so that was the genesis of the
DNA. Once I went down that path, it kind of liberated me to say Im going to go
with a name that is also humorous. So we became The Geek Squad.
You know, humour is a subtle form of self-confidence. If you poke fun at your-
self, it means you have knowledge about yourself and are comfortable with what
you are. It is also very useful in my industry, which often gets criticized for being
arrogant, talking down to customers and not revealing their knowledge. So this is
going to be a company that is comfortable with itself, it knows that the geek is in
the social ascent and the jock is reliant on the geek to get their technology to
work. But the way to win over the public is to make fun of ourselves because a
company can humanize its brand by using different forms of humour. We employ
a kind of British dry wit, which is essentially comedy with a straight face. The
names already funny, so were not going to be silly when we get to your home. In
fact, it works even better if we dont acknowledge the humour at all: we just show
up on time, were dressed smartly, we get the job done efficiently. So behind
what may look like marketing gimmicks are some very intricately woven
brand elements that are inseparable from the actual function we perform.

The Geek Squad promise

I never did much advertising in the beginning, but somebody gave me free adver-
tising for fixing their computer. So I had to create an ad. Ive got the name The
Geek Squad, Ive got the logo. But we needed a tag line and kept coming up with
stupid saccharine phrases like We solve problems, We help you and so on. This
is where companies go wrong they dont take a bold enough point of view.
Thats what brands are for, to make you distinct from other entities. So I said,
Well save your ass, and that became our brand promise.
I often say that marketing is a tax that you pay for being unremarkable. If
people know youre using traditional advertising, youre going to pay traditional
advertising rates. But if you come up with new ways to get your name and
message out there, the chances are that prices have not yet been set for those
new channels and therefore they are less expensive. In fact, now having gone
from one employee to over 19,000 in the Geek Squad worldwide, my advice is
and parents will understand this even if you have the resources to provide cash
for the organization, you should occasionally starve them so the organization
learns to fend and learn for itself. The danger is, the bigger you get the more
resources you have and the more you get lulled into a very lazy way of thinking
that is dangerous.
 Robert Stephens founder and chief inspector 149

Keeping the creative hunger

As we have gotten larger and merged with Best Buy and now Carphone Warehouse,
keeping this creative hunger is very important. The way we sustain our culture is
through a natural network effect. I think we all would agree that social networking
is nothing more than a technological version of a very primal human need to con-
nect with others, and a very natural tendency to maintain relationships. The larger
a company gets, the more diluted its culture can get, but you can avoid this by
putting a few things in place.
If the DNA is protected properly as the organization grows, it carries a copy of
itself in every person. Thats why it is so important to have a strong point of view,
to have a clear vision that is unambiguous in its language, so that if competitors
copy you, its very clear whose DNA theyve copied. When people try to copy the
Geek Squad, its usually done poorly because if you are going to copy me, Im
going to make you stick your neck all the way out. But the kind of person bold
enough to do that would probably do something original anyway, so, either way, it
ends up not being competitive. The DNA helps the organization withstand the
scaling-up of the entity so that the more geeks there are, the better the squad
becomes. If you take five employees in one city and you spread them around the
world, youve diluted the culture. But if you go from five employees in one city to
100 and then in another city you go from 100 to 500, youve actually increased the
nodes on the network; youve increased the strength of the DNA and the network
because the connecting points have all increased.

Agents 2011 The Geek Squad

150 Chapter Eight The Geek Squad

The importance of language and metaphor

Then you have to deal with what I call the Founders dilemma, and that is that the
organization will begin to weaken if it relies on a central point, which is me. A
founder, a strong leader, must work to fake their own death, as I refer to it, mean-
ing get the network to be self-healing, self-replicable. And the way you do this is
through the use of archetypes, myths, stories, rituals and your own unique language.
All industries, whether pharmacists, firemen, law enforcement officers, all have
their own language that helps to protect their culture, because language moves
like a virus.
In our case the fundamental unit of the Geek Squad is the Agent, and the use
of the word agent is very carefully chosen. In the US theres a company called
Target, a very well-known retailer, and they refer to all their customers as guests,
which changes the tone and context which I think is brilliant. I did the same thing.
Im not interested in perpetuating the stereotype of the word geek. I never use the
word geek outside the phrase Geek Squad because there are other companies
that use the word and they are diluting the meaning of it. Thats why I refer to them
as Agents, or theyre Counter-intelligence agents, or theyre Double agents. In
fact, we define any of our channels where a computer is fixed on a bench and not
in a customers home as technically a counter-intelligence operation. So we have
a master brand with distinct subcultures which are named accordingly. This allows
me to treat the departments as separate but equal. We have a group of people
called our Secret Weapon. The reason they are secret is because customers
dont know this department exists within the Geek Squad but all the Agents do.
When the Agents have trouble, they can call this number and it is the tech support
function for the agents. The calls are very quick because providing tech support
to another agent is very easy, it takes two seconds versus an hour to explain it
to a customer. Its a very efficient model that saves us a lot of money and helps
to bind the culture together. Its our secret weapon. So the way we use language
and titles simultaneously respects the individuality of the department or the indi-
vidual, but it also prevents company politics or comparisons of power because
you cant really compare a Secret Weapon to an Agent, to a Public Defender, to
a Counter-intelligence Agent, all of which are real titles. Theyre not chosen just
for the sake of humour. The words actually have functional meaning and define
who we are.

What do you stand for?

The most important thing a company can do is to stand for something. The
values the company possesses should be distinct and define the DNA. They should
be impactful, with specific language used to define those values; not the saccha-
rine, sugar-coated terms that are meant to please anybody and everybody. Brand
 Robert Stephens founder and chief inspector 151

is just as much about what you do as what you dont do. So the values are the light
source that attracts people to a company. I think people are drawn to Apple, theyre
drawn to Virgin; we have an intuitive sense about what we think their brand values
are and theyll be pretty close to what the founders probably intended; thats what
makes them strong brands.
Then the next question is, how true are you to those values? In the company
name, in the design of the logo, in the uniform, in the call centre experience. If you
have a phone number, if you have a website, if you have an e-mail address that you
communicate to customers; you are in the service business; the rest of what you do
is merely those things that help drive that experience. I realized early on that you
cannot design a customer experience because it is whatever the customer perceives
it to be. What you can do, though, is to influence that experience through the people
you hire. So what I began to realize is that I had a company name and logo that were
created to be distinct and supported by the uniforms and the cars, each taking a
functional role in creating our image, but that the key to success was that I had
to hand off that responsibility to each new employee and trust them to deliver
our brand.

Recruit for the experience

The brand became a filter. Some people turned down working at Geek Squad, they
didnt want to be called a geek, or they didnt want to wear a clip-on tie. Wearing
our uniform, if you think about it, is a type of forced humility. You really cant have
an ego, wearing our uniform, and thats when I began to realize that the brand is
almost more important in terms of how it attracts employees than customers.
I recruit for the customer experience I want to deliver. So thats why in the last
year I have moved away from the notion that we manage employees and distilled
my philosophy to the concept that we have temporary custody of talent. That
concept totally changes the management mindset away from management of
employees to one of servant leadership.
The question then becomes, How do I find the talent that will fit my brand?
I am currently negotiating with a film company in Hong Kong that owns the rights
to the old Shaw Brothers Kung-Fu movie collection, and my hope is to create a
Kung-Fu Film Festival in London and Minneapolis and eventually every city where
I hire. Why? Because how you hire people begins to tell people what you stand
for and what youre going to expect from them when they work for you. So it
actually costs less money to rent a movie theatre and to hold a film festival where
you show maybe an interesting double feature of the kind of films that only geeks
would be interested in watching, mainly sci-fi films and kung-fu films. The result is
you attract the kind of people who fit your brand and create a novel way of talking
to them. There are only two types of people that will show up for this type of film
festival: geeks and people who know geeks. So it is actually an efficient way to
152 Chapter Eight The Geek Squad

attract people from our target employee market and it costs less than an ad
on monster.com. I would argue that recruitment should be your most authentic
form of advertising.

Hire for values, train for skills

I believe if you ask enough questions you will pretty much come up with the same
three values that it took me 15 years to discover, and these are curiosity, ethics
and drive. Those are the three attributes that I cannot train for and therefore I must
hire for.
Im teaching these to my recruiting department, which I call the Inhuman
Resources department. I call it that because HR departments can feel inhuman,
so lets call a spade a spade and transform the typical meaning and bring it to
the forefront, which enables us to have a different kind of conversation with our
people.
Once I realized that the key to a great customer experience is to recruit for the
employee experience and that you only have temporary custody of talent, I then
realized that you have to figure out how to motivate them to stay with you. Our
goal while youre with us, whether that be for six weeks, six months or six years,
is to increase your value. How do you increase your own value? By showing up on
time, doing it right, doing it fast, doing it well, engaging in curiosity, ethics and drive.
If you do that, our brand will mean something on your CV, and therefore if you
leave us, you will be able to command a higher rate of pay. So our litmus test is
this: If you leave us and you go to work for a competitor for the same rate of pay,
we have utterly failed you as a company. That is the benchmark that you can hold
us to. We know that people leave managers, not companies, so you should expect
inspiring leadership. I tell the leaders, Your job is to inspire employees, which means
you must uphold curiosity, ethics and drive in service of that talent, and if you dont
increase their value, then you as a leader are not being effective.
The Geek Squad doesnt have my name on it, its about the squad, its about the
network, its about the crowd. Were seeing these trends towards self-organization.
Ive already started fading into the background. So now my focus is to detect
talent and then position them where theyll have the optimal effect. And thats
really my long-term plan.

Being bold can be as scary as hell

The boldest thing we did as a company was acquiring Best Buy. Its easy to be
creative when you only have 50 or 100 employees or even 500, but it is as scary
as hell when you have 15,000. The boldest thing I did personally as a founder was
to remain with the business. I have ideas for 10 other companies in my head, all
branded, all ready to go, but my work here is not completed yet. The world is
 Jeff Severts the promise maker 153

populated mainly by boring busi-

nesses but for creative people,
thats where the opportunities are to
contribute. If you think youre bold,
then I dare you to try and influence
150,000 people around the world
and get them to care about quality.
Thats what Im doing. It would be
easy for me to walk away and thumb
my nose at the leaders of the big Best Buy logo 2011 The Geek Squad
evil empires and say Its all their
fault, they made the company mediocre, versus saying, You know what, there
are two ways to change the world. You can either picket across the street from
the government office or you can take a job inside and begin to change the culture
one brick at a time. Ive chosen the latter because I think its a more creative
challenge.

For more on Robert Stephens and the Geek Squad story,

click here:
http://www.youtube.com/watch?v=EeZLfy_
C254&feature=related

Jeff Severts the promise maker

Jeff is minister of propaganda for the Geek Squad and currently working in
London on the introduction of the Best Buy brand into the UK market.

I call myself the promise maker in this business. We have promise makers and
promise keepers and my team worries about the promises we make to custom-
ers. So what should the customer experience be like? What price will we charge
for it? How should we communicate the promise? How do we talk about it with
our employees? What levels of quality do we expect from the promise keepers?
Our job is to answer all these questions.
Our most powerful channel for making promises is through our 150,000 retail
Best Buy employees. We have millions of customers walking into our stores
every year and coming into contact with those 150,000 employees. What those
employees say and believe about the Geek Squad is far more important than
any television ad that we might run.
154 Chapter Eight The Geek Squad

The Geek Squad difference

We have 20,000 Agents across the country. Its hard for me to promise the customer
that the individual Agent theyre talking to is always going to have the answer for
every problem for every type of technology in every circumstance. So we try not
to make that promise explicitly. The promise we do make, however, is that our
network of Agents can find you every answer to every technical problem every
time. They can do that because theyre connected to each other, and somewhere
within our huge network of employees weve got that answer. And thats what
differentiates us from our regional or local competitors, or maybe an independent
proprietor who is fantastically technically adept but is just one person. No matter
how smart you are, there are limitations to how much one person can know.
We have a couple of ways that we use the power of the network to deliver our
promise. We have what we refer to as Geek Squad Forums, which are web-
enabled meeting places in which thousands of our Agents can log in and share
information about challenges theyre struggling with or insights theyve discovered.
We also have our Secret Weapon, which is the collection of our best agents
across our different technical capabilities, who sit in Minneapolis waiting to take
calls from our Agents in the field, who might be stuck with a particular problem.
And our promise to the Agent is that either the Secret Weapon will know the
answer or they will find it out very quickly.

Relentless pursuit of a crazy vision

When I think about how the Geek Squad is different to other places I have worked,
I would say theres just a great audacity in some of the things we have done. So
asking thousands of employees to dress up in a uniform that most people would
view as ridiculous is audacious. Most business people would reject that idea as
something that employees would not be willing to do. I can imagine the conversa-
tion: You are asking employees to wear pants cut three inches short above the
ankles in order to show off their white socks? To wear a ridiculous short-sleeved
white shirt with a black clip-on tie? And carry a police-style badge? Forget it! Had
I been at a former employer, I would have been on that same side of the argument,
saying this is going to upset employees, hurt employee morale and consequently
retention. But the conventional business logic was just ignored because of Roberts
obstinacy and his relentless pursuit of this crazy vision hes always had, of what
the Geek Squad could look like.
 Jeff Severts the promise maker 155

Geek Squad cars 2011 The Geek Squad

Geek Squad City

Another example is that we built a repair centre in Louisville, Kentucky, and by any
conventional business wisdom this repair centre would have been called Repair
centre number 12 and would have been in a dingy building somewhere. The
people working there would have been allowed to wear whatever they wanted
to work because they are not meeting customers. Instead what we did, mainly
through Roberts encouragement, was create Geek Squad City. The head of Geek
Squad City is naturally called the Mayor and our 700 or so employees wear the
uniform, every day. Conventional business logic would never lead you to do that
because the argument would be, No customer is ever going to walk into this re-
pair facility. So why are you going to the expense of putting on this big show, when
nobody is going to see it?
156 Chapter Eight The Geek Squad

But this kind of thinking is what enables us to preserve our authenticity. There
are things we do today that we could do far more economically but might
sacrifice our authenticity and so we choose not to do them. For example, mould-
ing a Geek Squad logo on the bottom of our Agents shoes so that if they just
happen to walk in snow, the imprint of the logo will be left behind and people will
see that the Geek Squad was there. Consultants would take a look at the things
we do and describe them as waste. But they would be missing the point. We
generally believe that we have two icons that dramatize the brand. They are the
Geek Squad uniform and the black and white car. These are what consumers
remember and describe as differentiating us from competitors.

Measuring the experience

We pay enormous attention to measuring how well we are delivering our promise.
To put this in context, the Geek Squad represents roughly 5 per cent of Best Buy
in terms of size, yet we spend more money measuring the quality of the customer
experience than the whole of the rest of Best Buy put together.
We have a virtual customer council of several hundred customers that we con-
sult monthly. Well put out new concepts to them. Well ask for current pain points
and get feedback on the experience from them. Well ask for any new ideas that
they might have for us, on services we could offer against emerging technologies
or existing technologies. So we have sort of a constant virtual dialogue with an
established set of customers. And then we depend very heavily on our Agents
to help us develop new services, so they channel the customers needs to us,
because, you know, theyre dealing with customers eight hours a day.
Our ability to stay ahead of competitors also comes straight out of the Agent
community. Id like to say we are all strategic geniuses up here at the corporate
office, figuring out exactly what we should do to maintain our competitive edge,
but the truth is we are not, though we do have the luxury of having this community
full of energy and vitality thats always looking for ways to improve and grow. So
a lot of our best intelligence about whats going on with competitors just bubbles
up through the community and a lot of our best ideas for how we can compete
with new entrants comes from our community. Its just a natural part of the
dialogue we have.

Turning employees into heroes

Part of the magic that Robert discovered when he first built this business was that
there was already a community to tap into: this community of geeks, if you will, who
share the same values, who share similar life experiences, who share common
needs. And he created a kind of home for them: a place they could feel part of and
had been searching for. Robert says we have created a place for people who, for
 Jeff Severts the promise maker 157

most of their lives, may not have felt like superheroes but knew inside they had
superhero-like qualities. In coming to the Geek Squad, they felt like they came to
a company that understood and appreciated them.
I believe that this should be possible in nearly every business context if you
ask the question, How can I create a culture that makes employees feel like we
understand and appreciate them? And whether you are selling dental supplies
or cars it should be possible to create a culture that feels right for the people
who work there.
We have a tradition of using real Agents in almost all our advertising. For ex
ample, take our proposition Geek Dreams, which is about promising to find a
solution to your technical challenges. Say I want to watch my home theatre under-
water in my pool, is that possible? Our Agents will see if they can build a system
that would be capable of being viewed and heard underwater. This is consistent
with our culture of geeks rising to the challenge of whether or not technology
can be used to deliver an experience. We feature our Agents doing heroic things
for customers. This is just one example of how we use the Agents as our main
marketing weapon.
I am sceptical that senior management in a large company can ever really be
close enough to their customers. We all do our best to make sure were getting
out to the markets enough and that were talking to enough customers. Any good
manager will do that, but there are just physical limitations in running a large
organization that prevent you from ever being as smart as you should be. But if
you allow your employees to have a very loud voice on how youre running the
business they will take care of that for you.
If youre lucky enough in business to have your original visionary leader,
make sure you value their contribution appropriately and give them the right
voice in setting the direction for the company and maintaining the culture. You so
often read about companies starting up, getting to a certain size and then parting
from their founder because that person is an entrepreneur, not a professional man-
ager. I think its a very short-sighted approach that ends up costing the company
dearly because the value that that original visionary creates is very hard for an ac-
countant to capture, though it is real nonetheless.
We have a real community that people are proud to be part of in a way thats
very difficult to find in a lot of business environments. For example, I was talking
to an Agent in one of our stores recently who was among the first generation
of Agents who were hired. He was telling me that he had his badge framed and
fastened to the wall in his living room. I cant think of any other business where
an employee would be so proud to be part of that business, that they would take
a piece of their uniform and frame it and put it on a wall. That story made a big
impact on me because it sums up what the Geek Squad is all about.
158 Chapter Eight The Geek Squad

Lee Williams the Double Agent

Lee is an Agent with the Geek
Squad covering central London. He
has been with the brand for four
years.

Im a Double Agent Double Agent

5235. This means I go to custom-
ers homes and fix their computers
or any other problems they may
have with their technology, but I also
work in the store supporting our
Carphone Warehouse customers
when required. When people say
What do you do for a living? being
able to reply Im a Double Agent
is pretty cool.

Mission Impossible
My first impression of Geek Squad
Geek Squad Badge 2011 The Geek Squad
was the recruitment experience. I
came up from my home in Bath to London and the recruitment day was structured
like a Mission: Impossible task. I had to meet the interviewers for a briefing at an
advertising agency in central London which was kind of a daunting place to meet
for this little boy coming from Bath. Then we went for a drink somewhere and they
said, Okay, the next person you need to talk to is waiting at this location, so off
you go. And then when I got there I had to find my way to another location.
All I had was my phone to rely on but I found my way round London quite nicely.
I then realized that this was all part of the process: to see if I could navigate
around London and get to a place by a certain time just by using my phone. In
the end I was interviewed by seven people and that was the first time I felt
a company had taken the time to make it an experience rather than just
an interview.
The interviewers were different too. Some of them were just chatting about
technology, so I guess they got a sense of how genuine I was with micro-technology,
others just asked about movies and stuff, it was pretty cool. It was mostly How
you hanging, dude? you know, getting to me rather than looking at my CV and
going, Hmm, youve got this qualification, when did you get that? So yes, it was
quite cool.
 Lee Williams the Double Agent 159

Special Agent training

My training as an Agent was mostly culture training. There was not much technical
training as such. It was mainly learning about the brand and about Robert and his
philosophies. He came to our class and spoke to us mostly about how to treat
customers.
Robert says our uniform is like a filter for people. If you think youre too good to
wear short trousers and white socks, you probably dont belong in this company
anyway, so most people feel quite humbled being part of the Geek Squad. A lot of
technical people can be quite arrogant about their knowledge, so its really import
ant to understand that it just so happens that were experts in computing but its
not that special; customers pay us to impart our knowledge, not be spoken down
to. I think most guys think it was pretty cool to dress up like an FBI agent for your
job and be called Double Agent.
The training was good but generally I think the company just attracts some
really prime people. The way we treat our customers is just the way we treat people
normally in our lives and that just happens to be the right thing to do.
The thing with most computer repair people is that they dont look at actual
solutions, they just look at the end result and the quickest way to get there, which
I can kind of understand from a business point of view. But the thing about the
Geek Squad is the extra time that we spend to fix your computer, so we sort out
the culprits, the repair people who can give us a bad name, and get rid of them.
We have a warranty so if anything goes wrong within 30 days of us fixing it we
come back out and fix it for free. So you know youre genuinely paying for the
solution rather than just paying for the time that were there. We have a flat-rate
pricing scheme, whereas other people charge by the hour and that leads to some
sneaky people taking longer to do things than they need to. So our flat rate takes
the pressure off the Agents to perform quickly and gives them the time to achieve
a proper solution.
We are encouraged to think outside the box. For example, there was a cus-
tomer whose computer was overheating and failing each time he tried to use it.
We needed to get his data off the hard drive but each time we started it up it
died because of the overheating problem. So the Agent opened all the windows in
the house to make the house as cold as possible and then tried again. It worked;
he was able to retrieve that important data for the customer.

Getting the badge

After the training and induction period you are awarded your badge but only
once you have earned it because people really do have to prove that theyre
worthy of it. Generally its around the three- to six-month mark. Our standard prac-
tice is to be five minutes early to appointments to show customers that their time
is valuable to us. So often our supervisor, who is called Deputy Field Marshal will
160 Chapter Eight The Geek Squad

show up at the appointment and if you are there at least five minutes early you are
awarded your badge but if you are later than that he has gone!
Most of our bonus is based on the customers opinions of us. So well say to our
customers after weve visited them, How did you find the experience? and the big
question is Would you recommend us to a friend? And if we get a no to that bad
mark. If we get a yes to that a
good mark, so depending on how
many of those you get a month
thats what your bonus is based on.
The motivation for this job is
definitely not money or anything like
that but just seeing how you can
change peoples situations. It seems
quite simple to us to fix something
that can make such a difference to
someones life. So, you know, being
a hero is the big reward here...
when you save somebodys child-
hood pictures or something like that, Baby Geek 2011 The Geek Squad
its quite awesome.

Casper Thykier the customer

Casper is CEO of VEEMEE LLP, a computer gaming company based in London and
an enthusiastic advocate for the Geek Squad brand.

I came across Geek Squad about 18 months ago through positive word of mouth
from other people. I never really saw any marketing, Im not even sure if they do
any. It was one of those brands that had a mythology about it and friends said to
me, Hey, have you heard of this company called the Geek Squad? You really should
try them, theyre amazing.
Im not that technically gifted and there is a growing anxiety about the amount
of electronic equipment that you have in your home and the fact that you dont
know how to use any of it and cant get the different bits to talk to each other.
So friends whove had the Geek Squad treatment became evangelists about it.
I guess the notion of having someone come to your home and solve all your
technological problems in one fell swoop was just a huge gap in the market
that needed filling.

Expectations and experience

I think my first experience of the Geek Squad was one of those very rare occa-
sions in life where the reality exceeded the expectation, because its such a
 Casper Thykier the customer 161

singular message and such a pared-down approach that you cant believe it will
be as good as it is. Thinking about the customer journey I had, it was brilliant at
every single stage. So, from going on the website to making the initial phone call,
where the person I spoke to was extraordinarily helpful... you know, when trying
to figure out a time normally for support people you end up having to fit your diary
around theirs, whereas this was, When is it most convenient for you morning,
noon, night?
I was told who the Agent was who was going to turn up, what his name was,
where he was coming from and why he was coming. Lee Williams is his name.
(Editors note: the same Agent who was featured in the previous section.)

Double Agent Williams

He was actually late for the appointment but dealt with it brilliantly because he had
the courtesy and the common sense to phone to say that he was late but on his way
and he apologized despite the fact there had been a big issue with public trans-
port, which was out of his control. Normally somebody else would have just turned
up late and said, Oh, yes, sorry, but that would have mucked your plans up anyway.
He came in. He was in the gear, and, you know, his demeanour is one of some-
one who clearly knows that he is a geek and I dont mean that in a derogatory way.
So you got an instant sense that there was going to be no problem that he couldnt
solve. Our stress levels with the technology at that time were really high, so it
was like the knight in shining armour coming in.
You know, I can see how people can get cynical about the Geek Squad thing,
the uniforms, the badge, etc, because thats just the British way. But what I like
about it is that it does what it says on the tin, it reassures me that these people
are not like the rest of us. It just reinforces the fact that they are the guys who are
going to know what to do to solve my problem.
Its very hard to gain a level of trust with a brand, and a level of advocacy,
especially when youre a service business, because youre only as good as the
person who walks through the door or answers the phone. For example, we have
ADT come to our home to fix our security alarm. I dont know when they will turn
up or who will turn up. One of their people upset my wife. I phoned ADT to request
that this person does not come to my house. They still send him every time. They
come late, they dont fix the problem, the problem recurs once theyve visited and
it costs the same amount to call them again. I feel ripped off by ADT because
I hate the experience, whereas by the time Lee had come round and worked his
magic, Id have happily paid his charge twice over.
Its lovely to see someone from the Geek Squad come in and say, We can
make all this very simple for you. Dont worry about the details, because thats
why were called the Geek Squad. You know, thats the kind of service everyone
should aspire to.
162 Chapter Eight The Geek Squad

Bold lessons

Think big
The vision of the Geek Squad is The complete and total domination
of the computer support business. This was defined when Robert
Stephens was still working out of his home with start-up funds of just
$200, which shows that your vision need not be limited by your resources.

Be the brand
Robert Stephens is indistinguishable from his Agents. He wears the
Geek Squad uniform and is ready to swing into action at a moments
notice when there is a computer problem. Shaun was giving a keynote
speech at a conference and had a problem transferring the file. Robert
immediately came to the rescue and provided a complimentary Geek
Squad USB drive. Problem solved. Ass saved.

Engage with customers

The Geek Squad monitors social media like Twitter to identify when people
are having computer problems. When it does so it sends a tweet to the
person, offering to help. Robert was concerned that this may be construed
as spam and so sent a tweet himself asking for people to let him have
their views on whether it was okay or not to contact people directly.
The brand has used mainly viral marketing and word of mouth from
happy customers to create awareness but as it has grown larger it has
begun to use television ads to get its message across.

To see a Geek Squad advertisement, click here:

http://www.youtube.com/watch?v=q4I_
RlWs3TM&feature=related

Dramatize your brand through the experience

The strategy has been to create a distinctive end-to-end experience that
screams The Geek Squad. Each and every member of the organization
is a walking, talking advertisement for the brand. By stretching the Geek
superheroes and police squad metaphors to the extreme, the brand makes it
impossible for anyone to copy them without it being obvious they are doing so.
The experience is distinct, recognizable and memorable.
 Casper Thykier the customer 163

The importance of language and tone of voice

Although the Geek Squads success is founded on its excellent customer service,
it is its innovative use of language (both verbal and visual) that has distinguished
it and built its memorable brand personality. Robert Stephens intuitively understood
that language could be used not just to point out function or highlight a selling
point; it could be used to organize the business, attract and deter recruits, shape
and reinforce culture, and both set an expectation of experience for the customer
and enhance it. It has been a remarkably successful form of corporate
neuro-linguistic programming (and a geek would know what that means!).

Relentless authenticity
Many companies will take their marketing ideas so far, until they cost too much
money or until they perceive they have performed their useful function. The
Geek Squad are different; whether it be the logos on the soles of the shoes or
the naming and uniform of non-customer-facing staff, it pursues its distinctive
approach relentlessly regardless of whether any tangible return on investment
is generated. It does so because its entire approach is founded on a culture of
authenticity geeks behave like geeks even when the customer isnt watching.

Super-heroes need super-tools

One of the most customer-friendly innovations has been remote support, where
an Agent accesses the customers computer remotely and is able to carry out
maintenance or repairs without ever having to visit. For a small monthly fee,
customers can call up for help and get this service at any time.

To see remote access in action, click:

http://www.youtube.com/watch?v=cIuHXQAIN2k

The company DNA is the foundation for growth

Enormous attention is paid to the culture and the DNA of the brand through
hiring, training, titles and performance management. One of the ways that the
culture is reinforced is through a 24-hour hotline that Agents use to record and
share their customer stories. Another is through hiring people for personality
and their ability to innovate rather than technical ability they can be taught
that. The firm has steadied at a year-on-year growth rate of 40 per cent, but the
firm will not hire people unless they are right. The Geek Squad understands that
the company DNA is as important for the effective running of the company as
the source code is to the efficient working of the computers they repair.
164 Chapter Eight The Geek Squad

For more on the Geek Squad, click here:

http://www.geeksquad.co.uk/about-us

The Geek Squads insight was that you could turn peoples fear of and dependency
on technology into an opportunity to entertain as well as reassure them. They
created a wow experience by emphasizing a very human approach. Our next
bold brand, Zappos, has similarly taken what could be a remote, technologically
driven experience shopping online and found a way of ensuring it has a high
human factor. Its made the provision of wow its mantra. Like the Geek Squad it
encourages its people to be humble.
 CHAPTER NINE

Launch Support Equipment

"And let's not forget LUNCH following the LAUNCH"
- Bill All (N3KKM), Program Manager of NSBG

Chapter Objectives
1.0 Support Equipment To Collect ............................................................................... 1
2.0 Support Equipment To Be Built .............................................................................. 6
Good To Know - Lapse Rates, Dew Points, And A Stable Atmosphere............................... 26
Near Space Humor - Near Space Comix #1 .................................................................... 33

1.0 Support Equipment To Collect

The ground support equipment required to launch a near spacecraft consists of a mix of items, some
that are purchased ready to use and others that are constructed. The list below groups the ready-to-
use equipment into functions. Afterwards the function of various items is described in detail. Youll
notice that duct tape is used during balloon filling and stack assembly. One cannot go into near space
without duct tape.

Items For Balloon Filling

Two inexpensive bed sheets (sewn together)
Mark Conner recommends durable picnic blankets (available at Wal-Mart)
Tarp (for filling on gravel and wet grass)
Kneeling pads (recommended)
Tie down straps for the helium tanks (see Section 1.0.7)
Roll of duct tape
Sisal cord
Scissors
Electronic fish scale
Several pairs of soft cotton gloves (use brown jersey gloves without the beaded palms)

Items For Stack Assembly

Roll of duct tape
Nylon cord
Scissors
Electronic fish scale
Several 2 to 3 diameter metal craft rings
Several pairs of soft cotton jersey gloves
Solar powered calculator
Small white board and dry erase markers or note pads and pens and pencils
A selection of link lines in a storage box

Parallax Inc. 2006 Rev 1.0

Page 2 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

Bag of Styrofoam peanuts

Laundry bag

Launch Equipment
Lanyard release (see Section 2.3)
Kite winders (see Section 2.3)
Several pairs of leather gloves
Note: Do not use the leather gloves when working with the balloon

Items For Capsule Closeout

Folding table (recommended)
Jeweler's screwdrivers
One pair of straight slot and Phillips screwdrivers
Small wire cutters
Small parts like nuts, bolts, and coax barrel connectors
Plastic zip ties and twister seals
Lens brush
A digital multimeter
Flashlights(s)
Butane lighter
Shipping labels

Items For Organizing Equipment

Boxes for small parts
Several large gym bags or plastic tubs with lids
Labels for bags or tubs

1.1. 1.0.1 Ground Support Equipment

Equipment - Gloves, knee

pad, duct tape, etc.

Place two inexpensive bed sheets or picnic blankets on the ground before unpacking the balloon.
Even when filled indoors, the balloon must be protected from dirt on the ground. The abrasive nature

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 3 of 34

of dirt damages the balloon skin during filling. Sew the bed sheets or picnic blankets together to
create a large enough clean work space. For those mornings the grass is wet or when the balloon is
filled on a gravely surface, place a tarp on the ground before the bed sheets. This may not be
necessary with the picnic blankets as they may have a waterproof lining. Balloon Crews usually fill
the balloon working on their knees. Help them out by giving them kneeling pads. Kneeling pads are
available at garden supply stores. Capsule closeout can be performed on a launch tower or the bed of
a truck, but its easier if done on a table. Bring a portable folding table to the launch site if the
Launch Crew cannot closeout the near spacecraft on a truck bed.

1.2. 1.0.2 Balloon Filling Equipment

Duct tape is used to seal the balloon after inflation. Dont purchase inexpensive duct tape with a
weak adhesive. Instead, purchase a duct tape with a really strong and gooey adhesive. Sisal cord is
the brown natural fiber cord with lots of frayed threads. The fraying threads increase its surface
friction, making it more difficult for its knots to slip. While scissors are needed to cut cords, never let
them near the balloon. Have crews cut cord away form the balloon. Electronic fish scales are
available at fishing tackle stores. Electronic scales contain something like a strain gauge that
determines force acting on them and a digital read-out. The scale measures the weight of the modules
making up the near spacecraft and the lift generated by helium in the balloon. Use only one scale to
measure both weight and lift, as two separate scales may not be calibrated the same. Anyone
handling the balloon is required to wear soft cotton gloves. Gloves prevent abrasive skin from
contacting the balloon. Gloves also keep skin oils off the latex of the balloon, where it may weaken
the balloon (I dont know this for certain, but the concern has been expressed repeatedly by others).
In addition to protecting your investment in the balloon and helium, gloves make balloon filling more
comfortable for the balloon crew. Helium expanding out of the tank sucks up heat from filling
equipment. Even during the summer, balloon crews will want to wear warm gloves.

1.3. 1.0.3 Stack Assembly Equipment

All knots tied in the load line are taped over with duct tape. The Stack Crew and the Balloon Crew
can share the same duct tape, since it takes only a few moments to get the needed tape. Do not use
the inexpensive fake duct tape. Nylon cord forms the backbone of the near space stack and is
called the load line. Load line is used to unite the balloon to the parachute. An inexpensive twisted
nylon cord is sufficient. Scissors are needed to cut the load line. As mentioned earlier, use the same
electronic fish scale to measure the weight of each module as is used to measure the balloons lift.
Metal rings are used to as a pulley to raise (or lower) the balloon in preparation for launch. Craft
stores and the crafts department of retail stores carry a selection of two and three-inch diameter metal
rings that are suitable. The rings are made from approximately 1/8 diameter wire and are used for
crafts like macram. Even if the ends of the rings are welded together, cover the weld (or butt joint)
with a wrap or two of duct tape. The tape smoothes the joint and protects the lanyards from getting
cut as they brush against the sharp joint. One ring is needed for a launch and the ring is reusable if
the capsules are not separated from the balloon. If flight termination units (FTUs) are used on near
space missions, then one ring per launch is needed. Regardless, purchase at least half a dozen rings to
ensure having one on hand at launch. A roll of sisal line is needed to tie off the balloon. Sisal line
has a rough surface from its exposed threads and is less likely to come untied. The Launch Crew
needs access to a calculator with solar cell backup. Dont rely solely on the human brain for
mathematical calculations when attempting to launch a minimum lift balloon. Forgetting to carry the
one from an addition dooms the flight to bouncing across the ground rather than ascending into near
space. Rather than risk someone forgetting his or her calculator, toss an inexpensive calculator into
the launch equipment. If youre not supposed to rely solely on the human brain for calculations, you

Parallax Inc. 2006 Rev 1.0

Page 4 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

should not rely solely on the human memory to recall launch calculations and plans. Have a small
white board and dry erase marker handy at the launch site to record launch plans like weights and
lifts. Barring a white pad, at least have access to a note pad and pencil to document the same
information (dont forget to bring your notes back with you for use in a final report on the mission
please do not litter). Before launch the interior space of each module is filled with loose Styrofoam
peanuts. Purchase the real peanut-shaped peanuts and not the disc-shaped peanuts. Do not use the
cornstarch based biodegradable peanuts that turn to mush when exposed to water unless you want to
hose out the modules of the near spacecraft. Stores like Mailboxes Etc sell Styrofoam peanuts. Use
clean peanuts, do not use recycled ones. Finally purchase a mesh laundry bag to store the Styrofoam
peanuts. Along with some mesh laundry bags; stores sell an opened plastic frame for holding the bag
open. Purchase one of these if you can find it.

Styrofoam Peanuts - in mesh

laundry bag

1.4. 1.0.4 Miscellaneous Tools

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 5 of 34

Miscellaneous Tools

The miscellaneous tools are seldom needed, but at times they are needed, they will save a launch. A
set of wire cutters are needed to perform surgery on the avionics or to cut bad zip ties. If a nylon line
must be cut in the field, use the lighter to melt the cut end of the nylon line to prevent it from fraying.
There should be no frayed nylon lines, either melt them or cover them in duct tape. A quick wipe
with a lens brush before launch allows the near spacecraft to return higher quality images. Keep
spare shipping tags with the launch equipment to label capsules with lost or damaged shipping tags or
for those times when the original shipping tag needs to be updated at the last minute. A digital
multimeter (DMM) is needed to measure battery voltages before flight (ensuring discharged batteries
are not sent on a mission) and to troubleshoot last minute errors. A DMM is another one of those
tools that can literally save a launch. Flashlights are needed for those early morning launches. Dont
rely solely on headlights, as there are times you need to look into the airframe. Carry spare items
like, nuts, bolts, and washers, spare fuses (if used in the avionics), spare BNC barrel connectors for
antennas, and twist ties into the launch equipment also. Use a clear plastic container to hold these
small items as it makes it easier to determine if the container has the parts needed without having to
open it (and risk spilling small parts). Many of the miscellaneous tools are easy to lose, so pack them
inside a small box or durable bag. The storage box or bag needs to be closed tightly to keeps item
from falling out, so purchase a container with a lid or integral seal like a zip lock seal. Finally several
large gym bags or plastic boxes are needed to haul this equipment around.

Besides making it easy to move the launch equipment, boxes or bags keep the launch equipment
together and reduce the chances of losing them. Launch equipment lost or left at home ruins a
launch. Imagine how aggravating it is to fill a $50 balloon with $75 worth of helium and discover
there is no load line at the launch site. Do not use cardboard boxes to store equipment as they are not
durable enough in the long run and will eventually let equipment fall out. Divide the launch
equipment among several bags or boxes or else it becomes too difficult for one person to move the
bags. Besides, packing lots of equipment into a container too small results in broken equipment. The
bed sheet and kneepads are large enough to need their own bag. Besides, you dont want potentially
dirty bed sheets and kneepads inside the other equipment bags. The balloon should have its own bag
or box. Do not carry other equipment in the same container as the balloon.

Parallax Inc. 2006 Rev 1.0

Page 6 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

Balloon and Bag

The possibility of damage to the balloon is too great. Wrap the balloon with an inexpensive towel for
additional cushioning and remember to not remove a balloon from its shipping bag until it is ready to
be filled. Finally, label every bag or box with its contents or function. Keep all the launch equipment
containers stored together and only use them to launch balloons.

1.5. 1.0.5 Hauling Helium Bottles

The tanks of helium used to fill weather balloons weight around 120 pounds. With all that helium
inside of them, you would think they would be lighter. Hauling helium tanks that are free to roll
around creates one heck of a noise when they crash into each other. Never haul any pressured gas
cylinders this way. Cylinders must be restrained during transport and storage. Its safest to move
tanks in the back of an opened truck, as opposed to the confined volume of a car. Use strong nylon
tie-down straps to restrain the tanks if the car is durable. Some cars have tie down points that are too
weak for two 120-pound tanks. Rather than damage the car pack a blanket around the tanks to keep
them from banging into each other every time the car or truck makes a turn. Due to their restraining
system, welding supply stores move their helium tanks standing up.

Remove jewelry and watches when carrying tanks, as their weight will damage them (two people can
carry a tank). At the launch site, place the tanks on their side; do not leave them standing up.

2.0 Support Equipment To Be Built

There are several items that are not readily available off-the-shelf, and must be built before launching
your first near spacecraft. I recommend building them in conjunction with the near spacecraft so they
can be completed before the near spacecraft. Finishing them before the near spacecraft allows
training and gives launch crews the chance to practice procedures. Items to construct include the
following.

Weighing Frame
Balloon Filler
Two Launch Lanyards and a lanyard release
Launch Tower
Warning Signs

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 7 of 34

2.1. Weighing Frame

To make it easier to weigh the modules of the near spacecraft, construct the weighing frame described
below.

Weighing Frame - Mark

Conner (N9XTN) and Dan
Miller (KC7SLC) weighing
a module of a near
spacecraft

2.1.1. Materials
3/4" square pine trim, twelve inches long
Three inexpensive eyebolts, nuts, and washers
Four snap swivels (use the larger ones)
Woven Dacron kite line, #200 test
Printed sign and laminator
Two #6-32 mounting hardware, one-inch long

2.1.2. Procedure
Locate and mark the center of the pine trim
Mark one inch from both ends of the pine trim
Drill three holes at the marked locations that are large enough for the eyebolts
Bolt the eyebolts to the pine trim

Parallax Inc. 2006 Rev 1.0

Page 8 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

Eyebolts in Weighing
Frame

Note: The two eyebolts at the end terminate on the same side and the middle eyebolt
terminates on the opposite side of the pine trim
Cut two lengths of woven Dacron kite line three feet long and melt the cut ends
Find and mark the center of the kite lines
Tie the Dacron line at its middle into the two eyebolts located at the end of the pine trim
Note: Use a simple overhand knot
Tie the snap swivels to the ends of the woven Dacron
Note: This is one of the few places you can use snap swivels in a near space program
With a word processor, type a sign saying something to the effect,

Weigh the modules and any FTU and beacon

Add the weight of the parachute

Laminate the sign

Bolt the sign to the side of the weighing frame

2.1.3. Using The Weighing Frame

To use the weighing frame, slip the open snap swivels into the lift rings of the module to be weighed.
Hook the electronic scale into the center ring of the frame and lift the module. Read the weight after
the scale reading settles down. Record the measured weight on a white board or pad of paper. Don't
rely on memory, as an under filled balloon is only suitable for plowing a field and is very difficult to
correct.

2.2. Balloon Filler

A weather balloon requires on the order of 300 cubic feet of helium, an amount that a toy balloon
filler is incapable of providing in a reasonable amount of time. There is no known commercially
available filler capable of handling a 300 cubic foot balloon. The filler described here is design
specifically for weather balloons and is constructed with inert gas components (components use right-
handed threads).

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 9 of 34

Balloon Filler

2.2.1. Materials
Visit your local welding supply store for the following parts (support your local businesses when
possible). Your welding shop may use a different supplier, but the parts listed are commonly used in
welding.

One Helium RegulatorA

Ten Feet of 200 PSI 1/4" ID Oxygen Hose
One 541 barb to NPT (National Pipe Thread)
One BF 4HP Female to Female Bushing
One AW15A
One AW17 Nipple
One AW14A Nut
Two 7325 Ferrules

Oxygen Hose Diagram of Parts

Parallax Inc. 2006 Rev 1.0

Page 10 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

The remaining items are available at your local hardware store

A six-inch length of 1 1/4" PVC pipe
A 1 1/4" PVC cap
PVC cement
Epoxy or RTV
Sisal cord
Duct tape

2.2.2. Procedure
Regulator Modification
Remove the string cutter if the regulator has one
Note: The string cutter is used to cut strings for toy helium balloons and has a sharp edge
Remove the tilt valve
Note: The tilt valve is the rubber filler value for toy balloons. The tilt valve is sealed when
straight and lets helium flow when tilted

Oxygen Hose
Ask a Welding Shop to do the following
On one end of the hose, install the following:
7325 Ferrule
541 barb to NPT
This end of the hose connects to the PVC pipe

PVC Filler Side of

oxygen hose

On the other end of the hose, install the following

7325 Ferrule
AW17 Nipple
AW14A Nut
This end of the hose connects to the regulator through an AW15A

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 11 of 34

Regulator Side of
oxygen hose

PVC Filler
Screw AW15A into regulator end of oxygen hose
Screw AW15A into regulator
Find and mark the center of the PVC cap (don't sweat if its not perfect)
Drill a 1/4" hole in the bottom of the PVC end cap.
Pass the male pipe into the hole in the PVC cap
Bolt the PVC cap to the hose with a BF 4HP Bushing
Note: At this point you should have an hose that is secured tightly to a PVC cap
Fill around the BF 4HP Bush with either epoxy of RTV
Note: This makes an airtight seal around the hose where it enters the PVC cap
Note: Be careful you don't overfill the cap, as you still have to glue the PVC pipe into it.
Let the adhesives set over night
Look over the fill making sure there are no open gaps that can leak helium gas

Diagram of
PVC End of
hose

Sand the ends of the PVC pipe to make it free of burrs that may abrade the balloon nozzle
Test the fit of the cap and pipe and ensure the PVC pipe seats deeply into the cap where the
PVC cement can weld them together
Cement the PVC pipe into the PVC cap with PVC cement
Let the cement set for a couple of minutes before testing the connection.
Cut two feet of sisal cord
Fold the sisal in half and tie to the bottom of the PVC pipe, where it meets the PVC cap
Wrap the sisal in duct tape where it is tied around the PVC pipe

Parallax Inc. 2006 Rev 1.0

Page 12 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

Close-up of filler end

2.2.3. Using The Filler

Screw the regulator into the helium tank
Note: Only make this connection hand tight, it doesnt require tools to tighten the regulator
to the tank
Insert the PVC pipe of the filler into the nozzle of the balloon
Wrap the pipe and nozzle in duct tape to keep the balloon from slipping lose
Note: Some groups, like EOSS use a hose clamp for attaching the balloon nozzle to the
filler. KNSP used the same techniques for its first balloons. The author has not found them
necessary if quality duct tape is used. However, when using a hose clamp, only use a nut
driver to tighten and loosen the hose clamp and not a straight bladed screw driver (bad news
around a balloon).
Begin filling the balloon
Connect the electronic fish scale to the sisal loop to measure the balloon's lift
Note: The filler adds weight to the balloon, so the scales measured lift is a little lower than
the actual lift amount. Minimize the length of hose hanging from the filler when making the
lift measurement of the balloon. This is not as important of a factor if the balloon is filled
outdoors in a gentle breeze. The wind affects the lift measurement of the balloon, making it
less accurate.

2.2.4. Additional Optional Modifications

Large balloons like 2000 and 3000-gram balloons have larger nozzles than the balloons most
commonly used. The author has not found it necessary to change the diameter of the PVC pipe of the
filler for these larger balloons. The excess nozzle is wrapped around the pipe and the entire nozzle
tapped down. Other amateur near space groups use adapters to increase the diameter of the PVC pipe
when filling these larger balloons. If you desire to use this method, then purchase the next size lager
PVC pipe (1-1/2 PVC?) and an adapter for 1-1/4 to 1-1/2 PVC. Cement the adapter to the larger
PVC pipe and store in a bag along with the rest of the filler. To use it, tape the adapter to the filler
then attach the balloon nozzle. Taping the adapter ensures it does not come lose while filling the
balloon.

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 13 of 34

2.3. Launch Lanyards And Lanyard Release

It is difficult to control the raising of a balloon when your only grip is on the load line. Handling just
the load line increases the chances the balloon slipping free, resulting in a painful string burn.
Handling just the load line also places greater stress on the load line or near spacecraft. Using a set of
launch lanyards reduces the risk of string burn while launching a near spacecraft, reduces the chance
of breaking a load line due to G snaps, and makes it easier and less tiring to raise the stack. The
lanyards and lanyard ring act as pulleys that raise the stack off the ground without jerking snaps
acting on the load line.

The launch lanyard forms a tripod structure around the stack. One person on the lanyards (the one
with the diameter PVC pipe) is the lanyard release. The two other people control the kite winders.
Crews holding the winders raise the balloon keeping it under full control. There is virtually no risk of
string burn with this system. The lanyards and kite winders method allows the balloon to safely be
lowered if necessary.

In the past I have purchased a pair of kite winders for the lanyard lines ($7.00 for a small winder). To
keep the cost down I used small kite string winders. They were sufficient to hold the line, but being
small, they made it difficult to raise the stack smoothly. Every turn of the small winders sent
snapping shocks up the lanyards to the balloon neck. Ive made larger winders from wood in the past,
but those required tools that many people do not have at home. Mark Conner (N9XTN)
recommended I use winders made from PVC pipe similar to the ones he makes. So here are the
instructions I developed from Marks concept. Thanks for the idea, Mark. Modify these directions as
necessary.

PVC Winder

2.3.1. Materials
Six feet of PVC pipe
Eight Ts
Four caps
One can of PVC cement (use the smallest can)
Small saw (an Exacto saw or hack saw works well)
Beaded chain

Parallax Inc. 2006 Rev 1.0

Page 14 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

Two enclosed blade letter openers or seat belt cutters

Note: The total cost for PVC materials is about $6.00

2.3.2. Procedure
Cut the PVC pipe as follows
Four pieces 4 long
Four pieces 5 long
Four pieces 6 long

Divide the pieces into two piles. Each pile constructs a single winder.

PVC glue gives a few seconds to work with it. So first glue together the pieces that dont have
critical alignment issues.

Diagram of PVC Winder

Glue a cap onto the end of a five-inch length of pipe

Repeat for the second cap and five-inch pipe
Glue the open end of the five-inch pipe into one arm of a T
Repeat for the other five-inch pipe and a second T
Glue a six-inch pipe into the center port of the above T
Repeat for the second T and six-inch pipe
Glue a four-inch pipe into the remaining arm of the first T
Repeat for the second four-inch pipe and the second T

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 15 of 34

PVC Winder Assembly

Steps

In Rapid Succession
Glue the remaining Ts onto the open end of the four-inch pipe and them glue the two
halves together to finish the winder
Lay the completed winder on a flat surface and press down on its corners
Let the PVC cement set for an hour

Theres raised lettering on the center port of the Ts. Two of those will rub against the hands of crews
using the winder, so file off the raised lettering and smooth the surface of the Ts.

Cut one last piece of PVC pipe to a length of twelve inches (the lanyard release)
Sand the cut ends smooth

Winding String On The Kite Winders

First I have several notes about winding the lanyard lines to the winders. Its helpful for the Launch
Crew if the two lanyards lines use different colors. Do not tie the line to the winder as the winder
must fall free of the lanyard lines should the lanyard and winders accidentally get launched with the

Parallax Inc. 2006 Rev 1.0

Page 16 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

stack. When the winder is tied to the end of the lanyard, its weight may let the lanyard wrap around
the parachute and shroud lines. This is not a problem during the ascent of the stack, but is big trouble
during the descent after balloon burst. Without weights hanging off the ends of the lanyard cord, the
cord tends to swing away from the parachute when the stack spins during the ascent. If the lanyard
line is twisted when it is wound on the winder, then the lanyard line will untwist itself in the air when
released from the PVC release and will tie a knot in the lanyard ring. The untwisting lanyard can tie a
knot in the lanyard ring, snagging the lanyard ring and preventing the launch. So wrap the lanyard
lines on their winders by laying the lanyard on the ground first, and then wrap it onto the winder. Do
not wrap the line onto a stationary winder, but instead wind the winder, taking up the lanyard line.
This method will reduce the chances of twisting the lanyard lines as they are wound on the winders.

Since the near space stack is about 50" tall, wind at least 200' of 1/16" woven nylon line on
each winder
Melt the ends of the lanyards with a lighter
Tie a one-inch diameter loop at the free end of the lanyard line

An option to think about

The loop at the end of lanyard represents a place to snag the lanyard ring. Not tying a loop in the end
of the lanyard may be an option if the Launch Crewmember holding the lanyard release can hold the
ends of both lanyards to the lanyard pipe with his or her thumbs. Better yet is to wrap the ends of the
lanyards around a short length of separate PVC pipes. The combination of the friction from wrapping
of the lanyard and thumb pressure may keep the lanyard securely on the release pipe until it is time to
release the stack. At this time, this method has not been tested.

Finishing the winders by adding the emergency cutaways

Drill a small hole in one of the handles of each winder

Drill a hole in the letter opener or seat belt cutter
Use the beaded chain to attach a cutter to each winder

Finished
Winder

2.3.3. Using The Launch Lanyards

Follow this procedure after the balloon is filled and the lanyard ring is attached

Pass the looped ends of both lanyards through the same side of the lanyard ring

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 17 of 34

Pass the lanyard release through both loops

Raise the balloon (explained in detail in Chapter Ten, Section Six)
Release one lanyard at a time
Use the emergency cutaway to slice the lanyard should it get knotted on the stacks lanyard
ring or if unsafe wind levels begin picking up

2.4. Launch Tower

The Launch Tower Holding the assembled near

spacecraft. The modules and parachute are all
linked up. Mr. Bunny will skydive from 50,000 feet
on this mission.

Module closeout and testing can be performed on a table at the launch site. But when it comes time
to connect their link lines and umbilical together, invariably the link lines get twisted around one
another. To prevent the twisting, place the modules to a gantry-like structure that supports the
modules in a flight-like configuration. The spacing between the modules is shorter then they are in
flight, but good enough for linking the modules together correctly. The gantry also provides a
structure for mounting laptops, heaters, extension cords, and wheels. Why wheels? Once the near
spacecraft is assembled on the gantry, the gantry also acts like a dolly. One person can move the
entire near spacecraft around, if necessary.

The launch tower described in this section has two removable platforms for setting the near spacecraft
modules on. The platforms are removable to make it easier to transport the tower. On the back of the
gantry are two optional coat hooks for wrapping an extension cord. The extension cord provides
power to the tower if an outlet is available. Long Velcro straps attached to the side of the tower lock
the modules against the gantry. This way the modules can't shift around or fall off. The Velcro straps
extend around the back of the tower where they can strap the parachute to the back of the tower.

Parallax Inc. 2006 Rev 1.0

Page 18 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

2.4.1. Materials
square pine trim
1/8 plywood door skin
thick Styrofoam
At least twelve feet of one-inch wide Velcro tape (non-sticky back variety)
Assorted fasteners like screws and finishing nails
Wood glue
Paint (red and white is traditional)

70-pound bag of tube sand (traction sand for cars and trucks)

Note: The TVNSP launch tower was constructed in a kitchen using hand tools and clamps.

2.4.2. Procedure
Built the tower to suit your specifications. The directions below describe how TVNSP built its
launch tower.

Launch Tower Side View

(left) with sliding trays,
Front View (right) shows
rails for sliding trays.

Tower
The TVNSP launch tower is six feet tall and made up of six, one-foot tall repeating units. Cross
braces are glued diagonally between levels of the tower. Occasional sheets of 1/8 plywood door skin
covers sides to strengthen the tower. The TVNSP launch tower is not as wide as an airframe, so the
left and right sides of airframes extend beyond the edges of the launch tower.

Cut four lengths of pine to match the height of the tower (these are the vertical elements
of the tower)
Note: TVNSP extended the length of the front two vertical elements by six inches to

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 19 of 34

restrain the parachute ring

Determine the width and depth of the tower
Cut multiple units of pine to match the width and depth of the tower
Glue and nail the tower sides together
Measure the length needed for the diagonal cross braces
Note: The cross braces need 45-degree cuts in their ends to fit within the tower side
pieces
Cut the cross braces out and trim the corners
Glue and nail the cross braces into place
Cut four pieces of pine trim to form a rectangular ring for the bottom of the tower that is
the towers width but twice its depth (this will become the towers base)
Glue and clamp the ring together
Cut plywood door skin to the same dimensions of the ring
Glue and clamp the door skin to the bottom of the ring
Attach the ring to the base of the tower with the door skin on the very bottom
Measure the length of pine trim needed to make cross braces between the tower and its base
Glue and clamp the braces to the tower and its base
Cover with more door skin if necessary
Determine where you want to attach plywood door skin faces
Note: The TVNSP launch tower has them primarily at the top and bottom of the tower

Platforms
Measure the width of the launch tower
Note: You need an accurate measurement; dont use the planned dimensions of the tower,
measure it to be sure
Determine how long you want the platforms to be (platform depth)
Note: TVNSP platforms extend about ten-inches from the tower
Cut two pieces of Styrofoam to the measured width of the tower and to the desired
platform depth
Cut four pieces of pine trim to a length equal to depth of the tower and the extension length
of the platforms (platform rails)
Lay two pieces of pine beside the cut Styrofoam with one pine trim on each side
Measure the width of the Styrofoam and pine (the platform width)
Cut four pieces of 1/8 plywood door skin to a dimension of the platform width and the
platform depth
Glue and clamp the platform rails to the sides of the face of the platform with the Styrofoam
between the rails

X-Ray View of Launch

Tower Platform

Trim the extra Styrofoam from the platform

Note: TVNSP feed the platform through a sanding planner to make the Styrofoam and rails
the same thickness
Glue and clamp the second plywood door skin face to the platforms

Parallax Inc. 2006 Rev 1.0

Page 20 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

Note: A cast iron corn muffin pan comes in handy here

Cut eight pieces of pine trim to a length equal to the measured depth of the launch tower
Temporarily clamp the platforms to the launch tower at the desired heights
Note: You may have to trim the inside edges of the platform rail to get them to slide on and
off the launch tower
Note: Place an airframe on the platforms when doing this to ensure the spacing of the
platforms is sufficient
Use a carpenters level to make sure the platforms are level
Glue and clamp one piece of pine to the launch tower beneath each platform rail (remove the
platforms before they have a chance to glue to the tower)
Note: At this point you are attaching a two-piece railing to the tower for the platform rails to
ride in
After the glue dries, slide the platforms back onto the tower
Glue and clamp a second piece of pine above each platform rail (again remove the platforms
before they have a chance to glue to the launch tower)
After the glue dries, test fit the platforms
Add narrow pieces of plywood door skin above and below the tower rails to strengthen them
Strengthen all weak joints in the tower
Paint the tower
Note: Radio towers have seven bands of color, with red as the top and bottom color and
bands of white between

2.4.3. Using The Launch Tower

Note: Store the launch tower with the Velcro straps attached to each other
On launch day do the following

Set up the tower near the balloon

Lay a bag of sand on the feet of the launch tower for stability
Unstrap the Velcro bands
Insert the platforms
Place the modules on their proper platforms and proper orientation
Strap the modules to the tower with the Velcro bands
Attach link lines between modules
Attach the umbilical (if there is one) between the modules

Note: The umbilical should run along the same side of both modules and not wrap around

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 21 of 34

Umbilical Placement
The left diagram shows
bad umbilical placement,
resulting in a twisted
umbilical. On the right,
good umbilical placement,
keeping the entire
umbilical on the same
side.

Attach parachute to the top module

Note: Have a second crewmember verify the split key rings are still properly attached to the
Dacron loops of the abrasion jacket. Often, when someone is attaching a swivel to a split
ring, they begin disconnecting the split ring from the Dacron loops of the abrasions jacket.

Drape parachute over the back of the tower and Velcro it to the launch tower

2.4.4. Possible Modification To Launch Tower

Tower Wheels
Two balloon tires and handles can be added to the back of the gantry. Mounting tires to a wide base
increases the towers stability. One person can pull back on the handles and shift the weight of the
tower and modules to the wheels. Then the tower is moved around like a hand truck or dolly. Dont
mount the axle and balloon tires permanently to the tower. By removing the wheels, the tower can be
packed into the back of an SUV or truck without taking up as much space.

Wheel Diagram Make

two. Use wing nuts on
the wheels so they are
easy to remove.

Parallax Inc. 2006 Rev 1.0

Page 22 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

Front and Side

View of Wheels
Mounted to the
Launch Tower -
You could leave the
mounting bolts
permanently in the
tower.

Module Heaters

Launch Tower
Module Heater

If an extension cord is available at the launch site, then handheld hair driers can be operated at the
tower. The warmed air from a drier at low setting can be directed inside the modules while they wait
for launch. The higher internal temperature of the capsules at launch keeps them from cooling down
as much during the flight. Be sure the hair driers are set to a low enough temperature that they do not
melt the plastic inside the modules. Use flexible aluminum heater duct (or other materials able to
withstand the heat) to direct heat flow into the modules.

Laptop Table On Tower

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 23 of 34

Laptop Table Place the

laptop on the back of the
tower so its weight is
balanced by the near
spacecraft modules

Another possible modification to the launch tower is to incorporate a removable table or platform for
a laptop into the towers design. A laptop platform allows the closeout crew to monitor the near
spacecraft before launch while its still sitting on the launch tower. They can also load test programs
into the near spacecraft before launch, letting them test experiments on the near spacecraft. Be sure
the closeout crew loads the flight code into the flight computer before the launch. Launching the near
spacecraft with test code makes for a real bummer of a flight.

2.5. Warning Signs (optional)

While filling the balloon, people may not be cognizant of the hazards they pose to the balloon.
Placing warning signs near the balloon can help. The signs are designed to warn spectators and crews
that the balloon envelope is fragile and that no sharp objects are allowed near it. Bursting a filled
balloon can be a $150 loss! It's fine to lose the balloon once the stack makes 90,000 feet, but an
entirely different matter if the balloon makes an altitude of zero feet before the balloon bursts. Signs
should specifically state that rings, bracelets, and other jewelry are not permitted near the balloon.
The procedure described here explains how KNSP made a warning sign for balloons. TVNSP crews
fill balloons out of doors where spectators tend not to get close to the balloon, so they do not use
warning signs (at this time).

Parallax Inc. 2006 Rev 1.0

Page 24 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

Warning Sign "Caution Balloon Filling in Progress.

No sharp edges including rings and bracelets allowed
near the balloon. Wear cotton gloves before handling
balloon or lift line."

2.5.1. Materials
square pine trim
1/8 plywood door skin or masonite
Light duty steel chain
#6-32 mounting hardware
Fluorescent paper
Laminator

2.5.2. Procedure

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 25 of 34

Warning Easel Side View

Warning Easel Rear View

Cut three pieces of pine to a length of four feet (legs)

Parallax Inc. 2006 Rev 1.0

Page 26 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

Cut two pieces of pine to a length of 12 inches (hinge pieces)

Cut a piece of door skin into a triangle about 12 inches high and 18 inches wide (easel face)
Clamp two of the legs to the sides of the door skin so that they spread out at a 30-degree
angle
Find the center line of the door skin
Clamp the hinges to both sides of the third leg with their tops flush
Mark about 1-1/2 inches below their top
Drill a hole through the hinges and center leg
Run a #6-32 bolt, washers, and nut through the hinge and third leg
Clamp to the two hinges and third leg to the center line of the door skin, with hinge pieces
extending about three inches above top of the easel face
Verify the third leg can swing out from the door skin
Glue and screw the two side legs and the hinges to the door skin
Cut a length of pine trim to a length equal to the bottom width of the door skin
Glue and screw the pine to the bottom of the door skin
Bolt the chain to the legs, near their bottom, to keep them from spreading out too far

With a word processor, make a warning sign

Print the sign on fluorescent paper
Laminate the sign for durability
Clamp the sign to the folding easel

Good To Know - Lapse Rates, Dew Points, And A Stable Atmosphere

A less dense item floats in a denser medium because it displaces less weight than the same volume of
the medium. A warm parcel of air in the atmosphere is less dense than the colder air surrounding it.
The buoyancy of warm air keeps it floating (rising) as long as the air parcel remains warmer than the
air surrounding it, as can be seen with hot air balloons.

The troposphere is not heated by its exposure to the Sun, but instead by its contact with the ground.
As a result, the troposphere cools with increasing altitude. The amount the air temperature changes
per change in altitude is called the lapse rate. At the same time the buoyant air parcel rises, the
atmospheric pressure around it is lowering. The lowering air pressure forces the air parcel to expand.
The Ideal Gas LawB states that the temperature of an adiabatic parcel of air lowers as its pressure
exerted on it lowers. The term adiabatic means that no matter or energy enters the air parcel. While
not strictly true for an air parcel, it is close enough to adiabatic for our needs.

Now add another factor to the change in temperature of our buoyant (and expanding) parcel of air,
phase change. Does it take the same amount of energy to raise the temperature of a volume of water
by two degrees Celsius from 94 to 96 C as it takes to raise the temperature of the same volume of
water by two degrees Celsius from 96 to 98 C? The answer is yes, or close enough for our
purposes. How about raising the temperature of the same volume of water by two degrees Celsius
from 99 to 101 C. Does it take the same amount of energy as in the last two examples? The answer
is a definite no. The volume of water remained in the liquid phase in the first question, but changed
phases in the second question. Water cannot remain a liquid at 101 (this is an ideal example)C
without the water molecules first gaining enough energy to break free of each other (to go from a
liquid to a gas). The energy required to change the phase of any liquid to a vapor is called the heat of
vaporization. What happens to this energy after water changes phases? Its stored in the motion of
the water molecules. Until the water molecules in the water vapor slow down enough to begin

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 27 of 34

sticking together to form a liquid, the energy required to vaporize the liquid remains trapped in the
kinetic energy of the water molecules. Note that the water vapor is an invisible gas and not the white-
colored steam coming from a boiling teakettle.

Keeping this in mind, what happens when water vapor in the atmosphere changes phase from a gas to
a liquid? First we begin to see what was invisible water molecules begin to appear as tiny droplets of
liquid water. Since these droplets grow large enough to Rayleigh scatter all wavelengths of visible
light, the droplets appear to be white when seen in a large enough quantity. A large enough quantity
of water droplets is called a cloud. The second noticeable change is that energy trapped in the kinetic
energy of the water molecules in the vapor phase becomes available to warm the air. Remember that
the water molecules absorbed enough energy to change phases. Now that this energy is no longer
needed to maintain the vapor phase of water it has to go somewhere, since energy, like mass, cannot
be created or destroyed. As a result the energy is released and heats the surrounding air. The energy
released when vapor condenses to form a liquid is called the latent heat.

So now our picture looks like this. The Sun shines on the ground, warming it up. The air in contact
with the ground also begins to warm up. A warm air parcel containing water vapor begins rising
because it is warmer than the air surrounding it. As the air parcel rises it expands and cools. As long
as the air parcel remains warmer than the surround air, it continues rising and cooling until its dew
point is reached. At this point, the water molecules begin changing phase from a gas to a liquid. As
the water molecules change phase they warm the air parcel. The air parcel continues cooling as it
rises, but not as quickly as before the water began condensing. This process is most effective on
spring afternoons, when the ground is warming from the increasing hours of sunlight and the air aloft
is still chilly from the winter. The Suns light only warms the very top layer of ground, but thats
enough to warm the air.

In a stable atmosphere, any air parcel that becomes a bit warmer than the surrounding air will only
rise a short altitude before cooling down enough to stop being buoyant. A stable atmosphere has little
vertical mixing and if it has clouds, they tend to be stratoform types. In an unstable atmosphere, any
air parcel that becomes a bit warmer than the surrounding air will continue rising because it never
cools enough to stop being buoyant. An unstable atmosphere has lots of vertical mixing and if it has
clouds, they tend to be cumuliform types.

Just to add unnecessary complication, but to be excruciatingly correct, regardless of their temperature,
water molecules in the air are always changing from the liquid to the gas phase and back again. At
lower temperatures more water molecules transition from a gas to a liquid than transition from a
liquid to a gas. But for our purposes, water molecules begin condensing from a vapor a liquid once
the temperature drops below the dew point of the gas. The dew point by the way depends on the
amount of water vapor dissolved in the atmosphere. Dissolved? Yes, gases can dissolve into each
other just as metals are dissolved into each other in an alloy.

The decrease in atmospheric density and pressure ideally follows a simple rate of a 50% change in
pressure with every 18,000-foot change in altitude. As a result the temperature of a buoyant air parcel
decreases at a fixed rate for every fixed increase in altitude. However, because of the change in phase
when the temperature of the air parcel drops below its dew point, there are two lapse rates. Since the
air parcel is nearly isolated from the surrounding air, we can refer to these lapse rates as being
adiabatic. The lapse rate for an air parcel at a temperature above its dew point is called the dry
adiabatic lapse rate. The lapse rate for the same air parcel once its temperature is below its dew point
is called the moist adiabatic lapse rate.

Parallax Inc. 2006 Rev 1.0

Page 28 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

The dry adiabatic lapse rate is 5.4 F per 1000 feet and the moist adiabatic lapse rate is between 20
and 40 F per 1000 feet. For those practicing to use the metric system, the dry adiabatic lapse rate is
9.80 C per kilometer (close enough to 100 per kilometer) and the moist adiabatic lapse rate is between
40 and 70 C per kilometer.

Determining Lapse Rates From Environmental Sounders

After processing air temperature and altitude data from a near space flight, create a new column for
the lapse rate at each recorded altitude. Take the air temperature at a given altitude and subtract the
air temperature in the previous record. Divide the change in air temperate by the change in altitude
between the two records. Since the lapse rate is given in units of 1000 feet, multiple the results of the
previous division by 1000. The equation in each cell looks like this

(+D4 D5) / (+G4 G5) * 1000

Where:
The D column is the altitude in feet column
The G column is the air temperature in degrees F

Note: Graphing the lapse rate in the troposphere is more important than in the stratosphere.

Create graph of lapse rate and altitude. Place the altitude in feet in the vertical column and the lapse
rate for each altitude in the horizontal column. Label the graph as the Lapse Rate for that particular
flight. One thing youll notice is that the lapse rate is negative in the troposphere and positive in the
stratosphere (if calculated there). If you dont observe this, then youve done something wrong in
making your graph. Youll also notice that there are a lot of zigzags in the graph. Its not as smooth
as we would like. Part if this is due to the fact that the atmosphere really is messy. But its also due
to the fact that the temperature sensor and GPS altitude have errors that are constantly varying in
direction and magnitude.

Lapse Rate Graph Lapse

Rate (degrees/1000 feet) vs.
Altitude (feet)

So far, so good. But with this graph you can also determine of the if the atmosphere is stable or not.
Determine at what attitude the air temperature drops below the dew point. This can be determined by
a report from the National Weather Service at the time of launch. You can also determine the dew
point at launch yourself with a sling psychrometer. See the next section for directions for making a

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 29 of 34

sling psychrometer and how to use it to determining dew points. Once you have the dew point, insert
a new column to the spreadsheet, called the Ideal Lapse Rate. This column has one of two numbers
in it, either 5.4 or 3 (the average of the moist adiabatic lapse rates). Put a 5.4 in the column where the
row has an air temperature above the dew point and a 3 into the column with a row with an air
temperature below the dew point. Now update the Lapse Rate graph with a second series, the Ideal
Lapse Rate column and only plot it to the tropopause. Use a different color or different line style for
the two lapse rate columns. Now print this graph.

Lapse
Rate
Graph
with
Stability
Line

When the ideal lapse rate is smaller (more negative) than the lapse rate calculated from flight data, the
atmosphere is stable at that altitude. When the ideal lapse rate is greater (more positive) than the
calculated lapse rate, then the atmosphere is unstable at that altitude. So in the example above, the
atmosphere is stable until an altitude of around 8,000 feet when the dew point is 65 F.

Making a Sling Psychrometer

The sling psychrometer is an instrument for determining the atmospheres properties in regards to
water. Specifically, measurements of humidity and dew point are determined with the sling
psychrometer. If a current weather report is not available, then use a sling psychrometer determines
the dew point at the time and location of launch. The author has used the design given in this section
for a home weather station.

Theory of Operation
Energy is required to evaporate water. That energy can come from an object that the water is sitting
on when it evaporates. So a thermometer bulb covered in evaporating water indicates a lower
temperature when compared to a dry thermometer bulb because the heat of vaporization required
evaporating the water is coming from the thermometer. The amount of cooling depends on a
combination of how much heat flows out of the thermometer bulb into evaporating water and how
much heat is flowing into the bulb from the surrounding air. How fast water evaporates from the bulb
depends on the air temperature and the amount of water dissolved in the air (called the absolute
humidity and given in units of grams of water per kilogram of air). The maximum amount of water
that can be dissolved into the air depends on the atmosphere's temperature. The ratio of the maximum

Parallax Inc. 2006 Rev 1.0

Page 30 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

amount possible to the current absolute humidity is the relative humidity. If the absolute humidity
doesnt change, then as the air temperate rises, the relative humidity decreases and as air temperate
drops, the relative humidity increases. The temperature at which the air can hold no more water than
is currently dissolved in the air is called the dew point. Ideally, at the dew point, condensation
becomes visible on cars and grass. In reality, dew usually appears at temperatures above the dew
point because of the presence of condensation nuclei. Of course if the dew point is below the freezing
point of water, then you dont get dew, but frost instead.

The sling psychrometerD consists of two thermometers mounted in close proximity. One is exposed
to the air while the other one is covered in evaporating water. The sling psychrometer swings to force
air to pass over the thermometer bulbs. The temperatures of the two thermometers are referred to as
the dry bulb temperature and the wet bulb temperature. Columns on a table determine the dew point
and relative humidity of the air by comparing the dry bulb temperature to the difference between the
dry bulb and wet bulb measurements.

Completed Sling
Psychrometer

Materials
Two small thermometersE
Or
Two garden thermometersF
A base (use either a thick plastic or model aircraft plywood)
Short length of shoelace
Cotton thread
-20 bolt, 5 to 6 inches long
Brass or aluminum tubing, large enough to cover the bolt without binding
-20 nut
Three -20 washers
-20 acorn nut
4-40 hardware for bolting the thermometers to the base, include lock washers

Construction
You have two choices here. Either the thermometers can be mounted side by side or on opposite
sides of the psychrometer base. However, keep in mind the following. The handle needs to be
mounted to the top of the base. The handle is used to swing the psychrometer. Also, the thermometer

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 31 of 34

to become the wet bulb must be mounted such that its bulb extends beyond the base on the side
opposite from the handle.

Psychrometer Front View (left),

Side View (right)

Handle
Lay the thermometers on the material chosen to be the base
Determine the position of the bolt handle and the thermometers
Drill a hold for the -20 bolt
Attach the bolt to the base with washers and a nut
Attach the acorn nut to the end of the bolt and measure the open space between the bottom
nut and the base of the acorn nut
Cut the aluminum or brass tubing to this length
Remove the acorn nut and slide the tubing over the exposed threads of the -20 bolt
Place a washer over the end of the tubing and screw on the acorn nut
Note: The acorn nut will tighten enough against the tubing to keep it from spinning because
the tubing is too long
Estimate how much the tubing needs to be shortened before it can spin freely around the bolt
Remove the tubing and shorten it
Test the fit again
Note: You want the tubing just short enough that the tube can spin when the acorn nut is
tightly screwed to the bolt, but without bolt threads or metal edges exposed. The tubing
forms a comfortable grip for the sling psychrometer without exposing your hands to rapidly
spinning sharp metal edges.
Cut the tubing again if necessary

Thermometers

Parallax Inc. 2006 Rev 1.0

Page 32 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

Remember that the thermometer to become the wet bulb must be mounted such that its bulb extends
beyond the base. However, do not extend the thermometer bulb much beyond the base, just enough
to expose the bulb.

Carefully drill 4-40 holes in the thermometer bodies to mount the thermometers securely to
the psychrometer base
Use bolts and lock washers to ensure the thermometers are securely mounted
Note: You want to ensure the thermometers do not go flying away every time you take a
measurement
Take the sling psychrometer outside and spin it a few minutes

Note: Keep people and important possessions away from the potential ballistic trajectory of
a thermometer when swinging the sling psychrometer for the first time

Check the bolts after the test; make sure theyre still tight
Use the short length of cotton shoelace to cover the wet bulb like a sock
Use the cotton thread to tie the lace above and below the bulb.

If the bulbs do not indicate the same temperature, then write on the sling psychrometer the
correction needed.

Using The Psychrometer

Soak just the cotton lace of the wet bulb; do not get the dry bulb wet.
Spin the sling psychrometer for a few minutes
Note: After a few minutes the maximum difference between the wet and dry bulb should
occur
Record the dry bulb temperature and the difference between the dry and wet bulbs
Note: This difference is called the wet bulb depression.
Use the table below to determine the relative humidity and dew point

In this table, the dry bulb reading is found in the vertical column on the left, while the wet bulb
depression is found in the row on the top. Cross-reference the row and column and youll find two
numbers. The first number is the dew point and the second number is the relative humidity in
percent. Record the dew point at the time of launch.

Dew Point TableG

Air temp. (F) Depression of the wet-bulb thermometer

1 2 3 4 6 8 10 12 14 16 18 20 25 30
0 -7 -20
5 -1 -9 -24
10 5 -2 -10 -27
15 11 6 0 -9
20 16 12 8 2 -21
25 22 19 15 10 -3 -15
30 27 25 21 18 8 -7
35 33 30 28 25 17 7 -11

Parallax Inc. 2006 Rev 1.0

 Chapter Nine: Launch Support Equipment Page 33 of 34

40 38 35 33 30 25 18 7 -14
45 43 41 38 36 31 25 18 7 -14
50 48 46 44 42 37 32 26 18 8 -13
55 53 51 50 48 43 38 33 27 20 9 -12
60 58 57 55 53 49 45 40 35 29 21 11 -8
65 63 62 60 59 55 51 47 42 37 31 24 14
70 69 67 65 64 61 57 53 49 44 39 33 26 -11
75 74 72 71 69 66 63 59 55 51 47 42 36 15
80 79 77 76 74 72 68 65 62 58 54 50 44 28 -7
85 84 82 81 80 77 74 71 68 64 61 57 52 39 19
90 89 87 86 85 82 79 76 73 70 67 63 59 48 32
95 94 93 91 90 87 85 82 79 76 73 70 66 56 43
100 99 98 96 95 93 90 87 85 82 79 76 72 63 52

Example
Say you measure a dry bulb temperature of 70 F and a wet bulb temperature of 60 F.

Subtract the 60 from the 70 to get a wet bulb depression of 10 F. Go to the table and find the
intersection of the 70 F air temperature row with the 10 F wet bulb depression column. Youll find
the numbers 70/56. This indicates the relative humidity is 70% and the dew point is 56 F. The air is
holding 70% of the water vapor it can hold at this temperature and if the air temperature drops below
56 F, there will be dew. Note that if the dew point is below 32 F, then there will be no frost until
the air temperature drops below the dew point. So if can be below freezing without there being frost.
Be sure to add measuring the wet and dry bulb temperatures to your launch checklist (if these
measurements are needed).

Near Space Humor - Near Space Comix #1

Parallax Inc. 2006 Rev 1.0

Page 34 of 34 Near Space Exploration with the BASIC Stamp by Paul Verhage

A
One regulator is the Profill Balloon Regulator (Crammer Decker), model BR 3855 with gauge and hand
tightener. If the regulator does not have a hand tightener, then also purchase a crescent or box wrench.
B
As you no doubt recall from high school chemistry, the Ideal Gas Law is stated in the equation:
PV=nRT
Where:
P is the pressure exerted by gas
V is the volume occupied by a gas
n is the amount of gas (in moles) in a sample or parcel
R is Boltzmanns constant
T is the temperature in an absolute scale (Kelvin or Rankin)
C
Water can remain a liquid below 0 degrees Celsius or above 100 degrees Celsius. But eventually the water
does change phases, either with a little more time or with a further change in temperature. The purer the water,
the longer it can remain a liquid.
D
The name originates from psychro (cold), not psych (mind)
E
These are available at school supply stores or science catalogs
F
When purchasing thermometers, select two that read the same temperature
G
Table from http://www.jsu.edu/depart/geography/mhill/phygeogone/unit2/dewtablf.html, Dr. M.H. Hill,
Jacksonville State University

Parallax Inc. 2006 Rev 1.0

 Chapter 10
Motivation and Emotion

Outline

I. Instincts
A. Motivation is the process that arouses, directs, and maintains behavior.
B. In the past, behaviors often were explained in terms of an instinctan unlearned,
complex pattern of behavior that occurs in the presence of certain stimuli.
C. William McDougall believed that humans were motivated by 18 instincts.
1. As more behaviors needed explaining, the list of instincts grew.
2. Truth is, instincts may name, but they dont explain.
3. Still, the approach reminds us that we may engage in some behaviors for
reasons that are basically biological, physiological, and inherited.

II. Needs and Drives

A. In Clark Hulls system, a need is a lack or shortage of some biological essential
required for survival and resulting from deprivation.
1. Needs give rise to drives.
2. A drive is a state of tension, arousal, or activation resulting from an unlearned
need that arouses and directs an organisms behavior.
a. Primary drives are based on unlearned, physiological needs.
b. A secondary drive is a state of tension resulting from a learned or
acquired need that motivates an organisms behavior.
B. Maslows hierarchy of needs is a stage theory with the following levels:
1. Physiological
2. Safety and security
3. Love and belongingness
4. Esteem
5. Self-actualization.
6. Although logical and appealing, Maslows system seems only to fit Western
cultures and has actually received little research support.

III. Incentives
A. Incentives are stimuli an organism may be motivated to approach or avoid.
B. Whereas drives are said to push behaviors, incentives pull behaviors from
without
C. Many of the principles are similar to those of operant conditioning.
IV. Balance or Equilibrium
A. There are several theories that involve the concepts of balance or equilibrium.
B. Walter Cannons theory of homeostasis claimed that internal physiological conditions
seek a balanced set point.
C. Arousal theories suggest that for every task there is an optimal and balanced level of
activation required to complete the task well.
D. Leon Festingers theory of cognitive dissonance argues that we are motivated to
maintain a state of balance or equilibrium among cognitions.

V. Temperature Regulation
A. We are driven to regulate our body temperature.
B. When body temperature changes from normal, the first reaction is physiological.
1. The attempt is to return to set point by a process mediated by the
hypothalamus.
2. The hypothalamus is a small structure near the limbic system in the center of
the brain, associated with temperature regulation, feeding, drinking, and sex.
C. If automatic processes are insufficient, we are driven to engage in behavior to change
body temperature.

VI. The Thirst Drive and Drinking Behaviors

A. There are several cues to thirst.
1. Intracellular fluid loss is monitored by the hypothalamus.
2. A complex chain of events involving the kidneys monitors extracellular loss.
B. Sensory qualities (incentives) can give rise to external cues to thirst.

VII. The Hunger Drive and Eating Behaviors

A. There are several internal cues to feeling hungry.
1. The hypothalamus has both an eat (the lateral hypothalamus) and no-eat
(ventromedial hypothalamus) center.
2. We may be sensitive to levels of blood sugar or glucose.
3. The liver may be sensitive to levels of fat supply.
4. There may be an overall drive to maintain a set point body weight.
5. Genetic factors clearly are involved in weight gain and obesity.
6. An ob gene controls the amount of a hormone named leptin in the
bloodstream which tells the brain how much fat is stored in the body
B. Eating behaviors are influenced by non-physiological, external processes, such as
time-of-the-day, the appearance of food, and social pressures from others.
C. Obesity has become of epidemic proportions in the United States.
1. Technically, obesity is defined in terms of a body mass index of 30 or greater.
2. Nearly one-third of American adults are obese, and rates of childhood obesity
are skyrocketing.
3. Obesity seems to have a clear genetic basis.
4. There seems to be no quick and easy treatment for obesity.
a. The body protects better against weight loss than weight gain.
b. Fully 95 percent of those on a weight-loss program will be back at
their original weight within five years.
 c. The only useful plan seems to be a gradual shift in lifestyle: eat a bit
less; exercise a bit more.
D. Eating disorders afflict approximately 8 million Americans, 90 percent of whom are
women..
1. Anorexia nervosa is characterized by an inability (or refusal) to maintain
ones body weight through self-starvation and/or increased activity.
2. Bulimia is characterized by episodes of binge eating followed by purging.
a. Females most often experience eating disorders.
b. There are no known causes of eating disorders, but social/cultural
pressures and physiological predispositions are both suspected as
being important.
c. Prognosis for anorexia is poor with high degrees of relapse, but the
outlook for bulimia is better, and with new treatment techniques even
the prognosis for anorexia is improving..

VIII. Sexual Motivation and Human Sexual Behaviors

A. Sex can be an important motivator for humans and non-humans alike.
1. On a physiological level, the sex drive is unique.
a. The survival of the individual does not depend upon its completion.
b. It depletes rather than replenishes bodily energy.
c. It requires maturation before it is apparent.
d. Whereas in lower species internal physiological states are of prime
importance, they are less so for higher species such as humans.
2. For humans in particular, hormones are neither necessary nor sufficient to
account for sexual behaviors.
B. Men and women differ in but a few, but in important ways with regard to sexuality.
1. Men demonstrate more interest in sex, fantasize about it more and have a
decidedly greater interest in engaging in sex than do women, a reality that
even holds for men of advancing age nearly 20 percent of men over 100 say
that sex is important to them..
2. Whereas women seek commitment in sexual relationships, men tend to seek
sex first, relationships later.
3. Men are significantly more likely to be the sexual aggressor and initiate sexual
contact.

IX. SPOTLIGHT: Sexual Orientation

A. Homosexuality is a sexual orientation involving sexual attraction and arousal to
members of the same sex.
1. Heterosexuality is a sexual orientation involving sexual attraction to and
arousal by members of the opposite sex.
2. The terms gay, which is most often used to refer to males with a same-sex
orientation, and lesbian, the term for women with a same-sex orientation, are
preferred terms of reference.
3. Homosexuality and heterosexuality are not mutually exclusive categories.
B. In terms of sexual responsiveness, there is little difference between persons of
homosexual and heterosexual orientations.
 1. Homosexual orientation is related to an interaction among genetic, hormonal
and environmental factors.
2. There are no differences in sex hormone levels in adult homosexuals and adult
heterosexuals.
3. There are small but significant differences in the structure of the hypothalamus
between gay and heterosexual men.

X. Psychologically Based Motives

A. The need to achieve (nAch) is the acquired need to meet or exceed some standard of
excellence in ones behavior.
1. The Thematic Apperception Test, a projective personality test requiring a
subject to tell a series of short stories about a set of ambiguous pictures,
measures the nAch.
2. People with high nAch seek tasks in which success is not guaranteed but in
which there is a reasonable chance of success.
3. The need to achieve probably is learned, usually in childhood.
B. The need for power involves the need to be in control, to be in charge of the situation
and others.
1. The need for power in itself is neither good nor bad.
2. There are no reliable differences between men and women in measured needs
for power.
C. The need for affiliation is a need to be with others, to work with others toward some
end, and to form friendships and associations.
D. The need for intimacy is a need to form and maintain close, affectionate
relationships with others.
1. Intimacy involves self-disclosure.
2. Women are more likely than men to show high intimacy needs.
E. Loneliness is a psychological state arising when our actual social relationships are
discrepant from the relationships we would like to have.

XI. Defining and Classifying Emotions

A. There are four components of an emotional reaction.
1. One experiences a subjective feeling of affect.
2. One has a cognitive reaction, i.e., knows what happened.
3. There is an internal physiological reaction.
4. Finally, there is an overt behavioral reaction.
B. Emotions are motivators in that they arouse behaviors.
C. Classifying the subjective feeling component of emotion has been very difficult.
1. Wilhelm Wundt proposed three, intersecting dimensions.
2. Carroll Izard proposed six basic emotions two of them positive (joy and
interest) and four of them negative (sadness, anger, disgust, and fear).
3. Robert Plutchik argued for eight basic emotions, each related to survival and
adaptation.
4. Richard Lazarus defined basic emotion in terms of being motivated to
approach or avoid.
 D. The only issue on which there is any consensus is that emotions represent a valenced
state, meaning that they could be classified as positive or negative.

XII. Physiological Aspects of Emotion

A. Emotionality involves the autonomic nervous system (ANS).
1. The parasympathetic division of the ANS is actively involved in maintaining a
relaxed, calm, unemotional state.
2. When one is emotional, his or her sympathetic division of the ANS takes
over, producing several reactions.
a. The pupils of the eye dilate.
b. Heart rate and blood pressure are elevated.
c. Blood is diverted away from the digestive tract toward the limbs and
brain.
d. Respiration increases.
e. Moisture is brought to the surface of the skin in the form of
perspiration.
f. Blood sugar levels increase.
g. Blood will clot more readily than usual.
B. The two brain structures most involved in emotionality are the limbic system and the
hypothalamus.
C. The role of the cerebral cortex in emotionality seems to be largely inhibitory and
cognitive.

XIII. Theories of Emotion

A. Theories of emotion are systematic attempts to explain how we become emotional
and how the various components of an emotion interact.
B. Common sense tells us that we encounter a stimulus, become emotional, and then
react.
C. The James-Lange Theory claims reality is reversed: We encounter a stimulus, react to
it, and then (noting our reaction) experience an emotion.
D. The Cannon-Bard Theory claims that encountering an emotion-producing stimulus
produces an appropriate response and the experience of an emotion simultaneously.
E. In their theory, Schachter and Singer add a cognitive component, claiming that a
stimulus simultaneously produces a visceral reaction and a cognitive appraisal of the
situation and these two combined produce the experience of emotion.
F. Contemporary theories of emotion (for example, the cognitive appraisal theory) tend
to be cognitive in flavor.
G. Emotional reactions have been demonstrated to occur at an unconscious level
beyond ones awareness

XIV. Outward Expressions of Emotion

A. Charles Darwin was one of the first to understand that facial expressions provide
indicators of an organisms emotional state.
B. Whereas non-human animals have many instinctive patterns of behavior to
communicate emotional state, humans have language.
C. Much research by Paul Ekman has demonstrated a reliable relationship between
emotional states and facial expressions across cultures.
D. Emotions can have several behavioral manifestations.
1. Aggression is a behavior intended to inflict harm on another organism or a
symbol of that organism.
2. The frustration-aggression hypothesis argues that aggression is always a
consequence of frustration.
a. We now know that this hypothesis is too simplistic.
b. When frustration is accompanied by anger, aggression is more likely.
3. What factors lead to the arousal of anger?
a. How we judge the intent of a person who frustrates us
b. The perception that we have been treated unjustly
c. A need to restore justice and equity
d. Feelings of powerlessness
4. Aggressive drivers tend to be young, poorly educated males with a history of
violence and drug or alcohol problems.
 LEARNING OBJECTIVES

1. Discuss how instinct, drive, and incentive have been used to explain motivated
behaviors.

2. Explain how the concept of balance or equilibrium can be used to explain motivated
behaviors.

3. Describe how homeostasis relates to temperature regulation as a physiologically

based drive.

4. List the factors that may influence thirst and hunger.

5. Describe the symptoms of anorexia nervosa and bulimia, and explain the prognosis
for each disorder.

6. Discuss the ways in which the sex drive is a unique, physiologically based drive, and
explain how the cognitive and affective systems operate in the human sexual motive.

7. List the eight dispositional sexual motives.

8. Describe individual and mutual sexual behavior.

9. Explain the male and female sexual dysfunctions and their causes.

10. Discuss achievement motivation, and explain how it is measured.

11. Discuss the need for power, affiliation, and intimacy.

12. List four components that define emotional experience.

13. Discuss the different perspectives and controversies surrounding the classification of
emotion and the search for basic, or primary, emotions.

14. Describe the activities of the sympathetic division of the autonomic nervous system
during states of emotionality.

15. Describe the various brain centers involved in emotionality.

16. Explain the James-Lange and Cannon-Bard theories on the roots of emotion.

17. Discuss the two factors involved in the two-factor theory of emotion.

18. Define the cognitive appraisal theory of emotion and the content process model.

19. Explain how facial expressions help to convey emotion across cultures.
 Key Terms and Concepts
motivation____________________________________________________________________
______________________________________________________________________________
instincts______________________________________________________________________
______________________________________________________________________________
need__________________________________________________________________________
______________________________________________________________________________
drive_________________________________________________________________________
______________________________________________________________________________
secondary drive________________________________________________________________
______________________________________________________________________________
incentive______________________________________________________________________
______________________________________________________________________________
homeostasis___________________________________________________________________
______________________________________________________________________________
arousal_______________________________________________________________________
______________________________________________________________________________
cognitive dissonance____________________________________________________________
______________________________________________________________________________
obesity________________________________________________________________________
______________________________________________________________________________
anorexia nervosa_______________________________________________________________
______________________________________________________________________________
bulimia nervosa________________________________________________________________
______________________________________________________________________________
homosexuality_________________________________________________________________
______________________________________________________________________________
heterosexuality_________________________________________________________________
______________________________________________________________________________
gay___________________________________________________________________________
_____________________________________________________________________________
lesbian_______________________________________________________________________
______________________________________________________________________________
need to achieve (nAch)__________________________________________________________
______________________________________________________________________________
Thematic Apperception Test (TAT)_______________________________________________
______________________________________________________________________________
need for power_________________________________________________________________
______________________________________________________________________________
need for affiliation______________________________________________________________
______________________________________________________________________________
need for intimacy_______________________________________________________________
______________________________________________________________________________
loneliness_____________________________________________________________________
______________________________________________________________________________
emotion_______________________________________________________________________
_____________________________________________________________________________
valenced state__________________________________________________________________
______________________________________________________________________________
parasympathetic division (of the ANS)_____________________________________________
______________________________________________________________________________

sympathetic division (of the ANS)_________________________________________________

______________________________________________________________________________

aggression_____________________________________________________________________

______________________________________________________________________________
 Practice Test Questions

Multiple Choice

1. Which of these is LEAST involved in motivational states?

___a the arousal of behavior ___c. the directing of behavior
___b. the memory of behavior ___d. the maintenance of behavior

2. Of these, which psychological process is LEAST affected by ones motivations?

___a. memory ___c. perception
___b. learning ___d. sensation

3. What is the major problem with using the concept of instinct to explain human behavior?
___a. There are too many human instincts to keep track of.
___b. There are too few human behaviors that have a biological basis.
___c. Referring to instincts may describe behaviors, but it doesnt explain them.
___d. Too many human instincts have opposites, such as needs to socialize and needs to
be alone.

4. In Hulls theory (as an example) what gives rise to a drive?

___a. a need ___c. a behavior
___b. a motive ___d. a goal or incentive

5. Approaches to motivation that focus on stimuli outside the organism are approaches that
focus on
___a. incentives. ___c. arousal.
___b. drives. ___d. homeostasis.

6. Which of these terms is most like Cannons concept of homeostasis?

___a. drive ___c. fulfillment
___b. sensation-seeking ___d. balance

7. Which brain structure is MOST involved in temperature regulation?

___a. the hypothalamus ___c. the brain stem
___b. the limbic system ___d. the corpus callosum

8. Most of the water in our bodies is contained

___a. in our bloodstream.
___b. within the cells of our bodies.
___c. in sweat glands.
___d. in spaces between the cells in our bodies.

9. Which is likely to be LEAST involved in motivating us to eat or not eat?

___a. the physical appearance of food ___c. how empty our stomachs are
___b. our hypothalamus ___d. reactions of our liver
10. Which of these statements is TRUE about bulimia, but FALSE about anorexia nervosa?
___a. It is an eating disorder found mostly in young women.
___b. It involves a preoccupation with ones weight and body size.
___c. With proper treatment, the prognosis for a full recovery is good.
___d. In virtually all cases, at least some hospitalization will be required.

11. In what way is the sex drive in humans most different from the sex drive in rats?
___a. It does not appear until after puberty.
___b. Its satisfaction does not determine the survival of the individual.
___c. It is strongly affected by learning and experience.
___d. Its physiological basis is largely hormonal.

12. With regard to homosexuality, which observation is most TRUE?

___a. Sexual preference is a matter of choice freely made.
___b. Gay males have excess levels of female hormones in their systems.
___c. Homosexuality is a dimension, a matter of degree, not either/or.
___d. Most homosexuals (male and female) have not tried heterosexual sex.

13. At the moment, which of these can be taken as the most reasonable hypothesis for the
development of a homosexual orientation?
___a. genetic differences in X and Y chromosomes
___b. the lack of a father-figure in single-parent homes
___c. unsatisfying or frustrating sexual encounters in early adolescence
___d. hormonal imbalances that occur during prenatal development

14. If given a choice, a person with a high need to achieve (aAch) would probably chose a job in
which he or she
___a. could succeed with very little effort.
___b. would be in a position to control the fate of others.
___c. would be working with as many people as possible.
___d. could do well, but only with effort and hard work.

15. Of the following, which question reflects a current debate concerning the nature of
emotions?
___a. Do facial expressions express emotions?
___b. Are cognitions required for an emotional experience?
___c. Does becoming emotional involve the ANS?
___d. Do emotions serve any useful adaptive functions?

16. Which change is LEAST likely during an emotional reaction?

___a. Heart rate decreases. ___c. Blood flow is diverted to the limbs.
___b. Digestion stops. ___d. Pupils dilate.
17. According to the characterization presented in the text, which of these is NOT included in
our conceptualization of an emotional reaction?
___a. a subjective feeling, or affect
___b. a cognitive awareness of what is happening
___c. a judgment of whether the emotion is adaptive
___d. an overt behavioral reaction

18. How many basic emotions are there?

___a. 4 ____c. 9
___b. 8 ____d. It depends.

19. Which aspect of the brain is most directly involved in emotion?

___a. the thalamus ___c. the brain stem
___b. the limbic system ___d. the basal ganglia

20. In emotional states, the major role of the cerebral cortex seems to be to
___a. trigger reactions in lower centers, like the limbic system.
___b. increase heart rate and blood pressure.
___c. cognitively interpret the situation at hand.
___d. cause the organism to engage in fight or flight.

21. According to the James-Lange Theory of emotion, after an emotion-arousing stimulus is

perceived, the next reaction is
___a. a cognitive appraisal ___c. activation of the thalamus
___b. an appropriate internal response ___d. the experience of the emotion

22. Which expression of emotion is uniquely human?

___a. verbal description ___c. body language
___b. facial expression ___d. posture and gestures

True/False

1. ____True ____False The concepts of balance, equilibrium, and set-point refer only to
physiological conditions or physiological processes.

2. ____True ____False Arousal theory tells us that ones performance on a task will continue
to improve as ones level of arousal continues to increase.

3. ____True ____False Although our hypothalamus may inform us that we are hungry or
thirsty, learning and experience inform us about what to eat or drink.

4. ____True ____False The prognosis for anorexia nervosa is significantly better than the
prognosis for bulimia.

5. ____True ____False Whereas men are motivated by a need to achieve, women are
motivated by a fear of failure.
6. ____True ____False Emotions that we classify as negative seldom have any survival
value.

7. ____True ____False Most psychologists agree that there are four basic, or primary,
emotions.

8. ____True ____False The only emotions that appear to be universal are joy and fear.
 Answers to Practice Test Questions
Multiple Choice

1. b We say that motivation arouses, directs, and maintains behavior. Now memory is surely
involved in motivationits involved in nearly everythingbut it is less central than the
other three.
2. d Actually, the more complex, or higher a psychological process, the more likely that
motivation will be a significant factor. The best choice here is the nearly physiological
process of sensation.
3. c There is value to the notion of instinct, of course, but with regard to human behavior,
instinct tends more to name and describe than to explain anything.
4. a Dont get rattled just because I dropped a name in here. In virtually anyones system
including Hulls needs give rise to drives.
5. a Drives, arousal, and homeostasis all refer to conditions or states within the organism,
whereas incentives are thought of as being out there in the environment.
6. d By definition, homeostasis is a condition of balance or equilibrium.
7. a This one is fairly obvious. The hypothalamus would be the best guess if you werent sure
it seems to be involved in almost all physiological drives.
8. b Most of the water in our bodies is stored within the cells.
9. c Clearly it has an impact, but curiously, of these choices, the sense of fullness of the
stomach is the least important.
10. c Alternatives a and b are both true. Alternative d is true of anorexia, but not bulimia.
11. c Alternatives a, b, and d are true of the sex drive for both humans and rats. The sex drive
and sexual behaviors in rats do not seem to be much affected by learning or experience.
12. c Only the third alternative is true, and beyond that, the others are very false.
13. d The total picture is far from clear. Were quite sure that none of the observations made in
the first three alternatives are even relevant, and we are becoming quite convinced that
the fourth alternative makes the best statement that we can make right now..
14. d The best alternative here is the last one.
15. b I dont think that any of the others are debatable at all, but there is now quite a discussion
centering on the role or even the necessity of cognition for emotional experiences.
16. a Heart rate increases, it doesnt decrease in fact, think of this item in terms of what
would you like to have happen if you were faced with a bear in the woods?
17. c It is not possible to have an emotional reaction without the processes named in
alternatives a, b, and d. If there is ever a judgment made about the adaptive value of an
emotion, it would be made later.
18. d This one is sort of silly, isnt it? Different theorists have different ideas, and for now the
safest thing that we can say is that it depends mostly on which theorist youd like to
believe.
19. b As weve seen, all of the parts of the brain tend to work together and all of its aspects are
involved in all reactions, but having said that, it is the limbic system that is most involved
in emotionality.
20. c The main role of the cerebral cortex is to bring a cognitive, thoughtful analysis to the
situation that is being experienced.
21. b Remember, the James-Lange approach is nearly the opposite of common sense, arguing
that we respond (internally, perhaps) first, then experience an emotion.
22. a This has less to do with emotion than it does with the difference between humans and
nonhumans. Only humans can talk (verbalize) about how we feel.

True/False

1. F At one point in history, we might have been able to say that this statement was true, but
now we see that these concepts can be applied well beyond physiological functioning.
2. F Well, at first, maybe, but what makes this statement false is that if arousal continues to
rise, eventually it will become so high as to be debilitating.
3. T If the truth of this statement is not obvious, you only need think about it for a moment
longer.
4. F No, in fact, quite the opposite is true.
5. F With regard to achievement and failure, there is little evidence that there are any sensible
gender differences at all.
6. F Actually, some of the negative emotions fear, for example may have more
survival value than some of the positive emotions.
7. F Heres this one again. Previously, it was item #4 in multiple-choice form. Psychologists
have come to no general agreement on the number of primary emotionsand are
wondering if there even is such a thing as a primary, basic, emotion.
8. F The text addresses the universality of facial expressions of emotion and suggests that
anger, fear, disgust, sadness, and happiness are all expressed in the same way across
cultures. This does not mean that there are not even more emotional expressions in
common, as yet not confirmed.
 Experiencing Psychology

Joy to the World

This is a little exercise that demonstrates just how complex emotions can be particularly for
those who would like to organize or categorize basic human emotions.

JOY or HAPPINESS is commonly considered to be a basic emotion. There are a great many
words in our language that are somehow related to the basic concept of joy or happiness. Here is
a list of some of them:

BLISS AMUSEMENT CHEERFULNESS

GAIETY GLEE JOLLINESS
JOVIALITY DELIGHT GLADNESS
ENTHRALLMENT ENJOYMENT HAPPINESS
JUBILATION ELATION SATISFACTION
ECSTASY EUPHORIA ENTHUSIASM
ZEST EXCITEMENT THRILL
PLEASURE CONTENTMENT TRIUMPH
PRIDE EXHILARATION EAGERNESS
OPTIMISM HOPE JOY
RAPTURE MIRTH LEVITY
RELIEF GIDDYNESS

Write each of these (and any others you can think of) on a sheet of paper. Read each of the
following statements to others and ask them to indicate which of these words best describes the
situation being described. For which item(s) on the list was there the most agreement? Did you
notice any sex differences in which terms were chosen?

1. Colleen didnt think that she would be accepted at the college she most wanted to attend, but
she just received notification that she had been accepted.
2. Toms uncle told him that he was sending him a check. Tom was expecting a check for about
$100. He has just opened the envelope from his uncle and found a check for $3000.
3. One of Juanitas professors has just read her paper to the class as an example of a thoughtful
and well-written paper.
4. An instructor who usually dresses very conservatively has just walked into class wearing an
oversized T-shirt with a picture of Mickey Mouse on the front.
5. A male student that Mary admires very much has just asked her if she will meet with him and
help him with his math assignment.
6. Sams mother had a brain tumor surgically removed two days ago. Sam has just received
word from his father that the tumor was not malignant.
7. Jans best friend has just told her that she and Jack, whom Jan admires and likes a lot, are
planning to get married at the end of the term.
8. Julio worked hard campaigning for Alice Hawkins for student body president. He has learned
that she has been elected with 71 percent of the votes.
9. It is Fathers Day. The picnic is over, and Ralph is thinking about what great kids he and
Evelyn have and how beautiful his six grandchildren are.
10. Gail went to visit her friend Margaret. When she arrived, seventeen of her friends were there
to give her a surprise birthday party.
11. The party that Judy had worked so hard to plan was a great success. The guests have all
gone home, and Judy is exhausted. She is thinking about the party as she settles herself into
bed.
12. Joes favorite team just won the Super Bowl.
 Psychology on the Internet

1. WHAT MOTIVATES US?

What motivates us? What arouses, maintains, and directs our behaviors? As we saw in Chapter
Ten, the general answer is, Lots of things, and we reviewed many of them. Information about
the various approaches to motivation and the specific drives and motives covered in the chapter
also can be found on the Internet. Here is a sampling.

http://allpsych.com/psychology101/motivation.html
(the AllPsych Online websites are not very dramatic, are a bit wordy, but nonetheless provide good
summaries of important issues this can be said for this site on approaches to motivation)

http://changingminds.org/explanations/theories/a_motivation.htm
(the list of approaches to motivation found in Chapter Ten is not an exhaustive one (although we
do believe that we have included the most important). Here you will find a description of some
that we have sampled and several that we have not. Before clicking to the various theories
presented here, click on the link to ChangingMinds.org homepage to get a sense of why some
theories from Chapter Ten are listed, and why some are not.)

2. SPECIFIC DRIVES AND MOTIVES

Here we have another case where common sense suggests it best to address specific drive and
motives in one section. Most of the websites to found on the Internet cover hungerand
obesity.

http://www.purchon.com/biology/osmoregulation.htm
(a website on thirst and theres a lot of good information here. I suspect that some students may
be immediately attracted to the link, What happens if you drink too much beer?)

http://www.eufic.org/gb/food/pag/food12/food121.htm
(a few words and good advice from the European Food Information Center concerning the
necessity of maintaining levels of bodily fluids. It is short.)
http://www.i-sis.org.uk/ObesityEpidemic.php
(manyif not mostof the Internet websites with obesity in their URLs are commercial sites
trying to sell you something to either treat obesity or keep you from becoming obese. This non-
commercial website from the UK is a product of the Institute for Science in Society.)

http://www.obesity.org
(The American Obesity Association calls itself The Leading Organization for Advocacy and
Education on Obesity. Their homepage also claims that they think that theirs is the most
comprehensive site on obesity and overweight on the Internet. On this claim I am ready to
agree. Typical of good websites, the strength of this one is in the extensiveness of their links.)

http://www.cdc.gov/nccdphp/dnpa/obesity
(If the American Obesity Association (above) has not answered all of your questions, you can
turn here. It is from The Centers for Disease Control and Prevention. There are many links
here.)

http://www.niddk.nih.gov/index.htm
(the website of the National Institute of Diabetes and Digestive and Kidney Diseases of the
National Institutes of Health. It approaches obesity from a medical perspective their outline
looks like a copy of our issues in Chapter Ten.)

http://www.nationaleatingdisorders.org
http://www.edauk.com
(Both of these websites on eating disorders can be highly recommended.)

http://health.yahoo.com/health/centers/sexual_health/2390
(not terribly exciting, and focusing on problems with sexuality is this site, called the Sexual
Health Center)

3. PSYCHOLOGY AND EMOTIONS

Psychologists from the time of Wilhelm Wundt (and non-psychologist thinkers long before him)
have been intrigued by emotions. As central as the study of emotion has been in psychology,
answers to even basic questions have proven difficult to pin down. Are there basic, fundamental
emotions? Are any emotions universal? How are emotions expressed in a social context? There
is little doubt that emotions color our lives and give value to our experience. For the sadness of
loss there also is the joy of discovery. There are simply not all that many Internet websites
devoted to the psychology of emotion in the way we want approach as students of the science of
psychology. There are many more self-help and commercial sites.

Although your textbook divides the topic of emotion into five sections, from Defining and
Classifying Emotions to the Frustration-Aggression Hypothesis, the Internet has no such
organization. Hence, the following websites are presented more or less in order, and covering
the entire issue of emotionality.

http://plato.stanford.edu/entries/emotion
(the Stanford Encyclopedia of Philosophy includes this lengthy, relevant essay on the nature of
emotions. It is wordy, but it covers a lot of familiar ground. I suggest starting at the top and
working your way down through what is here. Toward the bottom is a link to Other Internet
Resources. Go there and then to Links on Emotion what comes up is nearly
overwhelming.)

http://emotion.nsma.arizona.edu/emotion.html
(the Emotion Home Page perhaps my best find so far. What a grand and glorious
repository if information about emotion research. You can easily spend hours exploring here,
overturning a new gem with nearly every click of your mouse.)

http://changingminds.org/explanations/emotions/emotions.htm
(Simple-appearing at first, this website has many very good links. It takes up issues such as
basic emotions, purpose of emotions, and the seven deadly sins and seven virtues all of
them emotions at that!)

http://www.paulekman.com
(this website is devoted to Paul Ekman and his research on the facial expression of emotions.
The homepage is simple and navigating it is easy. A good bit of what you will find here is
surprisingly commercial. You probably will find the link PUBLICATIONS most satisfying.)
 Chapter Eleven

Taylor Series

11.1 Power Series

Now that we are knowledgeable about series, we can return to the problem of
investigating the approximation of functions by Taylor polynomials of higher and higher
degree. We begin with the idea of a so-called power series. A power series is a series of
the form
n

k= 0
ck (x a) k .

A power series is thus a sequence of special polynomials: each term is obtained from
the previous one by adding a constant multiple of the next higher power of (x a).

Clearly the question of convergence will depend on x , as will the limit where there is one.
The k th term of the series is ck (x a)k so the Ratio Test calculation looks like

c k +1 (x a)k +1 c
r(x) = lim = x a lim k +1 .
k ck (x a) k k c
k

Recall that our series converges for r(x) < 1 and diverges for r(x) > 1 . Thus this

ck +1
series converges absolutely for all values of x if the number lim = 0 . Otherwise, we
k ck

ck
have absolute convergence for | x a | < lim and divergence for
k c
k +1

ck ck
| x a | > lim . The number R = lim is called the radius of convergence,
k c k ck +1
k +1

and the interval | x a |< R is called the interval of convergence. There are thus exactly
n
three possibilities for the convergence of our power series ck (x a) k :
k= 0

(i)The series converges for no value of x except x = a ; or

11.1
 (ii)The series converges for all values of x ; or
(iii)There is a positive number R so that the series converges for | x a |< R and
diverges for | x a | > R .

Note that the Ratio Test tells us nothing about the convergence or divergence of the
series at the two points where | x a |= R .

Example
n c k! 1
Consider the series k!x k . Then R = lim k = lim = lim = 0.
k=0 k c
k +1
k (k +1)! k k +1

Thus this series converges only when x = 0.

Another Example
n c 3k 1 1
Now look at the series 3k (x 1)k . Here R = lim k = lim k +1 = lim = .
k=0 k c
k +1
k 3 k 3 3
1 1
Thus, this one converges for | x 1|< and diverges for | x 1|> .
3 3

Exercises

Find the interval of convergence for each of the following power series:

n
1. (x + 5)k
k=0

n 1
2. (x 1)k
k=0 k

11.2
 n k
3. (x 4)k
k = 0 3k + 1

n 3k
4. (x +1) k
k = 0 k!

n k!
5. (x 9) k
k = 0 7(k +1)
2

11.2 Limit of a Power Series

n
If the interval of convergence of the power series ck (x a) k is | x a |< R , then,
k= 0

of course, the limit of the series defines a function f :

f (x) = c (x a)
k
k
, for | x a |< R .
k=0

It is known that this function has a derivative, and this derivative is the limit of the
derivative of the series. Moreover, the differentiated series has the same interval of
convergence as that of the series defining f . Thus for all x in the interval of convergence,
we have

f '( x) = kc (x a)
k
k 1
.
k=1

We can now apply this result to the power series for the derivative and conclude that
f has all derivatives, and they are given by

f (p ) (x) = k(k 1)K(k p + 1)c (x a) k
k p
.
k= p

Example

11.3

1
We know that = x k for | x |< 1 . It follows that
1 x k=0

1
(1 x) 2
= kx k 1 = 1+ 2x + 3x 2 + 4x3 +K
k =1

for | x |< 1 .

It is, miraculously enough, also true that the limit of a power series can be integrated,
and the integral of the limit is the limit of the integral. Once again, the interval of
convergence of the integrated series remains the same as that of the original series:
x
ck
f (t)dt = k +1(x a)
k +1
.
a k=0

Example
We may simply integrate the Geometric series to get

xk +1
log(1 x) = , for 1 < x < 1, or 0 < 1 x < 2.
k=0 k + 1

It is also valid to perform all the usual arithmetic operations on power series. Thus if

f (x) = c x k
k
and g(x) = d x k
k
for | x |< r , then
k=0 k=0

f (x) g(x) = (c k dk )x k , for | x |< r .
k =0

Also,

k
f (x)g(x) = ci dk i c kx k , for | x |< r .

k= 0 i= 0

The essence of the story is that power series behave as if they were infinite degree
polynomialsthe limits of power series are just about the nicest functions in the world.

11.4
Exercises

n
6. What is the limit of the series x 2k ? What is its interval of convergence?
k=0

n
7. What is the limit of the series 2(1) k kx 2k 1 ? What is its interval of convergence?
k =1

8. Find a power series that converges to tan 1 x on some nontrivial interval.

9. Suppose f (x) = c (x a)
k
k
. What is f (p ) (a) ?
k=0

11.3 Taylor Series

Our major interest in finding a power series that converges to a given function. The
obvious candidate for such a series is simply the sequence of Taylor polynomials of
increasing degree. Thus if f is a given function, and a is a point in the interior of the
domain of f, the Taylor Series for f at a is the series
n f (k ) (a)

k = 0 k!
(x a)k .

The Taylor Series is thus an infinite degree Taylor Polynomial>

In general, the Taylor series for a function may not converge on any nontrivial interval
to f , but, mercifully, for many sufficiently nice functions it does. In such cases, we are
provided with the nice answer to the question proposed back in Chapter Nine: Can we
approximate the function f as well as we like by a Taylor Polynomial for sufficiently large
degree?

11.5
Example
n x 2k + 1
The Taylor series for f (x) = sin x at x = a is simply (1)k . An easy
k=0 (2k +1)!

calculation shows us that the radius of convergence is infinite, or in other words, this
power series converges for all x . But is the limit sin x ? Thats easy to decide. From
Section 9.3, we know that
n
x 2 k+ 1 | x |2 n+ 3
sin x (1)k ,
k =0 (2k +1)! (2n + 3)!

and we know that

| x |2 n + 3
lim = 0,
n (2n + 3)!

no mater what x is. Thus we have

x 2 k+ 1
sin x = (1) k
, for all x .
k=0 (2k +1)!

Exercises

10. Find the Taylor Series at a = 0 for f (x) = ex . Find the interval of convergence and

show that the series converges to f on this interval.

11. Find the Taylor Series at a = 0 for f (x) = cos x . Find the interval of convergence and

show that the series converges to f on this interval.

12. Find the derivative of the cosine function by differentiating the Taylor Series you
found in Problem #11.

13. Find the Taylor Series at a = 1 for f (x) = logx . Find the interval of convergence and

show that the series converges to f on this interval.

11.6
14. Let the function f be defined by
0, for x = 0
f (x) = 1/ x 2 .
e , for x 0

Find the Taylor Series at a = 0 for f. Find the interval of convergence and the limit of
the series.

11.7
  odels of
M
12 Assessment
Janine Bolger and Patrick Walker

Key Themes
Assessment is a core activity of social work practices, which should be a process
capable of responding to dynamic factors in the lives of service users.
Assessment is underpinned by a series of principles that serve to guide and
direct practice.
The legal and policy context of assessment is essential to understand as this
sets a mandate for appropriate social work practice.
Models and frameworks for assessment provide guides for practitioners and are
underpinned by the skills and knowledge to inform the what, how and why of
assessment.
Assessment is founded on partnership with service users, but may be undertaken
in both voluntary and involuntary contexts.

INTRODUCTION
The concept of assessment is generally associated with notions of appraisal, making
judgements, forming opinions or calculating the value of something. Whilst these
provide a helpful starting point, they require much further examination when applied
to a social work context, where assessment is a discrete, core activity and a key skill.
This chapter will introduce you to elements of social work assessment, incorporating
principles, context, models, frameworks, skills and practice issues. It will draw reference
from across the range of service user groups and invites you to reflect on and critically
explore the material.
To begin to understand the meaning of assessment in social work, consider your
understanding of assessment in your day-to-day life. Everyone makes numerous
assessments every day in order to navigate their way through the daily interactions
and situations that they face. In making these day-to-day assessments you will use a
wide variety of perspectives that give meaning to the information that is presented,
or help sift the information that is presented or found. Perhaps personal experience

12_Lishman_Ch-12.indd 169 15/01/2014 2:31:54 PM

 170 Assessment

helps you, or perhaps your own cultural beliefs offer a way of interpreting situations or
environments. Some of these factors will have relevance to assessment in social work
and highlight the importance of being self-aware.

DEFINING ASSESSMENT IN SOCIAL WORK

Despite the large body of literature regarding assessment in social work, it remains a
much debated area, not least because of the variety of approaches, perspectives and
frameworks that are available. There is certainly agreement that assessment is a core
activity but less consensus on what actually constitutes a good assessment and whether
assessment is separate or integral to intervention. Coulshed and Orme (2012) describe
assessment as an ongoing process, which is participatory, seeks to understand the service
user and his/her situation and sets a basis for planning how change or improvement can
be achieved.
In a similar vein, Payne (2008) identifies assessment as something that is continuous
and ought to be part of a cycle. In this respect assessment is seen as a process rather than
an event; although Payne highlights that practice reality often does not reflect this.
Emphasising the process aspects, Milner and OByrne (2009) put forward a framework
for assessment with five key stages:

1 Preparing for the task.

2 Collecting data from all involved.
3 Applying professional knowledge to analyse, understand and interpret the informa-
tion gathered.
4 Making judgements.
5 Deciding and/or recommending.

For our purposes we would propose to define assessment in social work as a structured
activity with the characteristics shown in Figure 12.1.

PRINCIPLES OF ASSESSMENT
The purpose of this section is to offer some principles, or core common features, of
assessment. The discussion here cannot be prescriptive, but rather is indicative, for
reasons that will become apparent. Assessments are frequently context-specific and
consequently are shaped by the inclusion of particular elements and influenced by the
manner in which the assessment is undertaken.

Case Study
Consider the following three different types of assessment that may take place
under the auspices of services for children and young people:

12_Lishman_Ch-12.indd 170 15/01/2014 2:31:54 PM

 Models of Assessment 171

1 An assessment in a family centre may focus on elements of parenting capacity,

or parentchild interaction.
2 A comprehensive assessment for a Childrens Hearing (in Scotland) may require
capturing a much wider picture of the child in the context of his/her family,
school/community and social setting.
3 An assessment in a Youth Justice team may employ a standardised, structured
assessment focusing on specialised areas related to offending.

As a consequence, we need to be very clear about our role, remit and the context of any
assessment.
Guiding principles help clarify and direct practice in all areas of assessment. They
may be drawn from ethical frameworks, theoretical perspectives, legal obligations and
practice guidance and are important because, although various frameworks can be

Purposeful
and appropriate
Leads to
Dynamic and
planning and
responsive
future action

Involves
judgement to Based on
support engagement
decision-making and inclusion

Assessment

Holistic and Ethical and

comprehensive skilled

Highlights Grounded in the

protection and legal and policy
safe-guarding context
Empowering
and outcomes
focused

Figure 12.1 Structure of assessment in social work

12_Lishman_Ch-12.indd 171 15/01/2014 2:31:55 PM

 172 Assessment

used in assessment, it should be remembered that, as Statham and Kearney point out,
social work can never be a purely technical activity based simply on assessment formats,
models or methods (2007: 102, emphasis added). This suggests that some underpinning
and orientating principles are required when beginning and undertaking the assessment
process. Five key principles are presented here.

Principle 1: Understanding Need

Daniel (2007: 116) states clearly that need can offer a guiding principle for the social
worker; it orientates the practitioner towards exploring and understanding the service
users situation. In some cases, need will be immediate, for example in situations of child
or adult protection. In other cases, need may emerge over a longer period and relate to
support and quality of life, for example befriending to address social isolation.
Horder (2002: 117) notes that good assessment in social work has always been needs-
led, although he alerts the reader to the fact that need can be understood differently
by people and can become a contested concept. Horder goes on to suggest that need is
in most cases defined by others rather than as perceived by the person being assessed.
This is the practitioners dilemma: how to understand, take into account and respond to
the service users view of their needs, whilst also acting within employers requirements,
using professional theories and with normative concepts of need in mind.
Spicker (2012) offers the following:

The idea of need refers to:

the kinds of problem which people experience;
requirements for some particular kind of response; and
a relationship between problems and the responses available. A need is a claim for
service

and points the reader towards Bradshaws (1972) taxonomy of need, summarized as:

Normative need, which is identified according to a norm (or set standard); such
norms are generally set by experts. Benefit levels, for example, or standards of unfit-
ness in houses, have to be determined according to some criterion.
Comparative need concerns problems that emerge by comparison with others who
are not in need. One of the most common uses of this approach has been the com-
parison of social problems in different areas in order to determine which areas are
most deprived.
Felt need, which is based on the perspective of the people who have it.
Expressed need, which is need people say they have. People can feel need that they
do not express and they can express needs they do not feel.

Need, as a principle, can determine what the social worker may require to explore in
assessment. If children need, for example, a secure relationship with their parent or
caregiver, to grow and develop, then the absence of it becomes a risk to them. Similarly,

12_Lishman_Ch-12.indd 172 15/01/2014 2:31:55 PM

 Models of Assessment 173

if an adult with mental health problems needs support and counselling to manage
auditory hallucinations, then an absence of such support may significantly impact on
the persons health and well-being. By understanding need and drawing on broader
knowledge and theory, the practitioner is able to consider the persons situation and to
explore approaches to relieving the unmet need.

Exercise
The word need is used commonly by everyone, so the expression needs-led
should be fairly straightforward. However, think about when you and others use the
word and substitute the word want in place of need. How often is want a more
appropriate term, and how often is need the correct description of the situation?

Principle 2: Working with Systems and Ecology

Like most people, users of social work services very rarely live in a vacuum. Most of
us are part of systems, networks and connected relationships that serve to shape and
influence our lives in complex and multifaceted ways.
Bronfenbrenners Ecological Systems Theory (cited in Daniel, 2007: 116) suggests
that individuals are situated within layers of systems from immediate family up to wider
society and any assessment is required to take account of these layers of connections and
influences. Although this theory was developed in the context of child development,
the levels and systems identified are just as applicable in work with other service
user groups. The central aspect to draw on here is an understanding of how different
factors influence and are influenced by the individual. The ripple model proposes four
interconnected levels surrounding the individual:

Micro-system: the family, school, workplace etc.

Meso-system: the interaction of two different micro-systems
Exo-system: the community/external environment
Macro-system: the sociocultural context.

Bronfenbrenner later added a fifth level, the chrono-system, this being the dimension of
time relating to an individuals life events and environment for example the influence
of time in relation to reactions to the death of a parent, relocation, a relationship
breakdown and so on.
See the article by Hill (2002) on the Companion Website (www.sagepub.co.uk/
SocialWork) which reviews the social network approach to social work assessment.

Principle 3: Building on Strengths

If social workers are to work collaboratively with individuals and families the assessment
process must take account of capacities, strengths and protective factors. In doing so
social workers will take an assets-based approach which seeks to recognise resilience

12_Lishman_Ch-12.indd 173 15/01/2014 2:31:55 PM

 174 Assessment

and capacity for change (Daniel et al., 2010). An assessment that explores strengths
can reveal an individuals or familys ability to resolve their difficulties using their own
skills and expertise without becoming disempowered through service involvement. The
very process of assessment can help individuals or families to identify and utilise latent
strengths and thus reduce dependency on professionals.
This principle is relevant when working with all service users and in all domains of social
work, whether the practice base is termed Children and Families, Learning Disability
Services, Social Work with Older People and so on. Assessments that purely focus on
deficits are not only likely to be demoralising and incomplete, but also run the risk of
being oppressive, result in inappropriate labelling and potentially limit an individuals (and
their networks) abilities to resolve their difficulties themselves. Compensatory strengths
need to be explored, identified and added in to the equation whenever they are present.

Principle 4: Being Person-centred

We have outlined above the principle of systems and ecology and believe that such a
perspective is important. Whilst there is a requirement to keep this ecological perspective,
it must be emphasised that no assessment should lose sight of the fundamental needs
of the child or adult at the centre. Taking a person-centred or child-centred approach
sharpens the focus of social work practitioners to their primary concern. It can be, in
practice, all too easy to become side-tracked into the needs of others. It is a reality
that social workers are often engaged in working in complex situations where the
voices of these others are stronger and more articulate. A person-centred approach to
assessment should involve direct interaction with the adult or child and be informed by
the theoretical and knowledge base underpinning practice.

Exercise
Consider your own situation now. What are the positives and negatives in your own
situation? How do you interact with your family and how does your family interact
with neighbours, friends, work and the wider community? How much of this is relevant
to your current situation?

Principle 5: Taking an Interprofessional Approach

An interprofessional approach to assessment highlights the importance of recognising
that different professionals have particular areas of expertise. No one professional can
have the whole picture that makes up the lived existence of an individual. Given that
each profession will have unique insights and understandings, it is important to pull
these together and to make sense of them. The value of interprofessional practice is
brought into sharp relief through the following quotation from Bronstein (2004, cited
in McLean, 2007: 339). In it she describes interdisciplinary collaboration as:

12_Lishman_Ch-12.indd 174 15/01/2014 2:31:55 PM

 Models of Assessment 175

an effective interpersonal process that facilitates the achievement of goals that can-
not be reached when independent professions act on their own

Interprofessional assessment therefore brings together professional perspectives, and,

when these are collated and acted on, offers potential for a more comprehensive,
coherent and relevant approach to assessed need across multiple, but connected, areas
of peoples lives.
Of course, when we speak about taking an interprofessional approach (see Chapter
14) there is a tendency to focus on the contribution of professionals, but this is not the
whole story. By no means should service users and their families be excluded from such
an approach they are experts on themselves.

THE LEGAL AND POLICY CONTEXT OF ASSESSMENT

Across the four jurisdictions of the United Kingdom there is a raft of legislation and
policy concerned with social work and social care. Much of this legislation and policy
impacts directly on the assessment process, and consequently it is essential to acquire this
knowledge and learn to use it effectively. We shall address some of the broader points that
you need to be aware of, but for more detail you should also refer to Chapters 2 and 3.
One of the characteristics of assessment we identified earlier was that it should be
grounded in the legal and policy context. It is vital when undertaking assessment that
you are aware of what you can do, what you must do and what you may not be allowed
to do. In a legal context powers are what you can do in specific circumstances; duties
are what you must do in specific circumstances; and restrictions refer to any limitations
placed on the worker (Thompson, 2009). Whatever your field of practice, you need to
establish the legislative and policy framework within which your work takes place.
Given that the law generally regulates the activities of social work practitioners and
the organisations those practitioners work for, it is essential to recognise that the law
also holds those practitioners and organisations to account for the work undertaken.
This may seem on the one hand intimidating, but it is the natural companion of having
legislation and policy that gives you a mandate for practice, particularly in the context
of social work in statutory settings.
The third general point is that legislation and policy can act as a powerful tool in
empowering service users and promoting their rights. Often you will work with service
users who have a limited understanding of their rights and are not fully aware of the
options that are available to them. By understanding the law and what can and should
be done, you can assist service users to improve the quality of their lives, achieve their
outcomes and protect their interests ( Johns, 2011).
Lastly, it is important to be aware that whilst the law may appear prescriptive it is
applied in the context of individual lives, with all the complexity that brings. In this
sense, social workers still need to be able to make judgements and negotiate tensions
between legal principles and processes and the values and approaches that underpin
social work practice (Gordon and Davis, 2011: 1).

12_Lishman_Ch-12.indd 175 15/01/2014 2:31:55 PM

 176 Assessment

MODELS AND FRAMEWORKS OF ASSESSMENT

The increased emphasis on assessment in social work, particularly of risk, has created
more theories about the purpose, process and practice of assessment. Increased focus on
recording has resulted in the production of a number of proformas, many of which are
used by a range of professionals, including those in health, social work and education.
These standardised formats are supported with social work values and theories and so
can be viewed as value-based. The information gathered is from the workers perspective
and, therefore, the outcome of the assessment can be influenced by the attitudes and
values of the assessor. The social worker has a responsibility to the service user to be
both reflective (consciously looking backwards) and reflexive (using innate skills in the
moment) on their practice.
Agreement is required between what to do, how it can be done and why it needs to
be done. Above all the purpose of assessment must be clear (Doel and Shardlow, 2005).
Assessment frameworks do not ensure effective practice in their own right as they only
provide us with a framework to assist what is a complex activity. The process of assessment
must be underpinned by knowledge around current policy trends, professional codes of
practice, the attitudes of the workers, their managers, the organisations involved and
should be supported by good assessment skills (Statham and Kearney, 2007: 102).
The purpose of carrying out an assessment is usually to identify levels of need or
risk or to form an understanding when making first contact with the service user.
Depending on the kind of information we need to gather, Smale et al. (1993) offer
us three models the Procedural, the Questioning and the Exchange to guide us in
carrying out assessments.

The Procedural model, often associated with guidance related to legislation, involves
using systems that are devised to ensure consistency and thoroughness in data col-
lection. Consequently, eligibility for and allocation of services is often decided upon
as a result of the collection of such data. This can provide only a snapshot assessment,
directing the assessment away from examining the individuals strengths and abili-
ties, and can divert from individual rights or concerns over quality of life (Milner and
OByrne, 2009). The concern is that such systems can replace rather than support or
inform judgements made by professionals (Barry, 2007 cited in Milner and OByrne,
2009), and may be viewed as rigid, time-consuming (lots of forms) and one-way,
in that it meets the needs of the worker and agency rather than that of the service-
user. The difficulty arises when information is collected on an individual by different
professionals with a different focus (i.e., health, housing etc.) but stored separately.
This results in an inadequate understanding of the total experience of any individual
by any one professional. Workers can become caught up in the process of gathering
information rather than in trying to understand what the service user needs. On a
more positive note, this systematic manner of collecting large amounts of data has
also contributed to the evidence base for social work practice.
The Questioning model of assessment focuses on the nature of the questions and how
the information is used. Using this approach problems and solutions reside with the

12_Lishman_Ch-12.indd 176 15/01/2014 2:31:55 PM

 Models of Assessment 177

individual and the social workers task is to identify the problem and highlight the
most appropriate approach to resolve the issue. A criticism of this model is that it
can be seen as oppressive given that the social worker takes on the role of expert and
makes the final decision. However, if questions are asked in order to try to under-
stand what is impacting on the current situation, and if a range of perspectives are
sought, then this does not have to be the case.
When adopting the Exchange model the service user becomes the expert with regard
to their own needs and through their involvement in their own assessment becomes
empowered. It acknowledges that the workers expertise lies in their problem-solving
abilities. The aim, through development of trust, is to seek a compromise between
choices and needs through involvement of all parties. The worker takes on responsi-
bility for managing the process of assessment. The focus is on a holistic assessment
of the context in relation to the individual over time (Coulshed and Orme, 2012).

Specific frameworks have been outlined in the Case Study to demonstrate how models of
assessment can support particular frameworks or approaches to information gathering.

Case Study
1. The Common Assessment Framework (CAF) in England and Wales, the
UNOCINI Assessment Framework in Northern Ireland and the GIRFEC(GettingitRight
forEveryChild) approach in Scotland focus on how practitioners across all services
for children and adults can work together to ensure that children and young people
have their needs met with reference to a range of outcomes and indicators that
can be applied in any setting and circumstance. The approach is underpinned by
a set of common values and principles. The success of such approaches depends
on a standardised assessment and the application of shared tools and models. All
approaches require a lead professional.
The five outcomes of CAF concern being healthy, staying safe, enjoying and
achieving, making a positive contribution and achieving economic well-being
(Childrens Workforce Development Council, 2009). It consists of a pre-assessment
checklist to decide who would benefit from an assessment (focusing on the
development of the child/young person, parents and carers, and family and
environment); a standard recording format; and a process to enable practitioners in
the children and young peoples workforce to undertake a common assessment and
to move forward on the result through the development of an action plan.
In sharing information with other professionals, recording information on a single
system, identifying needs and services, establishing a plan and reviewing both the
plan and provision, it would appear that a Procedural model is being employed.
2. The SingleSharedAssessment(SSA) in Scotland, the SingleAssessmentProcess
(SAP) for older people in England, the Unified Assessment in Wales and the
Northern Ireland Single Assessment Tool (NISAT) combine elements of both
Procedural and Questioning models. SSA is the streamlining of the assessment
process to enable the needs and outcomes for the individual to be identified and
subsequent interventions and services put in place (Scottish Government, 2009b: 1).

(Continued)

12_Lishman_Ch-12.indd 177 15/01/2014 2:31:55 PM

 178 Assessment

(Continued)

The sharing of information across agencies is crucial and so the process encourages
joint working.
In a SSA a lead professional coordinates the gathering of information for the
assessment and ensures that a plan is made and reviewed and that the identified
services are delivered. Care Management is the name for this process, and it is
focused on the needs of individuals with complex or changing needs. Three different
types of assessment (Simple, Comprehensive or Specialist) can be carried out,
depending on the needs of the service user, and assessment is undertaken by
different professionals depending on their levels of training and expertise. The legal
context for Care Management is provided through the National Health Service and
Community Care Act 1990 and in Scotland is augmented by the Regulation of Care
(Scotland) Act 2001 and the Community Care and Health (Scotland) Act 2002. The
process of a SSA involves service users and carers and is intended to be person-
centred. However, for older people information is also gathered through an Indicator
of Relative Need questionnaire which consists of 12 multiple choice questions under
section headings: activities of daily living; personal care; food/drink preparation;
mental well-being and behaviour; and bowel management. The answers to each
question are scored and the totals for each section are calculated. The scores are
intended for planning purposes and not to determine eligibility for services.
3. Motivational interviewing (see Chapter 23), used in substance misuse counselling,
is both client-centred and semi-directive. The approach attempts to increase the
service users awareness of the consequences of their behaviour and to encourage
reflection on the benefits that might be achieved through change. The approach is
non-judgemental, non-adversarial and non-confrontational. The eight key interviewing
techniques: asking leading questions; reflecting resistance; acknowledging the
advantages of behaviours; raising awareness of discrepancy between the present
and the desired situation; elaborating on self-motivational statements; offering non-
dogmatic information; voicing the service users doubts and summarising selectively
(Miller and Rollnick, 1991) fit well with the Exchange model of assessment.

Having chosen a specific model and framework for assessment the social worker must
also consider the knowledge that underpins assessment. The range of knowledge used
to support the assessment should include an awareness of developmental theories,
social systems theories, policies, organisational knowledge and knowledge of research.
The point is to bring together information and resources in order to personalise the
provision (Statham and Kearney, 2007).

THE ASSESSMENT RELATIONSHIP

The task of assessment should be underpinned by skills that convey genuineness, warmth
and acceptance, encouragement and approval, empathy, responsiveness and sensitivity
(Lishman, 2009: 76). Cowager (1994) suggests that the strengths that the service user
brings are key to developing the helping relationship. Strength-based assessments may

12_Lishman_Ch-12.indd 178 15/01/2014 2:31:55 PM

 Models of Assessment 179

support the service user to draw on their own resources to examine alternative ways to
improve their situation and to build their confidence. As previously outlined, assessments
focusing on deficits may serve only to disempower the service user and reinforce
inequalities between them and the social worker. The social workers role is to develop
the service users capability to assist themselves. This is known as empowerment.
A good assessment relationship involves the social worker in:

examining the personal and environmental strengths of the service user and carrying
out a multidimensional assessment of such strengths
utilising meaningful and appropriate language
negotiating mutual agreement over the assessment
apportioning no blame. (Cowager, 1994)

In addition, we would add:

discovering the uniqueness of the service user by understanding an individuals iden-

tity and life choices which are formed by their life experiences, culture and ethnicity
and the way in which others have responded towards them.

Any attempt to form a genuine partnership will involve good skills of listening and
interviewing and will focus on the individual rather than the procedure.

REFLECTION ON SELF IN ASSESSMENT

Social workers must be aware of how their own attitudes, values and power based on
their gender, age, ethnicity and life experiences might impact on the process and/or the
outcome of the task. Through becoming aware of self (often assisted by education and
training), workers can consciously adapt their stance, if necessary, in order to practise in
an anti-oppressive manner. It is also important for the worker to gain an understanding
of how service users life experience may inform their perceptions of, and attitude
towards, the social workers involvement.
To develop your thinking about the role of self in assessment visit the Companion
Website (www.sagepub.co.uk/SocialWork) and explore the challenges Mark has been
facing in his practice.

SERVICE USER INVOLVEMENT

A key social work value concerns the involvement of service users in decisions about
their own situation and discussions upon other issues such as service provision and
agency policy. OSullivan (2011) identifies four levels of client involvement: where the
outcome of assessment is the result of decision-making by others; consultation, where
the service users opinions are taken into account; partnership, where joint decisions are
made between the service user and the social worker; being in control, where decisions

12_Lishman_Ch-12.indd 179 15/01/2014 2:31:55 PM

 180 Assessment

are made by the service user without the facilitation of a social worker (this is the highest
level of involvement). Decisions might be service user life decisions, decisions to protect
others, or decisions about resources or service delivery. Unless an individuals capacity is
in question or there is a concern that the safety of others might be compromised, service
users should have control over decisions about their own lives. The reason to choose a
lower rather than a higher level of involvement must be justified and limits should be
placed on the type of involvement only if there are grounds to do so (OSullivan, 2011).

Exercise
Read Chapter 15 and try to identify situations where service user involvement might
be problematic.

WORKING WITH RESISTANCE

Social work practice is often undertaken during challenging and stressful times, so it is
hardly surprising that service users are not always welcoming and appreciative of such
involvement (Taylor, 2011). The service user may experience difficulties in containing
the emotions elicited by their situation and the consequent involvement of social work
services. For example, the service user may experience feelings of failure or loss of
control over life events, and the individual response will be dependent on the nature of
their situation and on their preferred coping strategies (informed by their previous life
experiences). Social workers have to manage a range of behaviours, and aggressive and
violent reactions cannot be ruled out. Dockar-Drysdale (1968) suggests that violence
represents a breakdown in communication and is a symbolic way of finding someone to
help contain feelings of fear and anxiety.
In social work the term resistance is used to describe those service users who are
unwilling, or feel coerced into engaging with you (Taylor, 2011: 11). Taylor (2011)
suggests that individuals might be reluctant to become engaged because of a rigid
interpretation of life events that impedes consideration of other ways of thinking or
acting. The workers belief in the capacity of the service user to change, however, is
central to the helping process, as service users may display ambivalence (conflicting
emotions) or be reluctant to engage. The latter may be a result of a distrust of authority,
due to the workers potential role in relation to prosecution or removal of liberties (as a
result of offending behaviour, severe mental health problems or child protection cases).
There is a greater risk of experiencing aggressive or violent behaviour where:

The individual has experience of a subculture where violence is the norm.

The individual perceives that any unpleasantness generated is a deliberate and per-
sonal attack on them.
The person is disinhibited, e.g. through alcohol or drugs.

12_Lishman_Ch-12.indd 180 15/01/2014 2:31:55 PM

 Models of Assessment 181

There is an expectation that violence will be rewarded, i.e. by influencing the decision
or withdrawal of the worker.
There is a belief that no other action is possible, e.g. where there is evidence that
violence has been used frequently as a coping mechanism. (Breakwell, 1989)

Consideration of theories such as social learning theory, psychodynamic theory or

attribution theory can be helpful in assisting workers to understand the probable cause of
an individuals behaviour.To maximise the possibility of engaging an individual, sustaining
a relationship, or even calming a situation, it is crucial for the worker to demonstrate
empathy and to practise good communication skills, particularly active listening. Service
users and carers outlined a range of specific skills and values demonstrated by social
workers who were felt to be good communicators. These included:

Being polite and punctual

Listening to what is being said
Doing what is stated and agreed
Explaining what will happen and why without using jargon
Being honest. (Diggins, 2004)

In conclusion, resistance may be seen as a way in which service users attempt to regain
some of their perceived loss of power and control by refusing to recognise risks to self or
others, not accepting the need for change, or being unwilling to accept options presented
to them. The concept of principled negotiation might assist in finding a way forward.
By focusing on the interests rather than the attitudes of those involved, separating the
people from the problem and trying to find options for mutual interest before agreeing
criteria for evaluating the result of the negotiations, a resolution to any stalemate might
be found. However, legal and policy requirements might mean that negotiation is not
an option (e.g. because of protection issues) or that due to their personal values and
principles an individual might be unwilling to negotiate on certain matters (e.g. around
the use of alcohol). In planning a response discussions should take place with colleagues
and relevant agencies, involving the service user wherever possible. Any response should
recognise that safe practice is beneficial for both the worker and the service user.

Critical Thinking
In assessment work, collaborative approaches building on service users expertise are
vital. The current practice agenda, especially personalisation and self-directed support,
emphasises the role of self-assessment. Gardner (2011: 43) notes that there has been much
professional resistance to the concept and she highlights that we mistakenly assume that
self-assessment involves only the service user. In self-assessment, however, service users
are major participants because, quite simply, they know themselves best. The social worker
participates too, supporting, offering information and assistance. Gardners interpretation
of self-assessment is interesting and provides a useful point for you to consider.
During, or after your most recent period of practice learning, critically appraise your
practice in respect of a collaborative assessment.

12_Lishman_Ch-12.indd 181 15/01/2014 2:31:55 PM

 182 Assessment

To read about the findings of pilot projects focusing on self-assessment in adult social
work settings, visit the Companion Website (www.sagepub.co.uk/SocialWork) and see
the article by Abendstern et al. (2013).

CONCLUSION
This chapter has offered particular frameworks, methodologies and supporting
theoretical concepts which are integral in good social work assessment. The following
central themes should be borne in mind when undertaking assessment in work with
service users:

Assessment is a skilled activity that is crucial in setting the context of engagement

with service users.
A central theme is of partnership and empowerment but with a recognition that
at times assessments are carried out with service users who are either hesitant or
unwilling participants.
Assessment is underpinned by a broad knowledge and skill base, as well as a series of
guiding principles that support practitioners in their role.
Assessment in current practice contexts frequently involves working alongside other
professionals, allowing for the sharing of perspectives and a more comprehensive and
holistic approach.
The tools and frameworks that have been proffered in this chapter should be utilised
with both care and professional judgement rather than implemented in a technical
and formulaic manner. These tools offer a guide rather than a map.
Assessment is a dynamic activity that should always be viewed as a process rather
than a one-off event.

Reflective Questions
1 Considering the concepts of need and Bronfenbrenners Ecological Systems
Theory identify what a person requires from their immediate caregivers or family
in order to develop or progress. Go on to consider how a persons development
or progress may be influenced by the wider world and what difference being
part of a supportive community environment can make.
2 What questions might you ask during the assessment process in order to
explore strengths? Consider how the questions you ask interface with the
model of assessment being used.
3 As part of preparing for one of your social work placements or practice oppor-
tunities spend time researching the legal and policy context of the field you
will be working in. During placement, reflect on how legislation and policy
shape the work that your placement agency undertakes. Does working in dif-
ferent settings impact on the extent to which social work practice is statutorily
driven?

12_Lishman_Ch-12.indd 182 15/01/2014 2:31:55 PM

 Models of Assessment 183

RECOMMENDED READING
Milner, J. and OByrne, P. (2009) Assessment in Social Work. Basingstoke: Palgrave
MacMillan.
Parker, J. and Bradley, G. (2010) Social Work Practice: Assessment, Planning, Intervention
and Review (Transforming Social Work Practice), 3rd edn. Exeter: Learning Matters.
Walker, S. and Beckett, C. (2010) Social Work Assessment and Intervention. Lyme Regis:
Russell House Publications.

12_Lishman_Ch-12.indd 183 15/01/2014 2:31:55 PM

 Java

Socket Programming
This chapter presents key concepts of intercommunication between programs running on dierent computers
in the network. It introduces elements of network programming and concepts involved in creating network
applications using sockets. The chapter introduces the java.net package containing various classes re-
quired for creating sockets and message communication using two dierent protocols. It provides several
example programs demonstrating various capabilities supported by Java for creating network applications.

O A
er learning the contents of this chapter, the reader will be able to:
B Chapter
understand fundamental concepts of computer communication
J understand sockets and ports
E understand java.net package features
program Java Sockets
C
T
I
create comprehensive network applications using sockets
13
V
E
S

13.1 INTRODUCTION
Internet and WWW have emerged as global ubiquitous media for communication and changed the way
we conduct science, engineering, and commerce. They are also changing the way we learn, live, enjoy,
communicate, interact, engage, etc. The modern life activities are getting completely centered around or
driven by the Internet.
To take advantage of opportunities presented by the Internet, businesses are continuously seeking new
and innovative ways and means for offering their services via the Internet. This created a huge demand
for software designers and engineers with skills in creating new Internet-enabled applications or porting
existing/legacy applications to the Internet platform. The key elements for developing Internet-enabled
applications are a good understanding of the issues involved in implementing distributed applications and
sound knowledge of the fundamental network programming models.
 Socket Programming 347

13.1.1 Client/Server Communication

At a basic level, network-based systems consist of a server, client, and a media for communication
as shown in Fig. 13.1. A computer running a program that makes a request for services is called client
machine. A computer running a program that offers requested services from one or more clients is called
server machine. The media for communication can be wired or wireless network.

Fig. 13.1 Client Server communication

Generally, programs running on client machines make requests

to a program (often called as server program) running on a server
machine. They involve networking services provided by the
transport layer, which is part of the Internet software stack, often
called TCP/IP (Transport Control Protocol/Internet Protocol)
stack, shown in Fig. 13.2. The transport layer comprises two
types of protocols, TCP (Transport Control Protocol) and UDP
(User Datagram Protocol). The most widely used programming
interfaces for these protocols are sockets.
TCP is a connection-oriented protocol that provides a reliable
ow of data between two computers. Example applications that
use such services are HTTP, FTP, and Telnet.
UDP is a protocol that sends independent packets of data, called Fig. 13.2 TCP/IP so
ware stack
datagrams, from one computer to another with no guarantees
about arrival and sequencing. Example applications that use such services include Clock server and Ping.
The TCP and UDP protocols use ports to map incoming data to a particular process running on a
computer. Port is represented by a positive (16-bit) integer value. Some ports have been reserved to support
common/well known services:
ftp 21/tcp
telnet 23/tcp
smtp 25/tcp
login 513/tcp
http 80/tcp,udp
https 443/tcp,udp
User-level process/services generally use port number value >= 1024.
348 Object-Oriented Programming with Java

Fig. 13.3 TCP/UDP mapping of incoming packets to appropriate port/process

Object-oriented Java technologiesSockets, threads, RMI, clustering, Web serviceshave emerged as

leading solutions for creating portable, efcient, and maintainable large and complex Internet applications.

13.1.2 Hosts Identication and Service Ports

Every computer on the Internet is identied by a unique, 4-byte IP address. This is typically written in
dotted quad format like 128.250.25.158 where each byte is an unsigned value between 0 and 255. This
representation is clearly not user-friendly because it does not tell us anything about the content and then it is
difcult to remember. Hence, IP addresses are mapped to names like www.buyya.com or www.google.com,
which are easier to remember. Internet supports name servers that translate these names to IP addresses.
In general, each computer only has one Internet address. However, computers often need to communicate
and provide more than one type of service or to talk to multiple hosts/computers at a time. For example,
there may be multiple ftp sessions, web connections, and chat programs all running at the same time. To
distinguish these services, a concept of ports, a logical access point, represented by a 16-bit integer number
is used. That means, each service offered by a computer is uniquely identied by a port number. Each
Internet packet contains both the destination host address and the port number on that host to which the
message/request has to be delivered. The host computer dispatches the packets it receives to programs by
looking at the port numbers specied within the packets. That is, IP address can be thought of as a house
address when a letter is sent via post/snail mail and port number as the name of a specic individual to
whom the letter has to be delivered.

13.1.3 Sockets and Socket-based Communication

Sockets provide an interface for programming networks at the transport layer. Network communication
using Sockets is very much similar to performing le I/O. In fact, socket handle is treated like le handle.
The streams used in le I/O operation are also applicable to socket-based I/O. Socket-based
communication is independent of a programming language used for implementing it. That means, a socket
program written in Java language can communicate to a program written in non-Java (say C or C++) socket
program.
A server (program) runs on a specic computer and has a socket that is bound to a specic port. The
server listens to the socket for a client to make a connection request (see Fig. 13.4a). If everything goes
well, the server accepts the connection (see Fig. 13.4b). Upon acceptance, the server gets a new socket
bound to a different port. It needs a new socket (consequently a different port number) so that it can
continue to listen to the original socket for connection requests while serving the connected client.
 Socket Programming 349

Fig. 13.4 Establishment of path for two-way communication between a client and server

13.2 SOCKET PROGRAMMING AND JAVA.NET CLASS

A socket is an endpoint of a two-way communication link between two programs running on the network.
Socket is bound to a port number so that the TCP layer can identify the application that data is destined
to be sent. Java provides a set of classes, dened in a package called java.net, to enable the rapid
development of network applications. Key classes, interfaces, and exceptions in java.net package
simplifying the complexity involved in creating client and server programs are:
The Classes
ContentHandler
DatagramPacket
DatagramSocket
DatagramSocketImpl
HttpURLConnection
InetAddress
MulticastSocket
ServerSocket
Socket
SocketImpl
URL
URLConnection
URLEncoder
URLStreamHandler

The Interfaces
ContentHandlerFactory
FileNameMap
SocketImplFactory
URLStreamHandlerFactory
350 Object-Oriented Programming with Java

Exceptions
BindException
ConnectException
MalformedURLException
NoRouteToHostException
ProtocolException
SocketException
UnknownHostException
UnknownServiceException

13.3 TCP/IP SOCKET PROGRAMMING

The two key classes from the java.net package used in creation of server and client programs are:
ServerSocket
Socket
A server program creates a specic type of socket that is used to listen for client requests (server socket),
In the case of a connection request, the program creates a new socket through which it will exchange data
with the client using input and output streams. The socket abstraction is very similar to the le concept:
developers have to open a socket, perform I/O, and close it. Figure 13.5 illustrates key steps involved in
creating socket-based server and client programs.

Fig. 13.5 Socket-based client and server programming

A simple Server Program in Java The steps for creating a simple server program are:
1. Open the Server Socket:
ServerSocket server = new ServerSocket( PORT );
2. Wait for the Client Request:
Socket client = server.accept();
 Socket Programming 351

3. Create I/O streams for communicating to the client

DataInputStream is = new DataInputStream(client.getInputStream());
DataOutputStream os = new DataOutputStream(client.getOutputStream());
4. Perform communication with client
Receive from client: String line = is.readLine();
Send to client: os.writeBytes(Hello\n);
5. Close socket:
client.close();
An example program illustrating creation of a server socket, waiting for client request, and then
responding to a client that requested for connection by greeting it is given below:

Program 13.1
// SimpleServer.java: A simple server program.
import java.net.*;
import java.io.*;
public class SimpleServer {
public static void main(String args[]) throws IOException {
// Register service on port 1254
ServerSocket s = new ServerSocket(1254);
Socket s1=s.accept(); // Wait and accept a connection
// Get a communication stream associated with the socket
OutputStream s1out = s1.getOutputStream();
DataOutputStream dos = new DataOutputStream (s1out);
// Send a string!
dos.writeUTF(Hi there);
// Close the connection, but not the server socket
dos.close();
s1out.close();
s1.close();
}
}

A simple Client Program in Java The steps for creating a simple client program are:
1. Create a Socket Object:
Socket client = new Socket(server, port_id);
2. Create I/O streams for communicating with the server.
is = new DataInputStream(client.getInputStream());
os = new DataOutputStream(client.getOutputStream());
3. Perform I/O or communication with the server:
Receive data from the server: String line = is.readLine();
Send data to the server: os.writeBytes(Hello\n);
4. Close the socket when done:
client.close();
An example program illustrating establishment of connection to a server and then reading a message
sent by the server and displaying it on the console is given below:
352 Object-Oriented Programming with Java

Program 13.2
// SimpleClient.java: A simple client program.
import java.net.*;
import java.io.*;
public class SimpleClient {
public static void main(String args[]) throws IOException {
// Open your connection to a server, at port 1254
Socket s1 = new Socket(localhost,1254);
// Get an input file handle from the socket and read the input
InputStream s1In = s1.getInputStream();
DataInputStream dis = new DataInputStream(s1In);
String st = new String (dis.readUTF());
System.out.println(st);
// When done, just close the connection and exit
dis.close();
s1In.close();
s1.close();
}
}

Running Socket Programs Compile both server and client programs and then deploy server program
code on a machine which is going to act as a server and client program, which is going to act as a client.
If required, both client and server programs can run on the same machine. To illustrate execution of server
and client programs, let us assume that a machine called mundroo.csse.unimelb.edu.au on which we
want to run a server program as indicated below:
[raj@mundroo] java SimpleServer
The client program can run on any computer in the network (LAN, WAN, or Internet) as long as there
is no rewall between them that blocks communication. Let us say we want to run our client program on a
machine called gridbus.csse.unimelb.edu.au as follows:
[raj@gridbus] java SimpleClient
The client program is just establishing a connection with the server and then waits for a message. On
receiving a response message, it prints the same to the console. The output in this case is:
Hi there
which is sent by the server program in response to a client connection request.
It should be noted that once the server program execution is started, it is not possible for any other server
program to run on the same port until the rst program which is successful using it is terminated. Port
numbers are a mutually exclusive resource. They cannot be shared among different processes at the same
time.

13.4 UDP SOCKET PROGRAMMING

The previous two example programs used the TCP sockets. As already said, TCP guarantees the delivery of
packets and preserves their order on destination. Sometimes these features are not required and since they
 Socket Programming 353

do not come without performance costs, it would be better to use a lighter transport protocol. This kind of
service is accomplished by the UDP protocol which conveys datagram packets.
Datagram packets are used to implement a connectionless packet delivery service supported by the UDP
protocol. Each message is transferred from source machine to destination based on information contained
within that packet. That means, each packet needs to have destination address and each packet might be
routed differently, and might arrive in any order. Packet delivery is not guaranteed.
The format of datagram packet is:

| Msg | length | Host | serverPort |

Java supports datagram communication through the following classes:
DatagramPacket
DatagramSocket
The class DatagramPacket contains several constructors that can be used for creating packet object.
One of them is:
DatagramPacket(byte[] buf, int length, InetAddress address, int port);
This constructor is used for creating a datagram packet for sending packets of length length to the
specied port number on the specied host. The message to be transmitted is indicated in the rst argument.
The key methods of DatagramPacket class are:
byte[] getData()
Returns the data buffer.
int getLength()
Returns the length of the data to be sent or the length of the data received.
void setData(byte[] buf)
Sets the data buffer for this packet.
void setLength(int length)
Sets the length for this packet.
The class DatagramSocket supports various methods that can be used for transmitting or receiving
data a datagram over the network. The two key methods are:
void send(DatagramPacket p)
Sends a datagram packet from this socket.
void receive(DatagramPacket p)
Receives a datagram packet from this socket.
A simple UDP server program that waits for clients requests and then accepts the message (datagram)
and sends back the same message is given below. Of course, an extended server program can manipulate
clients messages/request and send a new message as a response.

Program 13.3
// UDPServer.java: A simple UDP server program.
import java.net.*;
import java.io.*;
public class UDPServer {
354 Object-Oriented Programming with Java

public static void main(String args[]){

DatagramSocket aSocket = null;
if (args.length < 1) {
System.out.println(Usage: java UDPServer <Port Number>);
System.exit(1);
}
try {
int socket_no = Integer.valueOf(args[0]).intValue();
aSocket = new DatagramSocket(socket_no);
byte[] buffer = new byte[1000];
while(true) {
DatagramPacket request = new DatagramPacket(buffer,
buffer.length);
aSocket.receive(request);
DatagramPacket reply = new DatagramPacket(request.getData(),
request.getLength(),request.getAddress(),
request.getPort());
aSocket.send(reply);
}
}
catch (SocketException e) {
System.out.println(Socket: + e.getMessage());
}
catch (IOException e) {
System.out.println(IO: + e.getMessage());
}
finally {
if (aSocket != null)
aSocket.close();
}
}
}
A corresponding client program for creating a datagram and then sending it to the above server and then
accepting a response is listed below.

Program 13.4
// UDPClient.java: A simple UDP client program.
import java.net.*;
import java.io.*;
public class UDPClient {

public static void main(String args[]){

// args give message contents and server hostname
DatagramSocket aSocket = null;
if (args.length < 3) {
System.out.println(
Usage: java UDPClient <message> <Host name> <Port number>);
System.exit(1);
 Socket Programming 355

}
try {
aSocket = new DatagramSocket();
byte [] m = args[0].getBytes();
InetAddress aHost = InetAddress.getByName(args[1]);
int serverPort = Integer.valueOf(args[2]).intValue();
DatagramPacket request =
new DatagramPacket(m, args[0].length(), aHost, serverPort);
aSocket.send(request);
byte[] buffer = new byte[1000];
DatagramPacket reply = new DatagramPacket(buffer, buffer.length);
aSocket.receive(reply);
System.out.println(Reply: + new String(reply.getData()));
}
catch (SocketException e) {
System.out.println(Socket: + e.getMessage());
}
catch (IOException e) {
System.out.println(IO: + e.getMessage());
}
finally {
if (aSocket != null)
aSocket.close();
}
}
}

13.5 MATH SERVER

It is time to implement a more comprehensive network application by using the socket programming APIs
you have learned so far. A sample math client-server interaction demonstrating online math server that can
perform basic math operations is shown in Fig. 13.6.

Fig. 13.6 A socket-based math server and clients

The basic math interface is shown as follows:
356 Object-Oriented Programming with Java

Program 13.5
// MathService.java: A basic math interface.
public interface MathService {
public double add(double firstValue, double secondValue);
public double sub(double firstValue, double secondValue);
public double div(double firstValue, double secondValue);
public double mul(double firstValue, double secondValue);
}

The implementation of this interface is not related to any network operation. The following code shows
a very simple implementation of this interface:

Program 13.6
// PlainMathService.java: An implementation of the MathService interface.
public class PlainMathService implements MathService {

public double add(double firstValue, double secondValue) {

return firstValue+secondValue;
}
public double sub(double firstValue, double secondValue) {
return firstValue-secondValue;
}
public double mul(double firstValue, double secondValue) {
return firstValue * secondValue;
}
public double div(double firstValue, double secondValue) {
if (secondValue != 0)
return firstValue / secondValue;
return Double.MAX_VALUE;
}
}

The implementation of the MathServer is quite straightforward, which looks pretty similar to the echo
server mentioned previously. The difference is that the MathServer have to consider the specic protocol
dened by the math server and client communication. The program uses a very simple protocol operator:
rst_value:second_value. It is the math servers responsibility to understand this protocol and delegate to
the proper methods such as add, sub, mul, or div.

Program 13.7
// MathServer.java : An implementation of the MathServer.
import java.io.*;
import java.net.*;

public class MathServer{

protected MathService mathService;
protected Socket socket;
 Socket Programming 357

public void setMathService(MathService mathService) {

this.mathService = mathService;
}
public void setSocket(Socket socket) {
this.socket = socket;
}
public void execute() {
try {
BufferedReader reader = new BufferedReader(
new InputStreamReader(socket.getInputStream()));
// read the message from client and parse the execution
String line = reader.readLine();
double result = parseExecution(line);
// write the result back to the client
BufferedWriter writer = new BufferedWriter(
new OutputStreamWriter(socket.getOutputStream()));
writer.write(+result);
writer.newLine();
writer.flush();
// close the stream
reader.close();
writer.close();
}
catch (Exception e) {
e.printStackTrace();
}
}
// the predefined protocol for the math operation is
// operator:first value:second value
protected double parseExecution(String line)
throws IllegalArgumentException {
double result = Double.MAX_VALUE;
String [] elements = line.split(:);
if (elements.length != 3)
throw new IllegalArgumentException(parsing error!);
double firstValue = 0;
double secondValue = 0;
try {
firstValue = Double.parseDouble(elements[1]);
secondValue = Double.parseDouble(elements[2]);
}
catch(Exception e) {
throw new IllegalArgumentException(Invalid arguments!);
}
switch (elements[0].charAt(0)) {
case +:
result = mathService.add(firstValue, secondValue);
break;
case -:
358 Object-Oriented Programming with Java

result = mathService.sub(firstValue, secondValue);

break;
case *:
result = mathService.mul(firstValue, secondValue);
break;
case /:
result = mathService.div(firstValue, secondValue);
break;
default:
throw new IllegalArgumentException(Invalid math operation!);
}
return result;
}

public static void main(String [] args)throws Exception{

int port = 10000;
if (args.length == 1) {
try {
port = Integer.parseInt(args[0]);
}
catch(Exception e){
}
}
System.out.println(Math Server is running...);
// create a server socket and wait for clients connection
ServerSocket serverSocket = new ServerSocket(port);
Socket socket = serverSocket.accept();
// run a math server that talks to the client
MathServer mathServer = new MathServer();
mathServer.setMathService(new PlainMathService());
mathServer.setSocket(socket);
mathServer.execute();
System.out.println(Math Server is closed...);
}
}

A test client program that can access the math server is shown below:

Program 13.8
// MathClient.java: A test client program to access MathServer.
import java.io.*;
import java.net.Socket;
public class MathClient {
public static void main(String [] args){
String hostname = localhost;
int port = 10000;
if (args.length != 2) {
System.out.println(Use the default setting...);
}
else {
 Socket Programming 359

hostname = args[0];
port = Integer.parseInt(args[1]);
}
try {
// create a socket
Socket socket = new Socket(hostname, port);
// perform a simple math operation 12+21
BufferedWriter writer = new BufferedWriter(
new OutputStreamWriter(socket.getOutputStream()));
writer.write(+:12:21);
writer.newLine();
writer.flush();
// get the result from the server
BufferedReader reader = new BufferedReader(
new InputStreamReader(socket.getInputStream()));
System.out.println(reader.readLine());
reader.close();
writer.close();
}
catch (Exception e) {
e.printStackTrace();
}
}
}

13.6 URL ENCODING

It is very important that the Web can be accessed via heterogeneous platforms such as Windows, Linux,
or Mac. The characters in the URL must come from a xed subset of ASCII in order to maintain the
interoperability between various platforms. Specically, the capital letters AZ, the lowercase letters az,
the digits 09 and the punctuation characters. Encoding is very simple, any characters that are not ASCII
numerals, letters, or the punctuation marks allowed are converted into bytes and each byte is written as a
percentage sign followed by two hexadecimal digits. For example, the following program helps encode a
query string if non-ASCII characters are present.

Program 13.9
// QueryStringFormatter.java: encodes a string with non-ASCII characters.
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;

public class QueryStringFormatter {

private String queryEngine;
private StringBuilder query = new StringBuilder();

public QueryStringFormatter(String queryEngine) {

this.queryEngine = queryEngine;
}
360 Object-Oriented Programming with Java

public String getEngine() {

return this.queryEngine;
}
public void addQuery(String queryString, String queryValue)
throws Exception {
query.append(queryString+=+
URLEncoder.encode(queryValue,UTF-8")+&);
}
public String getQueryString(){
return ?+query.toString();
}
public static void main(String[] args) throws Exception {
QueryStringFormatter formatter =
new QueryStringFormatter(http://www.google.com.au/search);
formatter.addQuery(newwindow,1);
formatter.addQuery(q,Xingchen Chu & Rajkumar Buyya);
System.out.println(formatter.getEngine()+
formatter.getQueryString());
}
}

The output of this program is shown as follows:

http://www.google.com.au/search?newwindow=1&q=Xingchen+Chu+%26+Rajkumar+Buyya&
It can be seen that the whitespace has been encoded as + and the & has been encoded as %26 the
percentage sign following by its byte value. Other characters remain the same. These conversions have been
performed by the URLEncoder class, which is part of the Java base class library and provides facilities for
encoding strings (urls) in different formats. There is also an URLDecoder class that performs the reverse
process. These two classes are useful not only for URLs but also for managing HTML form data.

13.6.1 Writing and Reading Data via URLConnection

Besides the socket and datagram introduced in the previous section, java.net package provides another
useful class that can be used to write and read data between the server and the client: URLConnetion
It is not possible to instantiate the URLConnection class directly. Instead you have to create the
URLConnection via the URL object.
URLConnection connection = new URL(www.yahoo.com).openConnection();
Then the URLConnection class provides the getInputStream and getOutputStream methods that
are very similar to the getInputStream and getOutputStream provided by the Socket class. The
following example shows how to use the URLConnection and the URLEncoder class to send queries to
the Yahoo search engine.

Program 13.10
// TextBasedSearchEngine.java:
import java.io.*;
import java.net.*;

public class TextBasedSearchEngine {

 Socket Programming 361

private String searchEngine;

public TextBasedSearchEngine(String searchEngine) {

this.searchEngine = searchEngine;
}
public void doSearch(String queryString) {
try {
// open a url connection
URL url = new URL(searchEngine);
URLConnection connection = url.openConnection();
connection.setDoOutput(true);
// write the query string to the search engine
PrintStream ps = new PrintStream(connection.getOutputStream());
ps.println(queryString);
ps.close();
// read the result
DataInputStream input =
new DataInputStream(connection.getInputStream());
String inputLine = null;
while((inputLine = input.readLine())!= null) {
System.out.println(inputLine);
}
}
catch(Exception e) {
e.printStackTrace();
}
}
public static void main(String[] args) throws Exception{
QueryStringFormatter formatter =
new QueryStringFormatter(http://search.yahoo.com/search);
formatter.addQuery(newwindow,1");
formatter.addQuery(q,Xingchen Chu & Rajkumar Buyya);
// search it via yahoo
TextBasedSearchEngine search =
new TextBasedSearchEngine(formatter.getEngine());
search.doSearch(formatter.getQueryString());
}
}
Output:
The program rst creates an encoded query string that can be used by the web application, then it utilises
the URLConnection class to send/receive data from the Yahoo search engine. The output in the entire
HTML page will be something like this:
<!doctype html public -//W3C/DTD HTML 4.01//EN http://www.w3.org/TR/
html4/strict.dtd>
<html>
<head>


</html>
362 Object-Oriented Programming with Java

13.7
Developing network applications is made possible in Java by using sockets, threads, RMI,
clustering, and Web services. These technologies allow for the creation of portable, ecient, and
maintainable large and complex Internet applications. The java.net package provides a powerful
and exible set of classes for implementing network applications.
Typically, programs running on client machines make requests to programs on a server
machine. These involve networking services provided by the transport layer. The most widely
used transport protocols on the Internet are TCP (Transmission control Protocol) and UDP
(User Datagram Protocol). TCP is a connection-oriented protocol providing a reliable ow of data
between two computers. It is used by applications such as the World Wide Web, e-mail, p, and
secure shell. On the other hand, UDP is a simpler message-based connectionless protocol which
sends packets of data known as datagrams from one computer to another with no guarantees of
arrival. Network applications using UDP include Domain Name Systems (DNS), streaming
media applications as IPTV, VoIP, and online games. Both protocols use ports for application-
to-application communication. A port is a logical access point represented by a positive 16-bit
integer value. Each service oered by a computer is uniquely identied by a port number. Each
Internet packet contains both destination IP address and the port to which the request is to be
delivered. Some ports are reserved for common services such as p, telnet, smtp, hp, hps, and
login. Port numbers >=1024 are generally used for user-level services.
Sockets provide an interface for programming networks at the transport layer. Using sockets,
network communication is very much similar to performing le I/O. A socket is an endpoint
of a two-way communication link between two programs running on the network. The source
and destination IP address, and the port numbers constitute a network socket. Two key classes
from java.net package used in creation of server and client programs are ServerSocket,
which represents a server socket, and Socket, an instantiation of which performs the
actual communication with the client. Datagram communication is also supported through
DatagramPacket and DatagramSocket classes. Writing and reading data between server and
the client is also supported through the URLconnection class.

13.8 EXERCISES

Objective Questions

13.1 ______ is a connection-oriented and reliable protocol, ______ is a less reliable protocol.
13.2 The TCP and UDP protocols use ______ to map incoming data to a particular process running on a
computer.
13.3 Datagrams are normally sent by ______ protocol.
13.4 Java uses ______ class representing a server and ______ class representing the client that uses TCP
protocol.
13.5 ______ is used to wait for accepting client connection requests.
13.6 Class ______ is used to create a packet of data for UDP protocol.
13.7 If something goes wrong related to the network, ______ will be thrown when dealing with TCP/
UDP programming in Java.
13.8 The main purpose of URL encoding is to maintain the ______ between various platforms.
13.9 Based on the URL encoding mechanism, www.test.com/test me&test you becomes ______.
 Socket Programming 363

13.10 ______ method is used to instantiate a URLConnection instance.

13.11 UDP is more reliable than TCP protocol: True or False.
13.12 The same port number can be reused many times when binding with sockets simultaneously: True
or False.
13.13 In order to create a client socket connecting to a particular server, the IP address must be given to
the client socket, otherwise, it cannot connect to the server: True or False.
13.14 Sockets provide an interface for programming networks at the transport layer: True or False.
13.15 Call Socket.close() method will close the TCP server that socket connects to: True or False.
13.16 The socket instance does not need to be explicitly closed to release all the resources it occupies, as
the Java Garbage Collection mechanism will do it automatically: True or False
13.17 The following line of code
Socket socket = new Socket(localhost, 1254);
will create a TCP server at localhost port 1254: True or False.
13.18 Java TCP Socket uses the InputStream/OutputStream to read/write data to the network channel:
True or False.
13.19 Java UDP sends/receives data as raw Java primitive types: True or False.
13.20 URL Encoding is useful for translating Chinese to English that presents in a particular URL: True
or False.

Review Questions

13.21 Explain the client-server architecture via a simple example.

13.22 Explain the TCP/IP stacks. Briey explain the TCP and UDP protocols and the difference between
the two protocols.
13.23 What is port? List some well-known ports and explain the applications associated with them.
13.24 Discuss the process of creation of server and client sockets with exceptions handled explicitly with
a suitable example.
13.25 What are URN, URL, and URI. Explain the differences between them.
13.26 Explain why the URL encoding is very important to web applications.

Programming Problems

13.27 Write a simple program that can read a host name and convert it to an IP address.
13.28 Write a URL-based program that pulls content from www.buyya.com.
13.29 Write a ping-pong client and server application. When a client sends a ping message to the server,
the server will respond with a pong message. Other messages sent by the client can be safely
dropped by the server.
13.30 Rewrite the math server application, instead of using the TCP socket, use the UDP datagram
socket.
13.31 Write an online dictionary application, users are able to search online dictionary and get the
meaning of the words. The online dictionary should maintain its indexing in order to enhance the
performance and scalability.
13.32 Write a simple crawler program that can parse certain HTML information and nd out the
hyperlinks within the HTML. (Use the URLConnection class to implement this application).
13.33 Write a Socket-based Java server program that responds to client messages as follows: When it
receives a message from a client, it simply converts the message into all uppercase letters and sends
back the same to the client. Write both client and server programs demonstrating this.
 Chapter 14 Work Programme Provider Guidance

Chapter 14 Communicating your Minimum Standards

This chapter covers:

Requirements for communicating your minimum service standards to

participants who are joining the Work Programme

High Level Must Do:

Produce a product which sets out your minimum service standards, and the first
step in your complaints process. Jobcentre Plus will give this to participants at
their referral interview to the Work Programme.

How To
1. The Government is clear that you, as a provider, are best placed to know
what works for participants. As such, we are not specifying what support
you should deliver. However, participants should know what they can
expect from you before they take part in your provision.

2. How you communicate and market your services is your decision, but we
do require that you provide a summary of the minimum service standards
that you will offer to all participants joining the Work Programme.

3. You are responsible for providing your local Jobcentre Plus contact(s) with
a product which sets out your minimum service standards for all
participants. The document should also clearly explain that you have a
complaints process, and the first step a participant should take if they are
not content with the service they are receiving while they are with you.

4. It will be given to the participant and discussed at their referral interview in

Jobcentre Plus. At the referral interview, the Jobcentre Plus adviser will
explain the participants rights and responsibilities whilst on the
programme. The summary of your minimum service standards will ensure
that participants understand what to expect from the programme and will
help enable their effective participation.

5. The minimum standards reflect what was in your successful bid; and must
be presented in a form that is accessible and easy for participants to
understand.

6. Further guidance on developing marketing products for participants can be

found at Generic Guidance Chapter 9 - Marketing and Customer
Communications.

1 V2.00
From the book Networks, Crowds, and Markets: Reasoning about a Highly Connected World.
By David Easley and Jon Kleinberg. Cambridge University Press, 2010.
Complete preprint on-line at http://www.cs.cornell.edu/home/kleinber/networks-book/

Chapter 15

Sponsored Search Markets

15.1 Advertising Tied to Search Behavior

The problem of Web search, as traditionally formulated, has a very pure motivation: it
seeks to take the content people produce on the Web and find the pages that are most
relevant, useful, or authoritative for any given query. However, it soon became clear that
a lucrative market existed within this framework for combining search with advertising,
targeted to the queries that users were issuing.
The basic idea behind this is simple. Early Web advertising was sold on the basis of
impressions, by analogy with the print ads one sees in newspapers or magazines: a company
like Yahoo! would negotiate a rate with an advertiser, agreeing on a price for showing its
ad a fixed number of times. But if the ad youre showing a user isnt tied in some intrinsic
way to their behavior, then youre missing one of the main benefits of the Internet as an
advertising venue, compared to print or TV. Suppose for example that youre a very small
retailer whos trying to sell a specialized product; say, for example, that you run a business
that sells calligraphy pens over the Web. Then paying to display ads to the full Internet-
using population seems like a very inefficient way to find customers; instead, you might want
to work out an agreement with a search engine that said, Show my ad to any user who
enters the query calligraphy pens. After all, search engine queries are a potent way to
get users to express their intent what it is that theyre interested in at the moment they
issue their query and an ad that is based on the query is catching a user at precisely this
receptive moment.
Originally pioneered by the company Overture, this style of keyword-based advertising
has turned out to be an enormously successful way for search engines to make money. At
Draft version: June 10, 2010

437
438 CHAPTER 15. SPONSORED SEARCH MARKETS

Figure 15.1: Search engines display paid advertisements (shown on the right-hand side of
the page in this example) that match the query issued by a user. These appear alongside
the results determined by the search engines own ranking method (shown on the left-hand
side). An auction procedure determines the selection and ordering of the ads.

present its a business that generates tens of billions of dollars per year in revenue, and it is
responsible, for example, for nearly all of Googles revenue. From our perspective, its also
a very nice blend of ideas that have come up earlier in this book: it creates markets out of
the information-seeking behavior of hundreds of millions of people traversing the Web; and
we will see shortly that it has surprisingly deep connections to the kinds of auctions and
matching markets that we discussed in Chapters 9 and 10.
Keyword-based ads show up on search engine results pages alongside the unpaid (or-
ganic or algorithmic) results. Figure 15.1 shows an example of how this currently looks
on Google for the query Keuka Lake, one of the Finger Lakes in upstate New York. The
algorithmic results generated by the search engines internal ranking procedure are on the
left, while the paid results (in this case for real estate and vacation rentals) are ordered on
the right. There can be multiple paid results for a single query term; this simply means that
the search engine has sold an ad on the query to multiple advertisers. Among the multiple
slots for displaying ads on a single page, the slots higher up on the page are more expensive,
since users click on these at a higher rate.
The search industry has developed certain conventions in the way it sells keyword-based
15.1. ADVERTISING TIED TO SEARCH BEHAVIOR 439

ads, and for thinking about this market its worth highlighting two of these at the outset.

Paying per click. First, ads such as those shown in Figure 15.1 are based on a cost-per-
click (CPC) model. This means that if you create an ad that will be shown every time a
user enters the query Keuka Lake, it will contain a link to your companys Web site
and you only pay when a user actually clicks on the ad. Clicking on an ad represents an
even stronger indication of intent than simply issuing a query; it corresponds to a user who
issued the query, read your ad, and is now visiting your site. As a result, the amount that
advertisers are willing to pay per click is often surprisingly high. For example, to occupy
the most prominent spot for calligraphy pens costs about $1.70 per click on Google as of
this writing; occupying the top spot for Keuka Lake costs about $1.50 per click. (For the
misspelling calligaphy pens, the cost is still about $0.60 per click after all, advertisers
are still interested in potential customers even if their query contains a small but frequent
typo.)
For some queries, the cost per click can be positively stratospheric. Queries like loan
consolidation, mortgage refinancing, and mesothelioma often reach $50 per click or
more. One can take this as an advertisers estimate that it stands to gain an expected value
of $50 for every user who clicks through such an ad to its site.1

Setting prices through an auction. There is still the question of how a search engine
should set the prices per click for different queries. One possibility is simply to post prices, the
way that products in a store are sold. But with so many possible keywords and combinations
of keywords, each appealing to a relatively small number of potential advertisers, it would
essentially be hopeless for the search engine to maintain reasonable prices for each query in
the face of changing demand from advertisers.
Instead, search engines determine prices using an auction procedure, in which they solicit
bids from the advertisers. If there were a single slot in which an ad could be displayed, then
this would be just a single-item auction such as we saw in Chapter 9, and there we saw
that the sealed-bid second-price auction had many appealing features. The problem is more
complicated in the present case, however, since there are multiple slots for displaying ads,
and some are more valuable than others.
We will consider how to design an auction for this setting in several stages.

(1) First, if the search engine knew all the advertisers valuations for clicks, the situation
could be represented directly as a matching market in the style that we discussed in
1
Naturally, you may be wondering at this point what mesothelioma is. As a quick check on Google reveals,
its a rare form of lung cancer that is believed to be caused by exposure to asbestos in the workplace. So if
you know enough to be querying this term, you may well have been diagnosed with mesothelioma, and are
considering suing your employer. Most of the top ads for this query link to law firms.
440 CHAPTER 15. SPONSORED SEARCH MARKETS

Chapter 10 essentially, the slots are the items being sold, and theyre being matched
with the advertisers as buyers.

(2) If we assume that the advertisers valuations are not known, however, then we need
to think about ways of encouraging truthful bidding, or to deal with the consequences
of untruthful bidding. This leads us directly to an interesting general question that
long predates the specific problem of keyword-based advertising: how do you design a
price-setting procedure for matching markets in which truthful bidding is a dominant
strategy for the buyers? We will resolve this question using an elegant procedure called
the Vickrey-Clarke-Groves (VCG) mechanism [112, 199, 400], which can be viewed as
a far-reaching generalization of the second-price rule for single-item auctions that we
discussed in Chapter 9.

(3) The VCG mechanism provides a natural way to set prices in matching markets, in-
cluding those arising from keyword-based advertising. For various reasons, however,
it is not the procedure that the search industry adopted. As a result, our third topic
will be an exploration of the auction procedure that is used to sell search advertising
in practice, the Generalized Second-Price Auction (GSP). We will see that although
GSP has a simple description, the bidding behavior it leads to is very complex, with
untruthful bidding and socially non-optimal outcomes. Trying to understand bidder
behavior under this auction turns out to be an interesting case study in the intricacies
of a complex auction procedure as it is implemented in a real application.

15.2 Advertising as a Matching Market

Clickthrough Rates and Revenues Per Click. To begin formulating a precise descrip-
tion of how search advertising is sold, lets consider the set of available slots that the
search engine has for selling ads on a given query, like the three advertising slots shown in
Figure 15.1. The slots are numbered 1, 2, 3, . . . starting from the top of the page, and users
are more likely to click on the higher slots. We will assume that each slot has a specific
clickthrough rate associated with it this is the number of clicks per hour that an ad placed
in that slot will receive.
In the models we discuss, we will make a few simplifying assumptions about the click-
through rates. First, we assume that advertisers know the clickthrough rates. Second, we
assume that the clickthrough rate depends only on the slot itself and not on the ad that is
placed there. Third, we assume that the clickthrough rate of a slot also doesnt depend on
the ads that are in other slots. In practice, the first of these assumptions is not particularly
problematic since advertisers have a number of means (including tools provided by the search
engine itself) for estimating clickthrough rates. The second assumption is an important is-
15.2. ADVERTISING AS A MATCHING MARKET 441

clickthrough slots advertisers revenues

rates per click
10 a x 3

5 b y 2

2 c z 1

Figure 15.2: In the basic set-up of a search engines market for advertising, there are a certain
number of advertising slots to be sold to a population of potential advertisers. Each slot has
a clickthrough rate: the number of clicks per hour it will receive, with higher slots generally
getting higher clickthrough rates. Each advertisers has a revenue per click, the amount of
money it expects to receive, on average, each time a user clicks on one of its ads and arrives
at its site. We draw the advertisers in descending order of their revenue per click; for now,
this is purely a pictorial convention, but in Section 15.2 we will show that the market in fact
generally allocates slots to the advertisers in this order.

sue: a relevant, high-quality ad in a high slot will receive more clicks than an off-topic ad,
and in fact we will describe how to extend the basic models to deal with ad relevance and ad
quality at the end of the chapter. The third assumption interaction among the different
ads being shown is a more complex issue, and it is still not well understood even within
the search industry.

This is the full picture from the search engines side: the slots are the inventory that it is
trying to sell. Now, from the advertisers side, we assume that each advertiser has a revenue
per click: the expected amount of revenue it receives per user who clicks on the ad. Here too
we will assume that this value is intrinsic to the advertiser, and does not depend on what
was being shown on the page when the user clicked on the ad.

This is all the information we need to understand the market for a particular keyword:
the clickthrough rates of the slots, and the revenues per click of the advertisers. Figure 15.2
shows a small example with three slots and three advertisers: the slots have clickthrough
rates of 10, 5, and 2 respectively, while the advertisers have revenues per click of 3, 2, and 1
respectively.
442 CHAPTER 15. SPONSORED SEARCH MARKETS

slots advertisers valuations prices slots advertisers valuations

a x 30, 15, 6 13 a x 30, 15, 6

b y 20, 10, 4 3 b y 20, 10, 4

c z 10, 5, 2 0 c z 10, 5, 2

(a) Advertisers valuations for the slots (b) Market-clearing prices for slots

Figure 15.3: The allocation of advertising slots to advertisers can be represented as a match-
ing market, in which the slots are the items to be sold, and the advertisers are the buyers.
An advertisers valuation for a slot is simply the product of its own revenue per click and
the clickthrough rate of the slot; these can be used to determine market-clearing prices for
the slots.

Constructing a Matching Market. We now show how to represent the market for a
particular keyword as a matching market of the type we studied in Chapter 10. To do this,
it is useful to first review the basic ingredients of matching market from Chapter 10.

The participants in a matching market consist of a set of buyers and a set of sellers.

Each buyer j has a valuation for the item offered by each seller i. This valuation can
depend on the identities of both the buyer and the seller, and we denote it vij .

The goal is to match up buyers with sellers, in such a way that no buyer purchases two
different items, and the same item isnt sold to two different buyers.

To cast the search engines advertising market for a particular keyword in this framework,
we use ri to denote the clickthrough rate of slot i, and vj to denote the revenue per click of
advertiser j. The benefit that advertiser j receives from being shown in slot i is then just
ri vj , the product of the number of clicks and the revenue per click.
In the language of matching markets, this is advertiser js valuation vij for slot i that
is, the value it receives from acquiring slot i. So by declaring the slots to be the sellers,
the advertisers to be the buyers, and the buyers valuations to be vij = ri vj , the problem
of assigning slots to advertisers is precisely the problem of assigning sellers to buyers in a
matching market. In Figure 15.3(a), we show how this conversion is applied to the example in
Figure 15.2, yielding the buyer valuations shown. As this figure makes clear, the advertising
set-up produces a matching market with a special structure: since the valuations are obtained
15.2. ADVERTISING AS A MATCHING MARKET 443

by multiplying rates by revenues, we have a situation where all the buyers agree on their
preferences for the items being sold, and where in fact the valuations of one buyer simply
form a multiple of the valuations of any other buyer.
When we considered matching markets in Chapter 10, we focused on the special case
in which the number of sellers and the number of buyers were the same. This made the
discussion simpler in a number of respects; in particular, it meant that the buyers and sellers
could be perfectly matched, so that each item is sold, and each buyer purchases exactly one
item. We will make the analogous assumption here: with slots playing the role of sellers
and advertisers playing the role of buyers, we will focus on the case in which the numbers
of slots and advertisers are the same. But it is important to note that this assumption
is not at all essential, because for purposes of analysis we can always translate a scenario
with unequal numbers of slots and advertisers into an equivalent one with equal numbers, as
follows. If there are more advertisers than slots, we simply create additional fictitious slots
of clickthrough rate 0 (i.e. of valuation 0 to all buyers) until the number of slots is equal to
the number of advertisers. The advertisers who are matched with the slots of clickthrough
rate 0 are then simply the ones who dont get assigned a (real) slot for advertising. Similarly,
if there are more slots than advertisers, we just create additional fictitious advertisers who
have a valuation of 0 for all slots.

Obtaining Market-Clearing Prices. With the connection to matching markets in place,

we can use the framework from Chapter 10 to determine market-clearing prices. Again, it
is worth reviewing this notion from Chapter 10 in a bit of detail as well, since we will be
using it heavily in what follows. Recall, roughly speaking, that a set of prices charged by
the sellers is market-clearing if, with these prices, each buyer prefers a different slot. More
precisely, the basic ingredients of market-clearing prices are as follows.

Each seller i announces a price pi for his item. (In our case, the items are the slots.)

Each buyer j evaluates her payoff for choosing a particular seller i: it is equal to the
valuation minus the price for this sellers item, vij pi .

We then build a preferred-seller graph as in Figure 15.3(b) by linking each buyer to

the seller or sellers from which she gets the highest payoff.

The prices are market-clearing if this graph has a perfect matching: in this case, we
can assign distinct items to all the buyers in such a way that each buyer gets an item
that maximizes her payoff.

In Chapter 10, we showed that market-clearing prices exist for every matching market, and
we gave a procedure to construct them. We also showed in Chapter 10 that the assignment
444 CHAPTER 15. SPONSORED SEARCH MARKETS

of buyers to sellers achieved by market-clearing prices always maximizes the buyers total
valuations for the items they get.
Returning to the specific context of advertising markets, market-clearing prices for the
search engines advertising slots have the desirable property that advertisers prefer different
slots, and the resulting assignment of advertisers to slots maximizes the total valuations of
each advertiser for what they get. (Again, see Figure 15.3(b).) In fact, it is not hard to work
out that when valuations have the special form that we see in advertising markets each
consisting of a clickthrough rate times a revenue per click then the maximum valuation
is always obtained by giving the slot with highest clickthrough rate to the advertiser with
maximum revenue per click, the slot with second highest rate to the advertiser with second
highest revenue per click, and so forth.
The connection with matching markets shows that we can in fact think about advertising
prices in the more general case where different advertisers can have arbitrary valuations for
slots they need not be the product of a clickthrough rate and a revenue per click. This
allows advertisers, for example, to express how they feel about users who arrive via an ad
in the third slot compared with those who arrive via an ad in the first slot. (And indeed, it
is reasonable to believe that these two populations of users might have different behavioral
characteristics.)
Finally, however, this construction of prices can only be carried out by a search engine if
it actually knows the valuations of the advertisers. In the next section we consider how to
set prices in a setting where the search engine doesnt know these valuations; it must rely
on advertisers to report them without being able to know whether this reporting is truthful.

15.3 Encouraging Truthful Bidding in Matching Mar-

kets: The VCG Principle
What would be a good price-setting procedure when the search engine doesnt know the
advertisers valuations? In the early days of the search industry, variants of the first-price
auction were used: advertisers were simply asked to report their revenues per click in the
form of bids, and then they were assigned slots in decreasing order of these bids. Recall from
Chapter 9 that when bidders are simply asked to report their values, they will generally
under-report, and this is what happened here. Bids were shaded downward, below their true
values; and beyond this, since the auctions were running continuously over time, advertisers
constantly adjusted their bids by small increments to experiment with the outcome and
to try slightly outbidding competitors. This resulted in a highly turbulent market and a
huge resource expenditure on the part of both the advertisers and the search engines, as the
constant price experimentation led to prices for all queries being updated essentially all the
time.
15.3. ENCOURAGING TRUTHFUL BIDDING IN MATCHING MARKETS: THE VCG PRINCIPLE445

In the case of a single-item auction, we saw in Chapter 9 that these problems are handled
by running a second-price auction, in which the single item is awarded to the highest bidder at
a price equal to the second-highest bid. As we showed there, truthful bidding is a dominant
strategy for second-price auctions that is, it is at least as good as any other strategy,
regardless of what the other participants are doing. This dominant strategy result means
that second-price auctions avoid many of the pathologies associated with more complex
auctions.
But what is the analogue of the second-price auction for advertising markets with multiple
slots? Given the connections weve just seen to matching markets in the previous section,
this turns out to be a special case of an interesting and fundamental question: how can we
define a price-setting procedure for matching markets so that truthful reporting of valuations
is a dominant strategy for the buyers? Such a procedure would be a massive generalization
of the second-price auction, which though already fairly subtle only applies to the case
of single items.

The VCG Principle. Since a matching market contains many items, it is hard to directly
generalize the literal description of the second-price single-item auction, in which we assign
the item to the highest bidder at the second-highest price. However, by viewing the second-
price auction in a somewhat less obvious way, we get a principle that does generalize.
This view is the following. First, the second-price auction produces an allocation that
maximizes social welfare the bidder who values the item the most gets it. Second, the
winner of the auction is charged an amount equal to the harm he causes the other bidders
by receiving the item. That is, suppose the bidders values for the item are v1 , v2 , v3 , . . . , vn
in decreasing order. Then if bidder 1 were not present, the item would have gone to bidder 2,
who values it at v2 . The other bidders still would not get the item, even if bidder 1 werent
there. Thus, bidders 2 through n collectively experience a harm of v2 because bidder 1 is
there since bidder 2 loses this much value, and bidders 3 through n are unaffected. This
harm of v2 is exactly what bidder 1 is charged. Indeed, the other bidders are also charged
an amount equal to the harm they cause to others in this case, zero, since no bidder is
affected by the presence of any of bidders 2 through n in the single-item auction.
Again, this is a non-obvious way to think about single-item auctions, but it is a principle
that turns out to encourage truthful reporting of values in much more general situations:
each individual is charged the harm they cause to the rest of the world. Or to put it
another way, each individual is charged a price equal to the total amount better off everyone
else would be if this individual werent there. We will refer to this as the Vickrey-Clarke-
Groves (VCG) principle, after the work of Clarke and Groves, who generalized the central
idea behind Vickreys second-price auction for single items [112, 199, 400]. For matching
markets, we will describe an application of this principle due to Herman Leonard [270] and
446 CHAPTER 15. SPONSORED SEARCH MARKETS

Gabrielle Demange [128]; it develops a pricing mechanism in this context that causes buyers
to reveal their valuations truthfully.

Applying the VCG Principle to Matching Markets. In a matching market, we have

a set of buyers and a set of sellers with equal numbers of each and buyer j has a
valuation of vij for the item being sold by seller i.2 We are assuming here that each buyer
knows her own valuations, but that these valuations are not known to the other buyers or
to the sellers. Also, we assume that each buyer only cares which item she receives, not
about how the remaining goods are allocated to the other buyers. Thus, in the language of
auctions, the buyers have independent, private values.
Under the VCG principle, we first assign items to buyers so as to maximize total valuation.
Then, the price buyer j should pay for seller is item in the event she receives it is the
harm she causes to the remaining buyers through her acquisition of this item. This is equal
to the total boost in valuation everyone else would get if we computed the optimal matching
without buyer j present. To give a better sense of how this principle works for matching
markets, we first walk through how it would apply to the example in Figure 15.3. We then
define the VCG price-setting procedure in general, and in the next section we show that it
yields truth-telling as a dominant strategy for each buyer, truth-telling is at least as good
as any other option, regardless of what the other buyers are doing.
In Figure 15.3, where the buyers are advertisers and the items are advertising slots,
suppose we assign items to maximize total valuation: item a to buyer x, item b to buyer
y, and item c to buyer z. What prices does the VCG principle dictate for each buyer? We
show the reasoning in Figure 15.4.
First, in the optimal matching without buyer x present, buyer y gets item a and buyer
z gets item b. This improves the respective valuations of y and z for their assigned
items by 20 10 = 10 and 5 2 = 3 respectively. The total harm caused by x is
therefore 10 + 3 = 13, and so this is the price that x should pay.

In the optimal matching without buyer y present, buyer x still gets a (so she is un-
affected), while buyer z gets item b, for an improved valuation of 3. The total harm
caused by y is 0 + 3 = 3, and so this is the price that y should pay.

Finally, in the optimal matching without buyer z present, buyers x and y each get the
same items they would have gotten had z been there. z causes no harm to the rest of
the world, and so her VCG price is 0.
With this example in mind, we now describe the VCG prices for a general matching
market. This follows exactly from the principle weve been discussing, but it requires a bit
2
As always, we can handle unequal numbers of buyers and sellers by creating fictitious individuals and
valuations of 0, as in Section 15.2.
15.3. ENCOURAGING TRUTHFUL BIDDING IN MATCHING MARKETS: THE VCG PRINCIPLE447

slots advertisers valuations

a x 30, 15, 6
If x weren't there, y
would do better by
20-10=10, and z would
do better by 5-2=3,
b y 20, 10, 4 for a total harm of 13.

c z 10, 5, 2

(a) Determining how much better off y and z would be if x were not present

slots advertisers valuations

a x 30, 15, 6
If y weren't there, x
would be unaffected,
and z would do better
by 5-2=3, for a total
b y 20, 10, 4 harm of 3.

c z 10, 5, 2

(b) Determining how much better off x and z would be if y were not present

Figure 15.4: The VCG price an individual buyer pays for an item can be determined by working out how
much better off all other buyers would be if this individual buyer were not present.

of notation due to the multiple items and valuations. First, let S denote the set of sellers and
B denote the set of buyers. Let VBS denote the maximum total valuation over all possible
perfect matchings of sellers and buyers this is simply the value of the socially optimal
outcome with all buyers and sellers present.
Now, let S i denote the set of sellers with seller i removed, and let B j denote the set
of buyers with buyer j removed. So if we give item i to seller j, then the best total valuation
Si
the rest of the buyers could get is VBj : this is the value of the optimal matching of sellers
and buyers when weve taken item i and buyer j out of consideration. On the other hand, if
buyer j simply didnt exist, but item i were still an option for everyone else, then the best
S
total valuation the rest of the buyers could get is VBj . Thus, the total harm caused by
448 CHAPTER 15. SPONSORED SEARCH MARKETS

buyer j to the rest of the buyers is the difference between how theyd do without j present
S Si
and how they do with j present; in other words, it is the difference VBj VBj . This is the
VCG price pij that we charge to buyer j for item i, so we have the equation

S Si
pij = VBj VBj . (15.1)

The VCG Price-Setting Procedure. Using the ideas developed so far, we can now
define the complete VCG price-setting procedure for matching markets. We assume that
there is a single price-setting authority (an auctioneer) who can collect information from
buyers, assign items to them, and charge prices. Fortunately, this framework works very well
for selling advertising slots, where all the items (the slots) are under the control of a single
agent (the search engine).
The procedure is as follows:

1. Ask buyers to announce valuations for the items. (These announcements need not be
truthful.)

2. Choose a socially optimal assignment of items to buyers that is, a perfect matching
that maximizes the total valuation of each buyer for what they get. This assignment
is based on the announced valuations (since thats all we have access to.)

3. Charge each buyer the appropriate VCG price: that is, if buyer j receives item i
under the optimal matching, then charge buyer j a price pij determined according to
Equation (15.1).

Essentially, what the auctioneer has done is to define a game that the buyers play: they
must choose a strategy (a set of valuations to announce), and they receive a payoff: their
valuation for the item they get, minus the price they pay. What turns out to be true, though
it is far from obvious, is that this game has been designed to make truth-telling in which
a buyer announces her true valuations a dominant strategy. We will prove this in the
next section; but before this, we make a few observations.
First, notice that theres a crucial difference between the VCG prices defined here, and
the market-clearing prices arising from the auction procedure in Chapter 10. The market-
clearing prices defined there were posted prices, in that the seller simply announced a price
and was willing to charge it to any buyer who was interested. The VCG prices here, on the
other hand, are personalized prices: they depend on both the item being sold and the buyer
it is being sold to, The VCG price pij paid by buyer j for item i might well differ, under
Equation (15.1), from the VCG price pik that buyer k would pay if it were assigned item i.3
3
Despite this, there is are deep and subtle connections between the two kinds of prices; we explore this
issue further in the final section of this chapter.
15.4. ANALYZING THE VCG PROCEDURE: TRUTH-TELLING AS A DOMINANT STRATEGY449

Another way to think about the relationship between the market-clearing prices from
Chapter 10 and the VCG prices here is to observe how each is designed to generalize a
different single-item auction format. The market-clearing prices in Chapter 10 were defined
by a significant generalization of the ascending (English) auction: prices were raised step-
by-step until each buyer favored a different item, and we saw in Section 10.5 that one could
encode the single-item ascending auction as a special case of the general construction of
market-clearing prices.
The VCG prices, on the other hand, are defined by an analogous and equally substantial
generalization of the sealed-bid second-price auction. At a qualitative level, we can see the
harm-done-to-others principle is behind both the second-price auction and the VCG prices,
but in fact we can also see fairly directly that the second-price auction is a special case of
the VCG procedure. Specifically, suppose there are n buyers who each want a single item,
and buyer i has valuation vi for it, where the numbers vi are sorted in descending order so
that v1 is the largest. Lets turn this into a matching market with n buyers and n sellers by
simply adding n1 fictitious items; all buyers have valuation 0 for each fictitious item. Now,
if everyone reports their values truthfully, then the VCG procedure will assign item 1 (the
real item the only one with any value) to buyer 1 (who has the highest valuation), and
all the rest of the buyers would get fictitious items of zero value. What price should buyer
S S1
1 pay? According to Equation (15.1), she should pay VB1 VB1 . The first term is buyer
2s valuation, since with buyer 1 gone the socially optimal matching gives item 1 to buyer
2. The second term is 0, since with both buyer 1 and item 1 gone, there are no remaining
items of any value. Thus, buyer 1 pays buyer 2s valuation, and so we have precisely the
pricing rule for second-price sealed-bid auctions.

15.4 Analyzing the VCG Procedure: Truth-Telling as

a Dominant Strategy
We now show that the VCG procedure encourages truth-telling in a matching market. Con-
cretely, we will prove the following claim.
Claim: If items are assigned and prices computed according to the VCG procedure,
then truthfully announcing valuations is a dominant strategy for each buyer, and
the resulting assignment maximizes the total valuation of any perfect matching of
slots and advertisers.
The second part of this claim (that the total valuation is maximized) is easy to justify:
if buyers report their valuations truthfully, then the assignment of items is designed to
maximize the total valuation by definition.
The first part of the claim is the more subtle: Why is truth-telling a dominant strategy?
Suppose that buyer j announces her valuations truthfully, and in the matching we assign
450 CHAPTER 15. SPONSORED SEARCH MARKETS

i j
j

S-i h
S-h
V V
B-j B-j

(a) vij +VBj

Si
is the maximum valuation of any (b) vhj + VBj
Sh
is the maximum valuation only
matching. over matchings constrained to assign h to j.

Figure 15.5: The heart of the proof that the VCG procedure encourages truthful bidding
comes down to a comparison of the value of two matchings.

her item i. Then her payoff is vij pij . We want to show that buyer j has no incentive to
deviate from a truthful announcement.
If buyer j decides to lie about her valuations, then one of two things can happen: either
this lie affects the item she gets, or it doesnt. If buyer j lies but still gets the same item
i, then her payoff remains exactly the same, because the price pij is computed only using
announcements by buyers other than j. So if a deviation from truth-telling is going to be
beneficial for buyer j, it has to affect the item she receives.
Suppose, therefore, that buyer j lies about her valuations and gets item h instead of item
i. In this case, her payoff would be vhj phj . Notice again that the price phj is determined
only by the announcements of buyers other than j. To show that there is no incentive to lie
and receive item h instead of i, we need to show that

vij pij vhj phj .

15.4. ANALYZING THE VCG PROCEDURE: TRUTH-TELLING AS A DOMINANT STRATEGY451

If we expand out the definitions of pij and phj using Equation (15.1), this is equivalent to
showing
S Si S Sh
vij [VBj VBj ] vhj [VBj VBj ].
S
Both sides of this inequality contain the term VBj , so we can add this to both sides; in this
way, the previous inequality is equivalent to showing

Si Sh
vij + VBj vhj + VBj . (15.2)

We now argue why this last inequality holds. In fact, both the left-hand side and the
right-hand side describe the total valuation of different matchings, as shown in Figure 15.5.
The matching on the left-hand side is constructed by pairing j with the item i she would
get in an optimal matching, and then optimally matching the remaining buyers and items.
In other words, it is a matching that achieves the maximum total valuation over all possible
perfect matchings, so we can write the left-hand side as

Si
vij + VBj = VBS . (15.3)

In contrast, the matching on the right-hand side of Inequality (15.2) is constructed by pairing
j with some other item h, and then optimally matching the remaining buyers and items. So
it is a matching that achieves the maximum total valuation only over those matchings that
pair j with h. Therefore,
Sh
vhj + VBj VBS .
The left-hand side of Inequality (15.2), the maximum valuation with no restrictions on who
gets any slot, must be at least as large as the right-hand side, the maximum with a restriction.
And this is what we wanted to show.
Nothing in this argument depends on the decisions made by other buyers about what to
announce. For example, it doesnt require them to announce their true values; the arguments
comparing different matchings can be applied to whatever valuations are announced by the
other buyers, with the same consequences. Thus we have shown that truthfully announcing
valuations is a dominant strategy in the VCG procedure.
To close this section, lets go back to the specific case of keyword-based advertising, in
which the buyers correspond to advertisers and the items for sale correspond to advertising
slots. Our discussion so far has focused on finding and achieving an assignment of advertisers
to slots that maximizes the total valuation obtained by advertisers. But of course, this is
not what the search engine selling the advertising slots directly cares about. Instead it cares
about its revenue: the sum of the prices that it can charge for slots. It is not clear that
the VCG procedure is the best way to generate revenue for the search engine. Determining
which procedure will maximize seller revenue is a current topic of research. It could be
that the best a seller can do is to use some procedure that generates an optimal matching
452 CHAPTER 15. SPONSORED SEARCH MARKETS

and potentially one that is better than VCG at converting more of the total valuation
into seller revenue. Or it could be that the seller is better off using a procedure that does
not always yield an optimal matching. And it may be that some version of a revenue-
equivalence principle such as we saw for single-item auctions in Chapter 9 holds here
as well, showing that certain classes of auction provide equivalent amounts of revenue to the
seller when buyers behave strategically.
In the next sections, we sample the general flavor of some of these revenue issues by
considering the alternative to VCG that the search industry has adopted in practice a
simple-to-describe auction called the Generalized Second Price auction that induces complex
bidding behavior.

15.5 The Generalized Second Price Auction

After some initial experiments with other formats, the main search engines have adopted a
procedure for selling advertising slots called the Generalized Second Price auction (GSP). At
some level, GSP like VCG is a generalization of the second-price auction for a single
item. However, as will see, GSP is a generalization only in a superficial sense, since it doesnt
retain the nice properties of the second-price auction and VCG.
In the GSP procedure, each advertiser j announces a bid consisting of a single number
bj the price it is willing to pay per click. (This would correspond, for example, to the
$1.70 for calligraphy pens or $1.50 for Keuka Lake that we saw at the beginning of the
chapter.) As usual, it is up to the advertiser whether or not its bid is equal to its true
valuation per click vj . Then, after each advertiser submits a bid, the GSP procedure awards
each slot i to the ith highest bidder, at a price per click equal to the (i + 1)st highest bid.
In other words, each advertiser who is shown on the results page is paying a price per click
equal to the bid of the advertiser just below them.
So GSP and VCG can be viewed in parallel terms, in that each asks for announced
valuations from the advertisers, and then each uses these announcements to determine an
assignment of slots to advertisers, as well as prices to charge. When there is a single slot,
both are equivalent to the second-price auction. But when there are multiple slots, their
rules for producing prices are different. VCGs rule is given by Equation (15.1). GSPs rule,
when the bids per click are b1 , b2 , b3 , . . . in descending order, is to charge a cumulative price
of ri bi+1 for slot i. This is because the ith highest bidder will get slot i at a price per click of
bi+1 ; multiplying by the clickthrough rate of ri gives a total price of ri bi+1 for all the clicks
associated with slot i.

Analyzing GSP. GSP was originally developed at Google; once it had been in use for
a while in the search industry, researchers including Varian [399] and Edelman, Ostrovsky,
15.5. THE GENERALIZED SECOND PRICE AUCTION 453

and Schwarz [144] began working out some of its basic properties. Their analysis formulates
the problem as a game, using the definitions from Chapter 6. Each advertiser is a player, its
bid is its strategy, and its payoff is its revenue minus the price it pays. In this game, we will
consider Nash equilibria we seek sets of bids so that, given these bids, no advertiser has
an incentive to change how it is behaving.4
First, well see that GSP has a number of pathologies that VCG was designed to avoid:
truth-telling might not constitute a Nash equilibrium; there can in fact be multiple possible
equilibria; and some of these may produce assignments of advertisers to slots that do not
maximize total advertiser valuation. On the positive side, we show in the next section
that there is always at least one Nash equilibrium set of bids for GSP, and that among
the (possibly multiple) equilibria, there is always one that does maximize total advertiser
valuation. The analysis leading to these positive results about equilibria builds directly on
the market-clearing prices for the matching market of advertisers and slots, thus establishing
a connection between GSP and market-clearing prices.
Hence, while GSP possesses Nash equilibria, it lacks some of the main nice properties of
the VCG procedure from Sections 15.3 and 15.4. However, in keeping with our discussion
from the end of the last section, the search engines ultimately have an interest in choosing a
procedure that will maximize their revenue (given the behavior of the advertisers in response
to it). Viewed in this light, it is not clear that GSP is the wrong choice, though it is also far
from clear that it is the right choice. As mentioned at the end of Section 15.4, understanding
the revenue trade-offs among different procedures for selling keyword-based advertising is
largely an open question, and the subject of current research.

Truth-telling may not be an equilibrium. It is not hard to make an example to show

that truth-telling may not be an equilibrium when the GSP procedure is used. One example
of this is depicted in Figure 15.6:

There are two slots for ads, with clickthrough rates of 10 and 4. In the figure, we
also show a third fictitious slot of clickthrough rate 0, so as to equalize the number of
advertisers and slots.

There are three advertisers x, y, and z, with values per click of 7, 6, and 1 respectively.

Now, if each advertiser bids its true valuation, then advertiser x gets the top slot at a
price per click of 6; since there are 10 clicks associated with this slot, x pays a cumulative
price of 6 10 = 60 for the slot. Advertiser xs valuation for the top slot is 7 10 = 70, so its
4
In order to analyze Nash equilibrium in the bidding game we will assume that each advertiser knows
the values of all other bidders. Otherwise, they do not know the payoffs to all players in the bidding game
and we could not use Nash equilibrium to analyze the game. The motivation for this assumption is that we
envision a situation in which these bidders have been bidding against each other repeatedly and have learned
each others willingnesses to pay for clicks.
454 CHAPTER 15. SPONSORED SEARCH MARKETS

clickthrough slots advertisers revenues

rates per click
10 a x 7

4 b y 6

0 c z 1

Figure 15.6: An example of a set of advertisers and slots for which truthful bidding is not
an equilibrium in the Generalized Second Price auction. Moreover, this example possesses
multiple equilibria, some of which are not socially optimal.

payoff is 70 60 = 10. Now, if x were to lower its bid to 5, then it would get the second slot
for a price per click of 1, implying a cumulative price of 4 for the slot. Since its valuation
for the second slot is 7 4 = 28, this is a payoff of 28 4 = 24, which is an improvement over
the result of bidding truthfully.

Multiple and non-optimal equilibria. The example in Figure 15.6 turns out to illus-
trate some other complex properties of bidding behavior in GSP. In particular, there is more
than one equilibrium set of bids for this example, and among these equilibria are some that
produce a socially non-optimal assignment of advertisers to slots.
First, suppose that advertiser x bids 5, advertiser y bids 4, and advertiser z bids 2. With
a little effort, we can check that this forms an equilibrium: checking the condition for z is
easy, and the main further things to observe are that x doesnt want to lower its bid below
4 so as to move to the second slot, and y doesnt want to raise its bid above 5 to get the
first slot. This is an equilibrium that produces a socially optimal allocation of advertisers to
slots, since x gets slot a, while y gets b and z gets c.
But one can also check that if advertiser x bids 3, advertiser y bids 5, and advertiser z
bids 1, then we also get a set of bids in Nash equilibrium. Again, the main thing to verify is
that x doesnt want to raise its bid above ys, and that y doesnt want to lower its bid below
xs. This equilibrium is not socially optimal, since it assigns y to the highest slot and x to
the second-highest.
There is much that is not understood in general about the structure of the sub-optimal
equilibria arising from GSP. For example, it is an interesting open question to try quantifying
15.5. THE GENERALIZED SECOND PRICE AUCTION 455

slots advertisers valuations

a x 70, 28, 0

b y 60, 24, 0

c z 10, 4, 0

Figure 15.7: Representing the example in Figure 15.6 as a matching market, with advertiser
valuations for the full set of clicks associated with each slot.

how far from social optimality a Nash equilibrium of GSP can be.

The Revenue of GSP and VCG. The existence of multiple equilibria also adds to the
difficulty in reasoning about the search engine revenue generated by GSP, since it depends
on which equilibrium (potentially from among many) is selected by the bidders. In the
example weve been working with, well show that depending on which equilibrium of GSP
the advertisers actually use, the revenue to the search engine can be either higher or lower
than the revenue it would collect by charging the VCG prices.
Lets start by determining the revenue to the search engine from the two GSP equilibria
that we worked out above.

With bids of 5, 4, and 2, the 10 clicks in the top slot are sold for 4 per click, and the 4
clicks in the second slot are sold for 2 per click, for a total revenue to the search engine
of 48.

On the other hand, with bids of 3, 5, and 1, the 10 clicks in the top slot are sold for 3
per click, and the 4 clicks in the second slot are sold for 1 per click, for a total revenue
to the search engine of 34.

Now, how do these compare with the revenue generated by the VCG procedure? To work
out the VCG prices, we first need to convert the example from Figure 15.6 into a matching
market, just as we did in Section 15.2: for each advertiser and each slot, we work out the
advertisers valuation for the full set of clicks associated with that slot. We show these
valuations in Figure 15.7.
456 CHAPTER 15. SPONSORED SEARCH MARKETS

prices slots advertisers valuations

40 a x 70, 28, 0

4 b y 60, 24, 0

0 c z 10, 4, 0

Figure 15.8: Determining market-clearing prices for the example in Figure 15.6, starting
with its representation as a matching market.

The matching used by the VCG procedure is the one which maximizes the total valuation
of all advertisers for the slot they get; this is achieved by assigning slot a to x, slot b to y,
and slot c to z. Now, we work out a price to charge each advertiser for the full set of clicks
in the slot it gets, by determining the harm each advertiser causes to all others. The harm x
causes to y and z can be computed as follows: without x present, y would move up one slot,
obtaining an increased valuation of 60 24 = 36, and z would move up one slot, obtaining
an increased valuation of 4 0 = 4. Therefore, x should pay 40 for the full set of clicks in
the first slot. Similarly, without y present, z would get 4 instead of 0, so y should pay 4
for the set of clicks in the second slot. Finally, since z causes no harm to anyone, it pays 0.
Thus, the total revenue collected by the search engine is 44.
So we find that in this example, the answer to the question, Does GSP or VCG provide
more revenue to the search engine? is indeed that it depends on which equilibrium of GSP
the advertisers use. With the first equilibrium of GSP that we identified, the revenue is
48, while with the second, the revenue is 34. The revenue from the VCG mechanism is in
between these two values, at 44.

15.6 Equilibria of the Generalized Second Price Auc-

tion
The examples in the previous section give a sense for some of the complex behavior of GSP.
Here, we show that there is nonetheless a natural connection between GSP and market-
clearing prices: from a set of market-clearing prices for the matching market of advertisers
15.6. EQUILIBRIA OF THE GENERALIZED SECOND PRICE AUCTION 457

and slots, one can always construct a set of bids in Nash equilibrium and moreover one
that produces a socially optimal assignment of advertisers to slots. As a consequence, there
always exists a set of socially optimal equilibrium bids for the GSP procedure.
To give the basic idea for how to construct an equilibrium, we do it first on the example
from Figure 15.6. In fact, weve just seen two equilibria for this example in the previous
section, but the point here is to see how a socially optimal one can be easily constructed
by following a few simple principles, rather than by trial-and-error or guesswork. Well then
identify the principles from this example that carry over to construct equilibria in general.

An Equilibrium for Figure 15.6. The basic idea is to use market-clearing prices to guide
us to a set of bids that produce these prices. To construct market-clearing prices, we first
convert the example from Figure 15.6 into a matching market by determining advertisers
valuations for each slot, as we did at the end of the previous section (in Figure 15.7). We
then determine market-clearing prices for this matching market, as shown in Figure 15.8.
These market-clearing prices are cumulative prices for each slot single prices that cover
all the clicks associated with that slot. We can easily translate back to prices per click by
simply dividing by the clickthrough rate: this produces a price per click of 40/10 = 4 for the
first slot, and 4/4 = 1 for the second slot. It will turn out not to be important how we price
the fictitious third slot per click, but it is fine to give it a price of 0.
Next, we find bids that result in these prices per click. This is not hard to do: the prices
per click are 4 and 1 for the two slots, so these should be the bids of y and z respectively.
Then the bid of x can be anything as long as its more than 4. With these bids, x pays 4
per click for the first slot, y pays 1 per click for the second slot, and z pays 0 per click for
the (fake) third slot and the allocation of advertisers to slots is socially optimal.
Having used the market-clearing prices to guide us toward a set of bids, we now use
the market-clearing property to verify that these bids form a Nash equilibrium. There are
several cases to consider, but the overall reasoning will form the general principles that
extend beyond just this example. First, lets argue that x doesnt want to lower its bid. If it
drops down to match ys bid, then it can get the second slot at the price that y is currently
paying. Similarly, it could match zs bid and get the third slot at the price that z is currently
paying. But since the prices are market-clearing, x doesnt want to do either of these things.
For similar reasons, y doesnt want to drop its bid to get the third slot at the price z is
currently paying.
Next, lets argue that y doesnt want to raise its bid. Indeed, suppose that it raised its
bid to get the first slot to do this, it would need to match xs current bid. But in this
case, x becomes the second-highest bidder, and so y would get the first slot at a price per
click equal to xs current bid, which is above 4. Because the prices are market-clearing, y
doesnt prefer the first slot to its current slot at a price per click of 4, so it certainly doesnt
458 CHAPTER 15. SPONSORED SEARCH MARKETS

prefer the first slot to its current slot at a higher price per click. Thus, y doesnt want to
raise its bid. Similar reasoning shows that z doesnt want to raise its bid.
This concludes the analysis: no advertiser wants to raise or lower its current bid, and so
the set of bids in this example forms a Nash equilibrium.
It is not hard to carry out the construction and the reasoning used here in general; we
show how to do this next.

GSP always has a Nash equilibrium: The General Argument Now lets consider
a general instance where we have a set of advertisers and a set of slots; by adding fake slots
of 0 value if necessary, we will assume that these two sets have the same size.
Lets suppose that the advertisers are labeled 1, 2, . . . , n in decreasing order of their
valuations per click, and lets suppose that the slots are labeled 1, 2, . . . , n in decreasing
order of their clickthrough rates. We first represent the set of advertisers and slots using a
matching market, and we consider any set of market-clearing prices for the slots, denoted
p1 , p2 , . . . , pn in order. Again, these are prices for the full set of clicks in each slot; we
will consider the price per click of each slot below. In Section 15.2, we argued that since
a perfect matching in the resulting preferred-seller graph maximizes the total valuation of
each advertiser for the slot it gets, it follows that the advertiser with the highest valuation
per click gets the top slot, the advertiser with next-highest valuation gets the second slot,
and so forth, with advertiser i getting slot i.
We now show how to get this outcome from an equilibrium set of bids in GSP. Our plan is
first to construct a set of bids that produces this same set of market-clearing prices, together
with the same socially optimal matching of advertisers to slots. Then, we will show that
these bids form a Nash equilibrium.

Constructing the bids. For the first step, we start by considering the prices per click
that we get from the market-clearing prices: pj = pj /rj . We start by arguing that these
prices per click decrease as we move down the slots: p1 p2 pn . To see why this
is true, lets compare two slots j and k, where j is numbered lower than k, and show that
pj pk .
Since the prices are market-clearing, advertiser k is at least as happy with slot k as it
would be with slot j. In slot k, its total payoff is the product of its payoff per click, vk pk ,
times the clickthrough rate rk . In slot j, its total payoff would be the product of its payoff
per click there, vk pj , times the clickthrough rate rj . Now, the clickthrough rate is higher
in slot j, yet slot k is preferred; so it must be that the payoff per click is smaller in slot j.
That is, vk pj is smaller than vk pk , or equivalently, pj pk . This inequality is precisely
the fact we were looking for.
Now that we have decreasing prices per click, we can construct the bids were looking
15.7. AD QUALITY 459

for. We simply have advertiser j place a bid of pj1 for each j > 1, and we have advertiser 1
place any bid larger than p1 . Notice that this is exactly what happened when we constructed
an equilibrium for the example in Figure 15.6. With these bids, we have all the desired
properties: for each j, advertiser j is assigned to slot j and pays a price per click of pj .

Why do the bids form a Nash equilibrium? To show why these bids form a Nash
equilibrium, we adapt the principles that we used in analyzing the equilibrium for Figure 15.6.
We first argue that no advertiser will want to lower its bid, and then that no advertiser will
want to raise its bid either.
Consider an advertiser j, currently in slot j. If it were to lower its bid, the best it could
do is to pick some lower slot k, bid just under the current bid of advertiser k, and thereby
get slot k at the price that advertiser k is currently paying. But since the prices are market-
clearing, j is at least as happy with its current slot at its current price as it would be with
ks current slot at ks current price. So in fact, this shows that no advertiser will want to
lower its bid.
How about raising a bid? The best advertiser j could do here is to pick some higher slot
i, bid just above the current bid of advertiser i, and thereby get slot i. What price would j
pay for slot i if it did this? Its forcing advertiser i one slot down, and so it would pay the
current bid of advertiser i. This is actually larger than what advertiser i is currently paying
for slot i: advertiser i is currently paying the bid of advertiser i + 1, which is lower. So the
upshot is that j would get slot i at a price higher than the current price of slot i. Since
the market-clearing condition says that j doesnt even prefer slot i at the current price, it
certainly wouldnt prefer it at a higher price. This shows that no advertiser wants to raise
its bid either, and so the set of bids indeed forms a Nash equilibrium.

15.7 Ad Quality
What weve discussed thus far forms part of the basic framework for thinking about search
advertising markets. Of course, there are numerous further issues that come up in the use
of this framework by the major search engines, and in this section and the next we briefly
discuss a few of these issues. We begin with the issue of ad quality.

The assumption of a fixed clickthrough rate. One of the assumptions weve made
throughout the analysis is that a fixed clickthrough rate rj is associated with each slot j
in other words, that the number of clicks this slot receives is independent of which ad you
place there. But in general this is not likely to be true: users will look at the thumbnail
description of an ad placed in a given slot (evaluating, for example, whether they recognize
the name of the company placing the ad), and this will affect whether they click on the ad.
460 CHAPTER 15. SPONSORED SEARCH MARKETS

And this, in turn, affects how much money the search engine makes, since its charging per
click, not per impression.
So from the search engines point of view, the worrisome scenario is that a low-quality
advertiser bids very highly, thus obtaining the first slot under GSP. Users are then not
interested in clicking through on this ad (maybe they dont trust the company, or the ad is
only minimally relevant to the query term). As a result, it sits at the top of the list as the
high bidder, but the search engine makes almost no money from it because users rarely click
on the ad. If the search engine could somehow expel this ad and promote the higher-quality
ads, it could potentially make more money.
Again, our model as described cant really address this, since it starts from the assumption
that an ad in position i will get clicks at rate ri , regardless of which ad it is. This pure
version of GSP, using the model from Sections 15.5 and 15.6 is essentially what the company
Overture used at the time it was acquired by Yahoo!, and hence what Yahoo! used initially
as well. And indeed, it suffers from exactly this problem advertisers can sometimes occupy
high slots without generating much money for the search engine.

The role of ad quality. When Google developed its system for advertising, it addressed
this problem as follows. For each ad submitted by an advertiser j, they determine an
estimated quality factor qj . This is intended as a fudge factor on the clickthrough rate: if
advertiser j appears in slot i, then the clickthrough rate is estimated to be not ri but the
product qj ri . The introduction of ad quality is simply a generalization of the model weve
been studying all along: in particular, if we assume that all factors qi are equal to 1, then
we get back the model that weve been using thus far in the chapter.
From the perspective of our matching market formulation, its easy to incorporate these
quality factors: we simply change the valuation of advertiser j for slot i, from vij = ri vj to
vij = qj ri vj . The rest of the analysis remains the same, using these new valuations.
Google has adapted the GSP procedure analogously. Rather than assigning advertisers to
slots in decreasing orders of their bids bj , it assigns them in decreasing order of the product of
their bid and quality factor qj bj . This makes sense, since this is the ordering of advertisers by
expected revenue to the search engine. The payments change correspondingly. The previous
rule paying the bid of the advertiser just below you can, in retrospect, be interpreted
more generally as paying the minimum bid you would need in order to hold your current
position. This rule carries over to the version with quality factors: each advertiser pays the
minimum amount it would need to keep its current position, when ranked according to qj bj .
With these changes, its possible to go back and perform the analysis of GSP at this
more general level. Close analogues of all the previous findings still hold here; while the
introduction of quality factors makes the analysis a little bit more complicated, the basic
ideas remain largely the same [144, 399].
15.8. COMPLEX QUERIES AND INTERACTIONS AMONG KEYWORDS 461

The mysterious nature of ad quality. How is ad quality computed? To a significant

extent, its estimated by actually observing the clickthrough rate of the ad when shown on
search results pages this makes sense, since the goal of the quality factor is to act as
a modifier on the clickthrough rate. But other factors are taken into account, including
the relevance of the ad text and the landing page that the ad links to. Just as with the
unpaid organic search engine results on the left-hand-side of the screen, search engines are
very secretive about how they compute ad quality, and will not reveal the function to the
advertisers who are bidding.
One consequence is that the introduction of ad quality factors makes the keyword-based
advertising market much more opaque to the advertisers. With pure GSP, the rules were
very simple: for a given set of bids, it was clear how the advertisers would be allocated to
slots. But since the ad quality factor is under the search engines control, it gives the search
engine nearly unlimited power to affect the actual ordering of the advertisers for a given set
of bids.
How does the behavior of a matching market such as this one change when the precise
rules of the allocation procedure are being kept secret from the bidders? This is an issue
that is actively discussed in the search industry, and a topic for potential research.

15.8 Complex Queries and Interactions Among Key-

words
At the outset, we observed that markets are being conducted simultaneously for millions of
query words and phrases. In our analysis, weve focused the model on what goes on in a
single one of these markets, for a single keyword; but in reality, of course, there are complex
interactions among the markets for different keywords.
In particular, consider the perspective of a company thats trying to advertise a product
using keyword-based advertising; suppose, for example, that the company is selling ski va-
cation packages to Switzerland. There are a lot of different keywords and phrases on which
the company might want to place bids: Switzerland, Swiss vacation, Swiss hotels,
Alps, ski vacation, European ski vacation, and many others (including grammatical
permutations of these). With a fixed advertising budget, and some estimates about user
behavior and the behavior of other advertisers, how should the company go about dividing
its budget across different keywords? This is a challenging problem, and one that is the
subject of current research [357].
Theres an analogous problem from the search engines perspective. Suppose advertisers
have placed bids on many queries relevant to Swiss ski vacations, and then a user comes and
issues the query, Zurich ski vacation trip December. Its quite likely that very few users
have ever issued this exact query before, and also very likely that no advertiser has placed
462 CHAPTER 15. SPONSORED SEARCH MARKETS

a bid on this exact phrase. If the rules of the keyword-based advertising market are defined
too strictly that the search engine can only show ads for words or phrases that have been
explicitly bid on then it seems as though both the search engine and the advertisers are
losing money: there clearly are advertisers who would be happy to be displayed for this
query.
The question of which ads to show, however, is quite a difficult problem. A simple
rule, such as showing the advertisers that placed the maximum bid for any of the words
in the query, seems like a bad idea: probably there are advertisers who have placed very
high bids on vacation (e.g. companies that sell generic vacation packages) and ski (e.g.
companies that sell skis), and neither of these seems like the right match to the query. It
seems important to take into account the fact that the query, through its choice of terms, is
specifying something fairly narrow.
Furthermore, even if relevant advertisers can be identified, how much should they be
charged for a click, given that they never expressed a bid on exactly this query? The main
search engines tend to get agreements from advertisers that theyll extrapolate from their
bids on certain queries to implied bids on more complex queries, such as in this example,
but working out the best way to do this is not fully understood. These issues are the subject
of active work at search engine companies, and again the subject of some very interesting
potential further research.

15.9 Advanced Material: VCG Prices and the Market-

Clearing Property
At the end of Section 15.3, we noted some of the differences between the two main ways weve
seen to assign prices to items in matching markets: the VCG prices defined in this chapter,
and the construction of market-clearing prices from Chapter 10. In particular, we observed
that the difference reflected a contrast between personalized and posted prices. VCG prices
are selected only after a matching between buyers and sellers has been determined the
matching that maximizes the total valuation of buyers for what they get. The VCG price
of an item thus makes use of information not just about the item itself, but also about who
is buying it in the matching. Market-clearing prices, in a sense, work the other way around:
the prices are chosen first, and they are posted prices that are offered to any buyer who
is interested. The prices then cause certain buyers to select certain items, resulting in a
matching.5
Given these significant differences, one might expect the prices to look different as well.
But a comparison of simple examples suggests that something intriguing might be going on.
5
In the discussion that follows, well refer to nodes on the left-hand side of the bipartite graph sometimes
as items and sometimes as sellers; for our purposes here, we treat these as meaning the same thing.
15.9. ADVANCED MATERIAL: VCG PRICES AND THE MARKET-CLEARING PROPERTY463

Prices Sellers Buyers Valuations

3 a x 12, 4, 2

1 b y 8, 7, 6

0 c z 7, 5, 2

Figure 15.9: A matching market, with valuations and market-clearing prices specified, and
a perfect matching in the preferred-seller graph indicated by the bold edges.

Consider for instance the matching market shown in Figures 15.3 and 15.4. In Figure 15.3
we see a set of market-clearing prices constructed using the procedure from Chapter 10. In
Figure 15.4, we see that these same prices arise as the VCG prices too.
Nor is it the special structure of prices arising from clickthrough rates and revenues per
click that causes this. For instance, lets go back to the example used in Figure 10.6 from
Chapter 10, which has valuations with a much more scrambled structure. Weve re-drawn
the final preferred-seller graph arising from the auction procedure in Figure 15.9, with the
(unique) perfect matching in this graph indicated using bold edges. This is the matching
that maximizes the total valuation of buyers for the item they get, so we apply the definitions
from earlier in the current chapter to determine the VCG prices. For example, to determine
the price that should be charged for seller as item, we observe

If neither a nor x were present, the maximum total valuation of a matching between
the remaining sellers and buyers would be 11, by matching y to c and z to b.

If x werent present but a were, then the maximum total valuation possible would be
14, by matching y to b and z to a.

The difference between these two quantities is the definition of the VCG price for item
a; it is 14 11 = 3.

We could perform the corresponding analysis to get the VCG prices for items b and c, and
wed see that the values are 1 and 0, respectively. In other words, we again find that the
VCG prices are also market-clearing prices.
464 CHAPTER 15. SPONSORED SEARCH MARKETS

In this section, we show that the relationship suggested by these examples holds in
general. Our main result is that despite their definition as personalized prices, VCG prices
are always market-clearing. That is, suppose we were to compute the VCG prices for a
given matching market, first determining a matching of maximum total valuation, and then
assigning each buyer the item they receive in this matching, with a price tailored for this
buyer-seller match. Then, however, suppose we go on to post the prices publicly: rather than
requiring buyers to follow the matching used in the VCG construction, we allow any buyer to
purchase any item at the indicated price. We will see that despite this greater freedom, each
buyer will in fact achieve the highest payoff by selecting the item she was assigned when the
VCG prices were constructed. This will establish that the prices are market-clearing under
the definition from Chapter 10.

First Steps Toward a Proof. Lets think for a minute about how you might prove such
a fact, once you start to suspect from simple examples that it might be true. Its tempting
to start with the very compact formula defining the VCG prices Equation (15.1) and
then somehow reason about this formula to show that it has the market-clearing property.
In fact, its tricky to make this approach work, and its useful to understand why. Recall
that Equation (15.1) says that if item i is assigned to buyer j in the optimal matching, then
we should charge a price of
S Si
VBj VBj ,
S Si
where VBj is the total valuation of an optimal matching with j removed, and VBj is the
S
total valuation of an optimal matching with both i and j removed. Now, the term VBj is in
fact a sum of many smaller terms, each consisting of the valuation of a distinct buyer for the
Si
item she is assigned in an optimal matching. VBj is similarly a sum of many terms. But
S Si
the key conceptual difficulty is the following: VBj and VBj arise from different matchings
potentially very different matchings and so there is no direct way to compare the sums
that they represent and easily subtract the terms of one from the other.
To make progress, we need to actually understand how the matchings that define these
S Si
two terms VBj and VBj relate to each other at a structural level. And to do this, we will
show that matchings achieving these respective quantities can in fact arise from a common
set of market-clearing prices: there is a single set of market-clearing prices on the set of
S Si
items S so that matchings achieving each of VBj and VBj arise as perfect matchings in the
preferred-seller graphs of related but slightly different matching markets. This will enable
us to see how the two matchings relate to each other and in particular how to build one
from the other in a way that lets us subtract the relevant terms from each other and thus
analyze the right-hand side of Equation (15.1).
For all this to work, we need to first understand which set of market-clearing prices
actually correspond to the VCG prices. There are many possible sets of market-clearing
15.9. ADVANCED MATERIAL: VCG PRICES AND THE MARKET-CLEARING PROPERTY465

prices, but with some checking, we can see that in our examples, the VCG prices have
corresponded to prices that are as small as possible, subject to having the market-clearing
property. So lets consider the following way to make this precise. Over all possible sets of
market-clearing prices, consider the ones that minimize the total sum of the prices. (For
example, in Figure 15.9, the total sum of prices is 3 + 1 + 0 = 4.) We will refer to such
prices as a set of minimum market-clearing prices. In principle, there could be multiple sets
of minimum market-clearing prices, but in fact we will see that there is only one such set,
and they form the VCG prices. This is the crux of the following result, proved by Leonard
[270] and Demange [128].

Claim: In any matching market, the VCG prices form the unique set of market-
clearing prices of minimum total sum.

This is the statement we will prove in this section.

The proof of this statement is quite elegant, but it is also arguably the most intricate piece
of analysis in the book; it has a level of complexity that involves bringing together several
non-trivial lines of argument as part of the overall proof. In approaching a proof with this
type of structure, it helps to proceed in two stages. First, we will outline a sequence of two
key facts that illuminate the structure of the underlying matchings. Each of these two facts
needs a proof, but we will first simply state the facts and show how the overall proof of the
claim follows directly from them. This provides a high-level overview of how the proof works,
in a way that is self-contained and contains the central ideas. After this, we will describe
how to prove the two facts themselves, which will fill in the remaining details of the proof.
Finally, here is one more point to note before beginning: As in a number of previous
places when we discussed matching markets, we will assume that all valuations are whole
numbers (0, 1, 2, . . .), and that all prices are whole numbers as well.

A. High-Level Overview of the Proof

S
Recall that our basic plan is to understand how matchings defining the quantities VBj and
Si
VBj relate to each other, by showing how they arise from a common structure. To do this,
the first step is to show that the preferred-seller graph for the minimum market-clearing
prices contains not only the edges of a perfect matching, but also enough extra edges that
we can easily assemble other matchings once we begin removing buyers in the ways suggested
by the VCG formula.

The First Fact: The Preferred-Seller Graph for Minimum Market-Clearing Prices.
The first of our two facts talks about the structure of the preferred-seller graph in the case
when a set of market-clearing prices has minimum total sum. As a first step, lets go back
466 CHAPTER 15. SPONSORED SEARCH MARKETS

i There is an alternating path,

beginning with a non-matching edge,
from i to an item (i*) of price 0.

h k

price 0 i* m

Figure 15.10: The key property of the preferred-seller graph for minimum market-clearing
prices: for each item of price greater than 0, there is an alternating path, beginning with a
non-matching edge, to an item of price 0.

to the initial example of market-clearing prices from Chapter 10, and in particular com-
pare preferred-seller graphs for two different sets of market-clearing prices on the same set
of valuations, shown in Figures 10.5(b) and 10.5(d). Notice that the prices in the first of
these, Figure 10.5(b), are larger and more spread out, while the prices in Figure 10.5(d)
in fact have minimum total sum. This corresponds to a difference in the structures of the
preferred-seller graphs as well. The preferred-seller graph in Figure 10.5(b) is very sparse,
with just three separate edges that constitute a perfect matching. The preferred-seller graph
in Figure 10.5(d) is much denser: although it too contains only one perfect matching, it has
additional edges that seem to serve as supports, anchoring the matching in place.
We now show that this anchoring effect is a general one: essentially, whenever a set of
market-clearing prices has minimum total sum, the preferred-seller graph must contain not
only a perfect matching, but also enough other edges to form a path linking each item to
an item of price 0. In fact, the paths we construct will be alternating paths in the sense
defined in Section 10.6: for a given perfect matching in the graph, the edges on the paths
will alternate between being part of the matching and not part of the matching. We will
refer to these two kinds of edges as matching edges and non-matching edges respectively.
Here is the exact statement of the first fact, shown schematically in Figure 15.10.

Fact 1: Consider the preferred-seller graph for a set of market-clearing prices

of minimum total sum, fix a particular perfect matching in this graph, and let
i be any item whose price is greater than 0. Then there is an alternating path,
15.9. ADVANCED MATERIAL: VCG PRICES AND THE MARKET-CLEARING PROPERTY467

Prices Sellers Buyers Valuations

4 a w 7, 5, 4, 2

3 b x 6, 5, 2, 1

1 c y 5, 6, 2, 2

0 d z 4, 4, 2, 1

Figure 15.11: A matching market with market-clearing prices of minimum total sum. Note
how from each item, there is an alternating path, beginning with a non-matching edge, that
leads to the item of zero price.

beginning with a non-matching edge, that connects i to some item of price 0.

For example, in Figure 15.9, with the matching indicated in bold, there is an alternating
path from b to y to c; this path begins with the non-matching b-y edge, and ends at the
zero-priced item c. Similarly, there is a longer alternating path from a through z, b, and
y, ending at c. Figure 15.11 shows a larger example, also with market-clearing prices of
minimum-total sum, and also with the matching indicated in bold; here too one can find
alternating paths from each of items a, b, and c down to the zero-priced item d.
Following our plan, we defer the proof of Fact 1 until later in the section. However, we can
give some intuition for the proof as follows. Roughly speaking, if there werent alternating
paths anchoring all the prices to items of price 0, then we could find a set of items that were
floating free of any relation to the zero-priced items. In this case, we could push the prices
of these free-floating items down slightly, while still preserving the market-clearing property.
This would yield a set of market-clearing prices of smaller total sum, contradicting our
assumption that we already have the minimum market-clearing prices. This contradiction
468 CHAPTER 15. SPONSORED SEARCH MARKETS

Sellers Buyers Valuations

a w 7, 5, 4, 2

b x 0, 0, 0, 0

c y 5, 6, 2, 2

d z 4, 4, 2, 1

Figure 15.12: If we start with the example in Figure 15.11 and zero out buyer x, the structure
of the optimal matching changes significantly.

will show that the minimum market-clearing prices are all anchored via alternating paths to
zero-priced items.

The Second Fact: Zeroing Out a Buyer. Our second main fact will relate the minimum
S
market-clearing prices to a matching that achieves the value VBj , the first term on the right-
hand side of Equation (15.1).
To explain how this fact works, we start with a useful way to think about the quantity
S S
VBj . Formally, VBj is the maximum total valuation of any matching in the market where
j has been removed, but where all items have been kept. But heres a different, equivalent
S
way to define VBj . Suppose that we were to change js valuations for every item to 0; well
call this the version of the matching market in which j has been zeroed out. To find an
optimal matching in this market with j zeroed out, we note that it doesnt matter which
item j gets (since j values them all at zero); therefore, we can first optimally match all the
other buyers with items, and then give j whatever is left over. The value of the resulting
S S
matching is VBj . In other words, VBj is the value of the optimal matching in the market
15.9. ADVANCED MATERIAL: VCG PRICES AND THE MARKET-CLEARING PROPERTY469

Prices Sellers Buyers Valuations

4 a w 7, 5, 4, 2

3 b x 0, 0, 0, 0

1 c y 5, 6, 2, 2

0 d z 4, 4, 2, 1

Figure 15.13: However, even after we zero out buyer x, the same set of prices remain market-
clearing. This principle is true not just for this example, but in general.

where j is zeroed out: she is still present, but all her valuations are now equal to 0.
Now, an optimal matching in the market with j zeroed out may have a very different
structure than an optimal matching in the original market different buyers may get
completely different items. For example, Figure 15.12 shows the unique optimal matching
in the market from Figure 15.11 after we zero out x: other than buyer y, who still gets
item b, the assignment of items to all other buyers has changed completely. This is another
reflection of the difficulty in reasoning about Equation (15.1): when we remove buyers or
items, the matchings can rearrange themselves in complex ways.
Despite this, there is an important connection between the original market and the zeroed-
out market: the minimum market-clearing prices for the original market are also market-
clearing for the zeroed-out market. We illustrate this for our example in Figure 15.13: keeping
the same prices that were used in Figure 15.11, we see that the preferred-seller graph still has
a perfect matching even after x has been zeroed out, and this means that the prices are still
market-clearing. Moreover, we can observe some additional features of this example. First,
x now receives an item of price 0. Second, consider the payoff of each other buyer, defined
470 CHAPTER 15. SPONSORED SEARCH MARKETS

as the valuation minus the price of the item she gets. For each other buyer, the payoff is the
same in Figures 15.11 and 15.13.
Our second fact shows that all these observations hold in general.

Fact 2: Consider any matching market with minimum market-clearing prices p,

and let j be any buyer.

(i) The prices p are also market-clearing for the market in which j is zeroed
out.

Moreover, for any perfect matching in the preferred-seller graph of the zeroed-out
market,

(ii) buyer j receives a zero-priced item; and

(iii) each buyer other than j obtains the same payoff that she did in the original
market.

Again, we defer the proof of Fact 2 to later in the section, but it is not hard to establish the
proof using Fact 1. Essentially, when we zero out j, we look at the item i that j formerly got
in the original market, before she was zeroed out. We follow the alternating path provided
by Fact 1 from i down to an item i of price 0. We then show that assigning item i to j, and
shifting the assignment to all other buyers using the edges on this alternating path, gives us
a perfect matching in the preferred-seller graph of the zeroed-out market at the same prices.
This shows that the same prices are in fact market-clearing for the zeroed-out market, and
will establish parts (ii) and (iii) of the claim as well.

Proving the Claim Using Facts 1 and 2. With Facts 1 and 2 in place, we can finish
the proof of our main claim, that the minimum market-clearing prices are defined by the
VCG formula.
To start, lets review some notation. As before, let vij denote the valuation that a buyer
j has for an item i. Let pi be the price charged for item i in our market-clearing prices, and
let P be the sum of the prices of all items. Suppose that buyer j is matched to item i in the
perfect matching in the preferred-seller graph. Buyer j receives a payoff of vij pi from this
item i; we will use zj to denote this payoff,

zj = vij pi , (15.4)

and Z to denote the sum of the payoffs of all buyers from the items they are matched with.
Next, lets recall two basic observations that were made in earlier sections. First, each
buyer j achieves a payoff of vij pi from the item to which she is matched. As we noted in
15.9. ADVANCED MATERIAL: VCG PRICES AND THE MARKET-CLEARING PROPERTY471

Chapter 10, if we add these expressions up over all buyers, we get the following relationship
for the matching M of buyers to items:

Total Payoff of M = Total Valuation of M Sum of all prices.

In our current notation, this is

Z = VBS P. (15.5)
Second, we argued in Section 15.4 that if i is matched to j in an optimal matching, then
Si
vij + VBj = VBS . (15.6)

This is Equation (15.3) from Section 15.4, and it follows simply because one way to achieve
an optimal matching is to first pair i with j (obtaining a valuation of vij ), and then optimally
match all the remaining buyers and items.
Finally, lets consider this same formula

Total Payoff of M = Total Valuation of M Sum of all prices,

for the market in which j has been zeroed out, using the same set of market-clearing prices
and a perfect matching in the preferred-seller graph that Fact 2 provides. The total valuation
S
of this matching is VBj , as we argued earlier. The prices havent changed, so their total
sum is still P . Finally, whats the total payoff? By part (ii) of Fact 2, the payoff for buyer
j has dropped from zj , which it was in the original market, to 0. By part (iii) of Fact 2,
the payoff for every other buyer has remained the same. Therefore, the total payoff in the
zeroed-out market is Z zi . Putting all these together, we have the equation
S
Z zi = VBj P. (15.7)

Since we now have equations that relate the two terms on the right-hand of Equation
(15.1) to a common set of quantities, we can finish the proof using a small amount of algebraic
manipulation. Lets first subtract Equation (15.7) from Equation (15.5): this gives us

zi = VBS VBj
S
.

Next, lets expand zi using Equation (15.4) and expand VBS using Equation (15.6). This
gives us
Si S
vij pi = vij + VBj VBj .
Canceling the common term of vij and negating everything, we get
S Si
pi = VBj VBj ,

which is the VCG formula we were seeking. This shows that the market-clearing prices of
minimum total sum are defined by the VCG formula, and hence proves the claim.
472 CHAPTER 15. SPONSORED SEARCH MARKETS

i j

h k
?
Figure 15.14: In order for a matching edge from a buyer k to an item h to leave the preferred-
seller graph when the price of i is reduced by 1, it must be that k now strictly prefers i.
In this case, k must have previously viewed i as comparable in payoff to h, resulting in a
non-matching edge to i.

B. Details of the Proof

The discussion so far provides a complete proof, assuming that we take Facts 1 and 2 as
given. To finish the proof, therefore, we need to provide proofs of Facts 1 and 2. The crux
of this is proving Fact 1, which will consist of an analysis of alternating paths in the style of
Section 10.6. After this analysis, establishing Fact 2 is relatively quick using Fact 1.

A First Step Toward Fact 1. To prove Fact 1, we consider a set of minimum market-
clearing prices, and an item i whose price is greater than 0, and we try to construct an
alternating path (beginning with a non-matching edge) from i to some zero-priced item.
As a first step toward this, to convey the idea at the heart of the argument, lets show
something simpler: that this item i, of price pi > 0, is connected to at least one non-matching
edge (in addition to its matching edge to the buyer j that obtains it). Clearly it will be
necessary to establish the presence of such a non-matching edge in any case, if we want
ultimately to show that i has an alternating path all the way down to a zero-priced item.
So suppose, by way of contradiction, that i is not connected to a non-matching edge: its
only edge is the matching edge to buyer j. In this case, we claim that we can subtract 1
from the price pi , and the resulting modified prices will still be market-clearing. This would
be a contradiction, since we assumed our market-clearing prices have minimum total sum.
Clearly if we subtract 1 from pi , it is still non-negative, so we just need to show that
the preferred-seller graph still contains a perfect matching. In fact, well show the stronger
fact that the preferred-seller graph still contains all the matching edges that it used to have.
Indeed, how could a matching edge leave the preferred-seller graph after the price reduction?
15.9. ADVANCED MATERIAL: VCG PRICES AND THE MARKET-CLEARING PROPERTY473

X
(1) No matching edges from a
buyer in X to an item not in X.
i
(2) No non-matching edges
from an item in X to a buyer
not in X.

h k

Figure 15.15: Consider the set X of all nodes that can be reached from i using an alternating
path that begins with a non-matching edge. As we argue in the text, if k is a buyer in X,
then the item to which she is matched must also be in X. Also, if h is an item in X, then
any buyer to which h is connected by a non-matching edge must also be in X. Here is an
equivalent way to phrase this: there cannot be a matching edge connecting a buyer in X to
an item not in X, or a non-matching edge connecting an item in X to a buyer not in X.

The only item that became more attractive was item i, so for a matching edge to leave the
preferred-seller graph, it must be that some buyer k other than j, who used to be matched to
an item h, drops its edge to h because it now strictly prefers i. This situation is pictured in
Figure 15.14. Now, since is price was only reduced by 1, and since all prices and valuations
are whole numbers, if k now strictly prefers i to h after the price reduction, it must have
formerly viewed them as tied. But this means that before the reduction in is price, k had a
preferred-seller edge to i. Since k was matched to h, this k-i edge would be a non-matching
edge in the preferred-seller graph, which is not possible since is only edge in the preferred-
seller graph was its matching edge to j. This completes the chain of conclusions we need:
no matching edge can leave the preferred-seller graph when is price is reduced by 1, so the
reduced prices are still market-clearing, and this contradicts the assumption that we had
474 CHAPTER 15. SPONSORED SEARCH MARKETS

X If n was matched to e but now strictly

prefers f, then:

(1) n must have had a

non-matching edge to f,
(2) f must be in X, and
(3) e must not be in X.

In this case, either the f-n edge or the

e-n edge causes a contradiction.

......

e n

Figure 15.16: We can reduce the prices of all items in X by 1 and still retain the market-
clearing property: as we argue in the text, the only way this can fail is if some matching
edge connects a buyer in X to an item not in X, or some non-matching edge connects an
item in X to a buyer not in X. Either of these possibilities would contradict the facts in
Figure 15.15.

minimum market-clearing prices.

A Proof of Fact 1. The argument above is the key to proving Fact 1; for the complete
proof, we need to move from simply showing the existence of a non-matching edge out of i
to a full alternating path, beginning with such an edge, all the way to a zero-priced item.
To do this, we start at the item i, and we consider the set X of all nodes in the bipartite
graph (both items and buyers) that can be reached from i, using an alternating path that
begins with a non-matching edge. Here are two simple observations about the set X.

(a) For any buyer k who is in X, the item h to which she is matched is also in X. Fig-
ure 15.15 helps make clear why this must be true. The alternating path that reached
k from i must have ended on a non-matching edge, so by adding the matching edge to
h to the end of this path, we see that h must also be in X.
15.9. ADVANCED MATERIAL: VCG PRICES AND THE MARKET-CLEARING PROPERTY475

(b) For any item h that is in X, and any buyer m connected to h by a non-matching edge
in the preferred-seller graph, the buyer m must also be in X. This is a direct companion
to the previous fact, and also illustrated by Figure15.15: the alternating path that
reached h from i must have ended on a matching edge, so by adding the non-matching
edge to m to the end of this path, we see that m must also be in X.

If this set X contains an item of price 0, we are done: we have the path we want. If
this set X doesnt contains an item of price 0, then we complete the proof using the same
price-reduction idea we saw earlier, in our warm-up to the proof of Fact 1: in this case, we
will reduce the price of each item in X by 1, show that the resulting prices are still market-
clearing, and thereby contradict our assumption that we had the minimum market-clearing
prices. It will follow that X must contain a zero-priced item.
Here is the main thing we need to show.

Suppose we reduce the price of each item in X by 1. Then all matching edges
that were in the preferred-seller graph before the price reduction remain in the
preferred-seller graph after the price reduction.

The argument is essentially the same as the one we used earlier, when we were reducing the
price of just item i. We ask: how could a matching edge leave the preferred-seller graph after
the reduction? Figure 15.16 shows what must happen for this to be possible: a buyer n was
formerly matched to an item e, and now some other item f has strictly higher payoff after
the price reduction. Since all valuations, prices, and payoffs are whole numbers, and no price
changed by more than 1, it must be that e and f used to be tied for the highest payoff to n
(so n had edges to both of them in the preferred-seller graph before the reduction) and f is
in the set X while e is not (so f had its price reduced while es price remained the same).
Now we get a contradiction to one of our basic observations (a) and (b) about the set X:
Since n was matched to e, and e is not in X, observation (a) says that n must not be in X;
but since n was not matched to f , and f is in X, observation (b) says that n must be in X.
This contradiction n must both be in X and not be in X shows that no matching edge
can leave the preferred-seller graph after the price reduction. And this in turn establishes
that the reduced prices are still market-clearing after the price-reduction, contradicting our
assumption that they were the minimum market-clearing prices.
This concludes the proof, and if we look back at how it worked, we can see that it bears
out our intuition for how the non-matching edges serve to anchor all the items via alternating
paths to the items of price 0. Specifically, if this anchoring did not happen, then there would
be a set X that was floating free of any connections to zero-priced items, and in this case
the prices of all items in X could be pushed further downward. This cant happen if the
market-clearing prices are already as low as possible.
476 CHAPTER 15. SPONSORED SEARCH MARKETS

When j is matched to i in the original

market, first find a path to a zero-
priced item i*.

i j

h k

i* m

Figure 15.17: The first step in analyzing the market with j zeroed out: find an alternating
path from item i to which buyer j was matched in the original market to a zero-priced
item i .

A Proof of Fact 2. To prove Fact 2, we start with a matching market with minimum
market-clearing prices p, and we consider the preferred-seller graph for these prices. Now,
suppose that we zero out a buyer j, but keep the prices the same. The resulting preferred-
seller graph is now different, but wed like to show that it still contains a perfect matching.
How does the preferred-seller graph change when we zero out j, keeping the prices fixed?
For buyers other than j, their edges remain the same, since they have the same valuations
and observe the same prices. For j, on the other hand, the zero-priced items are now the
only items that give her a non-negative payoff, so her edges in the preferred-seller graph now
go to precisely this set of zero-priced items. Notice, for example, that this is what happens
to the preferred-seller graph as we move from Figure 15.11 to Figure 15.13: the zeroed-out
buyer x has its preferred-seller edge shift from item b to the zero-priced item d.
Because we know that the preferred-seller graph in the original market has the structure
guaranteed by Fact 1, we can view this change to the preferred-seller graph in the way
suggested by Figures 15.17 and 15.18. Before zeroing out j, when it is matched to some item
i, there is an alternating path in the preferred-seller graph, beginning with a non-matching
edge, from i to a zero-priced item i . After zeroing out j, there is a preferred-seller edge
15.9. ADVANCED MATERIAL: VCG PRICES AND THE MARKET-CLEARING PROPERTY477

In the zeroed-out market, j loses its

preferred-seller edge to i, but acquires
a preferred-seller edge to i*.

i j

h k

i* m

Figure 15.18: The second step in analyzing the market with j zeroed out: build the new
preferred-seller graph by rewiring js preferred-seller edges to point to the zero-priced items.

from j directly to i (and to any other zero-priced items as well).

It is easy to see from this pair of pictures how to find a perfect matching in the preferred-
seller graph after this change to its structure. This is shown in Figure 15.19: for each buyer
other than j who is involved in the alternating path from i to i , we simply shift her edge
upward along the alternating path. This makes room for j to match with i , restoring the
perfect matching.
Since the preferred-seller graph has a perfect matching, this establishes that the prices
are still market-clearing for the zeroed-out market. We can also establish parts (ii) and (iii)
of Fact 2 directly from our construction. Part (ii) follows simply from the fact that j only
has edges to zero-priced items in the preferred-seller graph. For part (iii), note first of all
that it is a statement about the payoffs that buyers receive. Even when there are potentially
multiple perfect matchings in a preferred-seller graph, any given buyer obtains the same
payoff in every one of these perfect matchings, since all of her edges in the preferred-seller
graph yield the same, maximum payoff. As a result, it is enough to establish part (iii) for
the perfect matching we just constructed, and it will then apply to the payoff properties
of every perfect matching in the preferred-seller graph. So consider the matching we just
478 CHAPTER 15. SPONSORED SEARCH MARKETS

We can still find a perfect matching in

this new preferred-seller graph. This
means that the same prices are also
market-clearing for the zeroed-out
market.

i j

h k

i* m

Figure 15.19: The third and final step in analyzing the market with j zeroed out: observe
that the rewired preferred-seller graph still contains a perfect matching, in which j is now
paired with i .

constructed, and let k be any buyer other than j. Either k gets the same item she had in
the perfect matching for the original market, in which case she gets the same payoff or
else k shifts from one item to another along the alternating path. In this latter case, since
k had edges to both of these items in the preferred-seller graph of the original market, she
receives the same payoff from each of them, and so again ks payoff remains the same. This
completes the proof of Fact 2, and hence fills in the final details needed to complete the
proof of the overall claim.

15.10 Exercises
1. Suppose a search engine has two ad slots that it can sell. Slot a has a clickthrough
rate of 10 and slot b has a clickthrough rate of 5. There are three advertisers who are
interested in these slots. Advertiser x values clicks at 3 per click, advertiser y values
clicks at 2 per click, and advertiser z values clicks at 1 per click.
Compute the socially optimal allocation and the VCG prices for it. Give a brief
15.10. EXERCISES 479

explanation for your answer.

2. Suppose a search engine has three ad slots that it can sell. Slot a has a clickthrough
rate of 6, slot b has a clickthrough rate of 5 and slot c has a clickthrough rate of 1.
There are three advertisers who are interested in these slots. Advertiser x values clicks
at 4 per click, advertiser y values clicks at 2 per click, and advertiser z values clicks at
1 per click. Compute the socially optimal allocation and the VCG prices for it. Give
a brief explanation for your answer.

3. Suppose a search engine has three ad slots that it can sell. Slot a has a clickthrough
rate of 5, slot b has a clickthrough rate of 2, and slot c has a clickthrough rate of 1.
There are three advertisers who are interested in these slots. Advertiser x values clicks
at 3 per click, advertiser y values clicks at 2 per click, and advertiser z values clicks at
1 per click.
Compute the socially optimal allocation and the VCG prices for it. Give a brief
explanation for your answer.

4. Suppose a search engine has two ad slots that it can sell. Slot a has a clickthrough
rate of 4 and slot b has a clickthrough rate of 3. There are three advertisers who are
interested in these slots. Advertiser x values clicks at 4 per click, advertiser y values
clicks at 3 per click, and advertiser z values clicks at 1 per click.

(a) Suppose that the search engine runs the VCG Procedure to allocate slots. What
assignment of slots will occur and what prices will the advertisers pay? Give an expla-
nation for your answer.

(b) Now the search engine is considering the creation of a third ad slot which will have
a clickthrough rate of 2. Lets call this new ad slot c. Suppose that search engine does
create this slot and again uses the VCG Procedure to allocate slots. What assignment
of slots will occur and what prices will the advertisers pay? Give an explanation for
your answer.

(c) What revenue will the search engine receive from the VCG Procedure in parts (a)
and (b)? If you were running the search engine, given this set of advertisers and slots,
and could choose whether to create slot c or not, what would you do? Why? (In
answering this question assume that you have to use the VCG Procedure to allocate
any slots you create.)

5. Suppose a search engine has two ad slots that it can sell. Slot a has a clickthrough
rate of 12 and slot b has a clickthrough rate of 5. There are two advertisers who are
480 CHAPTER 15. SPONSORED SEARCH MARKETS

interested in these slots. Advertiser x values clicks at 5 per click and advertiser y values
clicks at 4 per click.
(a) Compute the socially optimal allocation and the VCG prices for it.
(b) Suppose the search engine decides not to sell slot b. Instead, it sells only slot a
using a sealed-bid, second-price auction. What bids will the advertisers submit for slot
a, who will win, and what price will they pay?
(c) Which of these two possible procedures (a) and (b) generate the greater revenue
for the search engine? By how much?
(d) Now lets see if the result in part (c) is general or not. That is, does it depend
on the clickthrough rates and values? Let there be two slots and two advertisers; let
the clickthrough rates be ra for slot a and rb for slot b, with ra > rb > 0; and let the
advertisers values be vx and vy , with vx > vy > 0. Can you determine which of the
two procedures generates the greater revenue for the search engine? Explain.

6. Chapter 15 discusses the relationship between the VCG Principle and second price
auctions. In particular, we saw that the VCG Principle is a generalization of the idea
behind second price auctions to a setting in which there is more than one object being
sold. In this problem we will explore this relationship in an example. Suppose that a
seller has one item, which well call item x. There are three buyers, whom well call a,
b, and c. The values that these buyers (a, b, and c) have for the item are 6, 3, and 1,
respectively.

(a) Suppose that the seller runs a second price auction for the item. Which buyer will
win the auction and how much will this buyer pay?

(b) Now lets suppose that the seller uses the VCG procedure to allocate the item.
Remember that the first step in the running the VCG procedure when there are more
buyers than items is to create fictional items, which each buyer values at 0, so that the
number of items to be allocated is the same as the number of bidders. Lets call these
additional (fictional) items y and z. Find the allocation that results from running the
VCG procedure. What are the prices charged to each buyer for the item that they
receive? Explain why the price that buyer a pays is the harm that he causes to the
remaining bidders by taking the item he is assigned.
16

Haemophilus influenzae
type b (Hib)
H. INFLUENZAE meningitis notifiable (except in scotland)

the disease
Haemophilus influenzae can cause serious invasive disease, especially in
young children. Invasive disease is usually caused by encapsulated strains of
the organism. Six typeable capsular serotypes (af) are known to cause
disease; non-typeable encapsulated strains can occasionally cause invasive
disease. Before the introduction of vaccination, type b (Hib) was the prevalent
strain. The proportion of typeable to non-typeable strains depends largely on
the prevalence of the type b strain. Non-encapsulated strains are mainly
associated with respiratory infections such as exacerbation of chronic bronchitis

influenzae type b (Hib)

and otitis media.

Haemophilus

march 2011
The most common presentation of invasive Hib disease is meningitis, frequently
accompanied by bacteraemia. This presentation accounts for approximately
60% of all cases (Anderson et al., 1995). Fifteen per cent of cases present with
epiglottitis, a potentially dangerous condition that presents with airway
obstruction. Bacteraemia, without any other concomitant infection, occurs in
10% of cases. The remainder is made up of cases of septic arthritis,
osteomyelitis, cellulitis, pneumonia and pericarditis. The sequelae following
Hib meningitis may include deafness, seizures, and intellectual impairment. In
studies conducted in Wales and Oxford, 8 to 11% had permanent neurological
sequelae (Howard et al., 1991; Tudor-Williams et al., 1989). The case fatality
rate from Hib meningitis is 45%.
Individuals can carry Hib bacteria in their nose and throat without showing
signs of the disease. Before Hib vaccine was introduced, about four in every
100 pre-school children carried the Hib organism; after the vaccine was
introduced, carriage rates fell below the level of detection (McVernon et al.,
2004). Hib is spread through coughing, sneezing or close contact with a
carrier or an infected person.

127

 Haemophilus influenzae type b (Hib)

History and epidemiology of the disease

Before the introduction of Hib immunisation, the estimated annual incidence of
invasive Hib disease was 34 per 100,000 children under five years of
age. One in every 600 children developed some form of invasive Hib disease before
their fifth birthday (Booy et al., 1994). The disease was rare in children under three
months of age, but the incidence rose progressively during the
first year, reaching a peak between 10 and 11 months of age. Thereafter, the
incidence declined steadily to four years of age after which infection was uncommon.
Vaccines against Hib were first produced in the early 1970s and they contained
purified capsular polysaccharide. These vaccines were effective in children
over 18 months of age, but failed to protect younger children, in whom the risk
of disease was highest. The development of conjugate Hib vaccines overcame
this problem. In conjugate vaccines, the capsular polysaccharides were linked
to proteins, improving the vaccines immunogenicity, particularly in children
less than one year of age. In 1992, Hib conjugate vaccine was introduced into
the routine UK immunisation schedule. Hib conjugate vaccine was originally
administered as a separate vaccine. In 1996, combination vaccines (DTwP/
Hib) were introduced, and in 2004, Hib vaccine combined with DTaP and IPV
(DTaP/IPV/Hib) became available.
influenzae type b (Hib)

The efficacy and safety of the conjugate Hib vaccines have been demonstrated
Haemophilus

in large field trials in Finland, the United States and in the UK, where
march 2011

efficacy ranged from 83 to 100% (Black et al., 1991a; Black et al., 1991b;
Eskola et al., 1990). Studies comparing different vaccines, using the present
UK primary schedule, have shown that 90 to 99% of children developed protective
levels of antibodies following three doses of vaccine (Booy et al., 1994). Cases
of invasive disease in fully vaccinated children (vaccine failures) have been
reported from some countries, including the UK (Heath and McVernon,
2002). A small proportion of such cases have underlying conditions, such as
immunoglobulin deficiency, predisposing the child to vaccine failure.
Since the introduction of Hib immunisation in the UK, disease incidence has
fallen (see Figure 16.1). In 1998, only 21 cases of invasive Hib were reported
in England and Wales in children under five years of age (0.65 per 100,000)
compared with 803 in 1991 (20.5 per 100,000). In infants under one year of
age, the highest risk age group for disease, reported cases fell by over 95%
(from 300 to 7). Notifications of H. influenzae meningitis for the same period
declined from 485 to 29. In 1998, coverage by the second birthday was 95%.

128

 Haemophilus influenzae type b (Hib)

1,000
Hib vaccine introduced
900

800

700 Under 5 years of age

All ages (including under 5 years)
600

500 Hib catch-up

400

300

200

100

0
1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005
Year

Figure 16.1 Laboratory reports of Hib disease in England and Wales (19902005)

From 1999, there was a small but gradual increase in the number of cases of

influenzae type b (Hib)

Hib disease reported, mostly in children less than four years of age. However,
this increase was most notable among children born in 2000 and 2001

Haemophilus

march 2011
(McVernon et al., 2003). Reasons for this increase in vaccine failures are
thought to include an effect of the DTaP/Hib combination vaccine which was
in use at that time and a waning of the impact of the catch-up programme
when the vaccine was introduced. In this latter group, who were immunised
at an older age, the efficacy was higher than in children vaccinated routinely
as infants.
In 2003, a booster campaign was implemented with call-back of children aged
six months to four years (Chief Medical Officer et al., 2004). Following the
campaign, cases have begun to return to the low levels achieved previously (see
Figure 16.1). In 2006, following studies that showed that protection against
Hib waned during the second year of life (Trotter et al., 2003), a booster dose
(combined with MenC as Hib/MenC) was introduced.

the Hib vaccination

Hib-containing vaccines are made from capsular polysaccharide that has been
extracted from cultures of Hib bacteria. The polysaccharide is linked
(conjugated) to a protein, according to the manufacturers methodology. In the
UK, Hib vaccines have been used that have been conjugated with either
129

 Haemophilus influenzae type b (Hib)

CRM197 (a non-toxic variant of diphtheria toxin) or tetanus toxoid. The

conjugation increases the immunogenicity, especially in young children, in
whom the plain polysaccharide vaccines are not immunogenic.
Some DTaP/Hib combination vaccines have been shown to attenuate the Hib
response in comparison with DTwP/Hib combinations (Trotter et al., 2003).
The Hib-containing vaccine (Pediacel) chosen for primary immunisation in the
UK programme has been shown not to have this problem (Miller et al., 2003).
The Hib vaccine is given as part of a combined product:
diphtheria/tetanus/acellular pertussis/inactivated polio vaccine/
H. influenzae type b (DTaP/IPV/Hib) vaccine, or
Hib/MenC conjugate.
The Hib/MenC conjugate vaccine is made from capsular polysaccharides of
H. influenzae type b and group C Neisseria meningitidis, which are conjugated
to tetanus toxoid. The vaccine has been shown to elicit booster responses to
both Hib and MenC when given in the second year of life to
children who were primed in infancy with Hib and MenC conjugate vaccines.
The above vaccines are thiomersal-free. They are inactivated, do not contain
live organisms and cannot cause the diseases against which they protect.
influenzae type b (Hib)

storage
Haemophilus

march 2011

Vaccines should be stored in the original packaging at +2C to +8C and

protected from light. All vaccines are sensitive to some extent to heat and
cold. Heat speeds up the decline in potency of most vaccines, thus reducing
their shelf life. Effectiveness cannot be guaranteed for vaccines unless they
have been stored at the correct temperature. Freezing may cause increased
reactogenicity and loss of potency for some vaccines. It can also cause hairline
cracks in the container, leading to contamination of the contents.

presentation
Hib vaccines are available as part of combined products DTaP/IPV/Hib or
Hib/MenC. The combined product, DTaP/IPV/Hib is supplied as a cloudy
white suspension either in a single dose ampoule or pre-filled syringe. The
suspension may sediment during storage and should be shaken to distribute
the suspension uniformly before administration.
Hib/MenC is supplied as a vial of white powder and 0.5ml of solvent in a
pre-filled syringe. The vaccine must be reconstituted by adding the entire
contents of the pre-filled syringe to the vial containing the powder. After

130

 Haemophilus influenzae type b (Hib)

addition of the solvent, the mixture should be shaken well until the powder is
completely dissolved. After reconstitution, the vaccine should be administered
promptly, or allowed to stand between +2C and +8C and used within 24 hours.

dosage and schedule

For children under one year of age:
First dose of 0.5ml of a Hib-containing vaccine.
Second dose of 0.5ml, one month after the first dose.
Third dose of 0.5ml, one month after the second dose.
A fourth booster dose of 0.5ml of a Hib-containing vaccine should be
given at the recommended interval (see below).
For children over one year of age and under ten years of age who have either
not been immunised or not completed a primary course of diphtheria, tetanus,
pertussis or polio, DTaP/IPV/Hib vaccination should be used. Children over one
year and under ten years of age who have completed a primary course of
diphtheria, tetanus, pertussis or polio should have Hib/MenC.

administration
Vaccines are routinely given intramuscularly into the upper arm or anterolateral

influenzae type b (Hib)

thigh. This is to reduce the risk of localised reactions, which are more
common when vaccines are given subcutaneously (Mark et al., 1999; Diggle

Haemophilus

march 2011
and Deeks, 2000; Zuckerman, 2000). However, for individuals with a
bleeding disorder, vaccines should be given by deep subcutaneous injection to
reduce the risk of bleeding.
Hib-containing vaccines can be given at the same time as other vaccines such
as MMR, MenC, hepatitis B, and pneumococcal. The vaccines should be
given at a separate site, preferably in a different limb. If given in the same
limb, they should be given at least 2.5cm apart (American Academy of
Pediatrics, 2003). The site at which each vaccine was given should be noted
in the patients records.

disposal
Equipment used for vaccination, including used vials or ampoules, should be
disposed of at the end of a session by sealing in a proper, puncture-resistant
sharps box (UN-approved, BS 7320).

131

 Haemophilus influenzae type b (Hib)

Recommendations for the use of the vaccine

The objective of the immunisation programme is to protect individuals under
ten years of age, and individuals older than this who may be at elevated risk
from invasive Hib disease.
To fulfil this objective, the appropriate vaccine for each age group is determined
also by the need to protect individuals against diphtheria, tetanus, pertussis, Hib
and polio.

primary immunisation
infants and children under ten years of age
The primary course of Hib vaccination in infants consists of three doses of a
Hib-containing product with an interval of one month between each dose.
DTaP/IPV/Hib is recommended for all children from two months up to ten
years of age. Although one dose of Hib vaccine is effective from one year of
age, three doses of DTaP/IPV/Hib should be given to children who have either
not been immunised or who have not completed a primary course, in order to
be fully protected against diphtheria, tetanus, pertussis and polio. If the
primary course is interrupted it should be resumed but not repeated, allowing
an interval of one month between the remaining doses.
influenzae type b (Hib)

Children of one to ten years of age who have completed a primary course of
Haemophilus

march 2011

diphtheria, tetanus, pertussis and polio but have not received Hib-containing
vaccines, should receive a single dose of Hib/MenC vaccine.

Reinforcing immunisation
A reinforcing (booster) dose of Hib/MenC is recommended at 12 months for
children who have received a complete primary course of three Hib-containing
vaccine injections. The Hib/MenC vaccine can be given at the same time as the
pneumococcal conjugate and MMR vaccines.

Vaccination of children with unknown or incomplete

immunisation status
Where a child born in the UK presents with an inadequate immunisation
history, every effort should be made to clarify what immunisations they may have
had (see Chapter 11, on immunisation schedule). A child who has not completed
the primary course should have the outstanding doses at monthly intervals.

132

 Haemophilus influenzae type b (Hib)

Children coming to the UK who have a history of completing immunisation

in their country of origin may not have been offered protection against all the
antigens currently used in the UK. They may not have received Hib-containing
vaccines in their country of origin (www-nt.who.int/immunization_monitor
ing/en/globalsummary/countryprofileselect.cfm).
Children coming from developing countries, from areas of conflict, or from
hard-to-reach population groups may not have been fully immunised. Where
there is no reliable history of previous immunisation, it should be assumed
that they are unimmunised and the full UK recommendations should be
followed (see Chapter 11).

children and adults with asplenia, splenic dysfunction

or complement deficiency
Children and adults with asplenia or splenic dysfunction may be at increased
risk of invasive Hib infection. Children and adults with early complement
deficiency (e.g. C1, 2, 3 or 4 deficiencies) may also be at increased risk of
invasive Hib infection (Figueroa et al., 1991).
Given the increased risk, additional vaccinations against Hib disease are
advised for individuals who develop asplenia or splenic dysfunction or when

influenzae type b (Hib)

complement deficiency is diagnosed depending on age and vaccination

Haemophilus
history. For the full list of immunisations for these groups, see Table 7.1 in

march 2011
chapter 7.

children under two years of age

These individuals should be vaccinated according to the UK routine childhood
schedule, which includes a booster of Hib/MenC and PCV given at 12 months
of age. A dose of MenACWY conjugate vaccine should be given at least one
month after the Hib/MenC and PCV boosters.
After the second birthday, an additional dose of Hib/MenC should be given.
If the individual received their routine pneumococcal booster dose as PCV7
(before April 2010) an additional dose of PCV13 should be offered at the same
time, followed by a dose of PPV two months later. If the child was routinely
boosted with PCV13 (after April 2010) a dose of PPV should be given with
the Hib/MenC booster.

fully vaccinated individuals over two and under five

years of age
These individuals should receive one additional dose of Hib/MenC and
PCV13 (as they will have received PCV7). One month after this, they

133

 Haemophilus influenzae type b (Hib)

should receive a dose of MenACWY conjugate vaccine. PPV should be

given at least two months after the last dose of PCV13.

previously unvaccinated individuals over two and

under five years of age
These individuals should receive one additional dose of Hib/MenC and
PCV13 (as they will have received PCV7). One month after this, they should
receive a dose of MenACWY conjugate vaccine. PPV should be given at least
two months after the last dose of PCV13.

individuals over five years of age regardless of

vaccination status
These individuals should receive one dose of Hib/MenC vaccine with a dose
of PPV. One month after this, a dose of MenACWY conjugate vaccine should
be given.

contraindications
There are very few individuals who cannot receive Hib-containing vaccines.
Where there is doubt, appropriate advice should be sought from a consultant
paediatrician, immunisation co-ordinator or consultant in communicable
influenzae type b (Hib)

disease control rather than withhold vaccine.

Haemophilus

march 2011

The vaccines should not be given to those who have had:

a confirmed anaphylactic reaction to a previous dose of a
Hib-containing vaccine, or
a confirmed anaphylactic reaction to any components of the vaccine.

Confirmed anaphylaxis occurs extremely rarely. Data from the UK, Canada
and the US point to rates of 0.65 to 3 anaphylaxis events per million doses of
vaccine given (Bohlke et al., 2003; Canadian Medical Association, 2002).
Other allergic conditions may occur more commonly and are not
contraindications to further immunisation. A careful history of the event will
often distinguish between anaphylaxis and other events that are either not due
to the vaccine or are not life-threatening. In the latter circumstance, it may be
possible to continue the immunisation course. Specialist advice must be
sought on the vaccines and circumstances in which they could be given. The
risk to the individual of not being immunised must be taken into account.

134

 Haemophilus influenzae type b (Hib)

precautions
Minor illnesses without fever or systemic upset are not valid reasons to
postpone immunisation. If an individual is acutely unwell, immunisation may
be postponed until they have recovered. This is to avoid confusing the
differential diagnosis of any acute illness by wrongly attributing any signs or
symptoms to the adverse effects of the vaccine.

systemic and local reactions following a previous

immunisation
This section gives advice on the immunisation of children with a history of a
severe or mild systemic or local reaction within 72 hours of a preceding
vaccine. Immunisation with Hib-containing vaccine should continue following
a history of:
fever, irrespective of its severity
hypotonic-hyporesponsive episodes (HHE)
persistent crying or screaming for more than three hours, or
severe local reaction, irrespective of extent.

In Canada, a severe general or local reaction to DTaP/IPV/Hib is not a

contraindication to further doses of the vaccine (Canadian Medical Association,

influenzae type b (Hib)

1998). Adverse events after childhood immunisation are carefully monitored in

Haemophilus
Canada (Le Saux et al., 2003) and their experience suggests that further doses

march 2011
were not associated with recurrence or worsening of the preceding events
(S Halperin and R Pless, pers. comm., 2003).

pregnancy and breast-feeding

Hib-containing vaccines may be given to pregnant women when protection is
required without delay. There is no evidence of risk from vaccinating pregnant
women or those who are breast-feeding with inactivated viral or bacterial
vaccines or toxoids (Plotkin and Orenstein, 2004).

premature infants
It is important that premature infants have their immunisations at the
appropriate chronological age, according to the schedule. The occurrence of
apnoea following vaccination is especially increased in infants who were born
very prematurely.
Very premature infants (born 28 weeks of gestation) who are in hospital
should have respiratory monitoring for 48-72 hrs when given their first
immunisation, particularly those with a previous history of respiratory
immaturity. If the child has apnoea, bradycardia or desaturations after the first

135

 Haemophilus influenzae type b (Hib)

immunisation, the second immunisation should also be given in hospital, with

respiratory monitoring for 48-72 hrs (Pfister et al., 2004; Ohlsson et al., 2004;
Schulzke et al., 2005; Pourcyrous et al., 2007; Klein et al., 2008).
As the benefit of vaccination is high in this group of infants, vaccination
should not be withheld or delayed.

immunosuppression and HiV infection

Individuals with immunosuppression and HIV infection (regardless of CD4
count) should be given Hib-containing vaccines in accordance with the
recommendations above. These individuals may not make a full antibody
response. Re-immunisation should be considered after treatment is finished and
recovery has occurred. Specialist advice may be required.
Further guidance is provided by the Royal College of Paediatrics and
Child Health (RCPCH) (www.rcpch.ac.uk), the British HIV Association
(BHIVA) Immunisation guidelines for HIV-infected adults (BHIVA, 2006)
and the Childrens HIV Association of UK and Ireland (CHIVA) immunisation
guidelines (www.bhiva.org/chiva).

neurological conditions
influenzae type b (Hib)

pre-existing neurological conditions

Haemophilus

The presence of a neurological condition is not a contraindication to

march 2011

immunisation. Where there is evidence of a neurological condition in a child,

the advice given in the flow chart in Figure 16.2 should be followed.
If a child has a stable pre-existing neurological abnormality such as spina
bifida, congenital abnormality of the brain or perinatal hypoxic-ischaemic
encephalopathy, they should be immunised according to the recommended
schedule. When there has been a documented history of cerebral damage in
the neonatal period, immunisation should be carried out unless there is
evidence of an evolving neurological abnormality.
If there is evidence of current neurological deterioration, including poorly
controlled epilepsy, immunisation should be deferred and the child should be
referred to a child specialist for investigation to see if an underlying cause can
be identified. If a cause is not identified, immunisation should be deferred
until the condition has stabilised. If a cause is identified, immunisation should
proceed as normal.
A family history of seizures is not a contraindication to immunisation. When
there is a personal or family history of febrile seizures, there is an increased
risk of these occurring after any fever, including that caused by immunisation.

136

 Haemophilus influenzae type b (Hib)

Evidence of a neurological
abnormality prior to immunisation

Is the condition stable?

Yes No

Defer immunisation.
Immunise as normal Consider referral to
paediatrician or paediatric
neurologist

Is there an identifiable cause?

influenzae type b (Hib)

Yes No

Haemophilus

march 2011
Defer and immunise
Immunise as normal once the condition has
stabilised

Figure 16.2 Flow chart for immunisation procedure if there is evidence of a

neurological condition before immunisation

Seizures associated with fever are rare in the first six months of life and most
common in the second year of life. After this age, the frequency falls and they
are rare after five years of age.
When a child has had a seizure associated with fever in the past, with no
evidence of neurological deterioration, immunisation should proceed as
recommended. Advice on the prevention and management of fever should be
given before immunisation.

137

 Haemophilus influenzae type b (Hib)

When a child has had a seizure that is not associated with fever, and there is
no evidence of neurological deterioration, immunisation should proceed as
recommended. When immunised with DTP vaccine, children with a family
or personal history of seizures had no significant adverse events and their
developmental progress was normal (Ramsay et al., 1994).
neurological abnormalities following immunisation
If a child experiences encephalopathy or encephalitis within seven days of
immunisation, the advice in the flow chart in Figure 16.3 should be followed.
It is unlikely that these conditions will have been caused by the vaccine and
should be investigated by a specialist. Immunisation should be deferred in
children where no underlying cause is found and the child does not recover
completely within seven days, until the condition has stabilised. If a cause is
identified or the child recovers within seven days, immunisation should
proceed as recommended.
If a seizure associated with a fever occurs within 72 hours of an immunisation,
further immunisation should be deferred until the condition is stable if no
underlying cause has been found and the child does not recover completely
within 24 hours. If a cause is identified or the child recovers within 24 hours,
immunisation should continue as recommended.
influenzae type b (Hib)

deferral of immunisation
Haemophilus

march 2011

There will be very few occasions when deferral of immunisation is required

(see above). Deferral leaves the child unprotected; the period of deferral
should be minimised so that immunisation can commence as soon as possible.
If a specialist recommends deferral, this should be clearly communicated to
the general practitioner and he or she must be informed as soon as the child is
fit for immunisation.

adverse reactions
Pain, swelling or redness at the injection site are common and may occur more
frequently following subsequent doses. A small, painless nodule may form at
the injection site; this usually disappears and is of no consequence. The incidence
of local reactions is lower with tetanus vaccines combined with acellular
pertussis vaccines than with whole-cell pertussis vaccines, and similar to that
after diphtheria (DT) vaccine (Miller, 1999; Tozzi and Olin, 1997).
Fever, convulsions, high-pitched screaming, and episodes of pallor, cyanosis
and limpness (HHE) occur with equal frequency after both DTaP and DT
vaccines (Tozzi and Olin, 1997).

138

 Haemophilus influenzae type b (Hib)

Refer to a paediatrician or paediatric

neurologist for investigation

Is there an identifiable cause

(e.g. a viral infection)?

Yes No

Immunise as normal Did the child recover

when stable completely within
seven days?

Yes No

influenzae type b (Hib)

Defer further
Immunise as normal
immunisations and

Haemophilus
fully investigate

march 2011
Immunise once the
condition
has stabilised

Figure 16.3 Flow chart for encephalitis or encephalopathy occurring within

seven days of immunisation

Confirmed anaphylaxis occurs extremely rarely. Data from the UK, Canada
and the US point to rates of 0.65 to 3 anaphylaxis events per million doses of
vaccine given (Bohlke et al., 2003; Canadian Medical Association, 2002).
Other allergic conditions may occur more commonly and are not
contraindications to further immunisation.
Hib/menc conjugate vaccine
Mild side effects such as irritability, loss of appetite, pain, swelling, redness
at the site of the injection and slightly raised temperature commonly occur.

139

 Haemophilus influenzae type b (Hib)

Less commonly crying, diarrhoea, vomiting, atopic dermatitis, malaise and

fever over 39.5C have been reported.

All suspected adverse reactions to vaccines occurring in children, or in

individuals of any age after vaccines labelled with a black triangle (),
should be reported to the Commission on Human Medicines using the
Yellow Card scheme. Serious suspected adverse reactions to vaccines in
adults should be reported through the Yellow Card scheme.

management of cases and contacts

Unimmunised cases up to the age of ten years should be immunised according
to their age-appropriate schedule after recovery from infection. Previously
vaccinated cases should have their convalescent antibody levels measured, and
booster vaccination may be advised. Where antibody testing is not possible, an
additional dose of Hib-containing vaccine should be given after recovery from
infection.

Household contacts of a case of invasive Hib disease have an increased risk of

contracting the disease. Unimmunised children under ten years of age are at
substantial risk. Contacts of cases should be managed following the advice of
influenzae type b (Hib)

the local health protection unit, as follows:

Haemophilus

march 2011

children who have never received any immunisations should receive

three doses of DTaP/IPV/Hib vaccine if below ten years of age.
children who have never received Hib vaccine, but who have been
immunised against diphtheria, tetanus, pertussis and polio, should
receive three doses of Hib/MenC vaccine if under one year, and one
dose if aged between one and ten years.
children aged between one and ten years who have received Hib vaccine
in infancy, but who did not receive a booster dose of Hib containing
vaccine after the age of 12 months, should receive a single dose of Hib/
MenC vaccine.

Where there is any individual in the household of a case who is also at risk, the
index case and all household contacts should be given rifampicin prophylaxis.
Those at risk in the household include all children under ten years of age and
vulnerable individuals of any age (e.g. those who are immunosuppressed or
asplenic) regardless of their immunisation status. The purpose of this
recommendation is to prevent transmission of Hib to vulnerable individuals
within a household. Further information is available at:

140

 Haemophilus influenzae type b (Hib)

www.hpa.org.uk/Topics/InfectiousDiseases/InfectionsAZ/Haemophilus
InfluenzaeTypeB/Guidance/

When a case occurs in a playgroup, nursery, crche or school, the opportunity

should be taken to identify and vaccinate any unimmunised children under ten
years of age. When two or more cases of Hib disease have occurred in a
playgroup, nursery, crche or school within 120 days, chemoprophylaxis
should be offered to all room contacts teachers and children. This is a
precautionary measure as there is little evidence that children in such settings
are at significantly higher risk of Hib disease than the general population of
the same age.

Vaccines
Pediacel (diphtheria/tetanus/5-component acellular pertussis/inactivated
polio vaccine/H. influenzae type b (DTaP/IPV/Hib) manufactured by
Sanofi Pasteur MSD.
Menitorix (Hib/MenC) manufactured by GlaxoSmithKline.
These vaccines are supplied by Healthcare Logistics (Tel: 0870 871 1890) as
part of the national childhood immunisation programme.

influenzae type b (Hib)

In Scotland, supplies should be obtained from local childhood vaccine holding

Haemophilus
centres. Details of these are available from Scottish Healthcare Supplies

march 2011
(Tel: 0141 282 2240).

References
American Academy of Pediatrics (2003) Active immunization. In: Pickering LK (ed.) Red
Book: 2003 Report of the Committee on Infectious Diseases, 26th edition. Elk Grove
Village, IL: American Academy of Pediatrics, p 33.
Anderson EC, Begg NT, Crawshaw SC et al. (1995) Epidemiology of invasive
Haemophilus influenzae infections in England and Wales in the pre-vaccination era
(19902). Epidemiol Infect 115: 89100.
British HIV Association (2006) Immunisation guidelines for HIV-infected adults:
www.bhiva.org/pdf/2006/Immunisation506.pdf.
Black SB, Shinefield HR, Fireman B et al. (1991a) Efficacy in infancy of oligosaccharide
conjugate Haemophilus influenzae type b (HbOC) vaccine in a United States population
of 61,080 children. Pediatr Infect Dis J 10: 97104.
Black SB, Shinefield H, Lampert D et al. (1991b) Safety and immunogenicity of oligosac
charide conjugate Haemophilus influenzae type b (HbOC) vaccine in infancy. Pediatr
Infect Dis J 10: 2.

141

 Haemophilus influenzae type b (Hib)

Bohlke K, Davis RL, Marcy SH et al. (2003) Risk of anaphylaxis after vaccination of
children and adolescents. Pediatrics 112: 81520.
Booy R, Hodgson S, Carpenter L et al. (1994) Efficacy of Haemophilus influenzae type
b conjugate vaccine PRP-T. Lancet 344 (8919): 3626.
Canadian Medical Association (1998) Pertussis vaccine. In: Canadian Immunization
Guide. 5th edition. Canadian Medical Association, p 133.
Canadian Medical Association (2002) General considerations. In: Canadian Immunization
Guide, 6th edition. Canadian Medical Association, p 14.
Chief Medical Officer, Chief Nursing Officer and Chief Pharmaceutical Officer
(2004) Planned Hib vaccination catch-up campaign further information. www.dh.gov.
uk/ cmo/letters/cmo0302.htm
Department of Health (2001) Health information for overseas travel, 2nd edition. London:
The Stationery Office.
Diggle L and Deeks J (2000) Effect of needle length on incidence of local reactions
to routine immunisation in infants aged 4 months: randomised controlled trial. BMJ
321: 9313.
Eskola J, Kayhty H, Takala AK et al. (1990) A randomised, prospective field trial of a
conjugate vaccine in the protection of infants and young children against invasive
Haemophilus influenzae type b disease. NEJM 323 (20): 13817.
influenzae type b (Hib)

Heath PT and McVernon J (2002) The UK Hib vaccine experience. Arch Dis Child
86:3969.
Haemophilus

march 2011

Howard AJ, Dunkin KT, Musser JM and Palmer SR (1991) Epidemiology of Haemophilus
influenzae type b invasive disease in Wales. BMJ 303: 4415.
Klein NP, Massolo ML, Greene J et al. (2008) Risk factors for developing apnea after
immunization in the neonatal intensive care unit. Pediatrics 121(3): 463-9.
McVernon J, Andrews N, Slack MPE and Ramsay ME (2003) Risk of vaccine failure after
Haemophilus influenzae type b (Hib) combination vaccines with acellular pertussis.
Lancet 361: 15213.
McVernon J, Howard AJ, Slack MP and Ramsay ME (2004) Long-term impact of
vaccination on Haemophilus influenzae type b (Hib) carriage in the United Kingdom.
Epidemiol Infect 132 (4): 7657.
Mark A, Carlsson RM and Granstrom M (1999) Subcutaneous versus intramuscular
injection for booster DT vaccination in adolescents. Vaccine 17: 206772
Miller E (1999) Overview of recent clinical trials of acellular pertussis vaccines.
Biologicals 27: 7986.
Miller E, Southern J, Kitchin N et al. (2003) Interaction between different meningococcal
C conjugate vaccines and the Hib component of concomitantly administered diphtheria/
tetanus/pertussis/Hib vaccines with either whole-cell or acellular pertussis

142

 Haemophilus influenzae type b (Hib)

antigens. 21st Annual Meeting of the European Society for Paediatric Infectious
Diseases, Sicily.
Ohlsson A and Lacy JB (2004) Intravenous immunoglobulin for preventing infection in
preterm and/or low-birth-weight infants. Cochrane Database Syst Rev(1): CD000361.
Pfister RE, Aeschbach V, Niksic-Stuber V et al. (2004) Safety of DTaP-based combined
immunization in very-low-birth-weight premature infants: frequent but mostly benign
cardiorespiratory events. J Pediatr 145(1): 58-66.
Plotkin SA and Orenstein WA (eds) (2004) Vaccines, 4th edition. Philadelphia: WB
Saunders Company, Chapter 8.
Pourcyrous M, Korones SB, Arheart KL et al. (2007) Primary immunization of premature
infants with gestational age <35 weeks: cardiorespiratory complications and C-reactive
protein responses associated with administration of single and multiple separate vaccines
simultaneously. J Pediatr 151(2): 167-72.
Ramsay M, Begg N, Holland B and Dalphinis J (1994) Pertussis immunisation in children
with a family or personal history of convulsions: a review of children referred for specialist
advice. Health Trends 26: 234.
Le Saux N, Barrowman NJ, Moore D et al. (2003) Canadian Paediatric Society/Health
Canada Immunization Monitoring Program Active (IMPACT). Decrease in hospital
admissions for febrile seizures and reports of hypotonic-hyporesponsive episodes presenting
to hospital emergency departments since switching to acellular pertussis vaccine in
Canada: a report from IMPACT. Pediatrics 112 (5): e348.
Schulzke S, Heininger U, Lucking-Famira M et al. (2005 ) Apnoea and bradycardia in
preterm infants following immunisation with pentavalent or hexavalent vaccines. Eur J
Pediatr 164(7): 432-5.
Tozzi AE and Olin P (1997) Common side effects in the Italian and Stockholm 1 Trials.
Dev Biol Stand 89: 1058.
Trotter CL, Ramsay ME and Slack MPE (2003) Rising incidence of Haemophilus influenzae
type b disease in England and Wales indicates a need for a second catch-up vaccination
campaign. Commun Dis Public Health 6: 558.
Tudor-Williams G, Frankland J, Isaacs D et al. (1989) Haemophilus influenzae type b
disease in the Oxford region. Arch Dis Child 64: 51719.
Zuckerman JN (2000) The importance of injecting vaccines into muscle. BMJ
321: 12378.

143
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 705
NUMERICS
0day, definition, 4
0day kernel vulnerabilities, Unix,
636642
A ASLR (Address Space Layout
AAAS (ASCII Armored Address
Space), 394395
abusing frame-based exception
handlers, 161166
existing handlers, 162164
alphanumeric filters, exploits,
writing, 205209
application layer attacks, database
software, 618619
arbitrary free vulnerabilities, 271
arbitrary size overflow, stack
overflows, 232233
architectural failures
asymmetry and, 511512
authentication and, 512
authorization and, 512
boundaries and, 508509
data translation and, 509511
archives, paper, 438
Index
arrays, 12
ASCII, converting to Unicode,
210211
ASCII Venetian implementation,
214217
Randomization), 313, 396399
asm( ) statements, 132
assembler references, 430
assembly language, 6
C code constructs, 710
C++ code constructs, 710
registers and, 6
asymmetry, problems, 511512
attack detection, bypassing
alternate encodings, 514
attack signatures, 517
file-handling features, 515517
length limitations, 517520
stripping bad data, 513514
auditing binaries. See binary
auditing
auditing source code
automated analysis tools, 484
705
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 706

706 Index AC

auditing source code (continued) B

methodology binary auditing
bottom-up approach, 485 C++ code constructs, 561
selective approach, 485486 calling conventions, 554555
top-down approach, 485 C calling convention, 555
tools Stdcall calling convention, 555
Cbrowser, 484 compiler-generated code
Cscope, 482483 for loops, 557558
Ctags, 483 function layouts, 556
editors, 483 if statements, 556557
vulnerabilities switch statements, 558560
different-sized integer while loops, 557558
conversions, 497498 IDA Pro, 550552
double free, 498 manual, 563566
extinct bug classes, 487 memcpy-like code constructs, 560
format strings, 487489 source code auditing, 550
generic logic errors, 486 stack frames, 552553
incorrect bounds-checking, BP-based, 553, 554
489490 frame pointer, 553554
integer-related vulnerabilities, strlen-like code constructs, 560561
495497 this pointer, 561562
loop constructs, 490 bit flipping, 469470
multithreading and, 500501 boundaries, problems, 518509
non-null termination issues, 492 bounds-checking, 489490
off-by-one, 490492 bridge building, 206
out-of-scope memory usage, 499 BSD, OS X and, 314
signed comparison buffer overflows
vulnerabilities, 494495 exploiting, 197202
skipping null-termination issues, heap-based, 173
493 exploiting, 178194
uninitialized variable usage, stack-based, 156
499500 buffers, 1213
use after free vulnerabilities, arrays, 12
499500 length, 95
authentication, problems, 512 overflowing on stack, 1823
Authentication Tokens, 116 bug discovery, exploitation and, 6
authorization, problems, 512
automated source code analysis C
tools, 484 C, code constructs, 710
C++, code constructs, 710
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 707

Index CD 707

canaries, 166, 388389 control registers, 7

Cbrowser, 484 cookies, 403
chained ret2code, 379 countermeasures, 601602
Check Heaps, 359 brute forcing, 602603
Cisco IOS, 339340 information leaks, 605606
crash dumps, 354355 local exploits, 603
exploiting OS/application fingerprinting,
heap overflows, 359361 603605
stack overflows, 357359 preparation, 602
GDB agent, 355356 cross-platform shellcode (OS X),
hardware platforms, 340 332333
images Cscope, 482483
diffing, 350351 Ctags, 483
taking apart, 349350
partial attacks
global variable overwrite, 363364 D
NVRAM invalidation, 362 data, instructions and, 4
reverse engineering, 348356 data translation, problems, 509511
runtime analysis, 351356 database software
ROMMON, 351354 application layer attacks, 618619
shellcode, bind shell, 370372 network layer attacks, 608618
shellcodes operating system commands
configuration changing, 364370 IBM DB2, 621623
runtime image patching, 370 Oracle, 620621
software packages, 340343 SQL Server, 619620
system architecture DB (define byte) directive, 56
IO memory, 346 DCE-RPC, 116
IOS heap, 344346 DCE-RPC tools, 118
memory layout, 343344 recon, 118120
vulnerabilities, 346347 SPIKE, 118120
command-line interface, 348 DCOM (Distributed Component
protocol parsing code, 347 Object), 111
security, 347348 DCE-RPC and, 116123
services, router, 347 exploitation, 120
class definitions, reconstructing debugging
vtables, 562563 debug trick, 433434
configuration related shellcode, 599 OllyDbg, 112114
continuation of execution, 441442 unhandled exception filter and, 186
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 708

708 Index DE

debugging (continued) EIP (Extended Instruction Pointer), 7

Windows controlling, 2223
kernel debugger, 124 ellipsis syntax, 84
Microsoft tool chain, 124 encryption, end-to-end, 299
OllyDbg, 124 end-to-end encryption, 299
SetDefaultExceptionHandler, 126 exception filter, unhandled,
shellcode, writing, 125 overwrite pointer to, 185191
SoftICE, 124 exception handlers
TlsSetValue( ), 126 abusing existing, 162164
VirtualProtect( ), 126 frame-based, 156161
Win XP, 127 triggering, continuation and, 441
Win2K, 126 exception handling
Win32, 124125 signal( ) system call, 122
Win9X/ME, 126 vectored, 181
WinDbg, 124 Vectored Exception Handling, 123
Windows 2003 Server, 127 Win32 and, 122123
Windows Vista, 127 Windows, searches and, 148153
WinNT, 126 Exception Registration Record, 392
WSASocket( ), 126 EXCEPTION_REGISTRATION
decoder, Unicode, 218221 structure, 156
delay slot, SPARC, 227 execution
device drivers, 683 continuation of, 441442
I/O control code components, controlling for exploitation, 7584
693694 execve function, 54
IOCTL handlers, 694695 execve syscal, 52
directives, DB (define byte), 56 exit( ) syscall, shellcode for, 4448
dlmalloc, 89 exploitation
DOS attacks, 521 Cisco IOS
double free vulnerabilities, 270, 498 heap overflows, 359361
dynamic analysis, fuzzing and, 470 stack overflows, 357359
dynamic heaps, 173 execution, controlling, 7584
dynamic linking heaps, OS X, 333335
single stepping dynamic linker, overruns, SQL level, SQL
281296 functions, 623625
SPARC ABI, 279 Solaris/SPARC, methodology,
dynamic string table, 280 263270
stack overflows, 236241
E exploits
EAX register, 177 countermeasures, 601606
EBP register, 15 definition, 4
editors, source code, 483 one-factor, 598
EFLAGS (Extended Flags), 7 planning, 439
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 709

Index EG 709

exploits (continued) format strings, 6163, 487489

root privileges and, 2535 bugs, 6367
stabilization and, 442443 reasons for, 8485
writing exploits, 6869
alphanumeric filters, 205209 crashing services, 6970
Unicode filters, 209211 information leakage, 7075
Extended Flags (EFLAGS), 7 techniques, 8588
extended stack pointer (ESP) frame pointers, 15
register, 6 frame-based exception handlers,
extinct bug classes, 487 156161
extproc overflow, 504508 abusing, 161166
existing, 162164
F fstat (BSD), 435
fault injection, 445446 functions
design, 447456 execve, 54
fault delivery, 455 HeapAllocate( ), 173
fuzzing and, 461 KiUserExceptionDispatcher, 161
heuristics, 455456 MyExceptionHandler, 158
input generation, 447448 printf, 6364
automated, 449 RtlImageNtheader, 161
fuzz generation, 449450 stack and, 1518
live capture, 449 fuzzers, 433
manual, 448 bit flipping, 469470
modification engines, 450451 Blackhat, 480
delimiting logic, 451453 CHAM, 480
input sanitization, 453454 definition, 4
Nagel algorithm, 455 Hailstorm, 480
state-based protocols, 456 open source, modifying, 470
stateless protocols, 456 weaknesses in, 468
timing, 455456 fuzzing, 99
fault monitoring, 456457 dynamic analysis and, 470
debuggers and, 457 fault injection and, 461
FaultMon, 457458 introduction, 461465
fault testing, 446 scalability, 466467
FIFO (first in first out), 5 sharefuzz, 462
filters static analysis versus, 466
alphanumeric, exploits, writing,
205209 G
Unicode, exploits, writing, 209211 gcc (GNU Compiler Collection), 430
Windows, 129130 gdb (GNU Debugger), 430431
foo( ) function, strcpy( ) call, 176 general-purpose registers, 6
generic logic errors, 486
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 710

710 Index GI

GetSystemTimeAsFileTime, 167 HeapAllocate( ), 115

global registers, SPARC, 225 HeapCreate( ), 114
GOT (Global Offset Table), 279 initialization, 5
GPG 1.2.2 randomness patch, 583 introduction, 90
malloc( ), 91
overview, 91
H process heap, 173177
hardware, Cisco IOS, 340 realloc( ), 91
heap overflows, 91 repairing, 191193
advanced exploitation, 105107 RtlHeapAllocate( ), 115
breakpoints, 96 RtlHeapFree( ), 115
buffer length, 95 strcpy( ), 176
Cisco IOS, 359361 threading and, 115116
partial attacks, 362364 host IDS related shellcode, 599
dlmalloc, 96
example, 271276
intermediate, 98105 I
limitations, Solaris, 266267 IBM DB2, operating system
ltrace output, 93 commands, 619620
malloc, 95 IDA Pro, 550552
Microsoft IIS, 92 ideal stack layout, 389394
repairing, 179 impersonation, tokens and, 120122
samba, 92 incorrect bounds-checking, 489490
Solaris, 92 input validation, bypassing
SPARC, 296299 alternate encodings, 514
heap protections, 399407 attack signatures, evading, 517
heap-based buffer overflows, 173 file-handling features, 515517
COM objects and, 193194 length limitations, 517520
exploiting, 178194 stripping bad data, 513514
logic program control data, 194 Instruction Pointer, 7
Solaris/SPARC, 241263 instructions, data and, 4
HeapAlloc, 177 integer-related vulnerabilities,
HeapAllocate( ) function, 173 495497
HeapFree, 177 Intel shellcode (OS X), 324327
HeapRealloc, 177 ret2libc, 327329
heaps, 114115 ret2str(l)cpy, 329321
default, 115 IPS (Intrusion Prevention System),
dynamic, 173 279
exploitation, OS X, 333335
GetDeafultHeap( ), 115
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 711
K Linux
kernel, Unix
exec_ibcs2_coff_prep_zmagic( )
vulnerability, 647652
breakpoints, calculating, 652654
execution, redirecting, 654655
kernel mode payload, 658665
offsets, calculating, 652654
process descriptor, 655658
return address, overwriting,
654655
root, 665672
overflows
0day kernel vulnerabilities,
636642
vulnerability types, 627636
Solaris vfs_getvfssw( ), 642344,
672678
kernel, Windows
introduction, 682683
kernel-mode payloads, 695
rootkit, 703
security, subverting, 701703
user-mode payload, 699701
user-mode processes, 696698
mode flaws, 681682
programming flaws, 683684
heap overflows, 688
repurposing attacks, 689
shared object attacks, 689
stack overflows, 684687
user-mode addresses, insufficient
validation, 688689
KiUserExceptionDispatcher
function, 161
L Software), 504
lazy binding, 280
LIFO (last in first out), 5
linking
dynamic, 279
unsafe unlinking, 402
Index KN 711
protections
ASLR, 418419
heap, 420
stack data, 419420
W^X, 417418
Windows comparison, 111114
loop constructs, 490
ltrace (Unix), 434
M
malloc, 89
memory, Cisco IOS, 343344
memory management, modem, 4
Microsoft toolchain, debugging and,
124
modeling network protocols, 469
SPIKE, 472480
modem, memory management, 4
mount( ) system call, 645646
MyExceptionHandler function, 158
MySQL, 1-bit patch, 578580
N
NASM (Netwide Assembler), 431
NetCat, 434
Network IDS (Intrusion Detection
System), 279
network layer attacks, database
software, 608618
network related shellcode, 599
networks, protocols, modeling, 469
NEXTSTEP, OS X and, 314
NGS (Next Generation Security
non-executable stacks
exploiting, 197202
Return to libc, 3538
non-null termination issues, 492
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 712

712 Index NP

NOP method Intel shellcode, 324327

offsets and, 33 ret2libc, 327329
SPARC, 231 ret2str(l)cpy, 329321
null-termination issues, 493 NEXTSTEP and, 314
nulls, 4849 open source, 314315
NVRAM validation, 362 passwords and, 316
PowerPC shellcode, 316324
protections
O ASLR, 423
objdump utility, 45 heap, 423
off-by-one vulnerabilities, 490492 stack data, 423
offset finder, 432433 W^X, 422
offsets, NOP Method and, 33 Unix and, 315316
OllyDbg, 112114, 124, 431 out-of-scope memory usage
one-factor exploits, 598 vulnerabilities, 499
opcodes, alphanumeric bytes, 206 overflows
open source buffer
modifying, fuzzers and, 470 heap-based, 173
OS X, 314315 stack-based, 156
OpenBSD, protections .data section, 194196
ASLR, 421 heap-based, Solaris/SPARC,
heap, 422 241263
stack data, 421422 off-by-one, 270
W^X, 421 stack-based buffer overflow
OpenSSH RSA authentication patch, methodologies, 232236
580581 static data overflows, 276
operating system commands, TEB/PEB, 196197
database software
IBM DB2, 621623
Oracle, 620621 P
SQL Server, 619620 Packetstorm, 599
Oracle paper archives, 438
extproc overflow, 504508 passwords, OS X and, 316
operating system commands, PaX, 382
619620 PE-COFF (Portable Executable-
OS X, 313 Common File Format), 111
Aqua, 314 Win32 and, 112114
BSD and, 314 PIC (Position Independent Code),
bugs, 335337 133
cross-platform shellcode, 332333 planning exploits, 439
exploits, resources, 337 PLT (Procedure Linkage Table), 280
heap exploitation, 333335 pointers, frame pointers, 15
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 713

Index PS 713

popping shells, Windows, 153154 repairing, heaps, 191193

PowerPC shellcode (OS X), 316324 ret2code, 378
printf functions, 6364 ret2data, 377
privilege related shellcode, 599 ret2dl-resolve, 380381
process heap, 173177 ret2gets, 378
Proglet server, 584 ret2libc, 377
ProPolice, 390 ret2plt, 380
protections, 375376 ret2strcpy, 377378
AAAS (ASCII Armored Address ret2syscall, 379
Space), 394395 ret2text, 380
ASLR (Address Space Layout reverse engineering Cisco IOS,
Randomization), 396399 348356
heap protections, 399407 Roman Exploit Writer, 214217
implementations ROMMON, Cisco IOS, 351354
Linux, 417420 root
OpenBSD, 421422 exploits and, 2535
OS X, 422423 shell, spawning, 26
Solaris, 423424 RtlAcquirePebLock( ), 178
Windows, 413417 RtlEnterCriticalSection( ), 178
kernel protections, 411412 RtlImageNtheader function, 161
non-executable stack, 376381 RtlLeaveCriticalSection( ), 178
pointer protections, 412413 RtlReleasePebLock( ), 178
stack data runtime analysis, Cisco IOS, 351356
canaries, 388389 runtime patching, 581583
ideal stack layout, 389394
W^X memory, 381387
Windows SEH protections, 407409 S
EEREAP, 409410 scalability, fuzzing, 466467
pdest, 410411 sea monkey data, 518
SEHInspector, 411 searches, Windows exception
protocols, network, modeling, 469 handling and, 148153
Python, 432 Security Cookies, 166
generating, 167
.set statements, 132133
R SetUnhandledExceptionFilter, 185
register windows, stack overflows sharefuzz, 462
and, 233 shellcode
registers, 67 Cisco IOS
control, 7 bind shell, 370372
EBP, 15 configuration changing, 364370
ESP (extended stack pointer), 6 runtime image patching, 370
SPARC, 224227 configuration related, 600
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 714

714 Index S

shellcode (continued) overflows, heap-based, 241263

exit( ) syscall, 4448 overwrite targets, 267270
Host IDS related, 600 protections, 423
injectable, 4850 ASLR, 424
introduction, 4142 heap, 424
libraries, 441 stack data, 424
network related, 599 W^X, 424
OS X, PowerPC, 316324 self-location determination,
Packetstorm, 599 228229
privilege related, 599600 shellcode, advanced, 299311
Solaris, advanced, 299311 stack frame, 231232
SPARC static data overflows, 276
advanced, 299311 system calls, 230
exec, 229230 vfs_getvfssw( ), 642344
self-location determination, source code
228229 auditing
thread related, 600601 binary auditing, 550
Unix, pitfalls, 299 Cbrowser, 484
Windows Cscope, 482483
heapoverflow.c, 132148 Ctags, 483
PEG, 132 editors, 483
setup, 131132 methodology, 485486
Windows debugging, 125 vulnerabilities, 486501
writing in inline assembler, automated analysis tools, 484
439441 SPARC (Scalable Processor
shells Architecture), 224
spawning, 5059 ABI (Application Binary Interface),
Windows, popping, 153154 279
signal( ) system call, 122 delay slot, 227
signed comparison vulnerabilities, exploit methodology, 263270
494495 NOP instruction, 231
single stepping dynamic linker, overflows, heap-based, 241263
281296 padding instructions, 231
Snort IDS, 299 register windows, 224227
SoftICE, 124 registers, 224227
software forced crash, 359 shellcode
software packages, Cisco IOS, advanced, 299311
340343 exec, 229230
Solaris self-location determination,
exploit methodology, 263270 228229
heap overflows stack frame, 231232
limitations, 266267 stack overflow exploitation,
methodology, 263 236241
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 715

Index ST 715

SPARC (continued) POP, 14

stack-based overflow
methodologies, 232236
synthetic instructions, 228
UltraSPARC processors, 224
spawning shells, 5059
SPIKE, 471472
modeling network protocols,
472480
SQL (Structured Query Language),
504
exploiting overruns, 623625
Transact-SQL, 504
SQL Server
3-byte patch, 575578
operating system commands,
619620
SQL-UDP, 522
stack data
canaries, 388389
ideal stack layout, 389394
stack frame, Solaris/SPARC,
231232
stack overflows
arbitrary size, 232233
Cisco IOS, 357359
exploitation, 236241
register windows and, 233
stack protection, Windows 2003
Server and, 166172
stack-based buffer overflows, 156
methodologies, 232236
StackGuard, 166
stacks, 1315
boundary, 1314
buffers, overflowing, 1823
EBP register, 15
functions and, 1518
initialization, 5
non-executable
exploiting, 197202
Return to libc, 3538
PUSH, 14
static analysis versus fuzzing, 466
static data overflows, 276
strace (Unix), 434
strcpy( ), 176
symbol resolution, 280
syntax
Windows, 129130
x86 AT&T versus Intel, 130
synthetic instructions, SPARC, 228
syscall proxies, 584586
problems with, 587596
sysfs( ) system call, 644645
Sysinternals Process Explorer,
tokens and, 121
system calls, 4244
arguments, 43
execve, 52
exit( ), 43
shellcode, 4448
Solaris, 230
Windows, attacking, 692
T
tcpdump (Unix), 435
TEB (Thread Environment Block),
190
TEB/PEB overflows, 196197
termination
non-null, 492
skipping null-termination issues,
493
thread related shellcode, 599
threading, heaps, 115116
Threading and Process Architecture,
116
TNS (Transparent Network
Substrate), 504
tokens
impersonation and, 120122
Sysinternals Process Explorer, 121
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 716

716 Index TV

tracing for vulnerabilities shellcode, pitfalls, 299

component design Solaris, vfs_getvfssw( ), 642344
data collection, 537538 strace, 434
function hooking, 534537 tcpdump, 435
machine-code analysis, 531534 truss, 434
process injection, 530 Wireshark, 435
fingerprint systems, 546547 unlinking, unsafe, 402
format bugs, 547 unsafe unlinking, 402
integer overflows, 547 use after free vulnerabilities,
VulnTrace, 538546 499500
Transact-SQL, 504 utilities, objdump, 45
truss (Unix), 434

V
U variables
UltraSPARC processors, 224 local, placement, 168
unhandled exception filter, uninitialized variable usage,
overwrite pointer to, 185191 499500
Unhandled Exception Filter Vectored Exception Handling, 123,
mechanism, 185 181
UnhandledExceptionFilter function, Venetion technique, 213217
168 ASCII implementation, 214217
Unicode, 210 versioning, 598599
converting from ASCII, 210211 Visual C++, 431
decoder, 218221 vtables, 562563
exploits, instruction set, 212213 vulnerabilities
Unicode filters, exploits, writing, arbitrary free, 271
209211 binary
Unicode-based vulnerabilities, IIS WebDAV, 568570
exploiting, 211213 LSDs RPC-DCOM, 567568
uninitialized variable usage, 499500 Microsoft SQL Server bugs,
Unix 566567
fstat (BSD), 435 Cisco IOS, 346347
kernel, command-line interface, 348
exec_ibcs2_coff_prep_zmagic( ) protocol parsing code, 347
vulnerability, 647672 security, 347348
kernel overflows services, router, 347
0day kernel vulnerabilities, definition, 4
636642 different-sized integer conversions,
vulnerability types, 627636 497498
ltrace, 434 double free, 270, 498
OS X and, 315316 extinct bug classes, 487
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 717

Index VW 717

vulnerabilities (continued) Microsoft tool chain, 124

fingerprint systems, 546547 OllyDbg, 124
format strings, 487489 SetDefaultExceptionHandler, 126
generic logic errors, 486 shellcode, writing, 125
heap-related, 270271 SoftICE, 124
incorrect bounds-checking, TlsSetValue( ), 126
489490 VirtualProtect( ), 126
integer-related vulnerabilities, Win XP, 127
495497 Win2K and, 126
loop constructs, 490 Win32, 124125
multithreading and, 500501 Win9X and, 126
non-null termination issues, 492 WinDbg, 124
off-by-one, 490492 Windows 2003 Server, 127
out-of-scope memory usage, 499 Windows Vista, 127
signed comparison vulnerabilities, WinME and, 126
494495 WinNT and, 126
skipping null-termination issues, WSASocket( ), 126
493 device drivers
tracing for, 525529 I/O control code components,
component design, 529543 693694
format bugs, 547 IOCTL handlers, 694695
integer overflows, 547 exception handling, searches and,
VulnTrace, 538546 148153
Unicode-based, exploiting, 211213 FileMon, 435
uninitialized variable usage, filters, 129130
499500 HandleEx, 435
use after free vulnerabilities, IDA pro disassembler, 436
499500 kernel
vulnerable arguments, 391 introduction, 682683
kernel-mode payloads, 695703
mode flaws, 681682
W programming flaws, 683689
W^X memory, 381387 Linux comparison, 111114
wide characters (Unicode), 210 ProcessExplorer, 435
Win32 protections
exception handling, 122123 ASLR, 414415
PE-COFF and, 112114 heap, 416
WinDbg, 124, 431 SEH, 416
Windows stack data, 415416
debugging W^X, 413414
kernel debugger, 124 RegMon, 435
80238bindex.qxd:WileyRed 7/11/07 7:42 AM Page 718

718 Index WX

Windows (continued) Windows 2003 Server, stack

shellcode protection and, 166172
heapoverflow.c, 132148 Wireshark (Unix), 435
PEB, 132
setup, 131132
shells, popping, 153154 X
SNMP DOS, 520521 x86 AT&T syntax versus Intel syntax,
syntax, 129130 130
system calls, 690691
attacking, 692
TCPView, 435