Co

m
pl
im
en
Container

ts
of
Networking
From Docker to Kubernetes

Michael Hausenblas
 The NGINX Application Platform
powers Load Balancers,
Microservices & API Gateways
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
Load
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
Balancing
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
https://www.nginx.com/solutions/adc/
Learn more at nginx.com
https://www.nginx.com/
https://www.nginx.com/
https://www.nginx.com/solutions/microservices/
https://www.nginx.com/solutions/microservices/
https://www.nginx.com/solutions/microservices/
https://www.nginx.com/solutions/microservices/
https://www.nginx.com/solutions/microservices/
https://www.nginx.com/solutions/microservices/
https://www.nginx.com/solutions/microservices/
https://www.nginx.com/solutions/microservices/
https://www.nginx.com/solutions/microservices/
https://www.nginx.com/solutions/microservices/
https://www.nginx.com/solutions/microservices/
Microservices
https://www.nginx.com/solutions/microservices/
https://www.nginx.com/solutions/microservices/
https://www.nginx.com/products/
FREE TRIAL
https://www.nginx.com/products/
https://www.nginx.com/products/
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/cloud/
Cloud
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/cloud/
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/solutions/application-security/
Security
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/solutions/application-security/
https://www.nginx.com/
LEARN MORE
https://www.nginx.com/
https://www.nginx.com/
https://www.nginx.com/solutions/web-mobile-acceleration/
https://www.nginx.com/solutions/web-mobile-acceleration/
https://www.nginx.com/solutions/web-mobile-acceleration/
https://www.nginx.com/solutions/web-mobile-acceleration/
https://www.nginx.com/solutions/web-mobile-acceleration/
https://www.nginx.com/solutions/web-mobile-acceleration/
https://www.nginx.com/solutions/web-mobile-acceleration/
Web & Mobile
https://www.nginx.com/solutions/web-mobile-acceleration/
https://www.nginx.com/solutions/web-mobile-acceleration/
Performance
https://www.nginx.com/solutions/web-mobile-acceleration/
https://www.nginx.com/solutions/web-mobile-acceleration/
https://www.nginx.com/
https://www.nginx.com/
https://www.nginx.com/solutions/api-gateway/
https://www.nginx.com/solutions/api-gateway/
https://www.nginx.com/solutions/api-gateway/
https://www.nginx.com/solutions/api-gateway/
https://www.nginx.com/solutions/api-gateway/
https://www.nginx.com/solutions/api-gateway/
https://www.nginx.com/solutions/api-gateway/
https://www.nginx.com/solutions/api-gateway/
https://www.nginx.com/solutions/api-gateway/
API
https://www.nginx.com/solutions/api-gateway/
https://www.nginx.com/solutions/api-gateway/
Gateway
https://www.nginx.com/solutions/api-gateway/
https://www.nginx.com/solutions/api-gateway/
 Container Networking
From Docker to Kubernetes

Michael Hausenblas

Beijing Boston Farnham Sebastopol Tokyo

Container Networking
by Michael Hausenblas
Copyright © 2018 O’Reilly Media. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online edi‐
tions are also available for most titles (http://oreilly.com/safari). For more information, contact our
corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com.

Editor: Nikki McDonald Proofreader: Charles Roumeliotis

Production Editors: Melanie Yarbrough Interior Designer: David Futato
and Justin Billing Cover Designer: Karen Montgomery
Copyeditor: Rachel Head Illustrator: Rebecca Demarest

May 2018: First Edition

Revision History for the First Edition

2018-04-17: First Release

The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Container Networking, the cover
image, and related trade dress are trademarks of O’Reilly Media, Inc.

While the publisher and the author have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and the author disclaim all responsi‐
bility for errors or omissions, including without limitation responsibility for damages resulting from
the use of or reliance on this work. Use of the information and instructions contained in this work is
at your own risk. If any code samples or other technology this work contains or describes is subject
to open source licenses or the intellectual property rights of others, it is your responsibility to ensure
that your use thereof complies with such licenses and/or rights.
This work is part of a collaboration between O’Reilly and NGINX. See our statement of editorial
independence.

978-1-492-03681-4
[LSI]
 Table of Contents

Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

1. Motivation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Introducing Pets Versus Cattle 1
Go Cattle! 2
The Container Networking Stack 3
Do I Need to Go “All In”? 4

2. Introduction to Container Networking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Single-Host Container Networking 101 5
Modes for Docker Networking 7
Administrative Considerations 10
Wrapping It Up 11

3. Multi-Host Networking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Multi-Host Container Networking 101 13
Options for Multi-Host Container Networking 13
Docker Networking 15
Administrative Considerations 16
Wrapping It Up 16

4. Orchestration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
What Does a Scheduler Actually Do? 19
Docker 20
Apache Mesos 21
Hashicorp Nomad 23
Community Matters 25
Wrapping It Up 25

v
5. Service Discovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
The Challenge 27
Technologies 28
Load Balancing 32
Wrapping It Up 34

6. The Container Network Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

History 38
Specification and Usage 38
Container Runtimes and Plug-ins 40
Wrapping It Up 41

7. Kubernetes Networking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
A Gentle Kubernetes Introduction 43
Kubernetes Networking Overview 45
Intra-Pod Networking 46
Inter-Pod Networking 47
Service Discovery in Kubernetes 50
Ingress and Egress 53
Advanced Kubernetes Networking Topics 55
Wrapping It Up 57

A. References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

vi | Table of Contents