Guide to

the Open Cloud

Current trends and open source projects

November 2016

AUTHORS
Libby Clark and Mark Hinkle
The Linux Foundation

www.linuxfoundation.org
 Contents
Introduction  3

What is the Open Cloud Today, Why It’s Important, and How It’s Advancing  5

Emerging Trends  6

Cloud Native Applications  6

Containers  7

Unikernels  7

Why is the Open Cloud Important?  7

Profile Methodology  9

Profiles  11

Infrastructure as a Service (IaaS)  11

Platform as a Service (PaaS)  12

Virtualization, Containers and Cloud Operating Systems  13

Micro or Minimalist OSes  13

Virtualization  14

Management and Automation  15

Unikernels  17

DevOps CI/CD  19

Complete CI/CD cycle  19

Configuration management  20

Logging and Monitoring  21

Software-Defined Networking (SDN)  22

Networking for containers  23

Software-Defined Storage  24

2 Guide to the Open Cloud: Current trends and open source projects
 Introduction
This third annual Guide to the Open Cloud aims to help companies stay
informed on the latest open source cloud technologies and trends. The
Linux Foundation’s cloud experts have curated a list of the most useful,
influential, and promising open source projects with which IT managers
and practitioners can build, manage, and monitor their current and
future mission-critical cloud resources. In doing so, we’re also helping
companies identify projects they can best leverage for external R&D
through strategic open source participation. Contributing knowledge
and code to open source projects not only helps companies meet their
business objectives, but it creates thriving communities that keep projects
strong and relevant over time, advances the technology, and benefits the
entire open source cloud ecosystem.

Cloud computing is the cornerstone of the digital economy. Companies across industries
now use the cloud—private, public or somewhere in between—to deliver their products and
services. Forty-one percent of all enterprise workloads are currently running in some type of
public or private cloud1. That number is expected to rise to 60 percent by mid-2018. And some
95 percent of companies are at least experimenting in the cloud2. Enterprises are continuing to
shift workloads to the cloud as their expertise and experience with the technology increases.
From banking and finance to automotive and healthcare, companies are facing the reality that
they’re now in the technology business. In this new reality, cloud strategies can make or break
an organization’s market success. And successful cloud strategies are built on Linux and open
source software.

Some of the most successful public companies today are built around cloud-native
applications—a fashionable term that simply means they’re designed to run in the cloud.
Netflix, Facebook, LinkedIn, Twitter, and Amazon have all leveraged open source components
within a distributed, microservices-based architecture to quickly deliver new products and
services that are cost-effective and responsive to market demands and changes. By breaking
applications up into microservices, or distinct, single-purpose services that are loosely coupled
with dependencies and explicitly described through service endpoints, they have significantly
increased the overall agility and maintainability of applications and used that to gain
competitive advantage.

1 451 Research, “Voice of the Enterprise: Cloud Transformation - Workloads and Key Projects,” Sept. 1, 2016.
2 RightScale, “2016 State of the Cloud Report,” January, 2016.

3 Guide to the Open Cloud: Current trends and open source projects
 The rest of the market has scrambled to replicate this architecture and approach, cobbling
together their own solutions using custom scripts and open source software—often using the
open source versions of these web giants’ own infrastructure (ie Google’s borg, which became
Kubernetes; Twitter’s Mesos project, VMware’s Cloud Foundry, etc.) This experimentation has
set off a chain of innovation with four notable trends, still playing out today:

• Increasing consumption of public cloud resources

• Adoption of container technologies like Docker and others (Fifty-three percent of
organizations are either investigating or using containers in development or in production3.)
• The rise of DevOps as the most effective method for application delivery in the cloud
• An explosion in available open source tooling as user companies like Walmart and Capital
One release their management software under open source licenses.

While deployment and management remain a challenge, microservices architecture is now

becoming mainstream. In a recent Nginx survey4 of 1,800 IT professionals, 44 percent said
they’re using microservices in development or in production. Adoption was highest among
small and medium-sized businesses. Not coincidentally, the use of public cloud is also
predominant among SMBs5, which are more nimble and faster to respond to market changes
than large enterprises with legacy applications and significant on-premise
infrastructure investments.

Many reports tout hybrid cloud as a fast-growing segment of the cloud. Demand is growing at
a compound rate of 27 percent, “far outstripping growth of the overall IT market,” according
to researcher MarketsandMarkets6. And IDC predicts that more than 80 percent of enterprise
IT organizations will commit to hybrid cloud architectures by 20177. However, this growth is
happening predominantly among large enterprises with legacy applications and the budget and
staffing to build private clouds. They turn to cloud for storage and scale-out capabilities, but
keep most critical workloads on premise. In the mid-market, hybrid cloud adoption stands at
less than 10 percent, according to 451 Research. Hybrid cloud is, then, a good transition point
for legacy workloads and experimenting with cloud implementation. But it suffers from several
challenges with more advanced cloud implementations, including management complexity
and cost.

“Most organizations are already using a combination of cloud services from different cloud
providers. While public cloud usage will continue to increase, the use of private cloud and
hosted private cloud services is also expected to increase at least through 2017. The increased
use of multiple public cloud providers, plus growth in various types of private cloud services,
will create a multi-cloud environment in most enterprises and a need to coordinate cloud usage
using hybrid scenarios.

3 Cloud Foundry, ”Hope Vs. Reality: Containers in 2016,” June 2016.

4 Nginx, “The Future of App Development and Delivery,” March 2016.
5 RightScale, “2016 State of the Cloud Report,” January, 2016.
6 Wall Street Journal, “Special Report: CIOs Say Hybrid Cloud Takes Off,” Oct. 2015.
7 IDC, “IDC FutureScape: Worldwide Cloud 2016 Predictions — Mastering the Raw Material of Digital Transformation,”
2016.

4 Guide to the Open Cloud: Current trends and open source projects
 “Although hybrid cloud scenarios will dominate, there are many challenges that inhibit working
hybrid cloud implementations. Organizations that are not planning to use hybrid cloud indicated
a number of concerns, including: integration challenges, application incompatibilities, a lack
of management tools, a lack of common APIs and a lack of vendor support,” according to
Gartner’s 2016 Public Cloud Services worldwide forecast.8

Over the long term, workloads are shifting away from hybrid cloud to a public cloud market
dominated by providers like AWS, Azure, and Google Compute. “The share of enterprise
workloads moved to the public cloud is expected to triple over the next five years,” from
16 percent to 41.3 percent of workloads running in the public cloud, according to a recent
JP Morgan survey of enterprise CIOs9. Among this group, 13 percent said they view AWS
as “intrinsic to future growth.” AWS still very much dominates the cloud industry and has
continued to grow at an incredible rate. It looks like hosted services are going to be a huge
second line of business for the company. Based on its July 2016 earnings release, Amazon’s
web business is on pace to be a $9 billion business this year.10

The shift to the public cloud will continue to accelerate as costs come down and security
concerns are allayed. Security is still a top concern among companies considering moving
workloads to the public cloud, according to Gartner, despite a strong track record of security
and increased transparency from cloud providers. Rather, security is still an issue largely due
to companies’ inexperience and improper use of cloud services. As companies gain more
experience with public cloud use, security concerns will become less of a hindrance.

As cloud adoption grows, open source technologies will continue to be the source of
innovation and the foundation for new markets and ecosystems. Each of the projects listed
in this report is actively involved in creating the future of IT infrastructure on which companies
will deliver their products and services, in the coming year and beyond. Let this report be your
guide to become familiar with these projects, the categories of technology in which they are
influential, and the ways in which they can help companies remain competitive in this age of
digital transformation.

What is the open cloud today, why it’s important, and

how it’s advancing
Many companies are still building their own cloud stacks—assembling open source
components to run a private or hybrid infrastructure. And you’ll find these components on our
list, as they were in the past. But the drive is no longer to build enterprise alternatives to large,
proprietary public clouds. Instead, more companies are turning to public cloud to run their
applications and services, and as they do so the nature of the open cloud today is changing.

8 Gartner, “Forecast Analysis: Public Cloud Services, Worldwide, 2Q16 Update,” Sept., 2016.
9 Business Insider, “CIOs See Momentous Shift in Public Cloud Usage,” April 2016.
10 Amazon, “Q2 2016 Financial Results,” July 2016.
11 Gartner, “Gartner Says Worldwide Public Cloud Services Market to Grow 17 Percent in 2016,” Sept. 2016.

5 Guide to the Open Cloud: Current trends and open source projects
 As cloud computing continues to evolve, the infrastructure is becoming more flexible and
programmable. There is a shift in the data center from finite hosted resources to an elastic
compute fabric that is more readily consumed by developers.

By the end of 2016 the public cloud services market will reach $208.6 billion in revenue,
growing by 172 percent from $178 billion in 201511. Cloud application services (software-as-a-
service or SaaS) is one of the largest segments of that and is expected to grow by 21.7 percent
in 2016 to reach $38.9 billion while Infrastructure-as-a-Service (IaaS) is projected to see the
most growth at 42.8 percent in 2016.

The public cloud itself is largely built on open source software. Offerings including Amazon
EC2, Google Compute Engine and OpenStack are all built on open source technologies.
They provide APIs that are well documented. They also provide a framework that is consistent
enough to allow users to duplicate their infrastructure from one cloud to another without a
significant amount of customization.

This allows for application portability, or the ability to move from one system to another without
significant effort. The less complex the application the more likely that it can remain portable
across cloud providers. And so the development practice that seems to be most suited for this
is to abstract things into their simplest parts.

A whole new class of open source cloud computing projects has now begun to leverage the
elasticity of the public cloud and enable applications designed and built to run on it. You’ll find
these new projects in our list as well.

EMERGING TRENDS
Early on in cloud computing there was a migration of existing applications to Amazon Web
Services, Google and Microsoft’s Azure. Virtually any app that ran on hardware in private data
centers could be virtualized and deployed to the cloud. Now with a mature cloud market more
applications are being written and deployed directly to the cloud and are often referred to as
being cloud native.

CLOUD NATIVE APPLICATIONS

While there is no textbook definition, the self-described cloud native in its simplest definition
indicates applications that have been designed to run on modern distributed systems
environments capable of scaling to tens of thousands of nodes. The old mantra, “No one ever
got fired for buying IBM (or Microsoft),” has changed to the new slogan of “No one is going
to get fired for moving to the cloud.” Rather than looking at hard and fast qualifiers for cloud-
native we need to look at the design patterns that are being applied to this evolving breed
of applications.

6 Guide to the Open Cloud: Current trends and open source projects
 In the pre-cloud days we saw virtualization take hold where entire operating systems were
portable inside of virtual machines. That way a machine could move from server to server
based on its compatibility with hypervisors like VMware, KVM or Xen Project. In recent years
we have seen the level of abstraction at the application layer where applications are container-
based and run in portable units that are easily moved from server to server regardless of
hypervisor due to container technologies like Docker and CoreOS-sponsored rkt
(pronounced rocket).

CONTAINERS
A more recent addition in the cloud era is the rise of the container, most notably Docker and
rkt. These application hosts are an evolution of previous innovations including Linux control
groups (cgroups) and LXC, and an even further abstraction to make applications more
portable. This allows them to be moved from development environments to production without
the need for reconfiguration.

Applications are now deployed either from registries or through continuous deployment
systems to containers that are orchestrated using tools like Ansible, Puppet or Chef.

Finally, to scale out these applications, the use of schedulers such as Kubernetes, Docker
Swarm, Mesos, and Diego coordinate these containers across machines and nodes.

UNIKERNELS
Another emerging technology that bears some similarity to containers are unikernels.
Unikernels are a pared down operating system (the unikernel) which are combined with a single
application into a unikernel application and are typically run within a virtual machine. They
are sometimes called a library operating system because they include libraries that enable
applications to use hardware and network protocols in combination with a set of policies for
access control and isolation of the network layer. There were systems in the 1990s called
Exokernel and Nemesis but today popular unikernels include MirageOS and OSv. Because
unikernel applications can be used independently and deployed across diverse environments,
unikernels can create highly specialized and isolated services and have become increasingly
used for developing applications in a microservices architecture.

WHY IS THE OPEN CLOUD IMPORTANT?

In previous years this report has distinguished between open source cloud and others. But
as cloud technologies have evolved it’s evident that any cloud without open source would be
the equivalent of an automobile without an engine. In 2006 we distinguished heavily between
public and private cloud and open source and closed. Today the conversation has evolved into
one cloud fabric of which open source has become an integral part.

7 Guide to the Open Cloud: Current trends and open source projects
 Perhaps what has most notably changed is that the initial cloud conversations about capex
(capital expenditures) versus opex (operating expenditures) and the actual costs to deploy
the cloud are now taking into account the advantages of improved agility and customization.
Where open source has traditionally sparked interest because of its free nature (as in no
acquisition cost) is now being lauded for the much harder to measure but much greater
benefits of faster speed to value.

We also see an improved return on investment for those companies that participate in open
source rather than only consume open source. They need to and are investing in the future
direction of the open technology they rely upon actively rather than only being passive
and opportunistic.

Besides open source software there is an overwhelming need for openness across cloud
architectures. And while emerging technologies and trends such as containers have done
a lot to improve interoperability among components and ensure application portability,
much work remains to ensure the trend toward openness and standardization continues.
To this end, foundations such as the Cloud Foundry Foundation, Cloud Native Computing
Foundation (CNCF) and Open Container Initiative (OCI) are actively bringing in new open source
projects and engaging member companies to create industry standards for new cloud-native
technologies. The goal is to help improve interoperability and create a stable base for container
operations on which companies can safely build commercial dependencies.

When work happens in the open, companies that participate are better able to compete in
rapidly changing markets and the entire industry benefits from the increased innovation. That
also means companies that do not use and participate in open source cloud projects will fall
behind. By harnessing the power of shared R&D companies that participate in open source
benefit from:

• Improved code quality

• Increased security with the ability to find and fix vulnerabilities
• Visibility into every layer of the infrastructure
• Code access in order to add features and influence the direction of the technology
• Insurance against lock-in through portability to other platforms
• Lower cost through shared development
• And more.

No single company could develop the technologies on this list on their own. Without open
source collaboration, the open cloud we know today would not exist.

8 Guide to the Open Cloud: Current trends and open source projects
 Profile Methodology
The Linux Foundation is creating the greatest shared technology asset in history, providing
professional open source management to more than 50 industry-wide projects including Cloud
Foundry, CNCF and OCI. By engaging with our members in our open source projects and
organizing leading technical conferences in North America, Europe and Asia, we help guide
many projects, technologies and companies that are driving the cloud. That puts us in a unique
position to create this list.

With new projects forming at a steady pace to innovate and define the next generation of
cloud-native infrastructure and web application deployment, it can be difficult to keep track
of what’s happening in the open source community. To be most helpful, we’ve limited the list
to the most important projects in each category including virtualization, containers, and cloud
operating systems; Infrastructure as a Service; Platform as a Service; DevOps continuous
integration/ continuous deployment tools (CI/CD); software-defined storage; and software-
defined networking (SDN).

Projects were selected for the list by The Linux Foundation’s cloud experts based on their
relevance, their relative maturity, and their relative visibility. More specifically, benchmarks for
consideration included:

• the project’s origins

• age of the project
• number of contributors
• number and frequency of commits
• diversity of contributions
• exposure
• demonstrated enterprise use
• expert opinions from IT practitioners

All profile data was collected from public sources, including project websites and source
code repositories.

Please note that as the technology is moving at cloud speed, we can’t keep an inclusive
directory of all open source cloud projects. But we can offer the notable ones. We have
focused on projects with which our experts, and the communities they work with, are most
familiar. As technology evolves, so does the list and we welcome input from the community.

This year, we’ve added a section on container management and automation tools, which
are a hot area for development as companies race to fill the growing need to manage highly
distributed, cloud-native applications. We’ve collected traditional DevOps CI/CD tools in
a separate category, though functionality can overlap. While CI/CD tools are well tested in
production and established for their purposes, most container management tools are still
early days.

9 Guide to the Open Cloud: Current trends and open source projects
 Since our last report, the SDN category has continued to rapidly evolve and gain maturity. We
are now beyond the conceptual phase of open source networking, and the companies that
were assessing the potential of these projects two years ago have begun
enterprise deployments.

The promise of lightweight cloud operating systems tailored to web applications, mobile and
embedded systems and specialized use cases has now also become a reality. Such operating
systems are ubiquitous in mobile computing and the Internet of Things, which has an entire
cloud ecosystem of its own.

10 Guide to the Open Cloud: Current trends and open source projects
 Profiles
Infrastructure as a Service (IaaS)

APACHE CLOUDSTACK
Apache CloudStack, an Apache Software Foundation project, is software designed to
deploy and manage large networks of virtual machines, as a highly available, highly scalable
Infrastructure as a Service (IaaS) cloud computing platform. It can be used to offer public
cloud services, to provide an on-premises (private) cloud offering, or as part of a hybrid cloud
solution. Users can manage their cloud with a Web interface, command line tools, and/or a full-
featured RESTful API.

CloudStack on GitHub

HPE HELION EUCALYPTUS

HPE Helion Eucalyptus is an open solution for building private clouds that are compatible with
Amazon Web Services (AWS). It provides open source implementations of many Amazon Web
Services to deploy AWS applications into a private cloud without changing tools, processes, or
application code.

HPE Helion Eucalyptus on GitHub

OPENNEBULA
OpenNebula is software to manage virtualized data centers for private, public, and hybrid
IaaS clouds. Use OpenNebula to manage data center virtualization, consolidate servers, and
integrate existing IT assets for computing, storage, and networking. Or provide a multi-tenant,
cloud-like provisioning layer on top of an existing infrastructure management solution.

OpenNebula on GitHub

OPENSTACK
OpenStack, an OpenStack Foundation project, is open source software for creating private
and public clouds. The software controls large pools of compute, storage, and networking
resources throughout a data center, and is managed through a dashboard or via the
OpenStack API. OpenStack works with other enterprise and open source technologies making
it ideal for heterogeneous infrastructure.

OpenStack on GitHub

11 Guide to the Open Cloud: Current trends and open source projects
 Platform as a Service (PaaS)

APACHE STRATOS
Apache Stratos, an Apache Software Foundation project, is a highly-extensible PaaS
framework that helps run Apache Tomcat, PHP, and MySQL applications and can be extended
to support many more environments on all major cloud infrastructures. For developers, Stratos
provides a cloud-based environment for developing, testing, and running scalable applications.
IT providers benefit from high utilization rates, automated resource management, and platform-
wide insight including monitoring and billing.

Stratos on GitHub

CLOUD FOUNDRY
Cloud Foundry, a Cloud Foundry Foundation project at The Linux Foundation, is an open
source cloud application platform that provides a choice of clouds, developer frameworks,
and application services. It supports applications built in virtually any programming language,
run as either Linux containers or Windows-based applications, and deployed across many
infrastructure types: Amazon Web Services (AWS), Microsoft Azure, Google Compute Platform
(GCP), OpenStack, VMware vSphere, VMware Photon Platform, IBM SoftLayer, and more.

Cloud Foundry on GitHub

DEIS WORKFLOW
Deis Workflow, is Engine Yard’s version 2.0 (beta) of the open source Deis PaaS that makes
it easy to deploy and manage applications on Kubernetes. It works in both public and private
clouds, as well as on bare metal.

Deis on GitHub

FLYNN
Flynn is an open source PaaS for running applications in production. In its 1.0 version, it’s
designed to run anything that can run on Linux, and includes built-in service discovery as well
as Postgres, MySQL, and MongoDB databases.

Flynn on GitHub

HEROKU
Heroku is a cloud platform that lets companies build, deliver, monitor and scale applications.
It’s based on a managed container system, with integrated data services and a powerful

12 Guide to the Open Cloud: Current trends and open source projects
 ecosystem, for deploying and running modern apps in multiple languages: Node, Ruby, Java,
Scala, PHP, and more.

Heroku on GitHub

OPENSHIFT
OpenShift is Red Hat’s PaaS that allows developers to quickly develop, host, and scale
applications in a cloud environment. Based on top of Docker containers and the Kubernetes
container cluster manager, OpenShift offers online, on-premise, and open source project
options. OpenShift Origin is the open source upstream of OpenShift.

OpenShift on GitHub

Virtualization, Containers and Cloud Operating Systems

MICRO OR MINIMALIST OSES

PROJECT ATOMIC
Project Atomic is Red Hat’s umbrella for many open source infrastructure projects to deploy
and scale containerized applications. It provides an operating system platform for a Linux
Docker Kubernetes (LDK) application stack, based on Fedora, CentOS and Red Hat
Enterprise Linux.

Project Atomic on GitHub

COREOS
A lightweight Linux operating system designed for clustered deployments providing
automation, security, and scalability for containerized applications. It runs on nearly any
platform whether physical, virtual, or private/public cloud.

CoreOS on GitHub

PHOTON OS
Photon OS is a minimal Linux operating system for cloud-native apps optimized for VMware’s
platforms. It runs distributed applications using containers in multiple formats including Docker,
rkt, and Garden.

Photon on GitHub

13 Guide to the Open Cloud: Current trends and open source projects
 RANCHEROS
RancherOS is a minimalist Linux distribution for running Docker containers. It runs Docker
directly on top of the kernel, replacing the init system, and delivers Linux services
as containers.

RancherOS on GitHub

VIRTUALIZATION

KVM
KVM (Kernel-based Virtual Machine) is a full virtualization solution for Linux on x86 hardware
containing virtualization extensions (Intel VT or AMD-V). It consists of a loadable kernel module,
kvm.ko, that provides the core virtualization infrastructure and a processor specific module,
kvm-intel.ko or kvm-amd.ko. It can run multiple virtual machines running unmodified Linux or
Windows images.

KVM mailing lists

LXC
Linux Containers (LXC) are lightweight virtual machines enabled by functions within the Linux
kernel, including cgroups, namespaces and security modules. Userspace tools coordinate
kernel features and manipulate container images to create and manage system or
application containers.

LXC on GitHub

LXD
LXD is Canonical’s container hypervisor and a new user experience for LXC. Developed in Go,
it runs unmodified Linux operating systems and applications with VM-style operations.

LXD on GitHub

XEN PROJECT
Xen Project, a Linux Foundation project, develops virtualization technologies for a number of
different commercial and open source applications including server virtualization, Infrastructure
as a Service (IaaS), desktop virtualization, security applications, embedded and hardware
appliances on x86 and ARM CPU architectures, and supports a wide range of guest
operating systems.

Xen Project Git repositories

14 Guide to the Open Cloud: Current trends and open source projects
 MANAGEMENT AND AUTOMATION

APACHE AURORA
Apache Aurora is a Mesos framework that runs applications and services across a shared pool
of machines, and is responsible for keeping them running, forever. When machines experience
failure, Aurora intelligently reschedules those jobs onto healthy machines.

Aurora Git repository

APACHE MESOS
Apache Mesos, an Apache Software Foundation project, abstracts CPU, memory, storage,
and other compute resources away from machines (physical or virtual), to build and run fault-
tolerant and elastic distributed systems.

Mesos Git repository

CLOUD FOUNDRY DIEGO

Diego is Cloud Foundry’s next-generation container management runtime system. Within CF,
the Diego system distributes the hosted app load over all of the host VMs, and keeps it running
and balanced through demand surges, outages, or other changes.

Diego on GitHub

DOCKER ENGINE
Docker Engine is Docker’s open source containerization technology combined with a workflow
for building and containerizing applications. Docker containers run on desktops, physical
servers, virtual machines, data centers, and public and private clouds.

Docker on GitHub

DOCKER SWARM (MACHINE AND COMPOSE)

Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts into a single,
virtual Docker host. Because Docker Swarm serves the standard Docker API, any tool that
already communicates with a Docker daemon can use Swarm to scale to multiple hosts. It
supports multiple tools including Dokku, Docker Compose, Docker Machine, and Jenkins.

Docker Swarm on GitHub

15 Guide to the Open Cloud: Current trends and open source projects
 KONTENA
Kontena is a platform to deploy, manage, scale, and monitor containerized applications and
microservices. It works on any public cloud, private data center, hybrid environment, virtualized
or bare metal.

Kontena on Github

KUBERNETES
Kubernetes, a Cloud-Native Computing Foundation project at The Linux Foundation, is
a container cluster manager for automating deployment, scaling, and management of
containerized applications, developed and open sourced by Google. It groups containers that
make up an application into logical units for easy management and discovery.

Kubernetes on GitHub

MANAGEIQ
ManageIQ is the open source project that powers Red Hat CloudForms. It automates,
optimizes, and controls cloud and virtualization services throughout an application’s lifecycle.

ManageIQ on GitHub

OVIRT
oVirt is a virtualization management application. It can manage hardware nodes, storage and
network resources, and deploy and monitor virtual machines running in a data center. It is the
upstream project of Red Hat’s Enterprise Virtualization product.

oVirt on GitHub

SKIPPBOX (KMACHINE AND KOMPOSE)

Kmachine is a Kubernetes tool from Skippbox that helps developers create a standalone,
single-node Kubernetes instance in public clouds, similar to Docker machine. Kompose takes
a Docker Compose file that describes a containerized application and automatically generates
the corresponding Kubernetes objects.

Kmachine on GitHub | Kompose on GitHub

SYSDIG
Sysdig is a distributed system monitoring and troubleshooting tool for Linux with native
container support. It captures system state and activity from a running Linux instance, then
saves, filters and analyzes. It’s scriptable in Lua and includes a command line interface and
a UI.

Sysdig on GitHub

16 Guide to the Open Cloud: Current trends and open source projects
 WEAVEWORKS
Weaveworks is a set of tools to cluster, view, and deploy microservices and cloud-native
applications across local and external networks. Weave Net and Flux connect and secure
applications across data centers and public clouds and work with any orchestrator. Weave
Scope monitors applications in real time.

Weaveworks on Github

WERCKER
Wercker is an automation platform to build and deploy containers for multi-tiered, cloud-native
applications. It can build containers automatically and deploy them to multiple places including
public and private Docker registries, and schedulers such as Marathon, DC/OS,
and Kubernetes.

Wercker on GitHub

UNIKERNELS

CLICKOS
ClickOS is NEC’s high-performance, virtualized software middlebox platform for network
function virtualization (NFV) built on top of MiniOS/ MirageOS.

ClickOS on GitHub

CLIVE
Clive is an operating system written in Go and designed to work in distributed and cloud
computing environments.

HALVM
The Haskell Lightweight Virtual Machine (HaLVM) is a port of the Glasgow Haskell Compiler
toolsuite that enables developers to write high-level, lightweight virtual machines that can run
directly on the Xen hypervisor.

HaLVM on GitHub

INCLUDEOS
IncludeOS is a unikernel operating system for C++ services running in the cloud. It provides a

17 Guide to the Open Cloud: Current trends and open source projects
 bootloader, standard libraries and the build- and deployment system on which to run services.
Test in VirtualBox or QEMU, and deploy services on OpenStack.

IncludeOS on GitHub

LING
Ling is an Erlang platform for building super-scalable clouds that runs directly on top of the Xen
hypervisor. It runs on only three external libraries — no OpenSSL — and the filesystem is read-
only to remove the majority of attack vectors.

Ling on GitHub

MIRAGEOS
MirageOS is a library operating system incubating under the Xen Project at The Linux
Foundation. It uses the OCaml language to construct unikernels for secure, high-performance
network applications across a variety of cloud computing and mobile platforms. Code can
be developed on a normal OS such as Linux or MacOS X, and then compiled into a fully-
standalone, specialised unikernel that runs under the Xen hypervisor.

MirageOS on GitHub

OSV
OSv is the open source operating system from Cloudius Systems designed for the cloud. It
supports applications written in Java, Ruby (via JRuby), JavaScript (via Rhino and Nashorn),
Scala, and others. And it runs on the VMware, VirtualBox, KVM, and Xen hypervisors.

OSv on GitHub

RUMPRUN
Rumprun is a production-ready unikernel that uses the drivers offered by rump kernels, adds
a libc and an application environment on top, and provides a toolchain with which to build
existing POSIX-y applications as Rumprun unikernels. It works on KVM and Xen hypervisors
and on bare metal and supports applications written in C, C++, Erlang, Go, Java, Javascript
(Node.js), Python, Ruby, Rust, and more.

Rumprun on GitHub

RUNTIME.JS
Runtime.js is an open source library operating system (unikernel) for the cloud that runs
JavaScript, can be bundled up with an application and deployed as a lightweight and
immutable VM image. It’s built on V8 JavaScript engine and uses event-driven and non-

18 Guide to the Open Cloud: Current trends and open source projects
 blocking I/O model inspired by Node.js. KVM is the only supported hypervisor.

Runtime.js on GitHub

UNIK
Unik is EMC’s tool for compiling application sources into unikernels (lightweight bootable disk
images) rather than binaries. It allows applications to be deployed securely and with minimal
footprint across a variety of cloud providers, embedded devices (IoT), as well as a developer
laptop or workstation. It supports multiple unikernel types, processor architectures, hypervisors
and orchestration tools including Cloud Foundry, Docker, and Kubernetes.

Unik on GitHub

DevOps CI/CD

COMPLETE CI/CD CYCLE

CONCOURSE
Concourse is a scalable continuous integration pipeline tool sponsored by Pivotal. It was built
for the needs of Cloud Foundry projects to test and deploy on different architectures and
platforms and remain compatible with previous versions.

Concourse on GitHub

DRONE
Drone is an open source continuous integration platform built on container technology for
GitHub and Bitbucket. It monitors code for bugs and integrates with Amazon, Heroku, Google
AppEngine, and more. It also works with SSH and shell scripts for custom deployments.

Drone on Github

HYGIEIA
Hygieia is a configurable DevOps dashboard to visualize near real-time status of the entire
software delivery pipeline, built and released as open source by Capital One.

Hygieia on GitHub

19 Guide to the Open Cloud: Current trends and open source projects
 JENKINS
Jenkins is an open source automation server. It provides hundreds of plugins to support
building, deploying, and automating any project across multiple platforms. It can be used as a
simple CI server or turned into a continuous delivery hub.

Jenkins on GitHub

SHIPPABLE
Shippable is a continuous deployment platform built on Docker that provides end-to-end
deployment pipelines for automation and integration. It integrates with multiple development
tools and deploys to public clouds or in a hybrid fashion across public and private clouds. It
also supports n-tier applications and cloud-native technologies such as Docker containers
and microservices.

Shippable on GitHub

TRAVIS-CI
Travis-CI is a hosted continuous integration and deployment system that integrates with
GitHub. It supports multiple languages, can test pull requests before they are merged, and can
deploy tested code to Amazon S3 and Heroku.

Travis-CI on GitHub

CONFIGURATION MANAGEMENT

ANSIBLE
Ansible is Red Hat’s open source IT automation engine for cloud provisioning, configuration
management, application deployment, intra-service orchestration, and other IT needs on multi-
tier architectures.

Ansible on GitHub

CHEF
Chef is a configuration management tool to automate infrastructure. It manages servers in the
cloud, on-premises, or in a hybrid environment.

Chef on GitHub

20 Guide to the Open Cloud: Current trends and open source projects
 PUPPET
Puppet is an open source server automation tool for configuration and management. It works
on Linux, Unix, and Windows systems and performs administrative tasks (such as adding
users, installing packages, and updating server configurations) based on a
centralized specification.

Puppet on GitHub

SALT OPEN
Salt Open is orchestration and configuration management software to manage infrastructure
and applications at scale. It’s the upstream open source project for SaltStack and runs on
Linux and Windows.

Salt on GitHub

LOGGING AND MONITORING

FLUENTD
Fluentd is an open source data collector for unified logging layer, sponsored by Treasure
Data. It structures data as JSON to unify all facets of processing log data: collecting, filtering,
buffering, and outputting logs across multiple sources and destinations.

Fluentd on GitHub

HEAPSTER
Heapster is a container cluster monitoring and performance analysis tool in Kubernetes. It
supports Kubernetes and CoreOS natively and can be adapted to run on OpenShift. It also
supports a pluggable storage backend: InfluxDB with Grafana, Google Cloud Monitoring,
Google Cloud Logging, Hawkular, Riemann and Kafka.

Heapster on GitHub

LOGSTASH
Logstash is Elastic’s open source data pipeline to help process logs and other event data from
a variety of systems. Its plugins can connect to a variety of sources and stream data at scale to
a central analytics system.

LogStash on GitHub

21 Guide to the Open Cloud: Current trends and open source projects
 PROMETHEUS
Prometheus is an open source systems monitoring and alerting toolkit, originally built at
SoundCloud and now a Cloud-Native Computing Foundation project at The Linux Foundation.
It fits both machine-centric and microservices architectures and supports multi-dimensional
data collection and querying.

Prometheus on GitHub

WEAVE SCOPE
Weave Scope is Weaveworks’ open source tool to monitor distributed applications and their
containers in real time. It integrates with Kubernetes and AWS ECS.

Weave Scope on GitHub

Software-Defined Networking (SDN)

ONOS
Open Network Operating System (ONOS), a Linux Foundation project, is a software-defined
networking OS for service providers that has scalability, high availability, high performance and
abstractions to create apps and services.

ONOS on GitHub

OPENCONTRAIL
OpenContrail is Juniper Networks’ open source network virtualization platform for the cloud.
It provides all the necessary components for network virtualization: SDN controller, virtual
router, analytics engine, and published northbound APIs. Its REST API configures and gathers
operational and analytics data from the system.

OpenContrail on GitHub

OPENDAYLIGHT
OpenDaylight, an OpenDaylight Foundation project at The Linux Foundation, is a
programmable, software-defined networking platform for service providers and enterprises.
Based on a microservices architecture, it enables network services across a spectrum of
hardware in multivendor environments.

OpenDaylight on GitHub

22 Guide to the Open Cloud: Current trends and open source projects
 OPEN VSWITCH
Open vSwitch, a Linux Foundation project, is a production-quality, multilayer virtual switch.
It’s designed for massive network automation through programmatic extension, while still
supporting standard management interfaces and protocols including NetFlow, sFlow, IPFIX,
RSPAN, CLI, LACP, and 802.1ag. It supports distribution across multiple physical servers
similar to VMware’s vNetwork distributed vswitch or Cisco’s Nexus 1000V.

OVS on GitHub

OPNFV
Open Platform for Network Functions Virtualization (OPNFV), a Linux Foundation project,
is a reference NFV platform for enterprise and service provider networks. It brings together
upstream components across compute, storage and network virtualization in order create an
end-to-end platform for NFV applications.

OPNFV on Bitergia

NETWORKING FOR CONTAINERS

FLANNEL
Flannel is an etcd-backed overlay network for containers from CoreOS. It was designed for
Kubernetes, but is a generic overlay network that can be used as an alternative to existing
software-defined networking solutions.

Flannel on GitHub

LIBNETWORK
Libnetwork is Docker’s project to create a multi-platform library for networking containers. It
enables network plugins by providing a consistent programming interface and the required
network abstractions for applications.

Libnetwork on GitHub

PROJECT CALICO
Calico is Tigera’s data center networking software that uses a pure Layer 3 approach.
It integrates with cloud orchestration systems (such as OpenStack) to enable secure IP
communication between virtual machines, containers, or bare metal workloads.

Project Calico on GitHub

23 Guide to the Open Cloud: Current trends and open source projects
 WEAVE NET
Weave Net is Weaveworks’ networking software for container applications across data centers
and public clouds. It supports peer-to-peer encryption, traffic isolation, and segmentation
for building secure control plane networks, running in production environments, or for joining
container deployments in public clouds over the internet.

Weave on GitHub

Software-Defined Storage

APACHE CASSANDRA
Apache Cassandra, an Apache Software Foundation project, is a scalable, high-availability
database for mission-critical applications. It runs on commodity hardware or cloud
infrastructure and replicates across multiple data centers for lower latency and fault tolerance.

Cassandra on GitHub

CEPH
Ceph is Red Hat’s distributed, highly scalable block, object, and file storage platform for
enterprises deploying public or private clouds. It’s commonly used with OpenStack.

Ceph on GitHub

COUCHDB
Apache CouchDB, an Apache Software Foundation project, is a single-node or clustered
database management system. It provides a RESTful HTTP API for reading and updating
database documents.

CouchDB on GitHub

DOCKER VOLUME PLUGINS

Docker Engine volume plugins enable Engine deployments to be integrated with external
storage systems and enable data volumes to persist beyond the lifetime of a single Engine
host. Volume plugins exist for multiple external storage systems including Azure File Storage,
NetApp, VMware vSphere, and more.

See individual plugins on GitHub

24 Guide to the Open Cloud: Current trends and open source projects
 GLUSTERFS
Gluster is Red Hat’s scalable network filesystem and data management platform. It can deploy
on-premise, in private, public, or hybrid clouds, and in Linux containers for media streaming,
data analysis, and other data- and bandwidth-intensive tasks.

GlusterFS on GitHub

MONGODB
MongoDB is a high performance document database designed for ease of development
and scaling.

MongoDB on GitHub

NEXENTA
NexentaStor is a scalable, unified software-defined file and block storage service that includes
data management functionality. It integrates with VMware and supports Docker
and OpenStack.

Nexenta on GitHub

REDIS
Redis is an in-memory data structure store, used as database, cache and message broker.
It supports multiple data structures and has built-in replication, Lua scripting, LRU eviction,
transactions and different levels of on-disk persistence.

Redis on GitHub

RIAK CS
Riak CS (Cloud Storage) is object storage software built on top of Riak KV, Basho’s distributed
database. It provides distributed cloud storage at any scale, and can be used to build public
or private cloud architectures or as storage infrastructure for heavy-duty applications and
services. Its API is Amazon S3 compatible and supports per-tenant reporting for use cases
involving billing and metering.

Riak CS on GitHub

SWIFT
Swift is OpenStack’s object storage system designed to store and retrieve unstructured data
with a simple API. It’s built for scale and optimized for durability, availability, and concurrency
across the entire data set.

Swift on GitHub

25 Guide to the Open Cloud: Current trends and open source projects
 Thank you for your interest in the 2016 Guide to the
Open Cloud!

Looking for training and other resources to help you keep up on cloud technologies? Check
out The Linux Foundation’s free Intro to Cloud Infrastructure Technologies course on
edX, and our other e-learning courses on OpenStack Administration Fundamentals and
Kubernetes Fundamentals.

Our cloud computing events are also a great place to meet the experts in the field and learn
the latest technologies and trends. The Linux Foundation organizes industry-wide events
including ContainerCon, CloudNativeCon, MesosCon, Cloud Foundry Summit, OpenDaylight
Summit, and more. Visit events.linuxfoundation.org for a list of upcoming events.

26 Guide to the Open Cloud: Current trends and open source projects
The Linux Foundation promotes, protects and standardizes Linux by
providing unified resources and services needed for open source to
successfully compete with closed platforms.

To learn more about The Linux Foundation or our other initiatives

please visit us at www.linuxfoundation.org