Transition From VM-based Architecture To Container Model

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 5

CASE STUDY

Everalbum

Everalbum transitioned from a more static, VM-based


architecture to a Docker-based model across dozens of
hosts and multiple cloud providers. In addition to
monitoring Docker containers, Everalbum leverages
Kubernetes and cloud provider-run services. They
needed a solution that covers server monitoring,
container monitoring and even custom statsd metrics
from their application code. Read on to learn about the
challenges Everalbum faced during this transition and
how they used Sysdig to assist in the change.
About Everalbum Transitioning to containers

Based in San Francisco, California, Everalbum is a data-intensive, Like many applications today, looks can be deceiving. What
mobile-first service designed to protect your most valuable appears to be a simple app on a user’s phone can (and should!)
memories. Everalbum protects your life's photos so you never hide the complexity of the backend. That’s certainly true for
have to worry about losing them. Across devices and photo Everalbum: the need to consistently and reliably store their
sources, Everalbum automatically backs up your photos and users’ photos from any number of devices requires them to have
videos so you can access them at any time. You can then free up a robust, distributed backend. It had to have high uptime,
space on your device by removing photos from your camera roll. scalability, and the ability to roll out new features quickly.

In order to achieve this, Everalbum now runs services both on


Amazon Web Services (AWS) and Google Container Engine (GKE),
as well as Heroku. They have dozens of hosts that they monitor,
in addition to services operated by the cloud providers
themselves.

“About a year ago we started to make the transition to


containers,” noted Jon Mumm, co-founder at Everalbum. “It just
so happened we ran across Sysdig at the same time.” Jon recalled
that they were looking at Docker containers for their potential to
accelerate development and deployment of new code.

Everalbum uses Kubernetes to orchestrate containers across


their environment, adding another important element to their
monitoring and operations strategy.

CASE STUDY | Everalbum | 2


Requirements

Given Everalbum has a reasonably complex backend, they


needed monitoring capabilities that could adapt. In particular
they needed:
“ monitoring would be different
We realized that docker

• Docker monitoring support: They required a system that on a number of levels, and we
could do more than just aggregate CPU/Memory/Network wanted to get ahead of that
stats from the Docker API and could give insight into
applications running inside their containers. problem. We knew that
• Support for non-containerized applications: Certain stateful services like AWS Cloudwatch
services don’t run inside containers, and Everalbum needed and Google Stackdriver were
to monitor those as well.
still fine for low level stuff, but
Kubernetes integration: Kubernetes understands both the
physical resources used by containers but also the logical services anything that we were running
that the containers represent. Everalbum required a container- inside a container would be an
native monitoring tool must be able to tap into that knowledge.
entirely different story.”
Simpler config for dynamic services: Everalbum knows less work
is better, especially in a Kubernetes world where containers can
come and go at any time of the day.

Custom metrics: There was a heavy bonus for the monitoring


system to capture statsd metrics as well so that they have one
less monitoring tool they need to worry about.

CASE STUDY | Everalbum | 3


Sysdig for Container-native monitoring

Jon recalled his earliest experiences with Sysdig. “At the time we Monitoring HTTP endpoints
had 6 or 7 HTTP services, and it was a pain to constantly be Jon talked a bit about being able to monitor applications and
tuning our monitoring system when there were changes. I was services, and not just Docker infrastructure metrics
really impressed with Sysdig’s instrumentation: it automatically
“For example, we keep a close eye on particular HTTP endpoints
figured out where my services were running, what the services
as early-warning indicators of application problems. Sysdig
they actually were, and what the appropriate metrics were to
provides out of the box HTTP metrics, which make it really easy to
collect. Sysdig could do this even if my services were in
get going. If you combine that with a service-level view using
containers, without agents inside the containers.
Kubernetes metadata, then you have a good sense of how your
service is performing. That’s a pretty powerful combination of

“ I also really liked the statsd capability. You can just send 

stuff to localhost and never worry about it - Sysdig just 

capabilities.”

Jon used to depend on CloudWatch for alerting on HTTP errors.


finds it. It’s an easy implementation that ensures the 
 “We would use CloudWatch whenever we saw a 500 on our ELBs.

development team will take advantage of it,” said Jon. But CloudWatch would never resolve when the 500 count goes to
zero. It sounds basic, but simple things like this make our lives
easier. Sysdig figured this out of course.”
Sysdig’s Kubernetes integration allows Everalbum to easily Jon noted where he wanted to take this functionality. “In addition
monitor at both the physical and the logical layers with their to Sysdig reporting on the top HTTP endpoints and the slowest
application. “The Kubernetes integration feels very natural. I can HTTP endpoints, they also allow measurement and reporting of
switch to a real-time logical view of my application using any path in the system. We’re going to implement that soon to
Kubernetes’ metadata for grouping containers. It also allows us to give us more coverage.”
divide up monitoring for our dev and prod namespaces. They’re
both important to monitor, but we treat them differently. Sysdig
makes it easy to do that.”

CASE STUDY | Everalbum | 4


Sysdig for Container-native monitoring

Monitoring Services that aren’t containerized What the future holds

“We run a bunch of our applications on top of GPUs,” Jon pointed Jon noted that he’s seen continually increasing adoption of Sysdig
out. “For most of this stuff, we typically depend on our Statsd across the company. “At first it was just me, but then when we
metrics, in addition to low level resource utilization metrics. started migrating alerts to Sysdig we saw more pickup. As we
Sysdig’s statsd capability allows us to aggregate all our Statsd exposed more and more statsd metrics there’s also a pickup.
metrics in one place for easy viewing and analysis.” Basically, as we leverage more and more functionality within
Sysdig we’re seeing more adoption. We want to keep that moving
In addition, Everalbum runs Postgres. “We don’t run this stateful
in the same direction.”
service in Docker. But that’s fine in terms of Sysdig.” Sysdig
automatically detects Postgres whether or not it’s running in a
container, and then collects the appropriate app-level metrics by
enabling the correct application check.

CASE STUDY | Everalbum | 5

You might also like