DAT208

Supercharge your real-time apps

with Amazon ElastiCache
Pratibha Suryadevara Henry Yang
GM, Amazon ElastiCache Software Engineer
Amazon Web Services Lyft

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What to expect

1 2 3
Hear Understand Plan
about what real time Internals on how how you would
applications need Amazon leverage this by
from a data store, ElastiCache appreciating how
what Amazon achieves Extreme Lyft and others use
ElastiCache provides Performance @ this to super charge
and the new features scale their applications
we launched
Purpose-built databases

Relational Key-value Document In-memory Graph Time-series Ledger Search

Amazon RDS DynamoDB DocumentDB ElastiCache Neptune Timestream QLDB Amazon

Elasticsearch
Service

Aurora Community Commercial

Data categories and common use cases

Relational Key-value Document In-memory Graph

CRM, Real-time bidding, Content Leaderboards, Fraud detection,

finance shopping cart, management, real-time social networking,
social, product personalization, analytics, caching recommendation
catalog, customer mobile engine
preferences
Redis: Real-time apps need low latency and high throughput

1 million+
TB
Global
Microseconds
Millions
Mobile, IoT, devices
Up-down-out-in
Pay as you go
Open Redis API
About Redis

Fast In-memory
<1 ms latency for most commands
key-value store
Open source
Powerful
~200 commands, Lua scripting,
Simple geospatial, and pub/sub

Highly available Various data structures

Strings, lists, hashes, sets, sorted sets,
Replication streams, bitmaps, and HyperLogLogs

Atomic operations Backup/restore

Supports transactions Enables snapshotting
Memcached: Fast caching

Fast Slab allocator

Open source Supports strings, objects

No persistence Multi-threaded

ElastiCache for Very established In-memory key-value

Memcached data store
Easy to scale
Choices
Redis Memcached
Primary use case In-memory database & cache Cache
Data model In-memory key-value In-memory key-value
Open source compatibility Redis Memcached

Cache databases compatibility All databases All databases

Strings, lists, sets, sorted sets,

Data structures Strings, objects
hashes, bit arrays, hyperloglog
Get, set, stats, add, replace,
Commands 200+
flush all
High availability & failover Yes No
Advanced eviction policies
no eviction, allkeys-lru, volatile-
Eviction policies Least recently used (LRU)
lru, allkeys-random, volatile-
random, volatile-ttl
Redis—#1 key-value store, most-loved database

* StackOverflow 2019 developer survey

Amazon ElastiCache
Redis and Memcached compatible, in-memory data store and cache

Redis & Memcached Extreme Secure Easily

compatible performance and reliable scalable

Fully compatible with In-memory data store Network isolation, Scale writes and
open-source Redis and cache for microsecond encryption at rest/transit, reads with sharding
and Memcached response times HIPAA, PCI, FedRAMP, multi and replicas
AZ, and automatic failover
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Extreme performance

Predictable microsecond latency

for real-time apps
What impacts performance?
Redis on optimized M5 and R5 instances

170 TB
250,000
in-memory capacity

Nitro
200,000
hardware
150,000

100,000
bare metal
50,000

0
Scalable
Large X Large 2X Large 4X Large 8X Large/12X 16X Large/24X
Large Large
Vanilla R4 Vanilla R5 Tuned R5
Redis has single thread architecture

Simplicity
No race conditions, no synchronizations
Easy to understand and extend
Easy to support rich functionality
Share-nothing architecture—scale by sharding.
Improves cache coherency
Redis connection processing: sequential

Main Redis thread

Redis port
Client 1 Client I/O 1 Redis
command

Redis server
Redis connection processing: sequential

Main Redis thread

Redis port
Client 2 Client 1 Client I/O 2 Client I/O 1 Redis
command

Redis server
Redis connection processing: sequential

Main Redis thread

Redis port
Client 3 Client 2 Client 1 Client I/O 3 Client I/O 2 Client I/O 1 Redis
command

Redis server
Redis connection processing: parallel

Redis port
Dedicated thread

Client I/O 1

Client 1
Main Redis thread

Redis server
Redis connection processing: parallel

Dedicated thread

Client I/O 2

Client 2
Redis port
Dedicated thread

Client I/O 1

Client 1
Main Redis thread

Redis server
Redis connection processing: parallel

Dedicated thread

Client I/O 3

Client 3
Dedicated thread

Client I/O 2

Client 2
Redis port
Dedicated thread

Client I/O 1

Client 1
Main Redis thread

Redis server
Optimized instances + enhanced I/O

Scale up to 170 TB of
in-memory capacity
450,000
AWS Nitro
400,000 hardware
350,000

300,000

250,000
bare metal
200,000

150,000 Custom Intel Xeon

Scalable processors up
100,000
to 3.1 GHz and AVX-512
50,000

0 Dynamic network
X Large 2X Large 4X Large 8X Large/12X Large 16X Large/24X Large processing to enhance
Vanilla R4 Vanilla R5 Tuned R5 (non enhanced I/O) Tuned R5 (enhanced I/O) input/output (I/O)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Easily scalable

Scale up to 170 TiB

of in-memory capacity
When to scale and how to scale?
Redis topologies
Vertically scaled

Cluster mode Slot 0

Keyspace
disabled Slot 1 …
Vertical scaling
Slot 16383
1 Primary 0…n replicas

Horizontally scaled
Cluster mode Slot 0-5461

Keyspace
enabled
Slot 5462-10922
Horizontal scaling
Slot 10923-16383
0…n replicas
1…n Primaries or shards
Redis cluster mode

Shard 1 Shard 2 Shard 3

0–5461 5462–10922 10923–16383

P P P

Configuration
endpoint

Reads/writes
App 1

Supports 250 node cluster

No downtime scale out: Amazon ElastiCache

Shard 1 Shard 2 Shard 3

0–5461 5462–10922 10923–16383

P P P

Reads/writes
App 1
No downtime scale out: Amazon ElastiCache

Shard 1 Shard 2 Shard 3

0–5461 5462–10922 10923–16383

P P P

Unified slot distribution across shards

Reads/writes
App 1 Added Added
Shard 4 Shard 5

No application interruption P P
No downtime scale in: Amazon ElastiCache

Shard 1 Shard 2 Shard 3

0–5461 5462–10922 10923–16383

P P P

Unified slot distribution across shards

Reads/writes
App 1 Removed Removed
Shard 4 Shard 5

No application interruption P P
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Caching
Challenge
Retrieving data from disk-based databases
and applications is slow. Workarounds such
as specialized hardware, proprietary in-
memory databases, and data reduction are
expensive, have high lock-in, don’t scale,
and are difficult to manage

Solution
Use Redis, an open source, in-memory data
store and cache, to store frequently used
data in-memory for microsecond latency
and high throughput

Use cases
Database query caching, persistent session
caching, and full-web page caching
Session store
Challenge
Need low latency and high
concurrency data stores to process
fast moving ephemeral session data
such as authentication tokens,
cookies, and session state

Solution
Use Redis as an in-memory key-value
store and cache with appropriate key
expiration strategies to store and
manage ephemeral session data for
microsecond responses

Apps
Web and mobile apps
Real-time streaming & analytics store
Challenge
Need a fast data store to
process fast moving streaming
data in real-time

Solution
Use Redis as an in-memory
data store with versatile data
structures such as streams, lists,
and sets to ingest, process, and
analyze real-time data with
sub-millisecond latency

Apps
Social media, ad targeting,
personalization, IoT, and
time-series data analytics
Geospatial apps
Challenge
Need low latency and high
concurrency data stores to process
fast moving location data in
real-time

Solution
Use Redis as an in-memory data
store with purpose-built data
structures such as sorted sets,
encoding/decoding of location
data, and APIs for location-specific
indexing, searching, and sorting

Apps
Ride-hailing, dating, social media,
gaming, eCommerce
Leaderboards
Challenge
Leaderboards need low latency and
high concurrency data stores to
process changing scores and update
ranks in real-time

Solution
Use Redis as an in-memory data
store with data structures such as
sorted sets and ranked lists that
ensure uniqueness of elements while
maintaining lists sorted by scores
with microsecond

Apps
Gaming leaderboards,
sales leaderboards
Chat & messaging
Challenge
Chat & messaging apps requires a
low latency and high throughput
communication channel over which a
client can send messages that are
redistributed to other participants

Solution
Use Redis as an in-memory store and
cache with Redis Streams and
PUB/SUB features to build high
performance chat & messaging apps

Apps
Chat rooms, comment streams,
machine to machine communication
ElastiCache for Redis customers
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Locations, locations, locations

5 sec
loop
PUT /v1/locations
{
"lat": 37.61,
"lng": -122.38
}

?
200 OK
low latency

6 -> 45 million RPS

Redis Infrastructure: the evolution

Twemproxy

Fast, stable Active + Passive Replication Reduced operation overhead

Health Check
Passive Health Check Hot shards/Hot keys Eventually consistent
Dynamic config
Unmaintained Scale read from replica
Consistent stats
Static configuration Live upgrade and scale
High availability
No hot restart No chaos monkey
Shadow traffic

Prefix routing

Consistency
Fully managed locations clusters
geohash -> [user_id]

(user_id, quality) -> {location info}

Migration
No code change
1. Shadow traffic
localhost:6379

2. Data validation
localhost:6380

3. Reverse shadow
traffic
4. Remove old cluster
Learnings

Managed service Great performance Envoy service mesh

100+ pagers per month 36 million rps on a single Unrestricted by language
→ ~0 pager cluster with 48% CPU or specific clients

Failover/online scaling in P99 latency from 0.5 ms Consistent stats across

5 mins without intervention (highly workloads dependent) the company

Rich feature set

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s new—2019

Fully managed Scalability Security & compliance OSS compatibility Extreme performance

Self service online Scale up/down Rename Command Redis 5.0.3 Enhanced I/O
migration from ec2
Reader endpoints Customer Managed Redis 5.0.4 T3 Instance Support
Self-service Master Key support
Maintenance updates Memcached 1.5.16
Auth Token Rotation
Redis 5.0.5
Coming soon…

AWS Identity
and Access
Management
Related Amazon ElastiCache Sessions
Session Type Title Time Location

DAT323-R What’s new with Amazon ElastiCache: We have been hard at 2019-12-04 13:00 Mirage, Events Center B2 Red
work this year. In this session, you’ll learn Online Migration
Breakout Tool, performance enhancement like Enhanced I/O, Security
like Customer Master Key, and scalability like Online Scaling
DAT323-R1 up/down cluster-mode), and more 2019-12-05 16:00 Aria, Level 3 West, Ironwood 5

Supercharge your Real-Time Apps with Amazon ElastiCache:

Learn how you can boost the performance of real-time apps
Venetian, Level 3, Murano
DAT208 Breakout by using the fully-managed ElastiCache for extreme 2019-12-04 08:30
3203
performance, high scalability, availability, and security. Lyft
will also share their experience with ElasitCache on stage.

Best practices for migrating your self-managed Redis

Clusters online from EC2 to fully managed Amazon
ElastiCache: Managing Redis Cluster on your own can be
DAT358 Breakout 2019-12-04 10:00 Bellagio, Monet 1
hard. With the Online Migration feature, you can now easily
move your data from self-hosted Redis on Amazon EC2 to
fully managed Amazon ElastiCache
Related Amazon ElastiCache Sessions
(Chalk talks and workshop)
Session ID Type Title Time Location
Real case on boosting performance with ElastiCache for Redis
MGM, Level 1,
DAT308 Chalk talk We’ll show you how to build a caching layer before your primary 2019-12-02 13:45
database to dramatically boost its performance Boulevard Ballroom 162

MGM, Level 3,
DAT349-R Chalk talk Beyond Caching: Advanced design patterns in Redis 2019-12-03 17:30
Redis is the "Swiss Army knife" of distributed apps. In this session
South Concourse 301
we will go through use-cases, and demos of how to solve
Chalk talk
DAT349-R1 problems like managing state, processing events and more. 2019-12-05 11:30 Aria, Level 1 East, Joshua 3

Best practices with Amazon ElastiCache

Learn best practices for setting up your Amazon ElastiCache for
DAT354 Chalk talk success with online scaling, high availability across Multi-AZ, and
2019-12-04 13:45 Aria, Level 1 East, Joshua 4
security configurations from our fully managed service.

Unlock the value of Redis in AWS

DAT353 Chalk talk In this session we will teach you how to accelerate your database 2019-12-04 16:45 Bellagio, Gauguin 2
and integrate Redis with other AWS services.

DAT406 R 2019-12-02 19:00 Venetian, Level 2, Veronese 2403

Hands-on with Amazon ElastiCache for Redis
Workshop
At the end of the session, you will learn how to build an end-to-
end real-time apps from ground. MGM, Level 3,
DAT406 R1 2019-12-04 14:30
Premier Ballroom 314
Thank you!
Pratibha Suryadevara, Henry Yang
suryadp@amazon.com, hyang@lyft.com

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.