ATC303-Cache Me If You Can Minimizing Latency While Optimizing Cost Through Advanced Caching Strategies.pdf

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
Cache Me If You Can
M i n i m i z i n g L a t e n c y W h i l e O p t i m i z i n g C o s t
Through Advanced Caching Strategies
M a r k u s O s t e r t a g , V P E n g i n e e r i n g , T e a m I n t e r n e t , @ o s t e r j o u r
C o n s t a n t i n G o n z a l e z , P r i n c i p a l S o l u t i o n s A r c h i t e c t , A W S , @ z a l e z
A T C 3 0 3
N o v e m b e r 2 7 , 2 0 1 7

What you’ll get out of this session
• Speed!
• Best practices
• Architecture patterns
• Real-world examples from
Team Internet
• Lower cost

AdTech
• Publishers, advertisers
• Bidding
• Limited time
• Hard cutoff
• Typical: x00 ms
Photo by Antonio Barroro on Unsplash© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

It‘s all about speed!
Photo by Jakub Gorajek on Unsplash© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Three ways to become faster
1.Increase rate
2.Parallelize
3.Do less

1.Increase rate Complex
2.Parallelize
3.Do less

2.Parallelize Complex
3.Do less

2.Parallelize Complex
3.Do less Easy!

1.Increase rate
2.Parallelize
3.Do less Caching

Caching
•Do time-consuming stuff once
•Re-use results many times
Because memory is cheaper and faster
than CPU

• Edge
• Web tier
• App tier
• Database
Photo by Annie Spratt on Unsplash© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Your app
Database App Web ClientEdge

Edge caching
Database App Web ClientEdgeEdge

Amazon CloudFront
• 107 Edge locations
• 55 cities
• 24 countries
• Static and
dynamic content
• Optimized last-mile
connectivity
• Lower cost than
directly from region
• PCI DSS compliant

Markus Ostertag
V P o f E n g i n e e r i n g , T e a m I n t e r n e t A G

Who is Team Internet?
• Domain monetization business
• 35 people
• HQ in Munich, Germany
• Tech focused

TONIC.

The challenge
•Need to support clients worldwide
•Need latency of <300 ms
•Need 100% consistency at the DB level
Should we go multi-region?
Or is there an alternative?
?

Multiple regions
EU
AZAZ
US
AZAZ
AP
AZAZ
Route 53
Some kind of syncing
between the regions
ELB ELB ELB

CloudFront dynamic content acceleration
AZAZ
CloudFront
EU APUS
ELB
US

Lambda@Edge
Available in all
Amazon CloudFront
edge locations
• Low-latency
request/response
customization
• Supports viewer
and origin events

CloudFront triggers for
Lambda@Edge functions
Origin
server
End user CloudFront
cache
Viewer response Origin response
Viewer request Origin request

Lambda@Edge use cases
Content
customization
Visitor
validation
A/B testing

Edge caching recap
• Use CloudFront for
reducing last-mile
latency
• Almost always a win!
• Can save cost, too
• Consider Lambda@Edge

ClientEdge
Web tier caching
Database App Web

ClientEdge
Web tier caching
Database App WebWeb Cache

Popular web cache solutions
• Varnish
• Nginx
• Apache with mod_cache/mod_proxy
• Squid
• Language/framework caches (APC, Zend)
• Cache in-memory
• Choose high RAM instances, like the R4 family

Amazon API Gateway: Serverless APIs
Internet
Mobile
apps
Websites
Partner
Services
AWS Lambda
functions
API
Gateway
response
cache
Endpoints on
Amazon EC2
Any publicly
accessible
endpoint
Amazon
CloudWatch
Amazon
CloudFront
API
Gateway

Web cache tips
• Cache all static content
• Cache logged out users
• Look for heavy hitters in your logs
• Monitor hit/miss ratios
• Choose TTLs wisely:
• Don‘t affect deployments
• Even small TTLs (for example, 60 seconds) help

Web caching recap
• Web caching makes sense, even
if you already use CloudFront
• Amazon API Gateway comes
with built-in caching: Easy
• Choose TTLs wisely

ClientEdge
App tier caching
Database App Web

What to cache
• Sessions
• Results
• Aggregations
• Templates
• Environments
• Configurations
In short: Everything!

Everything counts (in larger amounts)
• X0,000–X00,000 requests
• Per second
• Even microseconds add up!
• Everything worth caching
• Even for very short times
• 1 ms @ 10K rps
=> 10 app seconds per real
seconds saved
=> 7120 Instance h per month

Monitor everything

Think 80/20
• Find your heavy hitters
• Likely a very small percentage
• Have them in memory, all the time
• It pays off to do special things for
them
Vilfredo Pareto, 1848–1923
Image: sie-ase.org/

RAM
• Every instance type has RAM
• Use it!
• Duplicate data is good
• Pre-load popular data
• Leverage file system cache
• Use your language‘s caching
framework

In-memory key-value store
High-performance
Redis and Memcached
Fully managed; zero admin
Highly available and reliable
Hardened by Amazon
Amazon
ElastiCache
Add more RAM with Amazon ElastiCache
Memcached

Memcached—fast caching
Slab allocator
In-memory key-value datastore
Supports strings, objects
Multi-threaded
Fast!
Very established
No persistence
Open source
Easy to scale
Memcached

Redis—the in-memory leader
Powerful
~200 commands + Lua scripting
In-memory data structure server
Utility data structures
strings, lists, hashes, sets, sorted
sets, bitmaps, and HyperLogLogs
Simple
Atomic operations
supports transactions
Very fast!
Highly available
replication
Persistence
Open source

App caching recap
• Monitor everything!
• Find the 20% of users with 80%
impact
• Cache everything, in RAM!
• Consider using ElastiCache

ClientEdge
Database caching
Database App Web

Caching the traditional way
Amazon
DynamoDB
Amazon
RDS
Amazon
DynamoDB
Amazon
RDS
ElastiCache

Cache invalidation
“There are only two hard things in Computer Science: cache
invalidation and naming things.” –Phil Karlton
Two ways to cache:
• Time to Live (TTL) with invalidation
• Keep the cache in sync all the time

Synchronous writes
Sync/check in the app with after-write return values
ElastiCache
write/update always
write/update always
DynamoDB

After-write return with DynamoDB
var updateDynamo = function(value, key, callback) {
var params = {
Key: {key: {"S": key}},
TableName: "dynamodb_table",
AttributeUpdates: {
Action: "ADD", Value: {N: JSON.stringify(value)}
},
ReturnValues: "UPDATED_NEW"
};
dynamodb.updateItem(params, callback);
};

Compare after-write values
var calls = [
function(callback) {
updateDynamo(value, key, function(err, result) {
callback(err, result.Attributes["val"].N);
});
},
function(callback) { redis.incrbyfloat(key, value, callback); }
];
async.parallel(calls, function(err, result) {
if(Math.abs(parseFloat(result[0]) - parseFloat(result[1])) > 0)
logger.error("diffs @ the result"); // do something against it!
callback(err);
});

Uncoupled writes/invalidation
Sync/updates via AWS Lambda (uncoupled)
Downside: Small delay for cache updates
DynamoDB
or Amazon Aurora
ElastiCache
write/update always
no update
AWS Lambda
DDB Stream
or stored
procedure
update

Amazon DynamoDB Accelerator (DAX)
DynamoDB API compatible:
// If you have this...
var dynamodb = new AWS.DynamoDB();
// change it to
var dynamodb = new AmazonDaxClient({
endpoints: ["your-dax.amazonaws.com"],
region: "us-east-1"
});
Multiple tables with one DAX
cluster possible!
DynamoDB
DAX
DynamoDB

Average ~400–450 µs
Very consistent performance
Warming phase (average ~4500 µs on cold
keys)
No metrics per request
W i t h D A XW i t h o u t D A X
Performance of DAX
Average ~5000–6000 µs
Consistent performance
No warming phase
Detailed metrics per request

Cache hit ratio 25%–30% Cache hit ratio 89%–95%
Cache everything
With negative cachingWithout negative caching
Hit
Miss

Database caching recap
• Cache everything, even negative results!
• Consider cache auto-update with Lambda
• Can combine app with database cache
• Use DAX

Things to do:
• Check out Amazon CloudFront/API Gateway
• Monitor everything
• Understand your top 20% of users
• Consider adding a web cache
• Explore app caches with Amazon ElastiCache
• Use DAX
• Cache everything!

Like d this talk? Me e t u s ag ain!
ARC303–Running Lean Architectures:
How to Optimize for Cost Efficiency
Tu e sday, Nove mbe r 29, 2:30 p.m.
V e ne tian, Le ve l 2, V e ne tian E

Thank you!
m a r k u s @ t e a m i n t e r n e t . c o m
g l e z @ a m a z o n . d e
A T C 3 0 3

ATC303-Cache Me If You Can Minimizing Latency While Optimizing Cost Through Advanced Caching Strategies.pdf

Related slideshows

More Related Content

ATC303-Cache Me If You Can Minimizing Latency While Optimizing Cost Through Advanced Caching Strategies.pdf