Why Scale Matters and How the Cloud is Really Different (at scale)

Editor's Notes

1. Who in the room uses Dropbox? These statistics are from a recent Forbes Magazine article: Dropbox has over 100 million users Syncronising over 1 billion files per day They double their customer base in 12 months All of which is stored in the Cloud Notes: http://www.forbes.com/sites/victoriabarret/2012/11/13/dropbox-hits-100-million-users-says-drew-houston/
2. To do: Add what AWS services they use
3. Let’s take a look at a modern customer example - AirBnB
4. But by September they had more than doubled that, to 9 million customers
5. Background intro: Scaling is a big topic, with lots of opinions, guides, how-tos, and 3rd parties. If you are new to scaling on AWS, you might ask yourself this question: “So how do I scale?”
6. “And if you are like most people, its really hard to know where to start. Again, there are all these resources, twitter based experts, and blog posts preaching to you how to scale”.. “so again, where do we start?”
7. Background intro: Scaling is a big topic, with lots of opinions, guides, how-tos, and 3rd parties. If you are new to scaling on AWS, you might ask yourself this question: “So how do I scale?”
8. If you are like me, you’ll start where I usually start when I want to learn how to do something. A search engine. In this case I’ve gone and searched for “scaling on AWS” using my favorite search engine.
9. It’s important to note something about the results here. First off, there are a lot of things to read. This search was from a few months ago, and there were over half a million posts on how to scale on AWS.
10. Unfortunately for us and our search engine here however, the first response back is actually not what we are looking for. Auto-scaling IS an AWS services, and its great, but its not really the place to start
11. Back to Auto-scaling. “Auto-scaling is a tool and a destination for your infrastructure. It isn’t a single thing. Its not a check-box you can click when launching something. Your infrastructure really has to be built with the right properties in mind for Auto-scaling to work.”.. “So again, where do we start?”
12. What do we need first?
13. We need some basics to lay the foundations we’ll need to build our knowledge of AWS on top of.
14. Over 30 services as represented in groupings in the console
15. Over 30 services, the 29 major services as seen in the console
16. So let’s start from day one, user one, of our new infrastructure and application.
17. This here is the most basic set up you would need to serve up a web application. We have Route53 for DNS, an EC2 instance running our webapp and database, and an Elastic IP attached to the EC2 instance so Route53 can direct traffic to us. Now in scaling this infrastructure, the only real option we have is to get a bigger EC2 instance…
18. Scaling the one EC2 instance we have to a larger one is the most simple approach to start with. There are a lot of different AWS instance types to go with depending on your work load. Some have high I/O, CPU, Memory, or local storage. You can also make use of EBS-Optimized instances and Provisioned IOPs to help scale the storage for this instance quite a bit.
19. The key concern here, is that you WILL hit an endpoint, where we just don’t have a bigger instance class out yet, and so scaling this way while it can get you over an initial hump, really isn’t going to get us very far.
20. The first thing we can do to address the issues of too many eggs in one basket, and to over come the “no bigger boat” problem, is to split out our Webapp and Database into two instances. This gives us more flexibility in scaling these two things independently. And since we are breaking out the Database, this is a great time to think about maybe making use of a database services instead of managing this ourselves…
21. At AWS there are a lot of different options to running databases. One is to just install pretty much any database you can think of on an EC2 instance, and manage all of it yourself. If you are really comfortable doing DBA like activities, like backups, patching, security, tuning, this could be an option for you. If not, then we have a few options that we think are a better idea: First is Amazon RDS, or Relational Database Service. With RDS you get a managed database instance of either MySQL, Oracle, or SQL Server, with features such as automated daily backups, simple scaling, patch management, snapshots and restores, High availability, and read replicas, depending on the engine you go with. Next up we have DynamoDB, a NoSQL database, built ontop of SSDs. DynamoDB is based on the Dynamo whitepaper published by Amazon.com back in 2003, considered the grandfather of most modern NoSQL databases like Cassandra and Riak. DynamoDB that we have here at AWS is kind of like a cousin of the original paper. One of the key concepts to DynamoDB is what we call “Zero Administration”. With DynamoDB the only knobs to tweak are the reads and writes per second you want the DB to be able to perform at. You set it, and it will give you that capacity with query responses averaging in single millisecond. We’ve had customers with loads such as half a million reads and writes per second without DynamoDB even blinking. Lastly we have Amazon Redshift, a multi-petabyte-scale data warehouse service. With Redshift, much like most AWS services, the idea is that you can start small, and scale as you need to, while only paying for what scale you are at. What this means is that you can get 1TB of of data per year at less than a thousand dollars with Redshift. This is several times cheaper than most other dataware house providers costs, and again, you can scale and grow as your business dictates without you needing to sign an expensive contract upfront.
22. Given that we have all these different options, from running pretty much anything you want yourself, to making use of one of the database services AWS provides, how do you choose? How do you decide between SQL and NoSQL?
23. Read as is
24. Read as is
25. So Why start with SQL databases? Generally speaking SQL based databases are established and well worn technology. There’s a good chance SQL is older than most people in this room. It has however continued to power most of the largest web applications we deal with on a daily basis. There are a lot of existing code, books, tools, communities, and people who know and understand SQL. Some of these newer nosql databases might have a handful, tops, of companies using them at scale. You also aren’t going to break SQL databases in your first 10 million users. And yes there is an astrisk here, and we’ll get to that in a second. Lastly, there are a lot of clear patterns for scalability that we’ll discuss a bit through out this talk. So as for my point here at the bottom, I again strongly recommend SQL based technology, unless your application is doing something SUPER weird with the data, or you’ll have MASSIVE amounts of it, even then, SQL will be in your stack.
26. AH HA! You say. I said ‘massive amounts”, and we all assume we’ll have massive amounts, so that means that you must be the lone exclusion in this room… well lets clarify this a bit.
27. If your usage is such that you will be generating several terabytes ( greater than 5) of data in the first year, OR you will have an incredibly data intensive workload, then, you might need NoSQL
28. For those who aren’t familiar yet with ELB( Elastic Load Balancer ), it is a highly scalable load balancing service that you can put infront of tiers of your application where you have multiple instances that you want to share load across. ELB is a really great service, in that it does a lot for you without you having to do much. It will create a self-healing/self-scaling LB that can do things such as SSL termination, handle sticky Sessions, have multiple listeners. It will also do health checks back to the instances behind it, and puts a bunch of metrics into CloudWatch for you as well. This is a key service in building highly available infrastructures on AWS.
29. Read this slide.
30. but its not that efficient in both performance or cost, and since those are important too, let’s clean up this infrastructure a bit.
31. Talk about S3
32. Talk about CloudFront. Make sure to mention the two charts to the right. Static content will certainly speed up your site, but Static&Dynamic content is even better. The chart down below is showing data from a real customer who went from very little traffic, to a huge spike of over 60gigabits per second, without having to do anything on their side, or notify AWS at all.
33. Serving our static assets through CloudFront is going to be a massive performance boost to our end-users, but CloudFront can do much more.
34. Read slide
35. Read slide
36. Talk about auto-scaling.
37. If we add in auto-scaling, our caching layer(both inside, and outside our infrastructure), and the read-replicas with MySQL, we can now handle a pretty serious load. This could potentially even get us into the millions of users by itself if continued to be scaled horizontally and vertically.
38. Discuss lightly pros/cons of each. Elastic Beanstalk is easiest to start with, but offers less control. Opsworks gives you more tools, with a bit more work on your part. CloudFormation is a template driven tool with its own language, so a bit of a learning curve, but very very powerful. Lastly you could do all this manually, but at scale its nearly impossible without a huge team.
39. Read slide.
40. Read slide
41. This diagram is missing the other AZ, but we’ve only got so much room on the slide. But we can see we’ve added in some internal pools for different tasks perhaps. Maybe we’re not using SQS for something, and have SES for sending our out bound email. Again our users will still talk to Route53, and then to CloudFront to get to our site and our content hosted back by our ELB and S3.
42. Read slide
43. So, beyond 10mil
44. Read slide