Cloud storage with AWS
- 1. Cloud Storage with AWS: EFS vs EBS vs S3 AHMAD KARAWASH
- 2. Cloud Storage with AWS Cloud storage is a critical component of cloud computing, holding the information used by applications. Big data analytics, data warehouses, Internet of Things, databases, and backup and archive applications all rely on some form of data storage architecture.
- 3. AWS Cloud Storage Products Amazon Simple Storage Service (Amazon S3) Amazon Elastic File System (Amazon EFS) Amazon Elastic Block Storage (Amazon EBS) Amazon Glacier AWS Storage Gateway Cloud Data Migration Services
- 4. Amazon Simple Storage Service (Amazon S3) Object storage, to make data available through an API that can be accessed anywhere. It is secure, durable, and highly scalable, and applications can access storage directly. Amazon S3 is used for content storage and distribution, large-scale web development, data lakes for analytics workloads, and cloud-native applications going directly across the web.
- 5. Amazon Elastic File System (Amazon EFS) File storage, to make data available to one or more EC2 instances via a file system interface and file system access semantics. It delivers a file system interface with standard file system access semantics for Amazon EC2 instances. Amazon EFS grows and shrinks capacity automatically, and provides high throughput with consistently low latencies. Amazon EFS is ideal for Big Data analytics, media processing workflows, content management, web serving, and home directories.
- 6. Amazon Elastic Block Storage (Amazon EBS) Block storage, to make data available as a virtual volume attached to a single Amazon EC2 compute instance. Amazon EBS is designed for workloads that require persistent storage accessible by single EC2 instances. Typical use cases include boot volumes, transactional and NoSQL databases (like Microsoft Exchange, Cassandra and MongoDB), Big Data analytics platforms (like Hadoop, Amazon EMR, and HortonWorks), stream and log processing applications (like Kafka and Splunk), and data warehousing applications (like Vertica and Cassandra).
- 7. Amazon Glacier Highly affordable long-term storage that can replace tape for archive and regulatory compliance. Amazon Glacier is a solution for customers who want low-cost storage for infrequently accessed data. It can replace tape while assisting with compliance in highly regulated organizations like healthcare, life science, and financial services.
- 8. AWS Storage Gateway A software appliance that links your on-premises storage environment to Amazon cloud storage. It provides low-latency access to frequently accessed data and maintains durable remote volumes, to help you match the value of your storage with the value of your data. Use it with existing backup and recovery applications to replace tape automation without disrupting existing processes, or supplement “bursty” on- premises workloads with storage capacity on demand.
- 9. Cloud Data Migration Services A portfolio of services to help simplify and accelerate moving data of all types and sizes into and out of the AWS cloud. These services help you do things like securely and quickly move multi-petabyte archives, capture continuous streaming data from multiple sources, and accelerate network transfers on existing infrastructure.
- 10. Differences: EFS vs EBS vs S3 vs Glacier Why would I use EBS over EFS? They have the same use cases but minor semantic differences? Why choose S3 over EFS? They both store files, scale and are replicated.
- 11. Differences: EFS Generally Available (out of preview), but may not yet be available in your region Network filesystem (that means it may have bigger latency but it can be shared across several instances; even between regions) It is expensive compared to EBS (~10x more) but it gives extra features. It's a highly available service. It's a managed service You need an EC2 instance to attach it to Can be accessed by multiple EC2 instances simultaneously
- 12. Differences: EBS A block storage (so you need to format it). This means you are able to choose which type of file system you want. As it's a block storage, you can use Raid 1 (or 0 or 10) with multiple block storages It is really fast It is relatively cheap With the new announcements from Amazon, you can store up to 16TB data per storage on SSD-s. You can snapshot an EBS (while it's still running) for backup reasons But it only exists in a particular region. Although you can migrate it to another region, you cannot just access it across regions (only if you share it via the EC2; but that means you have a file server) You need an EC2 instance to attach it to
- 13. Differences: S3 An object store (not a file system). You can store files and "folders" but can't have locks, permissions etc like you would with a traditional file system This means, by default you can't just mount S3 and use it as your webserver But it's perfect for storing your images and videos for your website Great for short term archiving (e.g. a few weeks). It's good for long term archiving too, but Glacier is more cost efficient. Great for storing logs You can access the data from every region (extra costs may apply) Highly Available, Redundant. Basically data loss is not possible (99.999999999% durability, 99.9 uptime SLA) Much cheaper than EBS. You can serve the content directly to the internet, you can even have a full (static) website working direct from S3, without an EC2 instance
- 14. Differences: Glacier Long term archive storage Extremely cheap to store Potentially very expensive to retrieve Takes up to 4 hours to "read back" your data (so only store items you know you won't need to retrieve for a long time)
- 15. Comparison of Cost Cost Glacier: $0.007/Month S3: $0.03/Month S3-IA (announced in 2015.09): $0.0125/Month (+$0.01/gig retrieval charge) EBS: $0.05-0.1/Month (depends on speed - SSD or not) + IOPS costs EFS: $0.3/Month