research-article

Building a database on S3

Authors:

Matthias Brantner,

Daniela Florescu,

Donald Kossmann,

Tim KraskaAuthors Info & Claims

SIGMOD '08: Proceedings of the 2008 ACM SIGMOD international conference on Management of data

Pages 251 - 264

https://doi.org/10.1145/1376616.1376645

Published: 09 June 2008 Publication History

Abstract

There has been a great deal of hype about Amazon's simple storage service (S3). S3 provides infinite scalability and high availability at low cost. Currently, S3 is used mostly to store multi-media documents (videos, photos, audio) which are shared by a community of people and rarely updated. The purpose of this paper is to demonstrate the opportunities and limitations of using S3 as a storage system for general-purpose database applications which involve small objects and frequent updates. Read, write, and commit protocols are presented. Furthermore, the cost ($), performance, and consistency properties of such a storage system are studied.

References

[1]

M. Atwood. A storage engine for Amazon S3. MySQL Conference and Expo, 2007. http://fallenpegasus.com/code/mysql-awss3.

[2]

R. Bayer and M. Schkolnick. Concurrency of operations on B-trees. Acta Inf., 9(1):1--21, 1977.

Digital Library

[3]

H. Berenson et al. A critique of ANSI SQL isolation levels. In Proc. of ACM SIGMOD, pages 1--10, Jun 1995.

Digital Library

[4]

P. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency Control and Recovery in Database Systems. Addison Wesley, Reading, MA, 1987.

Digital Library

[5]

A. Biliris. The performance of three database storage structures for managing large objects. In Proc. of ACM SIGMOD, pages 276--285, Jun 1992.

Digital Library

[6]

D. Brunner. Scalability: Set Amazon?s servers on fire, not yours. Talk at ETech Conf., 2007. http://blogs.smugmug.com/don/files/ETech-SmugMug-Amazon-2007.pdf.

[7]

G. DeCandia et al. Dynamo: Amazon?s highly available key-value store. In Proc. of SOSP, pages 205--220, Oct 2007.

Digital Library

[8]

I. Foster and C. Kesselman, editors. The Grid 2: Blueprint for a new computing infrastructure. Elsevier, Amsterdam, 2004.

Digital Library

[9]

S. Garfinkel. An evaluation of Amazon?s grid computing services: EC2, S3, and SQS. Technical Report TR-08-07, Harvard University, 2007.

[10]

S. Gilbert and N. Lynch. Brewer?s conjecture and the feasibility of consistent, available, partition-tolerant Web services. SIGACT News, 33(2):51--59, 2002.

Digital Library

[11]

J. Gray and A. Reuter. Transaction Processing: Concepts and Techniques. Morgan Kaufmann, San Mateo, CA, 1994.

Digital Library

[12]

J. Hellerstein. Architectures and algorithms for interent-scale (P2P) data management. In Proc. of VLDB, page 1244, Aug 2004.

Digital Library

[13]

P. Lehman and B. Yao. Efficient locking for concurrent operations on B-trees. ACM TODS, 6(4):650--670, 1981.

Digital Library

[14]

D. Lomet. Replicated indexes for distributed data. In Proc. of PDIS, pages 108--119, Dec 1996.

Digital Library

[15]

M. McAuliffe, M. Carey, and M. Solomon. Towards effective and efficient free space management. In Proc. of ACM SIGMOD, pages 389--400, Jun 1996.

Digital Library

[16]

RightScale LLC. Redundant MySQL set-up for Amazon EC2, November 2007. http://info.rightscale.com/2007/8/20/redundant-mysql.

[17]

W. Ross and G. Westerman. Preparing for utility computing: The role of IT architecture and relationship management. IBM Systems Journal, 43(1):5--19, 2004.

Digital Library

[18]

M. Stonebraker. The case for shared nothing. IEEE Data Eng. Bulletin, 9(1):4--9, 1986.

[19]

A. Tanenbaum and M. van Steen. Distributed Systems: Principles and Paradigms. Prentice Hall, Upper Saddle River, NJ, 2002.

Digital Library

[20]

TPC. TPC Benchmark W. Specification Version 1.8 of TPC Council, 2002.

[21]

W. Vogels. Data access patterns in the Amazon.com technology platform. In Proc. of VLDB, page 1, Sep 2007.

Digital Library

[22]

G. Weikum and G. Vossen. Transactional Information Systems. Morgan Kaufmann, San Mateo, CA, 2002.

Digital Library

Cited By

Copik MCalotoiu AZhou PTaranov KHoefler TMencagli GDazzi PLowenthal DBadia R(2024)FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an ExampleProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658661(94-108)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658661
Guo YWang ZXue JShao Z(2024)A Spatio-Temporal Series Data Model with Efficient Indexing and Layout for Cloud-Based Trajectory Data Management2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00313(1171-1184)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00313
Durner DLeis VNeumann T(2023)Exploiting Cloud Object Storage for High-Performance AnalyticsProceedings of the VLDB Endowment10.14778/3611479.361148616:11(2769-2782)Online publication date: 24-Aug-2023
https://dl.acm.org/doi/10.14778/3611479.3611486
Show More Cited By

Index Terms

Building a database on S3

Recommendations

MATE-EC2: a middleware for processing data with AWS
MTAGS '11: Proceedings of the 2011 ACM international workshop on Many task computing on grids and supercomputers

Recently, there has been growing interest in using Cloud resources for a variety of high performance and data-intensive applications. While there is currently a number of commercial Cloud service providers, Amazon Web Services (AWS) appears to be the ...
DevOps patterns to scale web applications using cloud services
SPLASH '13: Proceedings of the 2013 companion publication for conference on Systems, programming, & applications: software for humanity

Scaling a web applications can be easy for simple CRUD software running when you use Platform as a Service Clouds (PaaS). But if you need to deploy a complex software, with many components and a lot users, you will need have a mix of cloud services in ...
WhoWas: A Platform for Measuring Web Deployments on IaaS Clouds
IMC '14: Proceedings of the 2014 Conference on Internet Measurement Conference

Public infrastructure-as-a-service (IaaS) clouds such as Amazon EC2 and Microsoft Azure host an increasing number of web services. The dynamic, pay-as-you-go nature of modern IaaS systems enable web services to scale up or down with demand, and only pay ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGMOD '08: Proceedings of the 2008 ACM SIGMOD international conference on Management of data

June 2008

1396 pages

ISBN:9781605581026

DOI:10.1145/1376616

General Chairs:
Laks V. S. Lakshmanan
University of British Columbia, Canada
,
Raymond T. Ng
University of British Columbia, Canada
,
Dennis Shasha
New York University, USA

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 June 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGMOD/PODS '08

Sponsor:

SIGMOD/PODS '08: SIGMOD/PODS '08 - International Conference on Management of Data

June 9 - 12, 2008

Vancouver, Canada

Acceptance Rates

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

184
Total Citations
View Citations
4,722
Total Downloads

Downloads (Last 12 months)143
Downloads (Last 6 weeks)7

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Copik MCalotoiu AZhou PTaranov KHoefler TMencagli GDazzi PLowenthal DBadia R(2024)FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an ExampleProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658661(94-108)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658661
Guo YWang ZXue JShao Z(2024)A Spatio-Temporal Series Data Model with Efficient Indexing and Layout for Cloud-Based Trajectory Data Management2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00313(1171-1184)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00313
Durner DLeis VNeumann T(2023)Exploiting Cloud Object Storage for High-Performance AnalyticsProceedings of the VLDB Endowment10.14778/3611479.361148616:11(2769-2782)Online publication date: 24-Aug-2023
https://dl.acm.org/doi/10.14778/3611479.3611486
Wang ZShao Z(2023)MirrorKV: An Efficient Key-Value Store on Hybrid Cloud Storage with Balanced Performance of Compaction and QueryingProceedings of the ACM on Management of Data10.1145/36267361:4(1-27)Online publication date: 12-Dec-2023
https://dl.acm.org/doi/10.1145/3626736
Kraska T(2023)Technical Perspective for Sherman: A Write-Optimized Distributed B+Tree Index on Disaggregated MemoryACM SIGMOD Record10.1145/3604437.360444752:1(44-44)Online publication date: 8-Jun-2023
https://dl.acm.org/doi/10.1145/3604437.3604447
Kang JTan SHe BZhang ZSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Real Time Index and Search Across Large Quantities of GNN Experts for Low Latency Online LearningProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599893(4308-4319)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599893
Baek HLee RJung SKim JPark S(2023)Real-Time High-Quality Visualization for Volumetric Contents Rendering: A Lyapunov Optimization FrameworkIEEE Open Journal of the Computer Society10.1109/OJCS.2023.33123714(243-252)Online publication date: 2023
https://doi.org/10.1109/OJCS.2023.3312371
Rashmi Ashtagi Jyoti R. Maranur Afsha Akkalkot Shweta M Madiwal Sridevi Hosmani Arunadevi Khaple (2022)Competitive Analysis for the Auditing Cloud ConsistencyInternational Journal of Scientific Research in Science and Technology10.32628/IJSRST229396(408-414)Online publication date: 1-Jun-2022
https://doi.org/10.32628/IJSRST229396
Guo ZZeng XWu KHwang WRen ZYu XBalakrishnan MBernstein P(2022)CornusProceedings of the VLDB Endowment10.14778/3565816.356583716:2(379-392)Online publication date: 23-Nov-2022
https://dl.acm.org/doi/10.14778/3565816.3565837
Berenberg ACalder B(2022)Deployment Archetypes for Cloud ApplicationsACM Computing Surveys10.1145/349833655:3(1-48)Online publication date: 3-Feb-2022
https://dl.acm.org/doi/10.1145/3498336
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents