research-article

Windows Azure Storage: a highly available cloud storage service with strong consistency

SOSP '11: Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles

Pages 143 - 157

https://doi.org/10.1145/2043556.2043571

Published: 23 October 2011 Publication History

Abstract

Windows Azure Storage (WAS) is a cloud storage system that provides customers the ability to store seemingly limitless amounts of data for any duration of time. WAS customers have access to their data from anywhere at any time and only pay for what they use and store. In WAS, data is stored durably using both local and geographic replication to facilitate disaster recovery. Currently, WAS storage comes in the form of Blobs (files), Tables (structured storage), and Queues (message delivery). In this paper, we describe the WAS architecture, global namespace, and data model, as well as its resource provisioning, load balancing, and replication systems.

References

[1]

J. Baker et al., "Megastore: Providing Scalable, Highly Available Storage for Interactive Services," in Conf. on Innovative Data Systems Research, 2011.

[2]

Eric A. Brewer. "Towards Robust Distributed Systems. (Invited Talk)," in Principles of Distributed Computing, Portland, Oregon, 2000.

Digital Library

[3]

M. Burrows, "The Chubby Lock Service for Loosely-Coupled Distributed Systems," in OSDI, 2006.

Digital Library

[4]

F. Chang et al., "Bigtable: A Distributed Storage System for Structured Data," in OSDI, 2006.

Digital Library

[5]

B. Cooper et al., "PNUTS: Yahoo!'s Hosted Data Serving Platform," VLDB, vol. 1, no. 2, 2008.

Digital Library

[6]

G. DeCandia et al., "Dynamo: Amazon's Highly Available Key-value Store," in SOSP, 2007.

Digital Library

[7]

Cristian Estan and George Varghese, "New Directions in Traffic Measurement and Accounting," in SIGCOMM, 2002.

Digital Library

[8]

S. Ghemawat, H. Gobioff, and S. T. Leung, "The Google File System," in SOSP, 2003.

Digital Library

[9]

J. Gray, P. Helland, P. O'Neil, and D. Shasha, "The Dangers of Replication and a Solution," in SIGMOD, 1996.

Digital Library

[10]

Albert Greenberg et al., "VL2: A Scalable and Flexible Data Center Network," Communications of the ACM, vol. 54, no. 3, pp. 95--104, 2011.

Digital Library

[11]

Y. Hu and Q. Yang, "DCD---Disk Caching Disk: A New Approach for Boosting I/O Performance," in ISCA, 1996.

Digital Library

[12]

H. T. Kung and John T. Robinson, "On Optimistic Methods for Concurrency Control," ACM Transactions on Database Systems, vol. 6, no. 2, pp. 213--226, June 1981.

Digital Library

[13]

Leslie Lamport, "The Part-Time Parliament," ACM Transactions on Computer Systems, vol. 16, no. 2, pp. 133--169, May 1998.

Digital Library

[14]

A. Malik and P. Lakshman, "Cassandra: a decentralized structured storage system," SIGOPS Operating System Review, vol. 44, no. 2, 2010.

Digital Library

[15]

M. McKusick and S. Quinlan, "GFS: Evolution on Fast-forward," ACM File Systems, vol. 7, no. 7, 2009.

Digital Library

[16]

S. Mysore, B. Agrawal, T. Sherwood, N. Shrivastava, and S. Suri, "Profiling over Adaptive Ranges," in Symposium on Code Generation and Optimization, 2006.

Digital Library

[17]

P. O'Neil, E. Cheng, D. Gawlick, and E. O'Neil, "The Log-Structured Merge-Tree (LSM-tree)," Acta Informatica - ACTA, vol. 33, no. 4, 1996.

Digital Library

[18]

H. Patterson et al., "SnapMirror: File System Based Asynchronous Mirroring for Disaster Recovery," in USENIX-FAST, 2002.

Digital Library

[19]

Irving S. Reed and Gustave Solomon, "Polynomial Codes over Certain Finite Fields," Journal of the Society for Industrial and Applied Mathematics, vol. 8, no. 2, pp. 300--304, 1960.

[20]

R. Renesse and F. Schneider, "Chain Replication for Supporting High Throughput and Availability," in USENIX-OSDI, 2004.

Digital Library

[21]

J. Terrace and M. Freedman, "Object Storage on CRAQ: High-throughput chain replication for read-mostly workloads," in USENIX'09, 2009.

Digital Library

[22]

D. Terry, K. Petersen M. Theimer, A. Demers, M. Spreitzer, and C. Hauser, "Managing Update Conflicts in Bayou, A Weakly Connected Replicated Storage System," in ACM SOSP, 1995.

Digital Library

[23]

W. Vogel, "All Things Distributed - Choosing Consistency," in http://www.allthingsdistributed.com/2010/02/strong_consistency_simpledb.html, 2010.

Cited By

Zhang WXu EWang QZhang XGu YLu ZOuyang TDai GPeng WXu ZZhang SWu DPeng YWang TZhang HWang JYan WDong YYao WWu ZZhu LShi CWang YLiu RWu JZhu JWu JMa XWon Y(2024)What's the story in EBS gloryProceedings of the 22nd USENIX Conference on File and Storage Technologies10.5555/3650697.3650714(277-292)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.5555/3650697.3650714
Dong HLiu S(2024)Asynchronous Consensus Quorum Read: Pioneering Read Optimization for Asynchronous Consensus ProtocolsElectronics10.3390/electronics1303048113:3(481)Online publication date: 23-Jan-2024
https://doi.org/10.3390/electronics13030481
Eldeeb TBurckhardt SBond RCidon AYang JBernstein P(2024)Cloud Actor-Oriented Database Transactions in OrleansProceedings of the VLDB Endowment10.14778/3685800.368580117:12(3720-3730)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.14778/3685800.3685801
Show More Cited By

Index Terms

Windows Azure Storage: a highly available cloud storage service with strong consistency
1. General and reference
  1. Cross-computing tools and techniques
    1. Measurement
2. Software and its engineering
  1. Software organization and properties

Recommendations

Erasure coding in windows azure storage
USENIX ATC'12: Proceedings of the 2012 USENIX conference on Annual Technical Conference

Windows Azure Storage (WAS) is a cloud storage system that provides customers the ability to store seemingly limitless amounts of data for any duration of time. WAS customers have access to their data from anywhere, at any time, and only pay for what ...
A RAMCloud Storage System based on HDFS: Architecture, implementation and evaluation

Few cloud storage systems can handle random read accesses efficiently. In this paper, we present a RAMCloud Storage System, RCSS, to enable efficient random read accesses in cloud environments. Based on the Hadoop Distributed File System (HDFS), RCSS ...
Middleware enabled data sharing on cloud storage services
MW4SOC '10: Proceedings of the 5th International Workshop on Middleware for Service Oriented Computing

With the emergence of public cloud storage platforms like Amazon, Microsoft and Google etc, individual applications and some enterprise storage are being increasingly deployed on Clouds. However, dynamic data sharing in public clouds face problems of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SOSP '11: Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles

October 2011

417 pages

ISBN:9781450309776

DOI:10.1145/2043556

General Chair:
Ted Wobber
MSR Silicon Valley
,
Program Chair:
Peter Druschel
MPI-SWS

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

INESC: Systems and Computer Engineering Institute
SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SOSP '11

Sponsor:

INESC
SIGOPS

SOSP '11: ACM SIGOPS 23nd Symposium on Operating Systems Principles

October 23 - 26, 2011

Cascais, Portugal

Acceptance Rates

Overall Acceptance Rate 131 of 716 submissions, 18%

Upcoming Conference

SOSP '25

Sponsor:
sigops

ACM SIGOPS 31st Symposium on Operating Systems Principles

October 13 - 16, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

585
Total Citations
View Citations
6,859
Total Downloads

Downloads (Last 12 months)315
Downloads (Last 6 weeks)28

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang WXu EWang QZhang XGu YLu ZOuyang TDai GPeng WXu ZZhang SWu DPeng YWang TZhang HWang JYan WDong YYao WWu ZZhu LShi CWang YLiu RWu JZhu JWu JMa XWon Y(2024)What's the story in EBS gloryProceedings of the 22nd USENIX Conference on File and Storage Technologies10.5555/3650697.3650714(277-292)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.5555/3650697.3650714
Dong HLiu S(2024)Asynchronous Consensus Quorum Read: Pioneering Read Optimization for Asynchronous Consensus ProtocolsElectronics10.3390/electronics1303048113:3(481)Online publication date: 23-Jan-2024
https://doi.org/10.3390/electronics13030481
Eldeeb TBurckhardt SBond RCidon AYang JBernstein P(2024)Cloud Actor-Oriented Database Transactions in OrleansProceedings of the VLDB Endowment10.14778/3685800.368580117:12(3720-3730)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.14778/3685800.3685801
Zhang WChen CWang QWang WYang SZhou BZhu HChen CZhao YHu YCheng MLi MTan HLiu MLin HZhang SZhang LBarcelo PSanchez-Pi NMeliou ASudarshan S(2024)BG3: A Cost Effective and I/O Efficient Graph Database in BytedanceCompanion of the 2024 International Conference on Management of Data10.1145/3626246.3653373(360-372)Online publication date: 9-Jun-2024
https://dl.acm.org/doi/10.1145/3626246.3653373
Zarnstorff JLebow LRemuck DRuiz CTseng LAgrawal KPetrank E(2024)Brief Announcement: Racos: A Leaderless Erasure Coding State Machine ReplicationProceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3626183.3660273(379-381)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3626183.3660273
Copik MCalotoiu AZhou PTaranov KHoefler TMencagli GDazzi PLowenthal DBadia R(2024)FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an ExampleProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658661(94-108)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658661
Li QXu LLi YLyu MWang WZuo PXu Y(2024)Enabling Efficient Erasure Coding in Disaggregated Memory SystemsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.333278235:1(154-168)Online publication date: Jan-2024
https://doi.org/10.1109/TPDS.2023.3332782
Sun RFan YLiu J(2024)A Repair-Efficient Piggybacking Design With Small Number of SubstripesIEEE/ACM Transactions on Networking10.1109/TNET.2023.332856032:2(1573-1583)Online publication date: Apr-2024
https://doi.org/10.1109/TNET.2023.3328560
Liu JDragojević AFleming SKatsarakis AKorolija DZablotchi INg HKalia ACastro M(2024)Honeycomb: Ordered Key-Value Store Acceleration on an FPGA-Based SmartNICIEEE Transactions on Computers10.1109/TC.2023.334517373:3(857-871)Online publication date: Mar-2024
https://doi.org/10.1109/TC.2023.3345173
Sheldon JZhu WAbdullah ABhupathiraju SSugawara TButler KIslam MRampazzi S(2024)AquaSonic: Acoustic Manipulation of Underwater Data Center Operations and Resource Management2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00201(331-349)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00201
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents