research-article

COSBench: cloud object storage benchmark

Authors:

Jiangang DuanAuthors Info & Claims

ICPE '13: Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering

Pages 199 - 210

https://doi.org/10.1145/2479871.2479900

Published: 21 April 2013 Publication History

Abstract

With object storage systems being increasingly recognized as a preferred way to expose one's storage infrastructure to the web, the past few years have witnessed an explosion in the acceptance of these systems. Unfortunately, the proliferation of available solutions and the complexity of each individual one, coupled with a lack of dedicated workload, makes it very challenging for one to evaluate and tune the performance of different systems. To help address this problem, we present the Cloud Object Storage Benchmark (COSBench). It is a benchmark tool that we have developed at Intel with the goal of facilitating both performance comparison and system optimization of these systems. In this paper, we describe the design and implementation of this tool, focusing on its extensibility and scalability. In addition, we discuss how people can use this tool to perform system characterization and how the latter can facilitate system comparison and optimization. To demonstrate the value of our tool, we report the results of our experiments conducted on two Swift setups we built in our lab. We also share some of our experiences in turning our setups to achieve higher performance.

References

[1]

905 billion objects and 650,000 requests/second. http://aws.typepad.com/aws.

[2]

Amazon s3. http://aws.amazon.com/s3.

[3]

Cdmi. http://www.snia.org/cdmi.

[4]

Equinox. http://www.eclipse.org/equinox.

[5]

Eucalyptus. http://www.eucalyptus.com.

[6]

Google cloud storage. http://www.google.com/enterprise/cloud.

[7]

Keystone. http://docs.openstack.org/developer/keystone.

[8]

Nimbus. http://www.nimbusproject.org.

[9]

Oauth. http://oauth.net.

[10]

Openstack. www.openstack.org.

[11]

Osgi. http://www.osgi.org/Main/HomePage.

[12]

Rackspace cloud files. http://www.rackspace.com/cloud.

[13]

Saml. https://www.oasis-open.org/committees/security.

[14]

Swauth. https://github.com/gholt/swauth.

[15]

Swift bench. https://github.com/openstack/swift.

[16]

N. Agrawal, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. Generating realistic impressions for file-system benchmarking. In FAST, 2009.

Digital Library

[17]

E. Anderson, M. Kallahalla, M. Uysal, and R. Swaminathan. Buttress: A toolkit for flexible and high fidelity i/o benchmarking. In FAST, 2004.

Digital Library

[18]

D. Beaver, S. Kumar, H. C. Li, J. Sobel, and P. Vajgel. Finding a needle in haystack: facebook's photo storage. In OSDI, 2010.

Digital Library

[19]

M. Brantner, D. Florescu, D. Graf, D. Kossmann, and T. Kraska. Building a database on s3. In SIGMOD, 2008.

Digital Library

[20]

B. Calder, J. Wang, A. Ogus, N. Nilakantan, A. Skjolsvold, S. McKelvie, Y. Xu, S. Srivastav, J. Wu, H. Simitci, J. Haridas, C. Uddaraju, H. Khatri, A. Edwards, V. Bedekar, S. Mainali, R. Abbasi, A. Agarwal, M. F. u. Haq, M. I. u. Haq, D. Bhardwaj, S. Dayanand, A. Adusumilli, M. McNett, S. Sankaran, K. Manivannan, and L. Rigas. Windows azure storage: a highly available cloud storage service with strong consistency. In SOSP, 2011.

Digital Library

[21]

J. Chen, C. Douglas, M. Mutsuzaki, P. Quaid, R. Ramakrishnan, S. Rao, and R. Sears. Walnut: a unified cloud object store. In SIGMOD, 2012.

Digital Library

[22]

B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking cloud serving systems with ycsb. In SoCC, 2010.

Digital Library

[23]

C. Dubnicki, L. Gryz, L. Heldt, M. Kaczmarczyk, W. Kilian, P. Strzelczak, J. Szczepkowski, C. Ungureanu, and M. Welnicki. Hydrastor: a scalable secondary storage. In FAST, 2009.

Digital Library

[24]

C. Huang, H. Simitci, Y. Xu, A. Ogus, B. Calder, P. Gopalan, J. Li, and S. Yekhanin. Erasure coding in windows azure storage. In USENIX ATC, 2012.

Digital Library

[25]

A. W. Leung, S. Pasupathy, G. Goodson, and E. L. Miller. Measurement and analysis of large-scale network file system workloads. In USENIX ATC, 2008.

Digital Library

[26]

D. Roselli, J. R. Lorch, and T. E. Anderson. A comparison of file system workloads. In USENIX ATC, 2000.

Digital Library

[27]

C. Ungureanu, B. Atkin, A. Aranya, S. Gokhale, S. Rago, G. Calkowski, C. Dubnicki, and A. Bohra. Hydrafs: a high-throughput file system for the hydrastor content-addressable storage system. In FAST, 2010.

Digital Library

[28]

M. Vrable, S. Savage, and G. M. Voelker. Bluesky: a cloud-backed file system for the enterprise. In FAST, 2012.

Digital Library

[29]

S. A. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long, and C. Maltzahn. Ceph: a scalable, high-performance distributed file system. In OSDI, 2006.

Digital Library

[30]

S. A. Weil, A. W. Leung, S. A. Brandt, and C. Maltzahn. Rados: a scalable, reliable storage service for petabyte-scale storage clusters. In PDSW, 2007.

Digital Library

[31]

N. Yezhkova, R. L. Villars, L. Conner, and B. Woo. Worldwide enterprise storage systems 20102014 forecast: Recovery, efficiency, and digitization shaping customer requirements for storage systems. IDC, 2010.

[32]

Q. Zheng, H. Chen, Y. Wang, J. Duan, and Z. Huang. Cosbench: A benchmark tool for cloud object storage services. In CLOUD, 2012.

Digital Library

Cited By

Sajal SZhu TUrgaonkar BSen S(2024)TraceUpscaler: Upscaling Traces to Evaluate Systems at High LoadProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3629581(942-961)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3629581
Jackowski AŚlusarczyk ŁLichota KWełnicki MWijata RKielar MKopeć TDubnicki CIwanicki K(2023)ObjDedup: High-Throughput Object Storage Layer for Backup Systems With Block-Level DeduplicationIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.325050134:7(2180-2197)Online publication date: Jul-2023
https://doi.org/10.1109/TPDS.2023.3250501
Bhimani JMaruf AMi NPandurangan RBalakrishnan V(2022)Auto-Tuning Parameters for Emerging Multi-Stream Flash-Based Storage Drives Through New I/O Pattern GenerationsIEEE Transactions on Computers10.1109/TC.2020.304830371:2(309-322)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TC.2020.3048303
Show More Cited By

Index Terms

COSBench: cloud object storage benchmark
1. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results

Recommendations

File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution
SOSP '19: Proceedings of the 27th ACM Symposium on Operating Systems Principles

For a decade, the Ceph distributed file system followed the conventional wisdom of building its storage backend on top of local file systems. This is a preferred choice for most distributed file systems today because it allows them to benefit from the ...
COSBench: A Benchmark Tool for Cloud Object Storage Services
CLOUD '12: Proceedings of the 2012 IEEE Fifth International Conference on Cloud Computing

With object storage services becoming increasingly accepted as replacements for traditional file or block systems, it is important to effectively measure the performance of these services. Thus people can compare different solutions or tune their ...
Hierarchical Storage from NVMe to Tapes
EMOSS '22: Proceedings of the 2022 Workshop on Emerging Open Storage Systems and Solutions for Data Intensive Computing

Hierarchical storage allows to build both high-performance and high-capacity storage systems by combining various technologies like NVMe, SSDs, HDDs, or tapes.

While most existing systems only manage a limited number of levels and technologies, the IO-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICPE '13: Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering

April 2013

446 pages

ISBN:9781450316361

DOI:10.1145/2479871

Editor:
Seetharami Seelam
IBM T.J. Watson Research Center, USA
,
General Chairs:
Petr Tůma
Charles University, Czech Republic
,
Giuliano Casale
Imperial College London, UK
,
Program Chairs:
Tony Field
Imperial College London, UK
,
José Nelson Amaral
University of Alberta, Canada

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMETRICS: ACM Special Interest Group on Measurement and Evaluation
SIGSOFT: ACM Special Interest Group on Software Engineering
SPEC: SPEC Research Group

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICPE'13

Sponsor:

ICPE'13: ACM/SPEC International Conference on Performance Engineering

April 21 - 24, 2013

Prague, Czech Republic

Acceptance Rates

ICPE '13 Paper Acceptance Rate 28 of 64 submissions, 44%;

Overall Acceptance Rate 252 of 851 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
651
Total Downloads

Downloads (Last 12 months)52
Downloads (Last 6 weeks)2

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sajal SZhu TUrgaonkar BSen S(2024)TraceUpscaler: Upscaling Traces to Evaluate Systems at High LoadProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3629581(942-961)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3629581
Jackowski AŚlusarczyk ŁLichota KWełnicki MWijata RKielar MKopeć TDubnicki CIwanicki K(2023)ObjDedup: High-Throughput Object Storage Layer for Backup Systems With Block-Level DeduplicationIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.325050134:7(2180-2197)Online publication date: Jul-2023
https://doi.org/10.1109/TPDS.2023.3250501
Bhimani JMaruf AMi NPandurangan RBalakrishnan V(2022)Auto-Tuning Parameters for Emerging Multi-Stream Flash-Based Storage Drives Through New I/O Pattern GenerationsIEEE Transactions on Computers10.1109/TC.2020.304830371:2(309-322)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TC.2020.3048303
Umare ABakhtawar Ahmed M(2021)Cloud Storage : A Way to Modify Data Storage in AdvanceInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology10.32628/CSEIT217137(182-187)Online publication date: 2-Feb-2021
https://doi.org/10.32628/CSEIT217137
Qin MFu Z(2021)A Heterogeneous Hybrid Storage Method Based on Ceph Erasure Code2021 Asia-Pacific Conference on Communications Technology and Computer Science (ACCTCS)10.1109/ACCTCS52002.2021.00044(182-186)Online publication date: Jan-2021
https://doi.org/10.1109/ACCTCS52002.2021.00044
Kang GKong DWang LZhan J(2021)OStoreBench: Benchmarking Distributed Object Storage Systems Using Real-World Application ScenariosBenchmarking, Measuring, and Optimizing10.1007/978-3-030-71058-3_6(90-105)Online publication date: 2-Mar-2021
https://doi.org/10.1007/978-3-030-71058-3_6
Kashiwazaki HTakakura HShimojo S(2020)A Quantitative Evaluation of a Wide-Area Distributed System with SDN-FIT2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC48688.2020.0-189(607-612)Online publication date: Jul-2020
https://doi.org/10.1109/COMPSAC48688.2020.0-189
Yoon SKim S(2020)Pattern analysis based data management method and memory-disk integrated system for high performance computingFuture Generation Computer Systems10.1016/j.future.2020.01.013Online publication date: Jan-2020
https://doi.org/10.1016/j.future.2020.01.013
Costa Filho JCavalcante DMoreira LMachado J(2020)An adaptive replica placement approach for distributed key‐value storesConcurrency and Computation: Practice and Experience10.1002/cpe.567532:11Online publication date: 10-Feb-2020
https://doi.org/10.1002/cpe.5675
Imran MDurad MKhan FDerhab A(2019)Reducing the effects of DoS attacks in software defined networks using parallel flow installationHuman-centric Computing and Information Sciences10.1186/s13673-019-0176-79:1(1-19)Online publication date: 1-Dec-2019
https://dl.acm.org/doi/10.1186/s13673-019-0176-7
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents