research-article

Towards distributed storage resource management using flow control

Authors:

Irfan AhmadAuthors Info & Claims

ACM SIGOPS Operating Systems Review, Volume 42, Issue 6

Pages 10 - 16

https://doi.org/10.1145/1453775.1453779

Published: 01 October 2008 Publication History

Abstract

Deployment of shared storage systems is increasing with rapid adoption of virtualization technologies to provide flexible sharing, isolation, better management and high utilization of resources. Quality of service (QoS) in such environments is quite desirable for meeting IO demands of virtual machines. The lack of QoS support at typical storage arrays, simultaneous access by multiple hosts and concerns regarding under-utilization of resources makes this problem quite challenging. In this paper, we study the problem of providing fairness among hosts accessing a storage array in a distributed manner while maintaining high efficiency. Towards this goal, we investigate whether local latency estimates at each host can be used to detect overload and whether limiting host issue queue lengths can provide fairness across hosts. In principle, the approach is similar to mechanisms used by TCP at each host for flow control. Initial experiments and simulation results for control mechanism provide encouragement to develop a complete framework.

References

[1]

Iometer. http://www.iometer.org.

[2]

Ajay Gulati and Irfan Ahmad. Towards distributed storage resource management using flow control. In SPEED (held in conjunction with HPCA), Feb 2008.

[3]

J. C. R. Bennett and H. Zhang. WF2Q: Worst-case fair weighted fair queueing. In Proc. of INFOCOM '96, pages 120--128, March 1996.

Digital Library

[4]

D. D. Chambliss, G. A. Alvarez, P. Pandey, D. Jadav, J. Xu, R. Menon, and T. P. Lee. Performance virtualization for large-scale storage systems. In Symposium on Reliable Distributed Systems, pages 109--118, Oct 2003.

[5]

P. M. Chen, E. K. Lee, G. A. Gibson, R. H. Katz, and D. A. Patterson. RAID: High-performance, reliable secondary storage. ACM Computing Surveys, 26(2), 1994.

Digital Library

[6]

A. Demers, S. Keshav, and S. Shenker. Analysis and simulation of a fair queuing algorithm. Journal of Internetworking Research and Experience, 1(1):3--26, September 1990.

[7]

S. Floyd and V. Jacobson. Random early detection gateways for congestion avoidance. IEEE/ACM Transactions on Networking, 1(4):397--413, 1993.

Digital Library

[8]

P. Goyal, H. M. Vin, and H. Cheng. Start-time fair queuing: A scheduling algorithm for integrated services packet switching networks. Technical Report CS-TR-96-02, UT Austin, January 1996.

Digital Library

[9]

A. Gulati, A. Merchant, and P. Varman. dClock: Distributed QoS in heterogeneous resource environments. In Proc. of ACM PODC (short paper), August 2007.

Digital Library

[10]

A. Gulati, A. Merchant, and P. Varman. pClock: An arrival curve based approach for QoS in shared storage systems. In Proc. of ACM SIGMETRICS, pages 13--24, June 2007.

Digital Library

[11]

L. Huang, G. Peng, and T. cker Chiueh. Multi-dimensional storage virtualization. In ACM SIGMETRICS, June 2004.

Digital Library

[12]

I. Ahmad et. al. An Analysis of Disk Performance in VMware ESX Server Virtual Machines. In IEEE Int. Workshop on Workload Characterization (WWC-6), Oct 2003.

[13]

C. Jin, D. Wei, and S. Low. FAST TCP: Motivation, Architecture, Algorithms, Performance. Proceedings of IEEE INFOCOM, March 2004.

[14]

W. Jin, J. S. Chase, and J. Kaur. Interposed proportional sharing for a storage service utility. In ACM SIGMETRICS, pages 37--48, June 2004.

Digital Library

[15]

J. R. Jump. Yacsim reference manual. http://www.owlnet.rice.edu/ elec428/yacsim/yacsim.man.ps.

[16]

M. Karlsson, C. Karamanolis, and X. Zhu. Triage: Performance differentiation for storage systems using adaptive control. Trans. Storage, 1(4):457--480, 2005.

Digital Library

[17]

C. Lumb, A. Merchant, and G. Alvarez. Façade: Virtual storage devices with performance guarantees. USENIX FAST, March 2003.

Digital Library

[18]

I. Stoica. Stateless core: A scalable approach for quality of service in the Internet, 2000.

Digital Library

[19]

VMware, Inc. Introduction to VMware Infrastructure. 2007. http://www.vmware.com/support/pubs/.

[20]

M. Wachs, M. Abd-El-Malek, E. Thereska, and G. R. Ganger. Argon: performance insulation for shared storage servers. In FAST'07, 2007.

Digital Library

[21]

Y. Wang and A. Merchant. Proportional-share scheduling for distributed storage systems. In Proc. of FAST, Feb 2007.

Digital Library

[22]

Y. Zhang et. al. Friendly virtual machines. In Proc. of VEE, June 2005.

Digital Library

[23]

J. Zhang, A. Sivasubramaniam, Q. Wang, A. Riska, and E. Riedel. Storage performance virtualization via throughput and latency control. In Proc. of MASCOTS, Sep 2005.

Digital Library

Cited By

Jain NLakshmi J(2015)PCOSProceedings of the 2015 International Conference on Cloud Computing and Big Data (CCBD)10.1109/CCBD.2015.24(145-152)Online publication date: 4-Nov-2015
https://dl.acm.org/doi/10.1109/CCBD.2015.24
Skowron PRzadca K(2014)Fair Share Is Not Enough: Measuring Fairness in Scheduling with Cooperative Game TheoryParallel Processing and Applied Mathematics10.1007/978-3-642-55195-6_4(38-48)Online publication date: 8-May-2014
https://doi.org/10.1007/978-3-642-55195-6_4
Costan A(2013)Grid Data HandlingIT Policy and Ethics10.4018/978-1-4666-2919-6.ch014(294-321)Online publication date: 2013
https://doi.org/10.4018/978-1-4666-2919-6.ch014
Show More Cited By

Index Terms

Towards distributed storage resource management using flow control
1. Software and its engineering
  1. Software creation and management
    1. Software development process management
      1. Software development methods
  2. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        Memory management
        Secondary storage

Recommendations

Towards Lightweight and Swift Storage Resource Management in Big Data Cloud Era
ICS '15: Proceedings of the 29th ACM on International Conference on Supercomputing

Workload IO behavior in modern data centers is fluctuating and unpredictable due to the rapidly adopted, public cloud environment. Nevertheless, existing storage resource management systems, such as VMware SDRS, are incapable of performing real time ...
SeRW: Adaptively Separating Read and Write upon SSDs of Hybrid Storage Server in Clouds
ICPP '20: Proceedings of the 49th International Conference on Parallel Processing

Nowadays, cloud providers embrace hybrid storage servers to reap both high IO performance of solid-state drives (SSDs) and low-cost of hard disk drives (HDDs). These hybrid storage servers generally employ SSDs as primary storage directly serving ...
Storage management in large distributed object-based storage systems

Comments

Information & Contributors

Information

Published In

cover image ACM SIGOPS Operating Systems Review

ACM SIGOPS Operating Systems Review Volume 42, Issue 6

October 2008

111 pages

ISSN:0163-5980

DOI:10.1145/1453775

Issue’s Table of Contents

Copyright © 2008 Authors.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2008

Published in SIGOPS Volume 42, Issue 6

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
410
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jain NLakshmi J(2015)PCOSProceedings of the 2015 International Conference on Cloud Computing and Big Data (CCBD)10.1109/CCBD.2015.24(145-152)Online publication date: 4-Nov-2015
https://dl.acm.org/doi/10.1109/CCBD.2015.24
Skowron PRzadca K(2014)Fair Share Is Not Enough: Measuring Fairness in Scheduling with Cooperative Game TheoryParallel Processing and Applied Mathematics10.1007/978-3-642-55195-6_4(38-48)Online publication date: 8-May-2014
https://doi.org/10.1007/978-3-642-55195-6_4
Costan A(2013)Grid Data HandlingIT Policy and Ethics10.4018/978-1-4666-2919-6.ch014(294-321)Online publication date: 2013
https://doi.org/10.4018/978-1-4666-2919-6.ch014
Skowron PRzadca KBlelloch GVöcking B(2013)Non-monetary fair schedulingProceedings of the twenty-fifth annual ACM symposium on Parallelism in algorithms and architectures10.1145/2486159.2486169(288-297)Online publication date: 23-Jul-2013
https://dl.acm.org/doi/10.1145/2486159.2486169
Skowron PBiskup MHeldt ŁDubnicki CKat RBaker MToledo S(2013)Fuzzy adaptive control for heterogeneous tasks in high-performance storage systemsProceedings of the 6th International Systems and Storage Conference10.1145/2485732.2485737(1-11)Online publication date: 30-Jun-2013
https://dl.acm.org/doi/10.1145/2485732.2485737
Liu HZhang YZhou YFu XYang L(2013)Receiving Buffer Adaptation for High-Speed Data TransferIEEE Transactions on Computers10.1109/TC.2012.10962:11(2278-2291)Online publication date: 1-Nov-2013
https://dl.acm.org/doi/10.1109/TC.2012.109
Costan A(2012)Grid Data HandlingComputational and Data Grids10.4018/978-1-61350-113-9.ch005(112-139)Online publication date: 2012
https://doi.org/10.4018/978-1-61350-113-9.ch005
Bjørgeengen JHaugerud HVan Drunen R(2010)Using TCP/IP traffic shaping to achieve iSCSI service predictabilityProceedings of the 24th international conference on Large installation system administration10.5555/1924976.1924986(1-13)Online publication date: 7-Nov-2010
https://dl.acm.org/doi/10.5555/1924976.1924986
Cristea VDobre CStratan CPop FCristea VDobre CStratan CPop FCostan A(2010)Data Storage, Retrieval and ManagementLarge-Scale Distributed Computing and Applications10.4018/978-1-61520-703-9.ch006(111-140)Online publication date: 2010
https://doi.org/10.4018/978-1-61520-703-9.ch006

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents