Cited By
View all- Skaperas SMamatas LChorti A(2020)Real-Time Algorithms for the Detection of Changes in the Variance of Video Content PopularityIEEE Access10.1109/ACCESS.2020.29726408(30445-30457)Online publication date: 2020
Resource Auto-Scaling and Sparse Content Replication for Video Storage Systems
Authors: 
Di Niu, 
Hong Xu, Baochun LiAuthors Info & Claims
Di Niu, Hong Xu, Baochun LiAuthors Info & ClaimsArticle No.: 19, Pages 1 - 30
Abstract
Many video-on-demand (VoD) providers are relying on public cloud providers for video storage, access, and streaming services. In this article, we investigate how a VoD provider may make optimal bandwidth reservations from a cloud service provider to guarantee the streaming performance while paying for the bandwidth, storage, and transfer costs. We propose a predictive resource auto-scaling system that dynamically books the minimum amount of bandwidth resources from multiple servers in a cloud storage system to allow the VoD provider to match its short-term demand projections. We exploit the anti-correlation between the demands of different videos for statistical multiplexing to hedge the risk of under-provisioning. The optimal load direction from video channels to cloud servers without replication constraints is derived with provable performance. We further study the joint load direction and sparse content placement problem that aims to reduce bandwidth reservation cost under sparse content replication requirements. We propose several algorithms, and especially an iterative L1-norm penalized optimization procedure, to efficiently solve the problem while effectively limiting the video migration overhead. The proposed system is backed up by a demand predictor that forecasts the expectation, volatility, and correlation of the streaming traffic associated with different videos based on statistical learning. Extensive simulations are conducted to evaluate our proposed algorithms, driven by the real-world workload traces collected from a commercial VoD system.
References
[1]
Amazon Web Services. 2015. Retrieved from http://aws.amazon.com.
[2]
Sharad Agarwal, John Dunagan, Navendu Jain, Stefan Saroiu, Alec Wolman, and Harbinder Bhogan. 2010. Volley: Automated data placement for geo-distributed cloud services. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI’10). 17--32.
[3]
Vaneet Aggarwal, Xu Chen, Vijay Gopalakrishnan, Rittwik Jana, K. K. Ramakrishnan, and Vinay A. Vaishampayan. 2011. Exploiting Virtualization for Delivering Cloud-based IPTV services. In Proceedings of the IEEE International Conference on Computer Communications Workshop on Cloud Computing (INFOCOM’11).
[4]
David Applegate, Aaron Archer, Vijay Gopalakrishnan Seungjoon Lee, and K. K. Ramakrishnan. 2010. Optimal content placement for a large-scale VoD system. In Proceedings of the ACM International Conference on Emerging Networking Experiments and Technologies (CoNEXT’10).
[5]
Hitesh Ballani, Paolo Costa, Thomas Karagiannis, and Ant Rowstron. 2011. Towards predictable datacenter networks. In Proceedings of the Association for Computing Machinery’s Special Interest Group on Data Communications (SIGCOMM’11).
[6]
M. Faizul Bari, Raouf Boutaba, Rafael Esteves, Lisandro Z. Granville, Maxim Podlesny, M. D. Golam Rabbani, Qi Zhang, and Mohamed Faten Zhani. 2013. Data center network virtualization: A survey. IEEE Commun. Surv. Tutor. 15, 2 (2013), 909--928.
[7]
Norman Bobroff, Andrzej Kochut, and Kirk Beaty. 2007. Dynamic placement of virtual machines for managing SLA violations. In Proceedings of the 10th IFIP/IEEE International Symposium on Integrated Network Management.
[8]
T. Bollerslev. 1986. Generalized autoregressive conditional heteroskedasticity. J. Econom. 31 (1986), 307--327.
[9]
Nicolas Bonvin, Thanasis G Papaioannou, and Karl Aberer. 2010. A self-organized, fault-tolerant and scalable replication scheme for cloud storage. In Proceedings of the 1st ACM Symposium on Cloud Computing. ACM, 205--216.
[10]
G. E. P. Box, G. M. Jenkins, and G. C. Reinsel. 2008. Time Series Analysis: Forecasting and Control. Wiley.
[11]
Emmanuel J. Candes, Michael B. Wakin, and Stephen P. Boyd. 2008. Enhancing sparsity by reweighted 1 minimization. J. Fourier Anal. Appl. 14, 5--6 (2008), 877--905.
[12]
Walter Enders. 2010. Applied Econometric Time Series (3 ed.). Wiley, Hoboken, NJ.
[13]
M. Fazel, H. Hindi, and S. P. Boyd. 2003. Log-det heuristic for matrix rank minimization with applications to hankel and euclidean distance matrices. In Proceedings of the American Control Conference.
[14]
D. Gmach, J. Rolia, L. Cherkasova, and A. Kemper. 2007. Workload analysis and demand prediction of enterprise data center applications. In Proceedings of the IEEE Symposium on Workload Characterization.
[15]
Zhenhuan Gong, Xiaohui Gu, and John Wilkes. 2010. PRESS: PRedictive elastic resource scaling for cloud systems. In Proceedings of the IEEE International Conference on Network and Services Management (CNSM’10).
[16]
Chuanxiong Guo, Guohan Lu, Helen J. Wang, Shuang Yang, Chao Kong, Peng Sun, Wenfei Wu, and Yongguang Zhang. 2010. SecondNet: A data center network virtualization architecture with bandwidth guarantees. In Proceedings of the ACM International Conference on Emerging Networking Experiments and Technologies (CoNEXT’10).
[17]
Gonca Gürsun, Mark Crovella, and Ibrahim Matta. 2011. Describing and forecasting video access patterns. In Proceedings of the IEEE International Conference on Computer Communications Mini-Conference (INFOCOM’11).
[18]
D. Kusic, J. O. Kephart, J. E. Hanson, N. Kandasamy, and G. Jiang. 2009. Power and performance management of virtualized computing environments via lookahead control. Cluster Comput. 12, 1 (March 2009), 1--15.
[19]
Minghong Lin, Adam Wierman, Lachlan L. H. Andrew, and Eno Thereska. 2011. Dynamic right-sizing for power-proportional data centers. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’11).
[20]
Zimu Liu, Chuan Wu, Baochun Li, and Shuqiao Zhao. 2010. UUSee: Large-scale operational on-demand streaming with random network coding. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’10).
[21]
Alexander McNeil, Rüdiger Frey, and Paul Embrechts. 2005. Quantitative Risk Management: Concepts Techniques and Tools. Princeton University Press.
[22]
Netflix. 2010. Four reasons we choose amazon’s cloud as our computing platform. The Netflix “Tech” Blog (December 14 2010). https://medium.com/netflix-techblog/four-reasons-we-choose-amazons-cloud-as-our-computing-platform-4aceb692afec.
[23]
Di Niu, Baochun Li, and Shuqiao Zhao. 2011a. Understanding demand volatility in large VoD systems. In Proceedings of the the 21st International workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV’11).
[24]
Di Niu, Zimu Liu, Baochun Li, and Shuqiao Zhao. 2011b. Demand forecast and performance prediction in peer-assisted on-demand streaming systems. In Proceedings of the IEEE International Conference on Computer Communications Mini-Conference (INFOCOM’11).
[25]
Di Niu, Hong Xu, Baochun Li, and Shuqiao Zhao. 2012. Quality-assured cloud bandwidth auto-scaling for video-on-demand applications. In Proceedings of the of IEEE International Conference on Computer Communications (INFOCOM’12).
[26]
Ashwin Rao, Arnaud Legout, Yeon-sup Lim, Don Towsley, Chadi Barakat, and Walid Dabbous. 2011. Network characteristics of video streaming traffic. In Proceedings of the 7th Conference on Emerging Networking Experiments and Technologies. ACM, 25.
[27]
Yuval Rochman, Hanoch Levy, and Eli Brosh. 2013. Resource placement and assignment in distributed network topologies. In Proceedings IEEE International Conference on Computer Communications (INFOCOM’13). IEEE, 1914--1922.
[28]
Chunqiang Tang, Malgorzata Steinder, Michael Spreitzer, and Giovanni Pacifici. 2007. A scalable application placement controller for enterprise data centers. In Proceedings of the ACM International World Wide Web Conference (WWW’07).
[29]
Bhuvan Urgaonkar, Prashant Shenoy, and Timothy Roscoe. 2002. Resource overbooking and application profiling in shared hosting platforms. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI’02).
[30]
M. Wang, X. Meng, and L. Zhang. 2011. Consolidating virtual machines with dynamic bandwidth demand in data centers. In Proceedings of the of IEEE International Conference on Computer Communications Mini-Conference (INFOCOM’11).
[31]
Di Xie, Ning Ding, Y. Charlie Hu, and Ramana Kompella. 2012. The only constant is change: Incorporating time-varying network reservations in data centers. In Proceedings of the ACM Special Interest Group on Data Communications (SIGCOMM’12).
[32]
Hong Xu and Baochun Li. 2013. Joint request mapping and response routing for geo-distributed cloud services. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’13). IEEE, 854--862.
[33]
Boyang Yu and Jianping Pan. 2015. Location-aware associated data placement for geo-distributed data-intensive applications. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’15).
- Resource Auto-Scaling and Sparse Content Replication for Video Storage Systems
Recommendations
A survey on auto-scaling: how to exploit cloud elasticity
Elasticity plays an essential role as far as the wide diffusion of cloud computing is concerned. It enables a cloud application deployment to 'scale' automatically, adapting to workload changes, guaranteeing the performance requirements with minimum ...
Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey
Web application providers have been migrating their applications to cloud data centers, attracted by the emerging cloud computing paradigm. One of the appealing features of the cloud is elasticity. It allows cloud users to acquire or release computing ...
A model driven method to deploy auto-scaling configuration for cloud services
RELENG 2016: Proceedings of the 4th International Workshop on Release EngineeringVendor lock-in is the issues in auto-scaling configuration; scaling configuration of a service cannot automatically transfer when the service is migrated from one cloud to another cloud. To facilitate fast service deployment, there is a need to ...
Comments
Information & Contributors
Information
Published In
December 2017
145 pages
ISSN:2376-3639
EISSN:2376-3647
DOI:10.1145/3133236
- Editors:
- Sem Borst,
- Carey Williamson
Copyright © 2017 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 November 2017
Accepted: 01 April 2017
Revised: 01 December 2016
Received: 01 December 2014
Published in TOMPECS Volume 2, Issue 4
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 167Total Downloads
- Downloads (Last 12 months)8
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
Cited By
View all- Skaperas SMamatas LChorti A(2020)Real-Time Algorithms for the Detection of Changes in the Variance of Video Content PopularityIEEE Access10.1109/ACCESS.2020.29726408(30445-30457)Online publication date: 2020
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in