research-article

Deconstructing Amazon EC2 Spot Instance Pricing

Authors:

Orna Agmon Ben-Yehuda,

Muli Ben-Yehuda,

Assaf Schuster,

Dan TsafrirAuthors Info & Claims

ACM Transactions on Economics and Computation (TEAC), Volume 1, Issue 3

Article No.: 16, Pages 1 - 20

https://doi.org/10.1145/2509413.2509416

Published: 01 September 2013 Publication History

Abstract

Cloud providers possessing large quantities of spare capacity must either incentivize clients to purchase it or suffer losses. Amazon is the first cloud provider to address this challenge, by allowing clients to bid on spare capacity and by granting resources to bidders while their bids exceed a periodically changing spot price. Amazon publicizes the spot price but does not disclose how it is determined.

By analyzing the spot price histories of Amazon’s EC2 cloud, we reverse engineer how prices are set and construct a model that generates prices consistent with existing price traces. Our findings suggest that usually prices are not market-driven, as sometimes previously assumed. Rather, they are likely to be generated most of the time at random from within a tight price range via a dynamic hidden reserve price mechanism. Our model could help clients make informed bids, cloud providers design profitable systems, and researchers design pricing algorithms.

References

[1]

Agmon Ben-Yehuda, O., Ben-Yehuda, M., Schuster, A., and Tsafrir, D. 2011. Deconstructing Amazon EC2 spot instance pricing. In Proceedings of the IEEE 3rd International Conference on Cloud Computing Technology and Science (CloudCom).

Digital Library

[2]

Altmann, J., Courcoubetis, C., Stamoulis, G., Dramitinos, M., Rayna, T., Risch, M., and Bannink, C. 2008. GridEcon: A market place for computing resources. In Grid Economics and Business Models, Lecture Notes in Computer Science Series, vol. 5206, Springer Berlin, 185--196.

Digital Library

[3]

Amazon. 2009. Amazon EC2 spot instances. http://aws.amazon.com/ec2/spot-instances/. (Accessed 8/11).

[4]

Amazon. 2010. Spot instance termination conditions? http://tinyurl.com/2dzp734. Online AWS Developer Forums discussion. (Accessed 4/11).

[5]

Andrzejak, A., Kondo, D., and Yi, S. 2010. Decision model for cloud computing under SLA constraints. In Proceedings of the IEEE/ACM International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS).

Digital Library

[6]

Brebner, P. and Liu, A. 2011. Performance and cost assessment of cloud services. In Service-Oriented Computing, Lecture Notes in Computer Science Series, vol. 6568, 39--50.

Digital Library

[7]

Chen, J., Wang, C., Zhou, B. B., Sun, L., Lee, Y. C., and Zomaya, A. Y. 2011. Tradeoffs between profit and customer satisfaction for service provisioning in the cloud. In Proceedings of HPDC.

Digital Library

[8]

Chohan, N., Castillo, C., Spreitzer, M., Steinder, M., Tantawi, A., and Krintz, C. 2010. See Spot run: Using spot instances for mapreduce workflows. In Proceedings of the USENIX Conference on Hot Topics in Cloud Computing (HotCloud).

Digital Library

[9]

Feitelson, D. Parallel workloads archive. http://www.cs.huji.ac.il/labs/parallel/workload/index.html.

[10]

Hellerstein, J. L., Cirne, W., and Wilkes, J. 2011. Google cluster data. http://code.google.com/p/googleclusterdata/.

[11]

Iosup, A., Li, H., Jan, M., Anoep, S., Dumitrescu, C., Wolters, L., and Epema, D. H. J. 2008. The grid workloads archive. Future Generation Comp. Syst. 24, 7, 672--686.

Digital Library

[12]

Iosup, A., Ostermann, S., Yigitbasi, N., Prodan, R., Fahringer, T., and Epema, D. 2011. Performance analysis of cloud computing services for many-tasks scientific computing. IEEE Trans. Paral. Distrib. Syst. 22, 6, 931--945.

Digital Library

[13]

Javadi, B. and Buyya, R. 2011. Comprehensive statistical analysis and modeling of spot instances in public cloud environments. Tech. rep. CLOUDS-TR-2011-1, Cloud Computing and Distributed Systems Laboratory, The University of Melbourne.

[14]

Katkar, R. and Reiley, D. H. 2006. Public versus secret reserve prices in ebay auctions: Results from a pokémon field experiment. B.E. J. Advances Economic Anal. Policy 5, 2. http://www.degruyter.com/view/j/bejeap.2006.5.2/bejeap.2006.6.2.1442/bejeap.2006.6.2.1442.xml.

[15]

Levy, M. and Solomon, S. 1997. New evidence for the power-law distribution of wealth. Physica A 242, 90--94.

[16]

Li, H. and Tan, G. 2000. Hidden reserve prices with risk-averse bidders. Tech. rep., University of British Columbia.

[17]

Li, T. and Perrigne, I. 2003. Timber sale auctions with random reserve prices. Rev. Econom. Statist. 85, 1, 189--200.

[18]

Liu, H. 2011. A measurement study of server utilization in public clouds. In Proceedings of the International Conference on Cloud and Green Computing (CGC).

Digital Library

[19]

Lossen, T. 2010. Cloud exchange. http://cloudexchange.org/. (Accessed 4/11). The dataset is available from http://files.evercu.be/cloudexchange.tgz}.

[20]

Lublin, U. and Feitelson, D. G. 2003. The workload on parallel supercomputers: Modeling the characteristics of rigid jobs. J. Parall. Distrib. Comput. 63, 1105--1122.

Digital Library

[21]

Mattess, M., Vecchiola, C., and Buyya, R. 2010. Managing peak loads by leasing cloud infrastructure services from a spot market. In Proceedings of the IEEE International Conference on High Performance Computing and Communications (HPCC).

Digital Library

[22]

Mazzucco, M. and Dumas, M. 2011. Achieving performance and availability guarantees with spot instances. In Proceedings of the IEEE International Conference on High Performance Computing and Communications (HPCC).

Digital Library

[23]

Medernach, E. 2005. Workload analysis of a cluster in a grid environment. In Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing.

Digital Library

[24]

Ortuno, F. M. and Harder, U. 2010. Stochastic calculus model for the spot price of computing power. In Proceedings of the Annual UK Performance Engineering Workshop (UKPEW).

[25]

Rahman, M. R., Lu, Y., and Gupta, I. 2011. Risk aware resource allocation for clouds. Tech. rep., University of Illinois at Urbana-Champaign.

[26]

Ramberg, C. 2002. Internet Marketplaces: The Law of Auctions and Exchanges Online. Oxford University Press.

Digital Library

[27]

Ryu, K. D., Zhang, X., Ammons, G., Bala, V., Berger, S., Da Silva, D. M., Doran, J., Franco, F., Karve, A., Lee, H., Lindeman, J. A., Mohindra, A., Oesterlin, B., Pacifici, G., Pendarakis, D., Reimer, D., and Sabath, M. 2011. RC2---living lab for cloud computing. In Proceedings of the USENIX Large Installation Systems Administration Conference (LISA).

Digital Library

[28]

Samovskiy, D. 2011. Amazon EC2 spot instances - a flop? http://tinyurl.com/somic11. (Accessed 9/11).

[29]

Souma, W. 2002. Physics of personal income. http://arxiv.org/pdf/cond-mat/0202388.

[30]

Toosi, A. N., Calheiros, R. N., Thulasiram, R. K., and Buyya, R. 2011. Resource provisioning policies to increase iaas provider s profit in a federated cloud environment. In Proceedings of the IEEE International Conference on High Performance Computing and Communications (HPCC).

Digital Library

[31]

Vanmechelen, K., Depoorter, W., and Broeckhove, J. 2011. Combining futures and spot markets: A hybrid market approach to economic grid resource management. J. Grid Comput. 9, 81--94.

Digital Library

[32]

Vermeersch, K. 2010. Spot watch. http://spotwatch.eu/input/. {Accessed Apr. 2011. The dataset and website code are available from https://s3-eu-west-1.amazonaws.com/ruben.ruben/SpotWatch.tar}.

[33]

Vermeersch, K. 2011. A broker for cost-efficient QoS aware resource allocation in EC2. M.S. thesis, Universiteit Antwerpen.

[34]

Vincent, D. R. 1995. Bidding off the wall: Why reserve prices may be kept secret. J. Econ. Theory 65, 2, 575--584.

[35]

Voorsluys, W., Garg, S. K., and Buyya, R. 2011. Provisioning spot market cloud resources to create cost-effective virtual clusters. In Proceedings of ICA3PP.

Digital Library

[36]

Wee, S. 2011. Debunking real-time pricing in cloud computing. In Proceedings of IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid).

Digital Library

[37]

Wieder, A., Bhatotia, P., Post, A., and Rodrigues, R. 2010. Brief announcement: modelling mapreduce for optimal execution in the cloud. In Proceedings of the ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing (PODC).

Digital Library

[38]

Yi, S., Kondo, D., and Andrzejak, A. 2010. Reducing costs of spot instances via checkpointing in the Amazon Elastic Compute Cloud. In Proceedings of the IEEE International Conference on Cloud Computing (CLOUD).

Digital Library

[39]

Yi, S., Andrzejak, A., and Kondo, D. 2011. Monetary cost-aware checkpointing and migration on Amazon cloud spot instances. IEEE Trans. Serv. Comput.

Digital Library

[40]

Zhang, Q., Gurses, E., Boutaba, R., and Xiao, J. 2011. Dynamic resource allocation for spot markets in clouds. In Proceedings of the Workshop on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services (Hot-ICE).

Digital Library

[41]

Zhao, H., Pan, M., Liu, X., Li, X., and Fang, Y. 2012. Optimal resource rental planning for elastic applications in cloud market. In Proceedings of the IEEE International Parallel & Distributed Processing Symposium (IPDPS).

Digital Library

Cited By

M. Prasanthi G.Chishma P. Padmavathi K. Reethika A. Chandra Sekhar (2024)Improving Amazon EC2 Spot Instances Price Prediction using Machine Learning AlgorithmInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology10.32628/CSEIT2410210210:2(713-720)Online publication date: 22-Apr-2024
https://doi.org/10.32628/CSEIT24102102
Kim KLee KChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Making Cloud Spot Instance Interruption Events VisibleProceedings of the ACM Web Conference 202410.1145/3589334.3645548(2998-3009)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645548
Zhu SLu JLyu BPan TZhang SSun XJia CCheng XKang DLv YYang FXue XYang XWang ZYang J(2024)Proactive Telemetry in Large-Scale Multi-Tenant Cloud Overlay NetworksIEEE/ACM Transactions on Networking10.1109/TNET.2024.338178632:4(3002-3017)Online publication date: Aug-2024
https://doi.org/10.1109/TNET.2024.3381786
Show More Cited By

Index Terms

Deconstructing Amazon EC2 Spot Instance Pricing
1. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
      1. Time series analysis

Recommendations

Deconstructing Amazon EC2 Spot Instance Pricing
CLOUDCOM '11: Proceedings of the 2011 IEEE Third International Conference on Cloud Computing Technology and Science

Cloud providers possessing large quantities of spare capacity must either incentivize clients to purchase it or suffer losses. Amazon is the first cloud provider to address this challenge, by allowing clients to bid on spare capacity and by granting ...
Perceptive bidding strategy for Amazon EC2 spot instance market

Spot pricing is an auction based market mechanism introduced by Amazon EC2 to sell available cloud resources at a low cost without any availability guarantee. Bid prices submitted by users directly affect reliability of the instance acquired. In ...
How Small and Medium Enterprises SMEs Should Bid for Spot Instances of Amazon's EC2 Cloud

In cloud service provisioning, spot instances are spare slots for which it has no pre-booking, unlike reserved or on-demand instances for which a cloud service provider CSP has a priori booking. CSPs like Amazon prefer spot instance approach to sell ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Economics and Computation

ACM Transactions on Economics and Computation Volume 1, Issue 3

September 2013

115 pages

ISSN:2167-8375

EISSN:2167-8383

DOI:10.1145/2509413

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2013

Accepted: 01 March 2012

Received: 01 November 2011

Published in TEAC Volume 1, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Technion Hasso Plattner Center

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
2,174
Total Downloads

Downloads (Last 12 months)101
Downloads (Last 6 weeks)7

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

M. Prasanthi G.Chishma P. Padmavathi K. Reethika A. Chandra Sekhar (2024)Improving Amazon EC2 Spot Instances Price Prediction using Machine Learning AlgorithmInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology10.32628/CSEIT2410210210:2(713-720)Online publication date: 22-Apr-2024
https://doi.org/10.32628/CSEIT24102102
Kim KLee KChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Making Cloud Spot Instance Interruption Events VisibleProceedings of the ACM Web Conference 202410.1145/3589334.3645548(2998-3009)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645548
Zhu SLu JLyu BPan TZhang SSun XJia CCheng XKang DLv YYang FXue XYang XWang ZYang J(2024)Proactive Telemetry in Large-Scale Multi-Tenant Cloud Overlay NetworksIEEE/ACM Transactions on Networking10.1109/TNET.2024.338178632:4(3002-3017)Online publication date: Aug-2024
https://doi.org/10.1109/TNET.2024.3381786
Chaturvedi AChekuri CLiu MRicha ARost MSchmid SWeber J(2024)Improved Throughput for All-or-Nothing Multicommodity Flows With Arbitrary DemandsIEEE/ACM Transactions on Networking10.1109/TNET.2023.332543732:2(1435-1450)Online publication date: Apr-2024
https://doi.org/10.1109/TNET.2023.3325437
Zhou YYe QHuang H(2024)DAS: A DRL-Based Scheme for Workload Allocation and Worker Selection in Distributed Coded Machine LearningIEEE Internet of Things Journal10.1109/JIOT.2024.339471411:15(26076-26090)Online publication date: 1-Aug-2024
https://doi.org/10.1109/JIOT.2024.3394714
Bandopadhyay ASwain SSingh RSarkar PBhattacharyya SMrsic L(2024)Game-Theoretic Resource Allocation and Dynamic Pricing Mechanism in Fog ComputingIEEE Access10.1109/ACCESS.2024.338433412(51704-51718)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3384334
Singhal RSharma VSinghal IBansal V(2024)Blockchain-enabled auction for cloud resource provisioning: a survey on trust and economyInternational Journal of System Assurance Engineering and Management10.1007/s13198-024-02314-x15:7(2787-2807)Online publication date: 15-Apr-2024
https://doi.org/10.1007/s13198-024-02314-x
Movsowitz Davidow DAgmon Ben-Yehuda ODunkelman OButt AMi NChard K(2023)Deconstructing Alibaba Cloud's Preemptible Instance PricingProceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3588195.3593001(253-265)Online publication date: 7-Aug-2023
https://dl.acm.org/doi/10.1145/3588195.3593001
Kim KPark SHwang JLee HKang SLee K(2023)Public Spot Instance Dataset Archive ServiceCompanion Proceedings of the ACM Web Conference 202310.1145/3543873.3587314(69-72)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543873.3587314
van der Mei Rvan den Berg HGanchev ITutschku KLeitner PLassila PBurakowski WLiberal FArvidsson ÅHoβfeld TWac KMelvin HGrbac THaddad YKey P(2018)State of the Art and Research Challenges in the Area of Autonomous Control for a Reliable Internet of ServicesAutonomous Control for a Reliable Internet of Services10.1007/978-3-319-90415-3_1(1-22)Online publication date: 25-May-2018
https://doi.org/10.1007/978-3-319-90415-3_1
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents