tutorial

On Estimating Minimum Bids for Amazon EC2 Spot Instances

Authors:

Mohan Baruwal Chhetri,

Ryszard KowalczykAuthors Info & Claims

CCGrid '17: Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

Pages 391 - 400

https://doi.org/10.1109/CCGRID.2017.76

Published: 14 May 2017 Publication History

Abstract

Consumers can realize significant cost savings by procuring resources from computational spot markets such as Amazon Elastic Compute Cloud (EC2) Spot Instances. They can take advantage of the price differentials across time slots, regions, and instance types to minimize the total cost of running their applications on the cloud. However, Spot markets are inherently volatile and dynamic, as a consequence of which Spot prices change continuously. As such, prospective bidders can benefit from intelligent insights into the Spot market dynamics that can help them make more informed bidding decisions. To enable this, we propose a descriptive statistics approach for the analysis of Amazon EC2 Spot markets to detect typical pricing patterns including the presence of seasonal components, extremes and trends. We use three statistical measures -- the Gini coefficient, the Theil index, and the exponential weighted moving average. We also devise a model for estimating minimum bids such that the Spot instances will run for specified durations with a probability greater than a set value based on different look back periods. Experimental results show that our estimation yields on average a bidding strategy that can reliably secure an instance at least 80% of the time at minimum target guarantee between 50% and 95%.

References

[1]

P. Mell and T. Granee, "The nist definition of cloud computing," 2011.

[2]

J. Weinman, "Cloud pricing and markets," IEEE Cloud Computing, vol. 2, no. 1, pp. 10--13, 2015.

[3]

O. Agmon Ben-Yehuda, M. Ben-Yehuda, A. Schuster, and D. Tsafrir, "Deconstructing amazon ec2 spot instance pricing," IEEE Third International Conference on Cloud Computing Technology and Science (CloudCom), 2011.

Digital Library

[4]

R. Wolski and J. Brevik, "Providing statistical reliability guarantees in the aws spot tier," in Proceedings of the 24th High Performance Computing Symposium. Society for Computer Simulation International, 2016, p. 13.

Digital Library

[5]

"Spot bid advisor," 2016, accesssed: 2016-10-21. {Online}. Available: https://aws.amazon.com/ec2/spot/bid-advisor/

[6]

A. K. Sen, On Economic Inequality. Oxford: Oxford University Press, 1973.

[7]

S. W. Roberts, "Control chart tests based on geometric moving averages," Technometrics, vol. 42, no. 1, pp. 97--101, 2000.

Digital Library

[8]

O. Goloshchapova and M. Lumpe, "On the application of inequality indices in comparative software analysis," in 22nd Australian Software Engineering Conference (ASWEC 2013). IEEE, 2013, pp. 117--126.

Digital Library

[9]

C. Gini, "Measurement of inequality of incomes," The Economic Journal, vol. 31, no. 121, pp. 124--126, 1921.

[10]

H. Theil, "A New Index Number Formula," The Review of Economics and Statistics, vol. 55, no. 4, pp. 498--502, Nov. 1973.

[11]

C. E. Shannon, "A mathematical theory of communication," ACM SIGMOBILE Mobile Computing and Communications Review, vol. 5, no. 1, pp. 3--55, 2001.

Digital Library

[12]

F. Husson, S. Lê, and J. Pagès, Exploratory Multivariate Analysis by Example Using R. CRC Press, 2011.

[13]

"Announcing amazon ec2 spot instances," 2009, accesssed: 2016-09-21. {Online}. Available: https://aws.amazon.com/about-aws/whats-new/2009/12/14/announcing-amazon-ec2-spot-instances/

[14]

"Improved pricing control for spot instances coming july 1st," 2011, accesssed: 2016-09-21. {Online}. Available: https://aws.amazon.com/about-aws/whats-new/2011/05/26/improved-pricing-control-for-spot-instances-coming-july-1st/

[15]

"Amazon web services puts a cap on spot instance pricing, takes away punchbowl," 2013, accesssed: 2016-09-21. {Online}. Available: https://www.mailchannels.com/2013/11/aws-puts-a-cap-on-spot-instance-pricing/

[16]

"Spot instance limits," 2016, accesssed: 2016-09-21. {Online}. Available: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-limits.html

[17]

O. Agmon Ben-Yehuda, M. Ben-Yehuda, A. Schuster, and D. Tsafrir, "Deconstructing amazon ec2 spot instance pricing," ACM Transactions on Economics and Computation, vol. 1, no. 3, p. 16, 2013.

Digital Library

[18]

B. Javadi, R. K. Thulasiram, and R. Buyya, "Characterizing spot price dynamics in public cloud environments," Future Generation Computer Systems, vol. 29, no. 4, pp. 988--999, 2013.

Digital Library

[19]

L. Zheng, C. Joe-Wong, C. W. Tan, M. Chiang, and X. Wang, "How to bid the cloud," ACM SIGCOMM Computer Communication Review, vol. 45, no. 4, pp. 71--84, 2015.

Digital Library

[20]

M. Zafer, Y. Song, and K.-W. Lee, "Optimal bids for spot vms in a cloud for deadline constrained jobs," in 5th IEEE International Conference onCloud Computing (CLOUD 2012). IEEE, 2012, pp. 75--82.

Digital Library

[21]

Y. Song, M. Zafer, and K.-W. Lee, "Optimal bidding in spot instance market," in 31st Annual IEEE International Conference on Computer Communications (INFOCOM 2012). IEEE, 2012, pp. 190--198.

[22]

P. Sharma, D. Irwin, and P. Shenoy, "How not to bid the cloud," University of Massachusetts Technical Report UMCS-2016-002, 2016.

[23]

S. Subramanya, A. Rizk, and D. Irwin, "Cloud spot markets are not sustainable: The case for transient guarantees," in 8th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 16). USENIX Association, 2016.

Digital Library

[24]

W. Chen, L. Qianlin, and B. Urgaonkar, "Identification and empirical analysis of amazon ec2 spot instance features for cost-effective tenant procurement," The Pennsylvania State University Technical Report CSE-16-006, 2016.

[25]

X. Ouyang, D. Irwin, and P. Shenoy, "Spotlight: An information service for the cloud," in 36th IEEE International Conference on Distributed Computing Systems (ICDCS 2016). IEEE, 2016, pp. 425--436.

Cited By

Kim KLee KChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Making Cloud Spot Instance Interruption Events VisibleProceedings of the ACM Web Conference 202410.1145/3589334.3645548(2998-3009)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645548
Iqbal SLi HBergsma SBeschastnikh IHu AGavrilovska AAltınbüken DBinnig C(2022)CoSpotProceedings of the 13th Symposium on Cloud Computing10.1145/3542929.3563499(540-556)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3542929.3563499
Baughman MCaton SHaas CChard RWolski RFoster IChard KNicolae BCostan ADuplyakin DGainaru AUta ACosta C(2019)Deconstructing the 2017 Changes to AWS Spot Market PricingProceedings of the 10th Workshop on Scientific Cloud Computing10.1145/3322795.3331465(19-26)Online publication date: 17-Jun-2019
https://dl.acm.org/doi/10.1145/3322795.3331465
Show More Cited By

Recommendations

Deconstructing Amazon EC2 Spot Instance Pricing

Cloud providers possessing large quantities of spare capacity must either incentivize clients to purchase it or suffer losses. Amazon is the first cloud provider to address this challenge, by allowing clients to bid on spare capacity and by granting ...
Deconstructing Amazon EC2 Spot Instance Pricing
CLOUDCOM '11: Proceedings of the 2011 IEEE Third International Conference on Cloud Computing Technology and Science

Cloud providers possessing large quantities of spare capacity must either incentivize clients to purchase it or suffer losses. Amazon is the first cloud provider to address this challenge, by allowing clients to bid on spare capacity and by granting ...
How Small and Medium Enterprises SMEs Should Bid for Spot Instances of Amazon's EC2 Cloud

In cloud service provisioning, spot instances are spare slots for which it has no pre-booking, unlike reserved or on-demand instances for which a cloud service provider CSP has a priori booking. CSPs like Amazon prefer spot instance approach to sell ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CCGrid '17: Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

May 2017

1167 pages

ISBN:9781509066100

Sponsors

Publisher

IEEE Press

Publication History

Published: 14 May 2017

Check for updates

Author Tags

Qualifiers

Tutorial
Research
Refereed limited

Conference

CCGrid '17

Sponsor:

SIGARCH

CCGrid '17: 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

May 14 - 17, 2017

Madrid, Spain

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
80
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kim KLee KChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Making Cloud Spot Instance Interruption Events VisibleProceedings of the ACM Web Conference 202410.1145/3589334.3645548(2998-3009)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645548
Iqbal SLi HBergsma SBeschastnikh IHu AGavrilovska AAltınbüken DBinnig C(2022)CoSpotProceedings of the 13th Symposium on Cloud Computing10.1145/3542929.3563499(540-556)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3542929.3563499
Baughman MCaton SHaas CChard RWolski RFoster IChard KNicolae BCostan ADuplyakin DGainaru AUta ACosta C(2019)Deconstructing the 2017 Changes to AWS Spot Market PricingProceedings of the 10th Workshop on Scientific Cloud Computing10.1145/3322795.3331465(19-26)Online publication date: 17-Jun-2019
https://dl.acm.org/doi/10.1145/3322795.3331465
Baughman MHaas CWolski RFoster IChard K(2018)Predicting Amazon Spot Prices with LSTM NetworksProceedings of the 9th Workshop on Scientific Cloud Computing10.1145/3217880.3217881(1-7)Online publication date: 11-Jun-2018
https://dl.acm.org/doi/10.1145/3217880.3217881
Chhetri MVo QKowalczyk RNepal SEl-Araby EEl-Ghazawi TPanda D(2018)Towards resource and contract heterogeneity aware rescaling for cloud-hosted applicationsProceedings of the 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing10.1109/CCGRID.2018.00030(153-162)Online publication date: 1-May-2018
https://dl.acm.org/doi/10.1109/CCGRID.2018.00030

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten