research-article

Open access

Expert-Calibrated Learning for Online Optimization with Switching Costs

Authors:

Shaolei RenAuthors Info & Claims

Proceedings of the ACM on Measurement and Analysis of Computing Systems, Volume 6, Issue 2

Article No.: 28, Pages 1 - 35

https://doi.org/10.1145/3530894

Published: 06 June 2022 Publication History

Abstract

We study online convex optimization with switching costs, a practically important but also extremely challenging problem due to the lack of complete offline information. By tapping into the power of machine learning (ML) based optimizers, ML-augmented online algorithms (also referred to as expert calibration in this paper) have been emerging as state of the art, with provable worst-case performance guarantees. Nonetheless, by using the standard practice of training an ML model as a standalone optimizer and plugging it into an ML-augmented algorithm, the average cost performance can be highly unsatisfactory. In order to address the "how to learn" challenge, we propose EC-L2O (expert-calibrated learning to optimize), which trains an ML-based optimizer by explicitly taking into account the downstream expert calibrator. To accomplish this, we propose a new differentiable expert calibrator that generalizes regularized online balanced descent and offers a provably better competitive ratio than pure ML predictions when the prediction error is large. For training, our loss function is a weighted sum of two different losses --- one minimizing the average ML prediction error for better robustness, and the other one minimizing the post-calibration average cost. We also provide theoretical analysis for EC-L2O, highlighting that expert calibration can be even beneficial for the average cost performance and that the high-percentile tail ratio of the cost achieved by EC-L2O to that of the offline optimal oracle (i.e., tail cost ratio) can be bounded. Finally, we test EC-L2O by running simulations for sustainable datacenter demand response. Our results demonstrate that EC-L2O can empirically achieve a lower average cost as well as a lower competitive ratio than the existing baseline algorithms.

References

[1]

Akshay Agrawal, Brandon Amos, Shane Barratt, Stephen Boyd, Steven Diamond, and J. Zico Kolter. 2019. Differentiable Convex Optimization Layers. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. dtextquotesingle Alché-Buc, E. Fox, and R. Garnett (Eds.), Vol. 32. Curran Associates, Inc.

[2]

Brandon Amos, Ivan Jimenez, Jacob Sacks, Byron Boots, and J. Zico Kolter. 2018. Differentiable MPC for End-to-end Planning and Control. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2018/file/ba6d843eb4251a4526ce65d1807a9309-Paper.pdf

[3]

Brandon Amos and J. Zico Kolter. 2017. OptNet: Differentiable Optimization as a Layer in Neural Networks. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70). PMLR, 136--145.

[4]

Keerti Anand, Rong Ge, Amit Kumar, and Debmalya Panigrahi. 2021. A Regression Approach to Learning-Augmented Online Algorithms. In Advances in Neural Information Processing Systems, A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (Eds.). https://openreview.net/forum?id=GgS40Y04LxA

[5]

Keerti Anand, Rong Ge, and Debmalya Panigrahi. 2020. Customizing ML Predictions for Online Algorithms. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119), Hal Daumé III and Aarti Singh (Eds.). PMLR, 303--313. https://proceedings.mlr.press/v119/anand20a.html

[6]

Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. 2016. Learning to learn by gradient descent by gradient descent. In Advances in neural information processing systems. 3981--3989.

[7]

Antonios Antoniadis, Christian Coester, Marek Eliás, Adam Polak, and Bertrand Simon. 2020. Online Metric Algorithms with Untrusted Predictions. In ICML. 345--355. http://proceedings.mlr.press/v119/antoniadis20a.html

[8]

Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. 2021. Machine Learning for Combinatorial Optimization: A methodological Tour D'Horizon. European Journal of Operational Research, Vol. 290, 2 (2021), 405--421. https://doi.org/10.1016/j.ejor.2020.07.063

[9]

Avrim Blum and Carl Burch. 2000. On-line learning and the metrical task system problem. Machine Learning, Vol. 39, 1 (2000), 35--58.

Digital Library

[10]

Niangjun Chen, Joshua Comden, Zhenhua Liu, Anshul Gandhi, and Adam Wierman. 2016. Using Predictions in Online Optimization: Looking Forward with an Eye on the Past. SIGMETRICS Perform. Eval. Rev., Vol. 44, 1 (June 2016), 193--206. https://doi.org/10.1145/2964791.2901464

Digital Library

[11]

Niangjun Chen, Gautam Goel, and Adam Wierman. 2018. Smoothed Online Convex Optimization in High Dimensions via Online Balanced Descent. In COLT .

[12]

Tianlong Chen, Xiaohan Chen, Wuyang Chen, Howard Heaton, Jialin Liu, Zhangyang Wang, and Wotao Yin. 2021. Learning to optimize: A primer and a benchmark. arXiv preprint arXiv:2103.12828 (2021).

[13]

Yutian Chen, Matthew W Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Timothy P Lillicrap, Matt Botvinick, and Nando Freitas. 2017. Learning to Learn Without Gradient Descent by Gradient Descent. In ICML .

[14]

Nicolas Christianson, Tinashe Handina, and Adam Wierman. 2022. Chasing Convex Bodies and Functions with Black-Box Advice. In Under Submission .

[15]

Joshua Comden, Sijie Yao, Niangjun Chen, Haipeng Xing, and Zhenhua Liu. 2019. Online Optimization in Cloud Resource Provisioning: Predictions, Regrets, and Algorithms. Proc. ACM Meas. Anal. Comput. Syst., Vol. 3, 1, Article 16 (March 2019), bibinfonumpages30 pages. https://doi.org/10.1145/3322205.3311087

Digital Library

[16]

Wei Cui, Kaiming Shen, and Wei Yu. 2019. Spatial deep learning for wireless scheduling. IEEE Journal on Selected Areas in Communications, Vol. 37, 6 (2019), 1248--1261.

[17]

Hanjun Dai, Elias B Khalil, Yuyu Zhang, Bistra Dilkina, and Le Song. 2017. Learning combinatorial optimization algorithms over graphs. arXiv preprint arXiv:1704.01665 (2017).

[18]

Roy De Maesschalck, Delphine Jouan-Rimbaud, and Désiré L Massart. 2000. The mahalanobis distance. Chemometrics and intelligent laboratory systems, Vol. 50, 1 (2000), 1--18.

[19]

Bingqian Du, Zhiyi Huang, and Chuan Wu. 2022. Adversarial Deep Learning for Online Resource Allocation. ACM Trans. Model. Perform. Eval. Comput. Syst., Vol. 6, 4, Article 13 (feb 2022), bibinfonumpages25 pages. https://doi.org/10.1145/3494526

Digital Library

[20]

Elbert Du, Franklyn Wang, and Michael Mitzenmacher. 2021. Putting the “Learning" into Learning-Augmented Algorithms for Frequency Estimation. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 2860--2869. https://proceedings.mlr.press/v139/du21d.html

[21]

Adam N. Elmachtoub and Paul Grigas. 2017. Smart “Predict, then Optimize”. CoRR, Vol. abs/1710.08005 (2017). https://arxiv.org/abs/1710.08005

[22]

Joel Friedman and Nathan Linial. 1993. On convex body chasing. Discrete & Computational Geometry, Vol. 9, 3 (1993), 293--321.

Digital Library

[23]

G. Ghatikar, V. Ganti, N. E. Matson, and M. A. Piette. 2012. Demand Response Opportunities and Enabling Technologies for Data Centers: Findings From Field Studies.

[24]

Gautam Goel, Yiheng Lin, Haoyuan Sun, and Adam Wierman. 2019. Beyond Online Balanced Descent: An Optimal Algorithm for Smoothed Online Optimization. In NeurIPS, Vol. 32. https://proceedings.neurips.cc/paper/2019/file/9f36407ead0629fc166f14dde7970f68-Paper.pdf

[25]

Gautam Goel and Adam Wierman. 2019. An Online Algorithm for Smoothed Online Convex Optimization. SIGMETRICS Perform. Eval. Rev., Vol. 47, 2 (Dec. 2019), 6--8.

Digital Library

[26]

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning .MIT Press. http://www.deeplearningbook.org .

Digital Library

[27]

Johannes Kirschner, Ilija Bogunovic, Stefanie Jegelka, and Andreas Krause. 2020. Distributionally Robust Bayesian Optimization. In AISTATS .

[28]

Zico Kolter, David Duvenaud, and Matt Johnson. http://implicit-layers-tutorial.org/. Deep Implicit Layers.

[29]

Weiwei Kong, Christopher Liaw, Aranyak Mehta, and D. Sivakumar. 2019. A New Dog Learns Old Tricks: RL Finds Classic Optimization Algorithms. In ICLR . https://openreview.net/forum?id=rkluJ2R9KQ

[30]

Daniel Kuhn, Peyman Mohajerin Esfahani, Viet Anh Nguyen, and Soroosh Shafieezadeh-Abadeh. 2019. Wasserstein distributionally robust optimization: Theory and applications in machine learning. In Operations Research & Management Science in the Age of Analytics. INFORMS, 130--166.

[31]

Ke Li and Jitendra Malik. 2017. Learning to Optimize. In ICLR .

[32]

Yingying Li, Xin Chen, and Na Li. 2019. Online Optimal Control with Linear Dynamics and Predictions: Algorithms and Regret Analysis .Curran Associates Inc., Red Hook, NY, USA.

[33]

Yingying Li and Na Li. 2020. Leveraging Predictions in Smoothed Online Convex Optimization via Gradient-based Algorithms. In NeurIPS, Vol. 33. https://proceedings.neurips.cc/paper/2020/file/a6e4f250fb5c56aaf215a236c64e5b0a-Paper.pdf

[34]

Yingying Li, Guannan Qu, and Na Li. 2020. Online optimization with predictions and switching costs: Fast algorithms and the fundamental limit. IEEE Trans. Automat. Control (2020).

[35]

F. Liang, C. Shen, W. Yu, and F. Wu. 2020. Towards Optimal Power Control via Ensembling Deep Neural Networks. IEEE Transactions on Communications, Vol. 68, 3 (2020), 1760--1776.

[36]

M. Lin, A. Wierman, L. L. H. Andrew, and E. Thereska. 2011. Dynamic right-sizing for power-proportional data centers. In INFOCOM .

[37]

Yiheng Lin, Gautam Goel, and Adam Wierman. 2020. Online Optimization with Predictions and Non-Convex Losses. Proc. ACM Meas. Anal. Comput. Syst., Vol. 4, 1, Article 18 (May 2020), bibinfonumpages32 pages. https://doi.org/10.1145/3379484

Digital Library

[38]

Heyuan Liu and Paul Grigas. 2021. Risk Bounds and Calibration for a Smart Predict-then-Optimize Method. In Advances in Neural Information Processing Systems, A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (Eds.). https://openreview.net/forum?id=pSitk34qYit

[39]

Sijia Liu, Pin-Yu Chen, Bhavya Kailkhura, Gaoyuan Zhang, Alfred O Hero III, and Pramod K Varshney. 2020. A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications. IEEE Signal Processing Magazine, Vol. 37, 5 (2020), 43--54.

[40]

Zhenhua Liu, Iris Liu, Steven Low, and Adam Wierman. 2014. Pricing data center demand response. In SIGMETRICS .

[41]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing Data Using t-SNE . Journal of machine learning research, Vol. 9, Nov (2008), 2579--2605.

[42]

Sulav Malla, Qingyuan Deng, Zoh Ebrahimzadeh, Joe Gasperetti, Sajal Jain, Parimala Kondety, Thiara Ortiz, and Debra Vieira. 2020. Coordinated Priority-aware Charging of Distributed Batteries in Oversubscribed Data Centers. In MICRO. IEEE.

[43]

Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. 2018. Foundations of Machine Learning .MIT press.

Digital Library

[44]

Francesco Orabona. 2019. A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019).

[45]

Daniel Paulin. 2015. Concentration inequalities for Markov chains by Marton couplings and spectral methods. Electronic Journal of Probability, Vol. 20 (2015), 1--32.

[46]

Daan Rutten, Nico Christianson, Debankur Mukherjee, and Adam Wierman. 2022. Online Optimization with Untrusted Predictions. CoRR, Vol. abs/2202.03519 (2022). showeprint[arXiv]2202.03519 https://arxiv.org/abs/2202.03519

[47]

Daan Rutten and Debankur Mukherjee. 2022. Capacity Scaling Augmented With Unreliable Machine Learning Predictions. SIGMETRICS Perform. Eval. Rev., Vol. 49, 2 (jan 2022), 24--26. https://doi.org/10.1145/3512798.3512808

Digital Library

[48]

Asis Sarkar and Dhiren Kumar Behera. 2012. Wind turbine blade efficiency and power calculation with electrical analogy. International Journal of Scientific and Research Publications, Vol. 2, 2 (2012), 1--5.

[49]

Manajit Sengupta, Yu Xie, Anthony Lopez, Aron Habte, Galen Maclaurin, and James Shelby. 2018. The national solar radiation data base (NSRDB). Renewable and Sustainable Energy Reviews, Vol. 89 (2018), 51--60.

[50]

R. K. Sharma, C. E. Bash, C. D. Patel, R. J. Friedrich, and J. S. Chase. 2005. Balance of Power: Dynamic Thermal Management for Internet Data Centers. IEEE Internet Computing, Vol. 9, 1 (2005), 42--49.

Digital Library

[51]

Michael Sheppy, Chad Lobato, Otto Van Geet, Shanti Pless, Kevin Donovan, and Chuck Powers. 2011. Reducing Data Center Loads for a Large-Scale, Low-Energy Office Building: NREL's Research Support Facility.

[52]

Dongqing Shi, Emmanuel G. Collins Jr, Brian Goldiez, Arturo Donate, Xiuwen Liu, and Damion Dunlap. 2008. Human-aware Robot Motion Planning with Velocity Constraints. In 2008 International Symposium on Collaborative Technologies and Systems. 490--497. https://doi.org/10.1109/CTS.2008.4543969

[53]

Guanya Shi, Yiheng Lin, Soon-Jo Chung, Yisong Yue, and Adam Wierman. 2020. Online Optimization with Memory and Competitive Control. In NeurIPS, Vol. 33. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2020/file/ed46558a56a4a26b96a68738a0d28273-Paper.pdf

[54]

Qihang Sun, Shaolei Ren, Chuan Wu, and Zongpeng Li. 2016. An Online Incentive Mechanism for Emergency Demand Response in Geo-Distributed Colocation Data Centers. In eEnergy .

[55]

U.S. DoE. 2006. Benefits of Demand Response in Electricity Markets and Recommendations for Achieving Them.

[56]

Can Wan, Jian Zhao, Yonghua Song, Zhao Xu, Jin Lin, and Zechun Hu. 2015. Photovoltaic and solar power forecasting for smart grid energy management. CSEE Journal of Power and Energy Systems, Vol. 1, 4 (2015), 38--46.

[57]

Kai Wang, Sanket Shah, Haipeng Chen, Andrew Perrault, Finale Doshi-Velez, and Milind Tambe. 2021. Learning MDPs from Features: Predict-Then-Optimize for Sequential Decision Making by Reinforcement Learning. In Advances in Neural Information Processing Systems, A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (Eds.). https://openreview.net/forum?id=-mGv2KxQ43D

[58]

Olga Wichrowska, Niru Maheswaranathan, Matthew W Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Nando Freitas, and Jascha Sohl-Dickstein. 2017. Learned optimizers that scale and generalize. In International Conference on Machine Learning . 3751--3760.

[59]

Adam Wierman, Zhenhua Liu, Iris Liu, and Hamed Mohsenian-Rad. 2014. Opportunities and Challenges for Data Center Demand Response. In IGCC .

[60]

Bryan Wilder, Bistra Dilkina, and Milind Tambe. 2019. Melding the data-decisions pipeline: Decision-focused learning for combinatorial optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 1658--1665.

Digital Library

[61]

Jingzhao Zhang, Aditya Krishna Menon, Andreas Veit, Srinadh Bhojanapalli, Sanjiv Kumar, and Suvrit Sra. 2021 b. Coping with Label Shift via Distributionally Robust Optimisation. In International Conference on Learning Representations . https://openreview.net/forum?id=BtZhsSGNRNi

[62]

Lijun Zhang, Wei Jiang, Shiyin Lu, and Tianbao Yang. 2021 a. Revisiting Smoothed Online Learning. In Advances in Neural Information Processing Systems, A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (Eds.). https://openreview.net/forum?id=sn0wj3Dci2J

[63]

Martin Zinkevich. 2003. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (icml-03) . 928--936.

Digital Library

Cited By

Piankarnka VLertbumroongchai KPiriyasurawong P(2023)A Digital Painting Learning Model Using Mixed-Reality Technology to Develop Practical Skills in Character Design for AnimationAdvances in Human-Computer Interaction10.1155/2023/52307622023Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1155/2023/5230762
Lechowicz AChristianson NZuo JBashir NHajiesmaili MWierman AShenoy P(2023)The Online Pause and Resume Problem: Optimal Algorithms and An Application to Carbon-Aware Load ShiftingProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/36267767:3(1-32)Online publication date: 12-Dec-2023
https://dl.acm.org/doi/10.1145/3626776
Li TYang RQu GShi GYu CWierman ALow S(2022)Robustness and Consistency in Linear Quadratic Control with Untrusted PredictionsACM SIGMETRICS Performance Evaluation Review10.1145/3547353.352265850:1(107-108)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3547353.3522658

Index Terms

Expert-Calibrated Learning for Online Optimization with Switching Costs
1. Computing methodologies
  1. Machine learning
  2. Symbolic and algebraic manipulation
    1. Symbolic and algebraic algorithms
      1. Optimization algorithms

Recommendations

Expert-Calibrated Learning for Online Optimization with Switching Costs
SIGMETRICS/PERFORMANCE '22: Abstract Proceedings of the 2022 ACM SIGMETRICS/IFIP PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems
Expert-Calibrated Learning for Online Optimization with Switching Costs
SIGMETRICS '22
Learning Global Optimization by Deep Reinforcement Learning
Intelligent Systems
Abstract
Learning to Optimize (L2O) is a growing field that employs a variety of machine learning (ML) methods to learn optimization algorithms automatically from data instead of developing hand-engineered algorithms that usually require hyperparameter ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Measurement and Analysis of Computing Systems

Proceedings of the ACM on Measurement and Analysis of Computing Systems Volume 6, Issue 2

POMACS

June 2022

499 pages

EISSN:2476-1249

DOI:10.1145/3543145

Editors:
Augustin Chaintreau
Columbia University
,
Leana Golubchik
University of Southern California
,
Zhi-Li Zhang
University of Minnesota

Issue’s Table of Contents

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 June 2022

Published in POMACS Volume 6, Issue 2

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF (National Science Foundation)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
338
Total Downloads

Downloads (Last 12 months)184
Downloads (Last 6 weeks)21

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Piankarnka VLertbumroongchai KPiriyasurawong P(2023)A Digital Painting Learning Model Using Mixed-Reality Technology to Develop Practical Skills in Character Design for AnimationAdvances in Human-Computer Interaction10.1155/2023/52307622023Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1155/2023/5230762
Lechowicz AChristianson NZuo JBashir NHajiesmaili MWierman AShenoy P(2023)The Online Pause and Resume Problem: Optimal Algorithms and An Application to Carbon-Aware Load ShiftingProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/36267767:3(1-32)Online publication date: 12-Dec-2023
https://dl.acm.org/doi/10.1145/3626776
Li TYang RQu GShi GYu CWierman ALow S(2022)Robustness and Consistency in Linear Quadratic Control with Untrusted PredictionsACM SIGMETRICS Performance Evaluation Review10.1145/3547353.352265850:1(107-108)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3547353.3522658

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents