Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Deep Learning Quantile Regression for Robustness, Confidence and Planning

  • Chapter
  • First Online:
Artificial Intelligence and Cybersecurity

Abstract

It is well known that the estimator that minimizes the mean square error loss is the mean. However, the latter is prune to outliers and additionally, in many practical use cases, such as regression confidence bounds for risk, one would like to predict other statistics such as the median, or a certain percentile. In this chapter we will introduce the concept of Quantile Regression, a method based on a unique loss function which allows predicting quantiles of the conditional distribution of the target given the input. Albeit the method is useful and the loss is differential, thus fits modern deep neural network approach, most data science practitioners are not aware for its existence. We hope this chapter will reveal the theory and the know-hows of this methods and will allow building better machine learning systems. In addition, we discuss some recently developed approaches of how to optimize a general loss function under a quantile constraint.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Abeywardana, S.: Deep quantile regression. https://towardsdatascience.com/deep-quantile-regression-c85481548b5a. Accessed 19 Sept 2021

  2. Bottou, L.: Stochastic gradient descent tricks. In: Neural networks: Tricks of the trade, pp. 421–436. Springer, Berlin (2012)

    Google Scholar 

  3. Cannon, A.J.: Quantile regression neural networks: Implementation in r and application to precipitation downscaling. Comput. Geosci. 37(9), 1277–1284 (2011)

    Article  Google Scholar 

  4. Chiu, C.-H., Wen, T.-H., Chien, L.-C., Yu, H.-L.: A probabilistic spatial dengue fever risk assessment by a threshold-based-quantile regression method. PloS One 9(10), e106334 (2014)

    Article  Google Scholar 

  5. Elad, M.: Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer, Berlin (2010)

    Book  MATH  Google Scholar 

  6. Evangelou, M., Adams, N.M.: An anomaly detection framework for cyber-security data. Comput. Secur. 97, 101941 (2020)

    Article  Google Scholar 

  7. Fazlyab, M., Robey, A., Hassani, H., Morari, M., Pappas, G.: Efficient and accurate estimation of Lipschitz constants for deep neural networks. In: Advances in Neural Information Processing Systems, pp. 11427–11438 (2019)

    Google Scholar 

  8. Gass, S.I.: Linear Programming: Methods and Applications. Courier Corporation (2003)

    Google Scholar 

  9. He, Y., Qin, Y., Wang, S., Wang, X., Wang, C.: Electricity consumption probability density forecasting method based on lasso-quantile regression neural network. Appl. Energy 233, 565–575 (2019)

    Article  Google Scholar 

  10. He, Y., Xu, Q., Wan, J., Yang, S.: Short-term power load probability density forecasting based on quantile regression neural network and triangle kernel function. Energy 114, 498–512 (2016)

    Article  Google Scholar 

  11. Koenker, R., Hallock, K.F.: Quantile regression. J. Econ. Perspect. 15(4), 143–156 (2001)

    Article  Google Scholar 

  12. Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Program. 45(1), 503–528 (1989)

    Article  MATH  Google Scholar 

  13. Meinshausen, N., Ridgeway, G.: Quantile regression forests. J. Mach. Learn. Res. 7(6) (2006)

    Google Scholar 

  14. Or, D.B., Kolomenkin, M., Shabat, G.: DL-DDA–Deep learning based dynamic difficulty adjustment with ux and gameplay constraints. In: IEEE Conference on Games (2021). https://ieee-cog.org/2021/assets/papers/paper_142.pdf

  15. Pradeepkumar, D., Ravi, V.: Forecasting financial time series volatility using particle swarm optimization trained quantile regression neural network. Appl. Soft Comput. 58, 35–52 (2017)

    Article  Google Scholar 

  16. Rodrigues, F., Pereira, F.C.: Beyond expectation: deep joint mean and quantile regression for spatiotemporal problems. In: IEEE Transactions on Neural Networks and Learning Systems (2020)

    Google Scholar 

  17. Ruder, S.: An overview of gradient descent optimization algorithms (2016). arXiv preprint arXiv:1609.04747

    Google Scholar 

  18. Silverman, B.W.: Some aspects of the spline smoothing approach to non-parametric regression curve fitting. J. R. Stat. Soc. Series B (Methodological) 47(1), 1–21 (1985)

    Google Scholar 

  19. Taylor, J.W.: A quantile regression neural network approach to estimating the conditional density of multiperiod returns. J. Forecasting 19(4), 299–311 (2000)

    Article  Google Scholar 

  20. Wang, H., Ruan, J., Zhou, B., Li, C., Wu, Q., Raza, M.Q., Cao, G.-Z.: Dynamic data injection attack detection of cyber physical power systems with uncertainties. IEEE Trans. Ind. Inform. 15(10), 5505–5518 (2019)

    Article  Google Scholar 

  21. Xu, Q., Deng, K., Jiang, C., Sun, F., Huang, X.: Composite quantile regression neural network with applications. Expert Syst. Appl. 76, 129–139 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Gil Shabat .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Or, D.B., Kolomenkin, M., Osokin, T., Shabat, G., Shteingart, H. (2023). Deep Learning Quantile Regression for Robustness, Confidence and Planning. In: Sipola, T., Kokkonen, T., Karjalainen, M. (eds) Artificial Intelligence and Cybersecurity. Springer, Cham. https://doi.org/10.1007/978-3-031-15030-2_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-15030-2_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-15029-6

  • Online ISBN: 978-3-031-15030-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics