Abstract
In recent years, distributionally robust optimization (DRO) has received a lot of interest due to its ability to reduce the worst-case risk when there is a perturbation to the data-generating distribution. A good statistical model is expected to perform well on both the normal and the perturbed data. On the other hand, variable selection and regularization is a research area that aims to identify the important features and remove the redundant ones. It helps to improve the prediction accuracy as well as the interpretability of the model. In this paper, we propose an optimization model that is a regularized version of the canonical distributionally robust optimization (DRO) problem where the ambiguity set is described by a general class of divergence measures that admit a suitable conic structure. The divergence measures we examined include several popular divergence measures used in the literature such as the Kullback–Leibler divergence, total variation, and the Chi-divergence. By exploiting the conic representability of the divergence measure, we show that the regularized DRO problem can be equivalently reformulated as a nonlinear conic programming problem. In the case where the regularization is convex and semi-definite programming representable, the reformulation can be further simplified as a tractable linear conic program and hence can be efficiently solved via existing software. More generally, if the regularization can be written as a difference of convex functions, we demonstrate that a solution for the regularized DRO problem can be found by solving a sequence of conic linear programming problems. Finally, we apply the proposed regularized DRO model to both simulated and real financial data and demonstrate its superior performance in comparison with some non-robust models.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
In the numerical experiment, computations were performed on the computational cluster Katana supported by the Research Technology Services at UNSW Sydney.
For example, Zhang et al. (2019) considered a non-robust model using \(\ell _2+\ell _{\theta }\) regularization with \(\theta \in (0,1)\), and solved this model via the smoothing projected gradient algorithm. However, this algorithm uses many model-tuning parameters, and its convergence in practice is very sensitive to the tuning of those parameters in our experiment. Thus, we do not include it in the comparison here.
Here, we do not include the quantile Huber loss in the experiment for our regularized DRO model. This is because, for the quantile Huber loss function to perform well, it often requires one to have some prior knowledge about the historic data, while we do not have such knowledge in this case. Therefore, for simplicity, we will use the quadratic loss function in this subsection.
References
Aravkin, A., Lozano, A., Luss, R., & Kambadur, P. (2014). Orthogonal matching pursuit for sparse quantile regression. IEEE International Conference on Data Mining, 2014, 11–19.
Ben-Tal, A., Ghaoui, L. E., & Nemirovski, A. (2009). Robust Optimization. Princeton University Press.
Ben-Tal, A., Den Hertog, D., De Waegenaere, A., Melenberg, B., & Rennen, G. (2013). Robust solutions of optimization problems affected by uncertain probabilities. Management Science, 59(2), 341–357.
Blanchet, J., Kang, Y., & Murthy, K. (2019). Robust Wasserstein profile inference and applications to machine learning. Journal of Applied Probability, 56(3), 830–857.
Chen, X., Xu, F., & Ye, Y. (2010). Lower bound theory of nonzero entries in solutions of \(\mathbf{\ell _2-\ell _p} \) minimization. SIAM Journal on Scientific Computing, 32(5), 2832–2852.
Dentcheva, D., Penev, S., & Ruszczyński, A. (2010). Kusuoka representation of higher order dual risk measures. Annals of Operations Research, 181, 325–335.
Duchi, J. C., & Namkoong, H. (2018). Variance-based regularization with convex objectives. Journal of Machine Learning Research, 19, 1–55.
Duchi, J. C., & Namkoong, H. (2021). Learning model with uniform performance via distributionally robust optimization. The Annals of Statistics, 49(3), 1378–1406.
Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360.
Grant, M., & Boyd, S. (2014). CVX: Matlab Software for Disciplined Convex Programming, version 2.1. http://cvxr.com/cvx.
Huber, P. J. (1964). Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1), 73–101.
Knight, K., & Fu, W. (2000). Asymptotics for Lasso-Type estimators. The Annals of Statistics, 28(5), 1356–1378.
Kocuk, B. (2021). Conic reformulations for Kullback-Leibler divergence constrained distributionally robust optimization and applications. An International Journal of Optimization and Control: Theories & Applications, 11(2), 139–151.
Le Thi, H. A., & Pham Dinh, T. (2018). DC programming and DCA: Thirty years of developments. Mathematical Programming, 169, 5–68.
Le Thi, H. A., Pham Ding, T., Le, H. M., & Vo, X. T. (2015). DC approximation approaches for sparse optimization. European Journal of Operational Research, 244, 26–46.
Le Thi, H. A., Huynh, V. N., & Pham Ding, T. (2018). Convergence analysis of difference-of-convex algorithm with subanalytic data. Journal of Optimization Theory and Applications, 179, 103–126.
Liu, Y., Meskarian, R., & Xu, H. (2017). Distributionally robust reward-risk ratio optimization with moment constraints. SIAM Journal of Optimization, 27(2), 957–985.
Loh, Po-Ling., & Wainwright, M. (2015). Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima. Journal of Machine Learning Research., 16, 559–616.
MOSEK, A.P.S. (2019). The MOSEK optimization toolbox for MATLAB manual. Version 9.0. http://docs.mosek.com/9.0/toolbox/index.html.
MOSEK, A.P.S. (2019). The MOSEK modelling cookbook. Version 3.2.3. https://docs.mosek.com/modeling-cookbook/index.html.
Pflug, G., & Wozabal, D. (2007). Ambiguity in portfolio selection. Quantitative Finance, 7(4), 435–442.
Pham Ding, T., & Le Thi, H. A. (1997). Convex analysis approach to D.C. programming: Theory, algorithms and application. ACTA Mathematica Vietnamica, 22, 289–355.
Rahminian, M., & Mehrotra, S. (2022). Frameworks and results in distributionally robust optimization. Open Journal of Mathematical Optimization, 3(4), 85.
Rockafellar, R. T. (1972). Convex analysis. Princeton University Press.
Shapiro, A. (2017). Distributionally robust stochastic programming. SIAM Journal on Optimization, 27(4), 2258–2275.
Takeda, A., Niranjan, M., Gotoh, J., & Kawahara, Y. (2012). Simultaneous pursuit of out-of-sample performance and sparsity in index tracking portfolios. Computational Management Science, 10(1), 21–49.
Tibshirani, R. (1996). Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society Series B, 58(1), 267–288.
Villani, C. (2009). Optimal transport: Old and New. Springer Verlag.
Wozabal, D. (2012). A framework for optimization under ambiguity. Annals of Operations Research, 193(1), 21–47.
Wu, L., Yang, Y., & Liu, H. (2014). Nonnegative-lasso and application in index tracking. Computational Statistics and Data Analysis, 70, 114–126.
Zhang, C., Wang, J., & Xiu, N. (2019). Robust and sparse portfolio model for index tracking. Journal of Industrial and Management Optimization, 15(3), 1001–1015.
Zhang, C. H. (2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38(3), 894–942.
Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society Series B, 67(2), 301–320.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors have no competing interests to declare that are relevant to the content of this article. This article does not contain any studies with human participants performed by any of the authors. The second author was partially supported by a Discovery Project (DP210101025) from the Australian Research Council (ARC).
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhao, L., Li, G. & Penev, S. Regularized distributionally robust optimization with application to the index tracking problem. Ann Oper Res 337, 397–424 (2024). https://doi.org/10.1007/s10479-023-05726-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-023-05726-3