Abstract
In ranking as well as in classification problems, the Area under the ROC Curve (AUC), or the equivalent Wilcoxon-Mann-Whitney statistic, has recently attracted a lot of attention. We show that the AUC can be lower bounded based on the hinge-rank-loss, which simply is the rank-version of the standard (parametric) hinge loss. This bound is asymptotically tight. Our experiments indicate that optimizing the (standard) hinge loss typically is an accurate approximation to optimizing the hinge rank loss, especially when using affine transformations of the data, like e.g. in ellipsoidal machines. This explains for the first time why standard training of support vector machines approximately maximizes the AUC, which has indeed been observed in many experiments in the literature.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Schölkopf, B., Smola, A., Müller, K.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)
Shivaswamy, P., Jebara, T.: Ellipsoidal machines. In: Proc. Int. Conf. on Artificial Intelligence and Statistics, pp. 481–488 (2007)
Rakotomamonjy, A.: Optimizing area ROC curve with SVMs. In: workshop ”ROC Analysis in AI” at the European Conference on Artificial Intelligence (2004)
Brefeld, U., Scheffer, T.: AUC maximizing support vector learning. In: workshop ”ROC Analysis in Machine Learning” at Int. Conf. on Machine Learning (2005)
Joachims, T.: A support vector method for multivariate performance measures. In: Proc. Int. Conf. on Machine Learning, pp. 377–384 (2005)
Caruana, R., Niculescu-Mizil, A.: Data mining in metric space: an empirical analysis of supervised learning performance criteria. In: Proc. Int. Conf. on Knowledge Discovery and Data Mining, pp. 69–78 (2004)
Hand, D.J., Till, R.J.: A simple generalization of the area under the ROC curve for multiple class classification problems. Machine Learning 45, 171–186 (2001)
Wu, S., Flach, P.: A scored AUC metric for classifier evaluation and selection. In: workshop ”ROC Analysis in Machine Learning” at Int. Conf. on Machine Learning (2005)
Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics 1, 80–83 (1945)
Mann, H.B., Whitney, D.R.: On a test whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18, 50–60 (1947)
Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20, 273–297 (1995)
Agarwal, S., Niyogi, P.: Stability and generalization of bipartite ranking algorithms. In: Proc. Conf. on Learning Theory, pp. 32–47 (2005)
Rudin, C., Cortes, C., Mohri, M., Schapire, R.: Margin-based ranking meets boosting in the middle. In: Proc. Conf. on Learning Theory, pp. 63–78 (2005)
Yan, L., Dodier, R., Mozer, M.C., Wolniewicz, R.: Optimizing classifier performance via the Wilcoxon-Mann-Whitney statistics. In: Proc. Int. Conf. on Machine Learning, pp. 848–855 (2003)
Herschtal, A., Raskutti, B.: Optimising the area under the ROC curve using gradient descent. In: Proc. Int. Conf. on Machine Learning, pp. 49–56 (2004)
Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: A statistical view of boosting. The Annals of Statistics 38, 337–374 (2000)
Freund, Y., Iyer, R., Schapire, R., Singer, Y.: An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research 4, 933–969 (2003)
Cortes, C., Mohri, M.: AUC optimization vs. error rate minimization. Advances in Neural Information Processing Systems 16, 313–320 (2003)
Cortes, C., Mohri, M.: Confidence intervals for the area under the ROC curve. Advances in Neural Information Processing Systems 17, 305–312 (2004)
Agarwal, S., Graepel, T., Herbrich, R., Har-Peled, S., Roth, D.: Generalization bounds for the area under the ROC curve. Journal of Machine Learning Research 6, 393–425 (2005)
Ferri, C., Flach, P., Hernandez-Orallo, J.: Learning decision trees using the area under the ROC curve. In: Proc. Int. Conf. on Machine Learning, pp. 139–146 (2002)
Herbrich, R., Graepel, T., Obermayer, K.: Support vector learning for ordinal regression. In: Proc. Int. Conf. on Neural Networks, pp. 97–102 (1999)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Steck, H. (2007). Hinge Rank Loss and the Area Under the ROC Curve. In: Kok, J.N., Koronacki, J., Mantaras, R.L.d., Matwin, S., Mladenič, D., Skowron, A. (eds) Machine Learning: ECML 2007. ECML 2007. Lecture Notes in Computer Science(), vol 4701. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74958-5_33
Download citation
DOI: https://doi.org/10.1007/978-3-540-74958-5_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74957-8
Online ISBN: 978-3-540-74958-5
eBook Packages: Computer ScienceComputer Science (R0)