Mixtures of large margin nearest neighbor classifiers
Pages 675 - 688
Abstract
The accuracy of the k-nearest neighbor algorithm depends on the distance function used to measure similarity between instances. Methods have been proposed in the literature to learn a good distance function from a labelled training set. One such method is the large margin nearest neighbor classifier that learns a global Mahalanobis distance. We propose a mixture of such classifiers where a gating function divides the input space into regions and a separate distance function is learned in each region in a lower dimensional manifold. We show that such an extension improves accuracy and allows visualization.
References
[1]
Xing, P.E., Ng, A.Y., Jordan, M.I., Russell, S.: Distance Metric Learning, with Application to Clustering with Side-Information. In: Advances in Neural Information Processing Systems 15, pp. 505-512. MIT Press, Cambridge (2002)
[2]
Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighbourhood Components Analysis. In: Advances in Neural Information Processing Systems 17, pp. 513-520. MIT Press (2004)
[3]
Salakhutdinov, R., Hinton, G.: Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure. In: 11th International Conference on Artificial Intelligence and Statistics, vol. 2, pp. 412-419 (2007)
[4]
Frome, A., Singer, Y., Malik, J.: Image Retrieval and Classification Using Local Distance Functions. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, vol. 19, pp. 417-424. MIT Press, Cambridge (2007)
[5]
Frome, A., Singer, Y., Sha, F., Malik, J.: Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification. In: 11st IEEE International Conference on Computer Vision, pp. 1-8 (2007)
[6]
Chang, H., Yeung, D.Y.: Locally Smooth Metric Learning with Application to Image Retrieval. In: 11th IEEE International Conference on Computer Vision, pp. 1-7 (2007)
[7]
Davis, J.V., Kulis, B., Jain, P., Sra, S., Dhillon, I.S.: Information-Theoretic Metric Learning. In: 24th International Conference on Machine Learning, pp. 209-216 (2007)
[8]
Weinberger, K.Q., Saul, L.K.: Distance Metric Learning for Large Margin Nearest Neighbor Classification. Journal of Machine Learning Research 10, 207-244 (2009)
[9]
Weinberger, K.Q., Saul, L.K.: Fast Solvers and Efficient Implementations for Distance Metric Learning. In: 25th International Conference on Machine Learning, pp. 1160-1167 (2008)
[10]
Torresani, L., Lee, K.C.: Large Margin Component Analysis. In: Advances in Neural Information Processing Systems, vol. 19, pp. 1385-1392 (2007)
[11]
Malisiewicz, T., Efros, A.A.: Recognition by Association via Learning Per-Exemplar Distances. In: 21st IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8 (2008)
[12]
Zhan, D.C., Li, M., Li, Y.F., Zhou, Z.H.: Learning Instance Specific Distances Using Metric Propagation. In: 26th International Conference on Machine Learning, pp. 1225-1232 (2009)
[13]
Chen, Q., Sun, S.: Hierarchical Large Margin Nearest Neighbor Classification. In: 20th International Conference on Pattern Recognition. 906-909 (2010)
[14]
Noh, Y.K., Zhang, B.T., Lee, D.D.: Generative Local Metric Learning for Nearest Neighbor Classification. In: Advances in Neural Information Processing Systems 23, pp. 1822-1830. MIT Press (2010)
[15]
Chang, C.C.: A Boosting Approach for Supervised Mahalanobis Distance Metric Learning. Pattern Recognition 45(2), 844-862 (2012)
[16]
Bunte, K., Schneider, P., Hammer, B., Schleif, F.M., Villmann, T., Biehl, M.: Limited Rank Matrix Learning, Discriminative Dimension Reduction and Visualization. Neural Networks 26, 159-173 (2012)
[17]
Wang, Y., Woznica, A., Kalousis, A.: Parametric Local Metric Learning for Nearest Neighbor Classification. In: Advances in Neural Information Processing Systems, vol. 25, pp. 1610-1618. MIT Press (2012)
[18]
Wu, L., Hoi, S.C.H., Jin, R., Zhu, J., Yu, N.: Learning Bregman Distance Functions for Semi-Supervised Clustering. IEEE Transactions on Knowledge and Data Engineering 24, 478-491 (2012)
[19]
Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive Mixtures of Experts. Neural Computation 3, 79-87 (1991)
[20]
Gönen, M., Alpaydın, E.: Localized Multiple Kernel Learning. In: 25th International Conference on Machine Learning, pp. 352-359 (2008)
[21]
Frank, A., Asuncion, A.: UCI Machine Learning Repository (2010), http://archive.ics.uci.edu/ml/
[22]
Chang, C.C., Lin, C.J.: LIBSVM Data: Classification, Regression, and Multi-label (2011), http://www.csie.ntu.edu.tw/cjlin/libsvmtools/datasets/
[23]
Sonnenburg, S., Ong, C.S., Henschel, S., Braun, M.: Machine Learning Data Set Repository (2011), http://mldata.org/
[24]
Alpaydın, E.: Combined 5×2 cv F Test for Comparing Supervised Classification Learning Algorithms. Neural Computation 11, 1885-1892 (1999)
Index Terms
- Mixtures of large margin nearest neighbor classifiers
Recommendations
Coarse to fine K nearest neighbor classifier
In this paper, we propose a coarse to fine K nearest neighbor (KNN) classifier (CFKNNC). CFKNNC differs from the conventional KNN classifier (CKNNC) as follows: CFKNNC first coarsely determines a small number of training samples that are ''close'' to ...
Design of nearest neighbor classifiers
The goal of designing optimal nearest neighbor classifiers is to maximize classification accuracy while minimizing the sizes of both reference and feature sets. A usual way is to adaptively weight the three objectives as an objective function and then ...
A new nearest neighbor classifier via fusing neighborhood information
The nearest neighbor (NN) classification is a classical and yet effective technique in machine learning and data mining communities. However, its performance depends crucially on the distance function used to compute distance between samples. In this ...
Comments
Information & Contributors
Information
Published In
September 2013
688 pages
ISBN:9783642409905
Sponsors
- XRCE: Xerox Research Centre Europe
- Winton Capital Management: Winton Capital Management
- Cisco Systems
- Yahoo! Labs
- CSKI: Czech Society for Cybernetics and Informatics
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 23 September 2013
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in