article

Selective Sampling for Nearest Neighbor Classifiers

Authors:

Michael Lindenbaum,

Shaul Markovitch,

Dmitry RusakovAuthors Info & Claims

Machine Learning, Volume 54, Issue 2

Pages 125 - 152

https://doi.org/10.1023/B:MACH.0000011805.60520.fe

Published: 01 February 2004 Publication History

Abstract

Most existing inductive learning algorithms work under the assumption that their training examples are already tagged. There are domains, however, where the tagging procedure requires significant computation resources or manual labor. In such cases, it may be beneficial for the learner to be active, intelligently selecting the examples for labeling with the goal of reducing the labeling cost. In this paper we present LSS—a lookahead algorithm for selective sampling of examples for nearest neighbor classifiers. The algorithm is looking for the example with the highest utility, taking its effect on the resulting classifier into account. Computing the expected utility of an example requires estimating the probability of its possible labels. We propose to use the random field model for this estimation. The LSS algorithm was evaluated empirically on seven real and artificial data sets, and its performance was compared to other selective sampling algorithms. The experiments show that the proposed algorithm outperforms other methods in terms of average error rate and stability.

References

[1]

Adler, R. J. (1981). The Geometry of Random Fields. John Wiley & Sons.]]

[2]

Aha, D. W., Kibler, D., & Albert, M. K. (1991). Instance-based learning algorithms. Machine Learning, 6:1, 37-66.]]

Digital Library

[3]

Angluin, D. (1988). Queries and concept learning. Machine Learning, 2:3, 319-342.]]

Digital Library

[4]

Blake, C., Keogh, E., & Merz, C. (1998). UCI Repository of machine learning databases, University of California, Irvine. {http://www.ics.uci.edu/~mlearn/MLRepository.html}.]]

[5]

Cohn, D. A., Atlas, L., & Lander, R. (1994). Improving generalization with active learning. Machine Learning, 15:2, 201-221.]]

[6]

Cohn, D. A., Ghahramani, Z., & Jordan, M. I. (1995). Active learning with statistical models. In G. Tesauro, D. Touretzky, & T. Leen (Eds.), Advances in Neural Information Processing Systems, vol. 7 (pp. 705-712). The MIT Press.]]

[7]

Cover, T. M., & Hart, P. E. (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13:1, 21-27.]]

Digital Library

[8]

Dagan, I., & Engelson, S. P. (1995). Committee-based sampling for training probabilistic classifiers. In A. Prieditis, & S. Russell (Eds.), Proceedings of the Twelfth International Conference on Machine Learning (pp. 150-157). Morgan Kaufmann.]]

[9]

Davis, D. T., & Hwang, J.-N. (1992). Attentional focus training by boundary region data selection. In Proceedings of International Joint Conference on Neural Networks, vol. 1 (pp. 676-681). IEEE Press.]]

[10]

DeGroot, M. H. (1986). Probability and Statistics, 2nd ed. Addison-Wesley.]]

[11]

Duda, R. O., & Hart, P. E. (1973). Pattern Classification and Scene Analysis. Wiley-Interscience.]]

[12]

Eldar, Y., Lindenbaum, M., Porat, M., & Zeevi, Y. Y. (1997). The farthest point strategy for progressive image sampling. IEEE Transactions on Image Processing, 6:9, 1305-1315.]]

Digital Library

[13]

Fedorov, V. V. (1972). Theory of Optimal Experiments. New York: Academic Press. Translation of Teoriia Optimalnogo Eksperimenta.]]

[14]

Freund, Y., Seung, H. S., Shamir, E., & Tishbi, N. (1997). Selective sampling using the query by committee algorithm. Machine Learning, 28:2/3, 133-168.]]

Digital Library

[15]

Hasenjager, M., & Ritter, H. (1996). Active learning of the generalized high-low-game. In C. von der Malsburg, W. von Seelen, J. C. Vorbrüggen, & B. Sendhoff (Eds.), Proceedings of the International Conference on Artificial Neural Networks, vol. 1112 of Lecture Notes in Computer Science (pp. xxv+922, 501-506). Springer-Verlag.]]

Digital Library

[16]

Hasenjager, M., & Ritter, H. (1998). Active learning with local models. Neural Processing Letters, 7:2, 107-117.]]

Digital Library

[17]

Lang, K. J., & Witbrock, M. J. (1988). Learning to tell two spirals apart. In D. Touretzky, G. Hinton, & T. Sejnowski (Eds.), Proceedings of the Connectionist Models Summer School (pp. 52-59). Morgan Kaufmann.]]

[18]

Lewis, D. D., & Catlett, J. (1994). Heterogeneous uncertainty sampling for supervised learning. In W. W. Cohen, & H. Hirsh (Eds.), Proceedings of the Eleventh International Conference on Machine Learning (pp. 148-156). New Brunswick, NJ: Rutgers University.]]

[19]

Lindenbaum, M., Markovitch, S., & Rusakov, D. (1999). Selective sampling for nearest neighbor classifiers. In Proceedings of the Sixteenth National Conference on Artificial Intelligence (pp. 366-371). Orlando, Florida: AAAI Press.]]

Digital Library

[20]

MacKay, D. J. (1998). Introduction to gaussian processes. Neural Networks and Machine Learning. NATO ASI Series. Series F, Computer and System Sciences, 168, 133-166.]]

[21]

MacKay, D. J. C. (1992). Information-based objective functions for active data selection. Neural Computation, 4:4, 590-604.]]

Digital Library

[22]

Moore, A. W., Schneider, J. G., Boyan, J. A., & Lee, M. S. (1998). Q2: Memory-based active learning for optimizing noisy continuous functions. In Proceedings of the Fifteenth International Conference on Machine Learning (pp. 386-394). Morgan Kaufmann.]]

Digital Library

[23]

Murthy, S., & Salzberg, S. (1995). Lookahead and pathology in decision tree induction. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (pp. 1025-1031).]]

[24]

Papoulis, A. (1991). Probability, Random Variables, and Stochastic Processes, 3rd ed. McGraw-Hill.]]

[25]

RayChaudhuri, T., & Hamey, L. (1995). Minimisation of data collection by active learning. In IEEE International Conference on Neural Networks Proceedings, vol. 3 (pp. 6 vol. 1+3219, 1338-1341). IEEE Press.]]

[26]

Russell, S. J., & Norvig, P. (1995). Artificial Intelligence, A Modern Approach. Prentice Hall.]]

Digital Library

[27]

Seung, H. S., Opper, M., & Sompolinsky, H. (1992). Query by committee. In Proceedings of the Fifth Annual ACM Workshop on Computational Learning Theory (pp. v+452, 287-294). New York: ACM.]]

Digital Library

[28]

Smyth, B., & McKenna, E. (1999). Building compact competent case-bases. In Lecture Notes in Artificial Intelligence , number 1650 in Lecture Notes in Computer Science (pp. 329-342). Springer.]]

Digital Library

[29]

Tan, M., & Schlimmer, J. C. (1990). Two case studies in cost-sensitive concept acquisition. In Proceedings of the Eighth National Conference on Artificial Intelligence (pp. 854-860). AAAI Press.]]

[30]

Turney, P. D. (1995). Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. Journal of Artificial Intelligence Research, 2, 369-409.]]

Digital Library

[31]

Williams, C. K. I., & Barber, D. (1998). Bayesian classification with Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20:12, 1342.]]

Digital Library

[32]

Wilson, D. R., & Martinez, T. R. (2000). Reduction techniques for instance-based learning algorithms. Machine Learning, 38:3, 257-286.]]

Digital Library

[33]

Wong, E., & Hajek, B. (1985). Stochastic Processes in Engineering Systems. Springer-Verlag.]]

[34]

Zhang, J., Yim, Y.-S., & Yang, J. (1997). Intelligent selection of instances for prediction functions in lazy learning algorithms. Artificial Intelligence Review, 11:1-5, 175-191.]]

Digital Library

Cited By

Zhu QZhou MHuang JZheng NGao HLi CXv YZhao FOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)FouriDownProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666743(14094-14112)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666743
Nguyen VShaker MHüllermeier E(2022)How to measure uncertainty in uncertainty sampling for active learningMachine Language10.1007/s10994-021-06003-9111:1(89-122)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1007/s10994-021-06003-9
Gao JBurghardt TCampbell N(2022)Label a Herd in Minutes: Individual Holstein-Friesian Cattle IdentificationImage Analysis and Processing. ICIAP 2022 Workshops10.1007/978-3-031-13324-4_33(384-396)Online publication date: 23-May-2022
https://dl.acm.org/doi/10.1007/978-3-031-13324-4_33
Show More Cited By

Index Terms

Selective Sampling for Nearest Neighbor Classifiers
1. Computing methodologies
  1. Machine learning
2. Mathematics of computing
  1. Mathematical analysis
    1. Quadrature

Recommendations

On kernel difference-weighted k-nearest neighbor classification
Special Issue: Non-parametric distance-based classification techniques and their applications

Nearest neighbor (NN) rule is one of the simplest and the most important methods in pattern recognition. In this paper, we propose a kernel difference-weighted k-nearest neighbor (KDF-KNN) method for pattern classification. The proposed method defines ...
Learning noisy linear classifiers via adaptive and selective sampling

We introduce efficient margin-based algorithms for selective sampling and filtering in binary classification tasks. Experiments on real-world textual data reveal that our algorithms perform significantly better than popular and similarly efficient ...
Selective sampling for trees and forests
Abstract
In this paper we describe selective sampling algorithms for decision trees and random forests and their contribution to the classification accuracy. In our selective sampling algorithms, the instance that yields the highest expected ...

Comments

Information & Contributors

Information

Published In

cover image Machine Language

Machine Language Volume 54, Issue 2

February 2004

80 pages

ISSN:0885-6125

Issue’s Table of Contents

Copyright © Copyright © 2004 Kluwer Academic Publishers.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 February 2004

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

54
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhu QZhou MHuang JZheng NGao HLi CXv YZhao FOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)FouriDownProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666743(14094-14112)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666743
Nguyen VShaker MHüllermeier E(2022)How to measure uncertainty in uncertainty sampling for active learningMachine Language10.1007/s10994-021-06003-9111:1(89-122)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1007/s10994-021-06003-9
Gao JBurghardt TCampbell N(2022)Label a Herd in Minutes: Individual Holstein-Friesian Cattle IdentificationImage Analysis and Processing. ICIAP 2022 Workshops10.1007/978-3-031-13324-4_33(384-396)Online publication date: 23-May-2022
https://dl.acm.org/doi/10.1007/978-3-031-13324-4_33
Chong QKnottenbelt WBhatia K(2021)Evaluation of Active Learning Techniques on Medical Image Classification with Unbalanced Data DistributionsDeep Generative Models, and Data Augmentation, Labelling, and Imperfections10.1007/978-3-030-88210-5_23(235-242)Online publication date: 1-Oct-2021
https://dl.acm.org/doi/10.1007/978-3-030-88210-5_23
Li LZhao KLi SSun RCai S(2020)Extreme Learning Machine for Supervised Classification with Self-paced LearningNeural Processing Letters10.1007/s11063-020-10286-952:3(1723-1744)Online publication date: 1-Dec-2020
https://dl.acm.org/doi/10.1007/s11063-020-10286-9
He ZLi LZheng SZou XSitu H(2019)Quantum speedup for pool-based active learningQuantum Information Processing10.1007/s11128-019-2460-x18:11(1-11)Online publication date: 1-Nov-2019
https://dl.acm.org/doi/10.1007/s11128-019-2460-x
Lv JZhao HChen RZhan JLi JLin KLi C(2019)Semi-supervised Batch Mode Active Learning for Multi-class ClassificationAdvances in Brain Inspired Cognitive Systems10.1007/978-3-030-39431-8_12(117-127)Online publication date: 13-Jul-2019
https://dl.acm.org/doi/10.1007/978-3-030-39431-8_12
Yoo SKalatzis AAmini NYe ZPourhomayoun M(2018)Interactive Predictive Analytics for Enhancing Patient Adherence in Remote Health MonitoringProceedings of the 8th ACM MobiHoc 2018 Workshop on Pervasive Wireless Healthcare Workshop10.1145/3220127.3220131(1-6)Online publication date: 26-Jun-2018
https://dl.acm.org/doi/10.1145/3220127.3220131
Zhang MWang CYu ZShen CBu J(2017)Active learning for Web accessibility evaluationProceedings of the 14th International Web for All Conference10.1145/3058555.3058559(1-9)Online publication date: 2-Apr-2017
https://dl.acm.org/doi/10.1145/3058555.3058559
Liu YLiu KZhang CWang JWang X(2017)Unsupervised feature selection via Diversity-induced Self-representationNeurocomputing10.1016/j.neucom.2016.09.043219:C(350-363)Online publication date: 5-Jan-2017
https://dl.acm.org/doi/10.1016/j.neucom.2016.09.043
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents