An improved nearest neighbour classifier
Abstract
A windowed version of the Nearest Neighbour (WNN) classifier for images is described. While its construction is inspired by the architecture of Artificial Neural Networks, the underlying theoretical framework is based on approximation theory. We illustrate WNN on the datasets MNIST and EMNIST of images of handwritten digits. In order to calibrate the parameters of WNN, we first study it on MNIST. We then apply WNN with these parameters to EMNIST resulting in an error rate of 0.76% which significantly outperforms traditional classification methods like Support Vector Machines. By expansions of the training set, an error rate down to 0.42% is achieved.
References
[1]
Bishop CM Pattern recognition and machine learning 2006 New York Springer-Verlag
[2]
Kruglyak N and Forchheimer R Design of classifiers based on ANN approximations of traditional methods Int J Circ Theor Appl 2021 49 1916-1931
[3]
Cohen G, Afshar S, Tapson J, van Schaik A (2017) EMNIST: an extension of MNIST to handwritten letters. arXiv:1702.05373
[4]
Le Cun Y, Cortes C, Burges CJC The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/
[5]
Fix E, Hodges Jr JL (1951) Discriminatory analysis, non-parametric discrimination. USAF School of Aviation Medicine, Randolph Field, Tex. Project 21-49-004, Report no 4, Contract AF41(128)-31
[6]
Silverman BW, Jones E, Fix MC, and Hodges JL An important contribution to nonparametric discriminant analysis and density estimation. Commentary on Fix and Hodges (1951) Int Stat Rev 1989 57 3 233-238
[7]
Cover TM and Hart PE Nearest neighbor pattern classification IEEE Trans Inf Theory 1967 13 1 21-27
[8]
Grover D and Toghi B Arai K and Kapoor S MNIST dataset classification utilizing k-NN classifier with modified sliding-window metric Advances in computer vision. CVC 2019 2019 Cham Springer 583-591 944
[9]
Keysers D, Deselaers T, Gollan C, and Ney H Deformation models for image recognition IEEE Trans Pattern Anal Mach Intell 2007 29 8 1422-1435
[10]
Baldominos A, Saez Y, and Isasi P A survey of handwritten character recognition with MNIST and EMNIST Appl Sci 2019 9 15 3169
[11]
Cavalin P and Oliveira L Vera-Rodriguez R, Fierrez J, and Morales A Confusion matrix-based building of hierarchical classification Progress in pattern recognition, image analysis, computer vision, and applications 2019 Cham Springer 271-278 11401
[12]
Pad P, Narduzzi S, Kündig C, Türetken E, Bigdeli SA, Dunbar LA (2020) Efficient neural vision systems based on convolutional image acquisition. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 12282–12291
[13]
Simard P, Le Cun Y, and Denker J Hanson S, Cowan J, and Giles C Efficient pattern recognition using a new transformation distance Advances in neural information processing systems 5 (NIPS 1992) 1993 USA Morgan Kaufmann 50-58
[14]
Shorten C and Khoshgoftaar TM A survey on image data augmentation for deep learning J Big Data 2019 6 1 1-48
[15]
Jayasundara V, Jayasekara S, Jayasekara H, Rajasegaran J, Seneviratne S, Rodrigo R (2019) Textcaps : handwritten character recognition with very small datasets. In: 2019 IEEE winter conference on applications of computer vision (WACV), pp 254–262
[16]
LeCun Y, Jackel LD, Bottou L, Cortes C, Denker JS, Drucker H, Guyon I, Müller UA, Sackinger E, Simard P, and Vapnik V Oh JH, Kwon C, and Cho S Learning algorithms for classification: a comparison on handwritten digit recognition Neural networks: the statistical mechanics perspective 1995 Singapore World Scientific 261-276 1
[17]
Ghadekar P, Ingole S, Sonone D (2018) Handwritten digit and letter recognition using hybrid DWT-DCT with KNN and SVM classifier. In: 2018 Fourth international conference on computing communication control and automation (ICCUBEA), pp 1–6
[18]
Dufourq E, Bassett BA (2017) EDEN: evolutionary deep networks for efficient machine learning. In: 2017 Pattern recognition association of South Africa and robotics and mechatronics (PRASA-RobMech), pp 110–115
[19]
Gesmundo A, Dean J (2022) An evolutionary approach to dynamic introduction of tasks in large-scale multitask learning systems. arXiv:2205.12755
[20]
Jeevan P, Viswanathan KA, Sethi A (2022) Wavemix: a resource-efficient neural network for image analysis. arXiv:2205.14375
[21]
Brudnyi Y Spaces defined by means of local approximations Trans Moscow Math Soc 1974 24 1422-1435
[22]
Peetre J A theory of interpolation of normed spaces 1968 Rio de Janeiro Instituto de Matemática Pura e Aplicada, Conselho Nacional de Pesquisas
[23]
Kislyakov S and Kruglyak N Extremal problems in interpolation theory, Whitney-Besicovitch coverings, and singular integrals 2013 Basel Birkhäuser
Index Terms
- An improved nearest neighbour classifier
Index terms have been assigned to the content through auto-classification.
Recommendations
Nearest neighbour cuckoo search algorithm with probabilistic mutation
Graphical abstractDisplay Omitted HighlightsWe extend a cuckoo search algorithm with the nearest neighbour and probabilistic mutation strategies.A suite of 20 functions are adopted to illustrate the improvement in effectiveness and efficiency of the ...
Analyze the effectiveness of an algorithm for identifying Polish characters in handwriting based on neural machine learning technologies
AbstractAn approach is presented that generalize OCR task including polish letters using deep learning technique. The paper extends EMNIST dataset so that two new classes of polish diacritics “Ą” and “Ć” are attached. Using this new dataset ...
Haar Wavelet Transform Based Fast K-Nearest Neighbor Search Algorithm
PCSPA '10: Proceedings of the 2010 First International Conference on Pervasive Computing, Signal Processing and ApplicationsThe task of k-nearest neighbor search is to find the k nearest neighbors of a query vector in the data set. Due to the orthogonality of the Haar wavelet transform, the k nearest neighbors, searching in the spatial domain, are the same as that in the ...
Comments
Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2025.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 15 January 2025
Accepted: 13 December 2024
Received: 18 September 2023
Author Tags
Author Tag
Qualifiers
- Research-article
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025