article

Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study

Authors:

S. Lazebnik, and

C. SchmidAuthors Info & Claims

International Journal of Computer Vision, Volume 73, Issue 2

Pages 213 - 238

https://doi.org/10.1007/s11263-006-9794-4

Published: 21 June 2007 Publication History

Abstract

Recently, methods based on local image features have shown promise for texture and object recognition tasks. This paper presents a large-scale evaluation of an approach that represents images as distributions (signatures or histograms) of features extracted from a sparse set of keypoint locations and learns a Support Vector Machine classifier with kernels based on two effective measures for comparing distributions, the Earth Mover's Distance and the ² distance. We first evaluate the performance of our approach with different keypoint detectors and descriptors, as well as different kernels and classifiers. We then conduct a comparative evaluation with several state-of-the-art recognition methods on four texture and five object databases. On most of these databases, our implementation exceeds the best reported results and achieves comparable performance on the rest. Finally, we investigate the influence of background correlations on recognition performance via extensive tests on the PASCAL database, for which ground-truth object localization information is available. Our experiments demonstrate that image representations based on distributions of local features are surprisingly effective for classification of texture and object images under challenging real-world conditions, including significant intra-class variations and substantial background clutter.

References

[1]

Agarwal, S. and Roth, D. 2002. Learning a sparse representation for object detection. In European Conference on Computer Vision , Vol. 4, pp. 113-130.

[2]

Berg, A., Berg, T., and Malik, J. 2005. Shape matching and object recognition using low distortion correspondences. In IEEE Conference on Computer Vision and Pattern Recognition , Vol. 1, pp. 26-33.

[3]

Brodatz, P. 1966. Textures: A Photographic Album for Artists and Designers . Dover: New York.

[4]

Caputo, B., Wallraven, C., and Nilsback, M.-E. 2004. Object categorization via local kernels. In International Conference on Pattern Recognition , Vol. 2, pp. 132-135.

[5]

Chapelle, O., Haffner, P., and Vapnik, V. 1999. Support vector machines for histogram-based image classification. IEEE Transactions on Neural Networks , 10(5):1055-1064.

Digital Library

[6]

Cohen, F.S., Fan, Z., and Patel, M.A.S. 1991. Classification of rotated and scaled textured images using Gaussian Markov field models. IEEE Transactions on Pattern Analysis and Machine Intelligence , 13(2):192-202.

Digital Library

[7]

Csurka, G., Dance, C., Fan, L., Willamowski, J., and Bray, C. 2004. Visual categorization with bags of keypoints. In ECCV Workshop on Statistical Learning in Computer Vision .

[8]

Cula, O.G. and Dana, K.J. 2001. Compact representation of bidirectional texture functions. In IEEE Conference on Computer Vision and Pattern Recognition , Vol. 1, pp. 1041-1047.

[9]

Dana, K.J., van Ginneken, B., Nayar, S.K., and Koenderink, J.J. 1999. Reflectance and texture of real world surfaces. ACM Transactions on Graphics , 18(1):1-34.

Digital Library

[10]

Deselaers, T., Keysers, D., and Ney, H. 2005. Discriminative training for object recognition using image patches. In IEEE Conference on Computer Vision and Pattern Recognition , Vol. 2, pp. 157-162.

[11]

Deselaers, T., Keysers, D., and Ney, H. 2005. Improving a discriminative approach to object recognition using image patches. In DAGM , pp. 326-333.

[12]

Dorkó, G. and Schmid, C. 2005. Object class recognition using discriminative local features. Technical Report RR-5497, INRIA - Rhône-Alpes.

[13]

Eichhorn, J. and Chapelle, O. 2004. Object categorization with SVM: kernels for local features. Technical report, Max Planck Institute for Biological Cybernetics, Tuebingen, Germany.

[14]

Everingham, M., Zisserman, A., Williams, C., Van Gool, L. et al. 2006. The 2005 PASCAL visual object classes challenge. In Selected Proceedings of the first PASCAL Challenges Workshop , F. d'Alche Buc, I. Dagan, and J. Quinonero (Eds), LNAI, Springer. http://www.pascal-network.org/challenges/ VOC/voc/index.html.

[15]

Fei-Fei, L., Fergus, R., and Perona, P. 2004. Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In IEEE CVPR Workshop on Generative-Model Based Vision .

[16]

Fei-Fei, L. and Perona, P. 2005. A Bayesian hierarchical model for learning natural scene categories. In IEEE Conference on Computer Vision and Pattern Recognition , Vol. 2, pp. 524- 531.

[17]

Felzenszwalb, P. and Huttenlocher, D. 2005. Pictorial structures for object recognition. International Journal of Computer Vision , 61(1):55-79.

Digital Library

[18]

Fergus, R., Perona, P., and Zisserman, A. 2003. Object class recognition by unsupervised scale-invariant learning. In IEEE Conference on Computer Vision and Pattern Recognition , Vol. 2, pp. 264-271.

[19]

Fischler, M. and Elschlager, R. 1973. The representation and matching of pictorial structures. IEEE Transactions on Computers , 22(1):67-92.

Digital Library

[20]

Fowlkes, C., Belongie, S., Chung, F., and Malik, J. 2004. Spectral grouping using the Nyström method. IEEE Transactions on Pattern Analysis and Machine Intelligence , 26(2):1-12.

Digital Library

[21]

Gårding, J. and Lindeberg, T. 1996. Direct computation of shape cues using scale-adapted spatial derivative operators. International Journal of Computer Vision , 17(2):163-191.

Digital Library

[22]

Grauman, K. and Darrell, T. 2005. Efficient image matching with distributions of local invariant features. In IEEE Conference on Computer Vision and Pattern Recognition , Vol. 2, pp. 627-634.

[23]

Grauman, K. and Darrell, T. 2005. Pyramid match kernels: Discriminative classification with sets of image features. In International Conference on Computer Vision , Vol. 2, pp. 1458-1465.

[24]

Hayman, E., Caputo, B., Fritz, M., and Eklundh, J.-O. 2004. On the significance of real-world conditions for material classification. In European Conference on Computer Vision , Vol. 4, pp. 253- 266.

[25]

Jing, F., Li, M., Zhang, H.-J., and Zhang, B. 2003. Support vector machines for region-based image retrieval. In IEEE International Conference on Multimedia and Expo. .

[26]

Johnson, A. and Hebert, M. 1999. Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence , 21(5):433-449.

Digital Library

[27]

Julesz, B. 1981. Textons, the elements of texture perception and their interactions. Nature , 290:91-97.

[28]

Jurie, F. and Triggs, B. 2005. Creating efficient codebooks for visual recognition. In International Conference on Computer Vision , Vol. 1, pp. 604-610.

[29]

Larlus, D., Dorkó, G., and Jurie, F. 2006. Création de vocabulaires visuels efficaces pour la catégorisation d'images. In Reconnaissance des Formes et Intelligence Artificielle .

[30]

Lazebnik, S., Schmid, C., and Ponce, J. 2004. Semi-local affine parts for object recognition. In British Machine Vision Conference , Vol. 2, pp. 959-968.

[31]

Lazebnik, S., Schmid, C., and Ponce, J. 2005. A sparse texture representation using local affine regions. IEEE Transactions on Pattern Analysis and Machine Intelligence , 27(8): 1265-1278.

Digital Library

[32]

Leibe, B. and Schiele, B. 2003. Analyzing appearance and contour-based methods for object categorization. In IEEE Conference on Computer Vision and Pattern Recognition , Vol. 2, pp. 409-415, http://www.mis.informatik.tu-darmstadt.de/Research/Projects/ categorization/eth80-db.html

[33]

Leung, T. and Malik, J. 2001. Recognizing surfaces using three-dimensional textons. International Journal of Computer Vision , 43(1):29-44.

[34]

Lindeberg, T. 1998. Feature detection with automatic scale selection. International Journal of Computer Vision , 30(2):79-116.

Digital Library

[35]

Llado, X., Marti, J., and Petrou, M. 2003. Classification of textures seen from different distances and under varying illumination direction. In IEEE International Conference on Image Processing , Vol. 1, pp. 833-836.

[36]

Lowe, D. 2004. Distinctive image features form scale-invariant keypoints. International Journal of Computer Vision , 60(2):91- 110.

[37]

Lyu, S. 2005. Mercer kernels for object recognition with local features. In IEEE Conference on Computer Vision and Pattern Recognition , Vol. 2, pp. 223-229.

[38]

Manjunath, B.S. and Ma, W.Y. 1996. Texture features for browsing and retrieval of image data. IEEE Transactions on Pattern Analysis and Machine Intelligence , 18(5):837-842.

Digital Library

[39]

Mao, J. and Jain, A. 1992. Texture classification and segmentation using multiresolution simultaneous autoregressive models. Pattern Recognition , 25(2):173-188.

Digital Library

[40]

McCallum, A. and Nigam, K. 1998. A comparison of event models for naive Bayes text classification. In AAAI-98 Workshop on Learning for Text Categorization , pp. 41-48.

[41]

Mikolajczyk, K. and Schmid, C. 2005. A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence , 27(10):1615-1630.

Digital Library

[42]

Mikolajczyk, K. and Schmid, C. 2002. An affine invariant interest point detector. In European Conference on Computer Vision , Vol. 1, pp. 128-142.

[43]

Mikolajczyk, K. and Schmid, C. 2004. Scale and affine invariant interest point detectors. International Journal of Computer Vision , 60(1):63-86.

Digital Library

[44]

Nene, S.A., Nayar, S.K., and Murase, H. 1996. Columbia object image library (COIL-100), Technical Report CUCS- 006-96, Columbia University, http://www1.cs.columbia.edu/ CAVE/research/softlib/coil-100.html

[45]

Niblack, W., Barber, R., Equitz, W., Fickner, M., Glasman, E., Petkovic, D., and Yanker, P. 1993. The QBIC project: Querying images by content using color, texture and shape. In SPIE Conference on Geometric Methods in Computer Vision II .

[46]

Nigam, K., Lafferty, J., and McCallum, A. 1999. Using maximum entropy for text classification. In IJCAI Workshop on Machine Learning for Information Filtering , pp. 61-67.

[47]

Nilsback, M.-E. and Caputo, B. 2004. Cue integration through discriminative accumulation. In IEEE Conference on Computer Vision and Pattern Recognition , Vol. 2, pp. 578-585.

[48]

Opelt, A., Fussenegger, M., Pinz, A., and Auer, P. 2004. Weak hypotheses and boosting for generic object detection and recognition. In European Conference on Computer Vision , Vol. 2, pp. 71-84.

[49]

Pelleg, D. and Moore, A. 2000. X-means: Exlending k -means with efficient estimation of the number of clusters. In International Conference on Machine Learning , pp. 727-734.

[50]

Pontil, M. and Verri, A. 1998. Support vector machines for 3D object recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence , 20(6):637-646.

Digital Library

[51]

Quelhas, P., Monay, F., Odobez, J.-M., Gatica, D., Tuytelaars, T., and Van Gool, L. 2005. Modeling scenes with local descriptors and latent aspects. In International Conference on Computer Vision , Vol. 2, pp. 883-890.

[52]

Rubner, Y., Tomasi, C., and Guibas, L. 2000. The Earth Mover's distance as a metric for image retrieval. International Journal of Computer Vision , 40(2):99-121.

Digital Library

[53]

Schiele, B. and Crowley, J. 2000. Recognition without correspondence using multidimensional receptive field histograms, International Journal of Computer Vision , 36(1):31- 50.

[54]

Schölkopf, B. and Smola, A. 2002. Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond . MIT Press: Cambridge, MA.

[55]

Sivic, J., Russell, B., Efros, A., Zisserman, A., and Freeman, W. 2005. Discovering objects and their location in images. In International Corference on Computer Vision , Vol. 1, pp. 370- 378.

[56]

Sivic, J. and Zisserman, A. 2003. Video Google: A text retrieval approach to object matching in videos, In International Conference on Computer Vision , Vol. 2, pp. 1470-1477.

[57]

Varma, M. and Zisserman, A. 2002. Classifying images of materials: Achieving viewpoint and illumination independence. In European Conference on Computer Vision , Vol. 3, pp. 255- 271.

[58]

Varma, M. and Zisserman, A. 2003. Texture classification: Are filter banks necessary? In IEEE Conference on Computer Vision and Pattern Recognition , Vol. 2, pp. 691-698.

[59]

Wallraven, C., Caputo, B., and Graf, A. 2003. Recognition with local features: the kernel recipe. In International Conference on Computer Vision , Vol. 1, pp. 257-264.

[60]

Weber, M., Welling, M., and Perona, P. 2000. Towards automatic discovery of object categories. In IEEE Conference on Computer Vision and Pattern Recognition , Vol. 2, pp. 2101-2109.

[61]

Willamowski, J., Arregui, D., Csurka, G., Dance, C.R., and Fan, L. 2004. Categorizing nine visual classes using local appearance descriptors. In ICPR Workshop on Learning for Adaptable Visual Systems .

[62]

Wu, J. and Chantler, M.J. 2003. Combining gradient and albedo data for rotation invariant classification of 3D surface texture. In International Conference on Computer Vision , Vol. 2, pp. 48-855.

[63]

Zhang, J., Marszalek, M., Lazebnik, S., and Schmid, C. Local features and kernels for classification of texture and object categories: An in-depth study. Technical Report RR-5737, INRIA Rhône-Alpes, November 2005. http://lear.inrialpes.fr/ pubs/2005/ZMLS05

Cited By

Lyra LFabris AFlorindo J(2024)A multilevel pooling scheme in convolutional neural networks for texture image recognitionApplied Soft Computing10.1016/j.asoc.2024.111282152:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.asoc.2024.111282
Pittaras NGiannakopoulos GStamatopoulos PKarkaletsis V(2023)Content-based and Knowledge-enriched Representations for Classification Across Modalities: A SurveyACM Computing Surveys10.1145/358368255:14s(1-40)Online publication date: 13-Feb-2023
https://dl.acm.org/doi/10.1145/3583682
Zhang YChen NQi SXue MHua Z(2023)Detection of Recolored Image by Texture Features in Chrominance ComponentsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/357107619:3(1-23)Online publication date: 25-Feb-2023
https://dl.acm.org/doi/10.1145/3571076
Show More Cited By

Index Terms

Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study
1. Computing methodologies

Recommendations

Object classification using a local texture descriptor and a support vector machine

Objects classification or object detection is one of the most challenging tasks in computer vision. Digital images taken of real-life scenes capture objects at different positions, rotations and scales. Furthermore, variations in lighting, shape, color ...
Read More
Random interest regions for object recognition based on texture descriptors and bag of features

In this work we propose a novel method for object recognition based on a random selection of interest regions, texture features (local binary/ternary patterns and local phase quantization) for describing each region, a bag-of-features approach for ...
Read More
Learning completed discriminative local features for texture classification

We propose a novel feature representation method, namely Completed Discriminative Local Features (CDLF), for texture classification.The CDLF learn transformation matrices for texture images that maximize the mutual information between the local features ...
Read More

Comments

Information & Contributors

Information

Published In

cover image International Journal of Computer Vision

International Journal of Computer Vision Volume 73, Issue 2

June 2007

114 pages

ISSN:0920-5691

Issue’s Table of Contents

Copyright © Copyright © 2007 Springer Science + Business Media, LLC.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 21 June 2007

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

409
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Lyra LFabris AFlorindo J(2024)A multilevel pooling scheme in convolutional neural networks for texture image recognitionApplied Soft Computing10.1016/j.asoc.2024.111282152:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.asoc.2024.111282
Pittaras NGiannakopoulos GStamatopoulos PKarkaletsis V(2023)Content-based and Knowledge-enriched Representations for Classification Across Modalities: A SurveyACM Computing Surveys10.1145/358368255:14s(1-40)Online publication date: 13-Feb-2023
https://dl.acm.org/doi/10.1145/3583682
Zhang YChen NQi SXue MHua Z(2023)Detection of Recolored Image by Texture Features in Chrominance ComponentsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/357107619:3(1-23)Online publication date: 25-Feb-2023
https://dl.acm.org/doi/10.1145/3571076
Nguyen VKong S(2023)Multimodal feature fusion for illumination-invariant recognition of abnormal human behaviorsInformation Fusion10.1016/j.inffus.2023.101949100:COnline publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1016/j.inffus.2023.101949
Lan SLiao XFan HHu SPan Z(2023)A multi-channel framework based Local Binary Pattern with two novel local feature descriptors for texture classificationDigital Signal Processing10.1016/j.dsp.2023.104124140:COnline publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1016/j.dsp.2023.104124
Chen XLi JZhang Y(2022)Multidirectional Gradient Feature With Shape Index for Effective Texture ClassificationInternational Journal on Semantic Web & Information Systems10.4018/IJSWIS.31218318:1(1-19)Online publication date: 20-Oct-2022
https://dl.acm.org/doi/10.4018/IJSWIS.312183
Zhou JWu ZJiang ZHuang KGuo KZhao S(2022)Background selection schema on deep learning-based classification of dermatological diseaseComputers in Biology and Medicine10.1016/j.compbiomed.2022.105966149:COnline publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1016/j.compbiomed.2022.105966
Boudra SYahiaoui IBehloul A(2022)Tree trunk texture classification using multi-scale statistical macro binary patterns and CNNApplied Soft Computing10.1016/j.asoc.2022.108473118:COnline publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1016/j.asoc.2022.108473
Zheng QGong MYou XTao D(2022)A Unified B-Spline Framework for Scale-Invariant Keypoint DetectionInternational Journal of Computer Vision10.1007/s11263-021-01568-3130:3(777-799)Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1007/s11263-021-01568-3
Kumar SPradhan JPal AIslam SKhan M(2022)Radiological image retrieval technique using multi-resolution texture and shape featuresMultimedia Tools and Applications10.1007/s11042-021-10525-881:10(13633-13660)Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1007/s11042-021-10525-8
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents