article

Scale & Affine Invariant Interest Point Detectors

Authors:

Krystian Mikolajczyk and

Cordelia SchmidAuthors Info & Claims

International Journal of Computer Vision, Volume 60, Issue 1

Pages 63 - 86

https://doi.org/10.1023/B:VISI.0000027790.02288.f2

Published: 01 October 2004 Publication History

Abstract

In this paper we propose a novel approach for detecting interest points invariant to scale and affine transformations. Our scale and affine invariant detectors are based on the following recent results: (1) Interest points extracted with the Harris detector can be adapted to affine transformations and give repeatable results (geometrically stable). (2) The characteristic scale of a local structure is indicated by a local extremum over scale of normalized derivatives (the Laplacian). (3) The affine shape of a point neighborhood is estimated based on the second moment matrix.

Our scale invariant detector computes a multi-scale representation for the Harris interest point detector and then selects points at which a local measure (the Laplacian) is maximal over scales. This provides a set of distinctive points which are invariant to scale, rotation and translation as well as robust to illumination changes and limited changes of viewpoint. The characteristic scale determines a scale invariant region for each point. We extend the scale invariant detector to affine invariance by estimating the affine shape of a point neighborhood. An iterative algorithm modifies location, scale and neighborhood of each point and converges to affine invariant points. This method can deal with significant affine transformations including large scale changes. The characteristic scale and the affine shape of neighborhood determine an affine invariant region for each point.

We present a comparative evaluation of different detectors and show that our approach provides better results than existing methods. The performance of our detector is also confirmed by excellent matching results; the image is described by a set of scale/affine invariant descriptors computed on the regions associated with our points.

References

[1]

Almansa, A. and Lindeberg, T. 2000. Fingerprint enhancement by shape adaptation of scale-space operators with automatic scale selection. IEEE Transactions on Image Processing, 9(12):2027- 2042.]]

Digital Library

[2]

Alvarez, L. and Morales, F. 1997. Affine morphological multiscale analysis of corners and multiple junctions. International Journal of Computer Vision, 2(25):95-107.]]

Digital Library

[3]

Baumberg, A. 2000. Reliable feature matching across widely separated views. In Proceedings of the Conference on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina, USA, pp. 774-781.]]

[4]

Borenstein, E. and Ullman, S. 2002. Class-specific, top-down segmentation. In Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark, pp. 202-215.]]

Digital Library

[5]

Brand, P. and Mohr, R. 1994. Accuracy in image measure. In Proceedings of the SPIE Conference on Videometrics III, S.F. El-Hakim (Ed.). Boston, Massachusetts, USA, vol. 2350, pp. 218- 228.]]

[6]

Bretzner, L. and Lindeberg, T. 1998. Feature tracking with automatic selection of spatial scales. Computer Vision and Image Understanding , 71(3):385-392.]]

Digital Library

[7]

Brown, M. and Lowe, D.G. 2002. Invariant features from interest point groups. In The 13th British Machine Vision Conference, Cardiff University, UK, pp. 253-262.]]

[8]

Chomat, O., de Verdière, V.C., Hall, D., and Crowley, J. 2000. Local scale selection for Gaussian based description techniques. In Proceedings of the 6th European Conference on Computer Vision, Dublin, Ireland, pp. 117-133.]]

Digital Library

[9]

Cottier, J. 1994. Extraction et appariements robustes des points d'intérêt de deux images non etalonnées.]]

[10]

Crowley, J. 1981. A representation for visual information. PhD thesis, Carnegie Mellon University.]]

Digital Library

[11]

Crowley, J. and Parker, A. 1984. A representation for shape based on peaks and ridges in the difference of low pass transform. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(2):156-170.]]

Digital Library

[12]

Deriche, R. and Giraudon, G. 1993. A computational approach for corner and vertex detection. International Journal of Computer Vision, 10(2): 101-124.]]

Digital Library

[13]

Duda, R. and Hart, P. 1973. Pattern Classification and Scene Analysis . Wiley-Interscience.]]

[14]

Dufournaud, Y., Schmid, C., and Horaud, R. 2000. Matching images with different resolutions. In Proceedings of the Conference on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina, USA, pp. 612-618.]]

[15]

Förstner, W. 1994. A framework for low level feature extraction. In Proceedings of the 3rd European Conference on Computer Vision, Stockholm, Sweden, pp. 383-394.]]

Digital Library

[16]

Förstner, W. and Gülchm, E. 1987. A fast operator for detection and precise location of distinct points, corners and centres of circular features. In Intercommission Conference on Fast Processing of Photogrammetric Data, Interlaken, Switzerland, pp. 281-305.]]

[17]

Freeman, W. and Adelson, E. 1991. The design and use of steerable filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(9):891-906.]]

Digital Library

[18]

Garding, J. and Lindeberg, T. 1994. Direct estimation of local surface shape in a fixating binocular vision system. In Proceedings of the 3rd European Conference on Computer Vision, Stockholm, Sweden, pp. 365-376.]]

Digital Library

[19]

Harris, C. and Stephens, M. 1988. A combined corner and edge detector. In Alvey Vision Conference, pp. 147-151.]]

[20]

Heitger, F., Rosenthaler, L., von der Heydt, R., Peterhans, E., and Kuebler, O. 1992. Simulation of neural contour mechanism: From simple to end-stopped cells. Vision Research, 32(5):963- 981.]]

[21]

Horaud, R., Skordas, T., and Veillon, F. 1990. Finding geometric and relational structures in an image. In Proceedings of the 1st European Conference on Computer Vision, Antibes, France, pp. 374- 384.]]

Digital Library

[22]

Kadir, T. and Brady, M. 2001. Scale, saliency and image description. International Journal of Computer Vision, 45(2):83-105,]]

Digital Library

[23]

Kanatani, K. 1998. Geometric information criterion for model selection. International Journal of Computer Vision, 26(3):171- 189.]]

Digital Library

[24]

Laptev, I. and Lindeberg, T. 2001. Tracking of multi-state hand models using particle filtering and a hierarchy of multi-scale image features. In Proceedings of Scale Space and Morphology Workshop , Vancouver, Canada, vol. 2106. Lecture Notes in Computer Science, pp. 63-74.]]

Digital Library

[25]

Lazebnik, S., Schmid, C., and Ponce, J. 2003. Sparse texture representation using affine-invariant neighborhoods. In Proceedings of the Conference on Computer Vision and Pattern Recognition, Madison, Wisconsin, USA, pp. 319-324.]]

[26]

Lindeberg, T. 1993. Detecting salient blob-like image structures and their scales with a scale-space primal sketch--A method for focus-of-attention. International Journal of Computer Vision, 11(3):283-318.]]

Digital Library

[27]

Lindeberg, T. 1998. Feature detection with automatic scale selection. International Journal of Computer Vision, 30(2):79-116.]]

Digital Library

[28]

Lindeberg, T. and Garding, J. 1997. Shape-adapted smoothing in estimation of 3-D shape cues from affine deformations of local 2- D brightness structure. Image and Vision Computing, 15(6):415- 434.]]

[29]

Lowe, D.G. 1999. Object recognition from local scale-invariant features. In Proceedings of the 7th International Conference on Computer Vision, Kerkyra, Greece, pp. 1150-1157.]]

Digital Library

[30]

Matas, J., Chum, O., Urban, M., and Pajdla, T. 2002. Robust wide baseline stereo from maximally stable extremal regions. In The 13th British Machine Vision Conference, Cardiff University, UK, pp. 384-393.]]

[31]

Mikolajczyk, K. 2002. Interest point detection invariant to affine transformations. PhD thesis, Institut National Polytechnique de Grenoble.]]

[32]

Mikolajczyk, K. and Schmid, C. 2001. Indexing based on scale invariant interest points. In Proceedings of the 8th International Conference on Computer Vision, Vancouver, Canada, pp. 525- 531.]]

[33]

Mikolajczyk, K. and Schmid, C. 2002. An affine invariant interest point detector. In Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark, vol. I, pp. 128-142.]]

Digital Library

[34]

Mikolajczyk, K. and Schmid, C. 2003. An performance evaluation of local descriptors. In Proceedings of the Conference on Computer Vision and Pattern Recognition, Madison, Wisconsin, USA, pp. 257-264.]]

[35]

Mikolajczyk, K. and Schmid, C. 2003. Shape recognition with edge based features. In Proceedings of the 13th British Machine Vision Conference, Norwich, UK, pp. 779-788.]]

[36]

Pritchett, P. and Zisserman, A. 1998. Wide baseline stereo matching. In Proceedings of the 6th International Conference on Computer Vision, Bombay, India. IEEE Computer Society Press, pp. 754- 760.]]

Digital Library

[37]

Rothganger, F., Lazebnik, S., Schmid, C., and Ponce, J. 2003. 3D Object modeling and recognition using affine-invariant patches and multi-view spatial constraints. In Proceedings of the Conference on Computer Vision and Pattern Recognition, Madison, Wisconsin, USA, pp. 272-277.]]

[38]

Schaffalitzky, F. and Zisserman, A. 2001. Viewpoint invariant texture matching and wide baseline stereo. In Proceedings of the 8th International Conference on Computer Vision, Vancouver, Canada, pp. 636-643.]]

[39]

Schaffalitzky, F. and Zisserman, A. 2002. Multi-view matching for unordered image sets. In Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark, pp. 414-431.]]

Digital Library

[40]

Schmid, C. and Mohr, R. 1997. Local grayvalue invariants for image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(5):530-534.]]

Digital Library

[41]

Schmid, C., Mohr, R., and Bauckhage, C. 2000. Evaluation of interest point detectors. International Journal of Computer Vision, 37(2): 151-172.]]

Digital Library

[42]

Tell, D. and Carlsson, S. 2002. Combining appearance and topology for wide baseline matching. In Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark, pp. 814- 828.]]

Digital Library

[43]

Triggs, B. 2001. Joint feature distributions for image correspondence. In Proceedings of the 8th International Conference on Computer Vision, Vancouver, Canada, pp. 201-208.]]

[44]

Tuytelaars, T. and Gool, L.V. 1999. Content-based image retrieval based on local affinely invariant regions. In Int. Conf. on Visual Information Systems, pp. 493-500.]]

Digital Library

[45]

Tuytelaars, T. and Van Gool, L. 2000. Wide baseline stereo matching based on local, affinely invariant regions. In The 11th British Machine Vision Conference, University of Bristol, UK, pp. 412-425.]]

Cited By

Bellavia F(2024)Image Matching by Bare HomographyIEEE Transactions on Image Processing10.1109/TIP.2023.334668233(696-708)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2023.3346682
He JZhang TZhang ZYu TZhang Y(2024)Efficient Dynamic Correspondence NetworkIEEE Transactions on Image Processing10.1109/TIP.2023.333459433(228-240)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2023.3334594
Kamoona AGostar AWang XEaston MBab-Hadiashar AHoseinnezhad R(2024)Anomaly detection of defect using energy of point pattern features within random finite set frameworkEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107706130:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107706
Show More Cited By

Index Terms

Scale & Affine Invariant Interest Point Detectors
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Interest point and salient region detections
        Object recognition
  2. Computer graphics
    1. Image manipulation

Recommendations

An Affine Invariant Interest Point Detector
ECCV '02: Proceedings of the 7th European Conference on Computer Vision-Part I

This paper presents a novel approach for detecting affine invariant interest points. Our method can deal with significant affine transformations including large scale changes. Such transformations introduce significant changes in the point location as ...
Read More
ASIFT: A New Framework for Fully Affine Invariant Image Comparison

If a physical object has a smooth or piecewise smooth boundary, its images obtained by cameras in varying positions undergo smooth apparent deformations. These deformations are locally well approximated by affine transforms of the image plane. In ...
Read More
Invariant Gabor-based interest points detector under geometric transformation

This paper proposes a novel approach for detecting interest points invariant to rotation and scale using Gabor wavelet. Our scale and rotational invariant interest points detector is produced based on a multi-scale representation by selecting points at ...
Read More

Comments

Information & Contributors

Information

Published In

cover image International Journal of Computer Vision

International Journal of Computer Vision Volume 60, Issue 1

October 2004

86 pages

ISSN:0920-5691

Issue’s Table of Contents

Copyright © Copyright © 2004 Kluwer Academic Publishers.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 October 2004

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

765
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Bellavia F(2024)Image Matching by Bare HomographyIEEE Transactions on Image Processing10.1109/TIP.2023.334668233(696-708)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2023.3346682
He JZhang TZhang ZYu TZhang Y(2024)Efficient Dynamic Correspondence NetworkIEEE Transactions on Image Processing10.1109/TIP.2023.333459433(228-240)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2023.3334594
Kamoona AGostar AWang XEaston MBab-Hadiashar AHoseinnezhad R(2024)Anomaly detection of defect using energy of point pattern features within random finite set frameworkEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107706130:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107706
Tsai TWang RTung N(2024)Hardware architecture design for real-time SIFT extraction with reduced memory usageMultimedia Tools and Applications10.1007/s11042-023-15789-w83:2(6297-6317)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11042-023-15789-w
Liao YDi YZhu KZhou HLu MZhang YDuan QLiu J(2024)Local feature matching from detector-based to detector-free: a surveyApplied Intelligence10.1007/s10489-024-05330-354:5(3954-3989)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s10489-024-05330-3
Wang WZhang LHuang HWilliams BChen YNeville J(2023)Revisiting unsupervised local descriptor learningProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v37i3.25367(2680-2688)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1609/aaai.v37i3.25367
Xie YGuan L(2023)Sparsity-Guided Discriminative Feature Encoding for Robust Keypoint DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3628432Online publication date: 17-Oct-2023
https://dl.acm.org/doi/10.1145/3628432
Bickel BBächer MOtaduy MRichard Lee HPfister HGross MMatusik W(2023)Design and Fabrication of Materials with Desired Deformation BehaviorSeminal Graphics Papers: Pushing the Boundaries, Volume 210.1145/3596711.3596798(829-838)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1145/3596711.3596798
Xue NWu TBai SWang FXia GZhang LTorr P(2023)Holistically-Attracted Wireframe Parsing: From Supervised to Self-Supervised LearningIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.331274945:12(14727-14744)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TPAMI.2023.3312749
Chen WLiu YWang WBakker EGeorgiou TFieguth PLiu LLew M(2023)Deep Learning for Instance Retrieval: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.321859145:6(7270-7292)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TPAMI.2022.3218591
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents