research-article

Retrieving landmark and non-landmark images from community photo collections

Authors:

Yannis Avrithis,

Yannis Kalantidis,

Giorgos Tolias,

Evaggelos SpyrouAuthors Info & Claims

MM '10: Proceedings of the 18th ACM international conference on Multimedia

Pages 153 - 162

https://doi.org/10.1145/1873951.1873973

Published: 25 October 2010 Publication History

Abstract

State of the art data mining and image retrieval in community photo collections typically focus on popular subsets, e.g. images containing landmarks or associated to Wikipedia articles. We propose an image clustering scheme that, seen as vector quantization compresses a large corpus of images by grouping visually consistent ones while providing a guaranteed distortion bound. This allows us, for instance, to represent the visual content of all thousands of images depicting the Parthenon in just a few dozens of scene maps and still be able to retrieve any single, isolated, non-landmark image like a house or graffiti on a wall. Starting from a geo-tagged dataset, we first group images geographically and then visually, where each visual cluster is assumed to depict different views of the the same scene. We align all views to one reference image and construct a 2D scene map by preserving details from all images while discarding repeating visual features. Our indexing, retrieval and spatial matching scheme then operates directly on scene maps. We evaluate the precision of the proposed method on a challenging one-million urban image dataset.

References

[1]

S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski. Building Rome in a day. In ICCV, 2009.

[2]

H. Bay, T. Tuytelaars, and L. Van Gool. SURF: Speeded up robust features. In ECCV, 2006.

Digital Library

[3]

O. Chum and J. Matas. Large-scale discovery of spatially related images. PAMI, 32(2):371--377, 2010.

Digital Library

[4]

O. Chum, J. Matas, and J. Kittler. Locally optimized RANSAC. In DAGM, page 236. Springer Verlag, 2003.

[5]

O. Chum, J. Philbin, J. Sivic, M. Isard, and A. Zisserman. Total recall: Automatic query expansion with a generative feature model for object retrieval. In ICCV, 2007.

[6]

D. Crandall, L. Backstrom, D. Huttenlocher, and J. Kleinberg. Mapping the world's photos. In WWW, 2009.

Digital Library

[7]

M. Fischler and R. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381--395, 1981.

Digital Library

[8]

S. Gammeter, L. Bossard, T. Quack, and L. V. Gool. I know what you did last summer: Object-level auto-annotation of holiday snaps. In ICCV, 2009.

[9]

R. Hartley and A. Zisserman. Multiple View Geometry. Cambridge university press Cambridge, UK, 2000.

Digital Library

[10]

J. Hays and A. A. Efros. IM2GPS: Estimating geographic information from a single image. In CVPR, 2008.

[11]

H. Jegou, M. Douze, and C. Schmid. Hamming embedding and weak geometric consistency for large scale image search. In ECCV, 2008.

Digital Library

[12]

B. Johansson and R. Cipolla. A system for automatic pose-estimation from a single image in a city scene. In Proc. IASTED Int. Conf. Signal Processing, Pattern Recognition and Applications, 2002.

[13]

E. Kalogerakis, O. Vesselova, J. Hays, A. A. Efros, and A. Hertzmann. Image sequence geolocation with human travel priors. In ICCV, 2009.

[14]

L. Kennedy, M. Naaman, S. Ahern, R. Nair, and T. Rattenbury. How Flickr helps us make sense of the world: Context and content in community-contributed media collections. In ACM Multimedia, volume 3, pages 631--640, 2007.

Digital Library

[15]

C. Lampert. Detecting objects in large image collections and videos by efficient subimage retrieval. In ICCV, 2009.

[16]

X. Li, C. Wu, C. Zach, S. Lazebnik, and J.-M. Frahm. Modeling and recognition of landmark image collections using iconic scene graphs. In ECCV, pages 427--440. Springer, 2008.

Digital Library

[17]

Y. Li, D. J. Crandall, and D. P. Huttenlocher. Landmark classification in large-scale image collections. In ICCV, 2009.

[18]

D. Lowe. Local feature view clustering for 3D object recognition. In CVPR, 2001.

[19]

D. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60(2):91--110, 2004.

Digital Library

[20]

J. Matas, O. Chum, M. Urban, and T. Pajdla. Robust wide-baseline stereo from maximally stable extremal regions. Image and Vision Computing, 2004.

[21]

K. Mikolajczyk and C. Schmid. A performance evaluation of local descriptors. Pattern Analysis and Machine Intelligence, 27(10):1615--1630, 2005.

Digital Library

[22]

M. Muja and D. Lowe. Fast approximate nearest neighbors with automatic algorithm configuration. In ICCV, 2009.

[23]

D. Nister and H. Stewenius. Scalable recognition with a vocabulary tree. In CVPR, 2006.

Digital Library

[24]

M. Perdoch, O. Chum, and J. Matas. Efficient representation of local geometry for large scale object retrieval. In CVPR, 2009.

[25]

J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman. Object retrieval with large vocabularies and fast spatial matching. In CVPR, 2007.

[26]

T. Quack, B. Leibe, and L. Van Gool. World-scale mining of objects and events from community photo collections. In CIVR, pages 47--56, 2008.

Digital Library

[27]

G. Schindler, M. Brown, and R. Szeliski. City-scale location recognition. In CVPR, 2007.

[28]

C. Silpa-Anan and R. Hartley. Optimised KD-trees for fast image descriptor matching. In CVPR, 2008.

[29]

I. Simon, N. Snavely, and S. Seitz. Scene summarization for online image collections. In ICCV, 2007.

[30]

J. Sivic and A. Zisserman. Video Google: A text retrieval approach to object matching in videos. In ICCV, pages 1470--1477, 2003.

Digital Library

[31]

N. Snavely, S. Seitz, and R. Szeliski. Photo tourism: Exploring photo collections in 3D. In Computer Graphics and Interactive Techniques, 2006.

Digital Library

[32]

U. Steinhoff, D. Omercevic, R. Perko, B. Schiele, and A. Leonardis. How computer vision can help in outdoor positioning. In European Conference on Ambient Intelligence, 2007.

Digital Library

[33]

M. Tipping and B. Scholkopf. A kernel approach for vector quantization with guaranteed distortion bounds. In Artificial Intelligence and Statistics, pages 129--134, 2001.

Cited By

Liang HKambhamettu C(2023)Edge-Guided Image Inpainting with TransformerAdvances in Visual Computing10.1007/978-3-031-47966-3_22(285-296)Online publication date: 3-Dec-2023
https://doi.org/10.1007/978-3-031-47966-3_22
Tahmasebzadeh GHakimov SEwerth RMüller-Budack E(2023)Multimodal Geolocation Estimation of News PhotosAdvances in Information Retrieval10.1007/978-3-031-28238-6_14(204-220)Online publication date: 2-Apr-2023
https://dl.acm.org/doi/10.1007/978-3-031-28238-6_14
Arbinger CBullin MHenrich A(2022)Exploiting Geodata to Improve Image Recognition with Deep LearningCompanion Proceedings of the Web Conference 202210.1145/3487553.3524645(648-655)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3487553.3524645
Show More Cited By

Index Terms

Retrieving landmark and non-landmark images from community photo collections
1. Information systems
  1. Information retrieval
    1. Document representation
    2. Search engine architectures and scalability
      1. Search engine indexing
  2. Information storage systems

Recommendations

VIRaL: Visual Image Retrieval and Localization

New applications are emerging every day exploiting the huge data volume in community photo collections. Most focus on popular subsets, e.g., images containing landmarks or associated to Wikipedia articles. In this work we are concerned with the problem ...
Clustering near-duplicate images in large collections
MIR '07: Proceedings of the international workshop on Workshop on multimedia information retrieval

Near-duplicate images introduce problems of redundancy and copyright infringement in large image collections. The problem is acute on the web, where appropriation of images without acknowledgment of source is prevalent. In this paper, we present an ...
Geo-based automatic image annotation
ICMR '12: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval

A huge number of user-tagged images are daily uploaded to the web. Recently, a growing number of those images are also geotagged. These provide new opportunities for solutions to automatically tag images so that efficient image management and retrieval ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '10: Proceedings of the 18th ACM international conference on Multimedia

October 2010

1836 pages

ISBN:9781605589336

DOI:10.1145/1873951

General Chairs:
Alberto del Bimbo
University of Florence, Italy
,
Shih-Fu Chang
Columbia University, USA
,
Program Chair:
Arnold Smeulders
University of Amsterdam, NL

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '10

Sponsor:

SIGMM

MM '10: ACM Multimedia Conference

October 25 - 29, 2010

Firenze, Italy

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

93
Total Citations
View Citations
728
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)2

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liang HKambhamettu C(2023)Edge-Guided Image Inpainting with TransformerAdvances in Visual Computing10.1007/978-3-031-47966-3_22(285-296)Online publication date: 3-Dec-2023
https://doi.org/10.1007/978-3-031-47966-3_22
Tahmasebzadeh GHakimov SEwerth RMüller-Budack E(2023)Multimodal Geolocation Estimation of News PhotosAdvances in Information Retrieval10.1007/978-3-031-28238-6_14(204-220)Online publication date: 2-Apr-2023
https://dl.acm.org/doi/10.1007/978-3-031-28238-6_14
Arbinger CBullin MHenrich A(2022)Exploiting Geodata to Improve Image Recognition with Deep LearningCompanion Proceedings of the Web Conference 202210.1145/3487553.3524645(648-655)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3487553.3524645
Humenberger MCabon YPion NWeinzaepfel PLee DGuérin NSattler TCsurka G(2022)Investigating the Role of Image Retrieval for Visual LocalizationInternational Journal of Computer Vision10.1007/s11263-022-01615-7130:7(1811-1836)Online publication date: 1-Jul-2022
https://dl.acm.org/doi/10.1007/s11263-022-01615-7
Chen MWu H(2021)Landmark Dataset Development and RecognitionInternational Journal of Multimedia Data Engineering & Management10.4018/IJMDEM.202110010312:4(38-51)Online publication date: 1-Oct-2021
https://dl.acm.org/doi/10.4018/IJMDEM.2021100103
Kordopatis-Zilos GGalopoulos PPapadopoulos SKompatsiaris ICheng WKankanhalli MWang MChu WLiu JWorring M(2021)Leveraging EfficientNet and Contrastive Learning for Accurate Global-scale Location EstimationProceedings of the 2021 International Conference on Multimedia Retrieval10.1145/3460426.3463644(155-163)Online publication date: 24-Aug-2021
https://dl.acm.org/doi/10.1145/3460426.3463644
Alfarrarjeh AYang XJabal AKim SShahabi C(2021)Exploring the Spatial-Visual Locality of Geo-tagged Urban Street Images2021 IEEE 4th International Conference on Multimedia Information Processing and Retrieval (MIPR)10.1109/MIPR51284.2021.00023(104-110)Online publication date: Sep-2021
https://doi.org/10.1109/MIPR51284.2021.00023
Hochschild VBraun ASommer CWarth GOmran A(2021)Visualizing Landscapes by Geospatial TechniquesModern Approaches to the Visualization of Landscapes10.1007/978-3-658-30956-5_4(47-78)Online publication date: 1-Feb-2021
https://doi.org/10.1007/978-3-658-30956-5_4
Wu HChen M(2020)Chinese Landmark Recognition2020 International Conference on Computing, Networking and Communications (ICNC)10.1109/ICNC47757.2020.9049717(24-28)Online publication date: Feb-2020
https://doi.org/10.1109/ICNC47757.2020.9049717
Weyand TAraujo ACao BSim J(2020)Google Landmarks Dataset v2 – A Large-Scale Benchmark for Instance-Level Recognition and Retrieval2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR42600.2020.00265(2572-2581)Online publication date: Jun-2020
https://doi.org/10.1109/CVPR42600.2020.00265
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents