research-article

Global annotation on georeferenced photographs

Authors:

B. S. ManjunathAuthors Info & Claims

CIVR '09: Proceedings of the ACM International Conference on Image and Video Retrieval

Article No.: 12, Pages 1 - 8

https://doi.org/10.1145/1646396.1646413

Published: 08 July 2009 Publication History

Abstract

We present an efficient world-scale system for providing automatic annotation on collections of geo-referenced photos. As a user uploads a photograph a place of origin is estimated from visual features which the user can refine. Once the correct location is provided, tags are suggested based on geographic and image similarity retrieved from a large database of 1.2 million images crawled from Flickr. The system effectively mines geographically relevant terms and ranks potential suggestion terms by their posterior probability given observed visual and geocoordinate features. A series of experiments analyzes the geocoordinate prediction accuracy and precision-recall metric of tags suggestions based on information retrieval techniques. The system is novel in that it fuses geographic and visual information to provide annotations for uploaded photographs taken anywhere in the world in a matter of seconds.

References

[1]

G. Carneiro, A. B. Chan, P. J. Moreno, and N. Vasconcelos, "Supervised Learning of Semantic Classes for Image Annotation and Retrieval," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007.

Digital Library

[2]

S. Feng, R. Manmatha, and V. Lavrenko, "Multiple Bernoulli Relevance Models for Image and Video Annotation," in CVPR, 2004.

Digital Library

[3]

L. Kennedy, M. Naaman, S. Ahern, R. Nair, and T. Rattenbury, "How Flickr Helps Us Make Sense of the World: Context and Content in Community-Contributed Media Collections," in ACM Multimedia, 2007.

Digital Library

[4]

L. Kennedy and M. Naaman, "Generating Diverse and Representative Image Search Results for Landmarks," in WWW, 2008.

Digital Library

[5]

J. Hays and A. Efros, "IM2GPS: Estimating Geographic Information from a Single Image," in CVPR, 2008.

[6]

D. Joshi and J. Luo, "Inferring Generic Activities and Events from Image Content and Bags of Geo-tags," in CIVR, 2008.

Digital Library

[7]

X.-J. Wang, L. Zhang, F. Jing, and W.-Y. Ma, "AnnoSearch: Image Auto-Annotation by Search," in CVPR, 2006.

Digital Library

[8]

E. Moxley, T. Mei, X.-S. Hua, W.-Y. Ma, and B. Manjunath, "Automatic Video Annotation Through Search and Mining," in ICME, 2008.

[9]

E. Moxley, J. Kleban, and B. S. Manjunath, "SpiritTagger: A Geo-Aware Tag Suggestion Tool Mined from Flickr," in MIR, 2008.

Digital Library

[10]

P. Salembier and T. Sikora, Introduction to MPEG-7: Multimedia Content Description Interface, B. Manjunath, Ed. New York, NY, USA: John Wiley&Sons, Inc., 2002.

Digital Library

[11]

A. Oliva and A. Torralba, "Building the Gist of a Scene: The Role of Global Image Features in Recognition," in Progress in Brain Research, 2006.

[12]

D. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," IJCV, 2004.

Digital Library

[13]

E. Nowak, F. Jurie, and B. Triggs, "Sampling Strategies for Bag-of-Features Image Classification," in ECCV, 2006.

Digital Library

[14]

D. Nister and H. Stewenius, "Scalable Recognition with a Vocabulary Tree," in CVPR, 2006.

Digital Library

[15]

"http://people.csail.mit.edu/torralba/code/spatialenvelope," Torralba GIST implementation.

[16]

"http://vision.ucla.edu/~vedaldi/code/bag/bag.html), note = UCLA Bag of Features."

[17]

S. Ahern, M. Naaman, R. Nair, and J. H. Yang, "World Explorer: Visualizing Aggregate Data from Unstructured Text in Geo-Referenced Collections," in Conference on Digital Libraries, 2007.

Digital Library

[18]

T. Tran, R. Wehrens, and L. M. Buydens, "KNN Density-Based Clustering for High Dimensional Multispectral Images," Computational Statistics and Data Analysis, 2006.

Digital Library

[19]

M. Asefi, "Classification-Based Adaptive Search Algorithm for Video Motion Estimation," Ph.D. dissertation, University of Waterloo, Waterloo, Ontario, Canada, 2006.

[20]

O. Boiman, E. Shechtman, and M. Irani, "In Defense of Nearest-Neighbor Based Image Classification," in CVPR, 2008.

[21]

S. Tsu and W. Hsieh, "Quadtree-Based Perceptual Watermarking Scheme," in ASIACCS, 2006.

Digital Library

[22]

L. A. Consularo and R. M. Cesar, "Quadtree-Based Inexact Graph Matching for Image Analysis," in SIBGRAPI, 2005.

Digital Library

[23]

S. Wu, M. Rahman, and T. Chow, "Content-Based Image Retrieval Using Growing Hierarchical Self-Organizing Quadtree Map," Pattern Recognition, 2005.

Digital Library

[24]

L. Grady and E. L. Schwartz, "Faster Graph-Theoretic Image Processing via Small-World and Quadtree Topologies," CVPR, 2004.

Digital Library

[25]

"http://www.flickr.com/services/api/flickr.places. findByLatLon.html," Flickr Places API Services.

Cited By

Yin YZhang YLiu ZWang SShah RZimmermann R(2022)GPS2Vec: Pre-Trained Semantic Embeddings for Worldwide GPS CoordinatesIEEE Transactions on Multimedia10.1109/TMM.2021.306095124(890-903)Online publication date: 2022
https://doi.org/10.1109/TMM.2021.3060951
Yin YZhang YLiu ZLiang YWang SShah RZimmermann RShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)Learning Multi-context Aware Location Representations from Large-scale Geotagged ImagesProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475268(899-907)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475268
Yin YLiu ZZhang YWang SShah RZimmermann R(2019)GPS2VecProceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems10.1145/3347146.3359067(416-419)Online publication date: 5-Nov-2019
https://dl.acm.org/doi/10.1145/3347146.3359067
Show More Cited By

Index Terms

Global annotation on georeferenced photographs
1. Information systems
  1. Information retrieval
    1. Document representation

Recommendations

Tag recommendation for georeferenced photos
LBSN '11: Proceedings of the 3rd ACM SIGSPATIAL International Workshop on Location-Based Social Networks

This paper presents methods for annotating georeferenced photos with descriptive tags, exploring the annotations for other georeferenced photos which are available at online repositories like Flickr. Specifically, by using the geospatial coordinates ...
Disinformation in Multimedia Annotation: Misleading Metadata Detection on YouTube
iV&L-MM '16: Proceedings of the 2016 ACM workshop on Vision and Language Integration Meets Multimedia Fusion

Popularity of online videos is increasing at a rapid rate. Not only the users can access these videos online, but they can also upload video content on platforms like YouTube and Myspace. These videos are indexed by user generated multimedia annotation, ...
Leveraging geo-referenced digital photographs

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIVR '09: Proceedings of the ACM International Conference on Image and Video Retrieval

July 2009

383 pages

ISBN:9781605584805

DOI:10.1145/1646396

Conference Chairs:
Yiannis Kompatsiaris
CERTH-ITI, Greece
,
Stephane Marchand-Maillet
Univ. of Geneva, Switzerland
,
Program Chairs:
Yannis Avrithis
NTUA, Greece
,
Noel O Connor
DCU, Ireland
,
Daniel Gatica-Perez
Idiap Research Institute, Switzerland
,
Tat-Seng Chua
National University of Singapore, Singapore

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

In-Cooperation

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Division of Graduate Education

Conference

CIVR '09

Sponsor:

SIGMM

CIVR '09: CIVR '09 - International Conference on Image and Video Retrieval

July 8 - 10, 2009

Santorini, Fira, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
359
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yin YZhang YLiu ZWang SShah RZimmermann R(2022)GPS2Vec: Pre-Trained Semantic Embeddings for Worldwide GPS CoordinatesIEEE Transactions on Multimedia10.1109/TMM.2021.306095124(890-903)Online publication date: 2022
https://doi.org/10.1109/TMM.2021.3060951
Yin YZhang YLiu ZLiang YWang SShah RZimmermann RShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)Learning Multi-context Aware Location Representations from Large-scale Geotagged ImagesProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475268(899-907)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475268
Yin YLiu ZZhang YWang SShah RZimmermann R(2019)GPS2VecProceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems10.1145/3347146.3359067(416-419)Online publication date: 5-Nov-2019
https://dl.acm.org/doi/10.1145/3347146.3359067
Shao JHu GSong JLiu XShen H(2019)Towards Accurate Georeferenced Video Search With Camera Field of View ModelingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2018.284820029:6(1844-1855)Online publication date: Jun-2019
https://doi.org/10.1109/TCSVT.2018.2848200
Shen M(2018)Social image tag enrichment based on textual similarity modelingMultimedia Tools and Applications10.1007/s11042-017-5184-x77:3(3659-3676)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1007/s11042-017-5184-x
Yeo JCho HPark JHwang S(2017)Multimodal KB Harvesting for Emerging Spatial EntitiesIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2017.265180529:5(1073-1086)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1109/TKDE.2017.2651805
Qian XLu XHan JDu BLi X(2017)On Combining Social Media and Spatial Technology for POI Cognition and Image LocalizationProceedings of the IEEE10.1109/JPROC.2017.2731600105:10(1937-1952)Online publication date: Oct-2017
https://doi.org/10.1109/JPROC.2017.2731600
Zhu ZXu C(2017)Organizing photographs with geospatial and image semanticsMultimedia Systems10.1007/s00530-014-0426-523:1(53-61)Online publication date: 1-Feb-2017
https://dl.acm.org/doi/10.1007/s00530-014-0426-5
Li ZLi Z(2017)Personalized Tag RecommendationUnderstanding-Oriented Multimedia Content Analysis10.1007/978-981-10-3689-7_4(75-99)Online publication date: 27-May-2017
https://doi.org/10.1007/978-981-10-3689-7_4
Bartolini ICiaccia PPatella M(2017)Multimedia, Similarity, and Preferences: Adding Flexibility to Your Information NeedsA Comprehensive Guide Through the Italian Database Research Over the Last 25 Years10.1007/978-3-319-61893-7_8(127-141)Online publication date: 31-May-2017
https://doi.org/10.1007/978-3-319-61893-7_8
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents