research-article

Multimedia multimodal geocoding

Authors:

Daniel Carlos Guimarães Pedronette,

Jurandy Almeida,

Otávio A. B. Penatti,

Rodrigo Tripodi Calumby,

Ricardo da S. TorresAuthors Info & Claims

SIGSPATIAL '12: Proceedings of the 20th International Conference on Advances in Geographic Information Systems

Pages 474 - 477

https://doi.org/10.1145/2424321.2424393

Published: 06 November 2012 Publication History

Abstract

This work is developed in the context of the placing task of the MediaEval 2011 initiative. The objective is to geocode (or geotag) a set of videos, i.e., automatically assign geographical coordinates to them. This paper presents an architecture for multimodal geocoding that exploits both visual and textual descriptions associated with videos. This work also describes our efforts regarding the implementation of this architecture to demonstrate its applicability. Conducted experiments show how our multimodal approach enhances the results compared to relying on a single modality.

References

[1]

J. Almeida, N. J. Leite, and R. da S. Torres. Comparison of video sequences with histograms of motion patterns. In ICIP, pages 3673--3676, 2011.

[2]

R. Candeias and B. Martins. Associating relevant photos to georeferenced textual documents through rank aggregation. In Int. Semantic Web Conf. - Terra Cognita Workshop, 2011.

Digital Library

[3]

J. Choi, H. Lei, and G. Friedland. The 2011 ICSI video location estimation system. In Working Notes Proc. MediaEval Workshop, volume 807, 2011.

[4]

W. B. Croft. Combining approaches to information retrieval. In Adv. in Inf. Retrieval, volume 7, pages 1--36. Springer US, 2002.

[5]

F. A. Faria, A. Veloso, H. M. de Almeida, E. Valle, R. da S. Torres, M. A. Gonçalves, and W. M. Jr. Learning to rank for content-based image retrieval. In ACM MIR, pages 285--294, 2010.

Digital Library

[6]

M. Friendly. Corrgrams: Exploratory displays for correlation matrices. The American Statistician, 56(4):316--324, 2002.

[7]

J. Hays and A. A. Efros. im2gps: estimating geographic information from a single image. In CVPR, 2008.

[8]

C. B. Jones and R. S. Purves. Geographical information retrieval. Int. J. Geo. Info. Science, 22(3):219--228, 2008.

Digital Library

[9]

Y. Kalantidis, G. Tolias, Y. Avrithis, M. Phinikettos, E. Spyrou, P. Mylonas, and S. Kollias. Viral: Visual image retrieval and localization. Mult. Tools and App., 51:555--592, 2011.

Digital Library

[10]

P. Kelm, S. Schmiedeke, and T. Sikora. A hierarchical, multi-modal approach for placing videos on the map using millions of flickr photographs. In Workshop on Social and behavioural networked media access, pages 15--20, 2011.

Digital Library

[11]

M. Larson, M. Soleymani, P. Serdyukov, S. Rudinac, C. Wartena, V. Murdock, G. Friedland, R. Ordelman, and G. J. F. Jones. Automatic tagging and geotagging in video collections and communities. In ICMR, pages 51:1--51:8, 2011.

Digital Library

[12]

L. T. Li, J. Almeida, and R. da S. Torres. RECOD working notes for placing task MediaEval 2011. In Working Notes Proc. MediaEval Workshop, volume 807, 2011.

[13]

J. Luo, D. Joshi, J. Yu, and A. Gallagher. Geotagging in multimedia and computer vision--a survey. Mult. Tools and App., 51:187--211, 2011.

Digital Library

[14]

C. D. Manning, P. Raghavan, and H. Schtze. Introduction to Information Retrieval. Cambridge University Press, New York, NY, USA, 2008.

Digital Library

[15]

D. C. G. Pedronette and R. da S. Torres. Exploiting clustering approaches for image re-ranking. J. Vis. Lang. and Comp., 22(6):453--466, 2011.

Digital Library

[16]

D. C. G. Pedronette, R. da S. Torres, and R. T. Calumby. Using contextual spaces for image re-ranking and rank aggregation. Mult. Tools and App., pages 1--28, 2012.

[17]

O. A. B. Penatti, L. T. Li, J. Almeida, and R. da S. Torres. A Visual Approach for Video Geocoding using Bag-of-Scenes. In ICMR, 2012.

Digital Library

[18]

A. Rae, V. Murdock, P. Serdyukov, and P. Kelm. Working notes for the placing task at MediaEval 2011. In Working Notes Proc. MediaEval Workshop, volume 807, 2011.

[19]

O. Van Laere, S. Schockaert, and B. Dhoedt. Finding locations of flickr resources using language models and similarity search. In International Conference on Multimedia Retrieval, pages 48:1--48:8, 2011.

Digital Library

Cited By

Warch DStellbauer PNeis P(2024)Advanced Techniques for Geospatial Referencing in Online Media RepositoriesFuture Internet10.3390/fi1603008716:3(87)Online publication date: 1-Mar-2024
https://doi.org/10.3390/fi16030087
Yin YZhang LZimmermann RHauptmann ANgo CXue XJiang YSnoek CVasconcelos N(2015)Exploiting Spatial Relationship between Scenes for Hierarchical Video GeotaggingProceedings of the 5th ACM on International Conference on Multimedia Retrieval10.1145/2671188.2749354(363-370)Online publication date: 22-Jun-2015
https://dl.acm.org/doi/10.1145/2671188.2749354
Fox ELeidig J(2014)Digital Libraries Applications: CBIR, Education, Social Networks, eScience/Simulation, and GISSynthesis Lectures on Information Concepts, Retrieval, and Services10.2200/S00565ED1V01Y201401ICR0326:1(1-175)Online publication date: 28-Mar-2014
https://doi.org/10.2200/S00565ED1V01Y201401ICR032
Show More Cited By

Index Terms

Multimedia multimodal geocoding

Recommendations

A rank aggregation framework for video multimodal geocoding

This paper proposes a rank aggregation framework for video multimodal geocoding. Textual and visual descriptions associated with videos are used to define ranked lists. These ranked lists are later combined, and the resulting ranked list is used to ...
Bidirectional Joint Representation Learning with Symmetrical Deep Neural Networks for Multimodal and Crossmodal Applications
ICMR '16: Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval

Common approaches to problems involving multiple modalities (classification, retrieval, hyperlinking, etc.) are early fusion of the initial modalities and crossmodal translation from one modality to the other. Recently, deep neural networks, especially ...
Multimodal and Crossmodal Representation Learning from Textual and Visual Features with Bidirectional Deep Neural Networks for Video Hyperlinking
iV&L-MM '16: Proceedings of the 2016 ACM workshop on Vision and Language Integration Meets Multimedia Fusion

Video hyperlinking represents a classical example of multimodal problems. Common approaches to such problems are early fusion of the initial modalities and crossmodal translation from one modality to the other. Recently, deep neural networks, especially ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGSPATIAL '12: Proceedings of the 20th International Conference on Advances in Geographic Information Systems

November 2012

642 pages

ISBN:9781450316910

DOI:10.1145/2424321

General Chairs:
Isabel Cruz
University of Illinois at Chicago
,
Craig Knoblock
University of Southern California
,
Program Chairs:
Peer Kröger
Ludwig-Maximilians-Universität München, Germany
,
Egemen Tanin
University of Melbourne, Australia
,
Peter Widmayer
ETH Zürich, Switzerland

Copyright © 2012 Authors.

Sponsors

SIGSPATIAL: ACM Special Interest Group on Spatial Information

In-Cooperation

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGSPATIAL'12

Sponsor:

SIGSPATIAL

SIGSPATIAL'12: SIGSPATIAL 2012 International Conference on Advances in Geographic Information Systems

November 6 - 9, 2012

Redondo Beach, California

Acceptance Rates

Overall Acceptance Rate 257 of 1,238 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
151
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Warch DStellbauer PNeis P(2024)Advanced Techniques for Geospatial Referencing in Online Media RepositoriesFuture Internet10.3390/fi1603008716:3(87)Online publication date: 1-Mar-2024
https://doi.org/10.3390/fi16030087
Yin YZhang LZimmermann RHauptmann ANgo CXue XJiang YSnoek CVasconcelos N(2015)Exploiting Spatial Relationship between Scenes for Hierarchical Video GeotaggingProceedings of the 5th ACM on International Conference on Multimedia Retrieval10.1145/2671188.2749354(363-370)Online publication date: 22-Jun-2015
https://dl.acm.org/doi/10.1145/2671188.2749354
Fox ELeidig J(2014)Digital Libraries Applications: CBIR, Education, Social Networks, eScience/Simulation, and GISSynthesis Lectures on Information Concepts, Retrieval, and Services10.2200/S00565ED1V01Y201401ICR0326:1(1-175)Online publication date: 28-Mar-2014
https://doi.org/10.2200/S00565ED1V01Y201401ICR032
Larson MKelm PRae AHauff CThomee BTrevisiol MChoi JVan Laere OSchockaert SJones GSerdyukov PMurdock VFriedland G(2014)The Benchmark as a Research Catalyst: Charting the Progress of Geo-prediction for Social MultimediaMultimodal Location Estimation of Videos and Images10.1007/978-3-319-09861-6_2(5-40)Online publication date: 5-Oct-2014
https://doi.org/10.1007/978-3-319-09861-6_2
Park MChen YShafique KKnoblock CSchneider MKröger PKrumm JWidmayer P(2013)Tag configuration matcher for geo-taggingProceedings of the 21st ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems10.1145/2525314.2525441(384-387)Online publication date: 5-Nov-2013
https://dl.acm.org/doi/10.1145/2525314.2525441
Li LPenatti OFox ETorres RDownie JMcDonald RCole TSanderson RShipman F(2013)Domain-specific image geocodingProceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries10.1145/2467696.2467727(363-366)Online publication date: 22-Jul-2013
https://dl.acm.org/doi/10.1145/2467696.2467727

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents