research-article

Multimodal geo-tagging in social media websites using hierarchical spatial segmentation

Authors:

Sebastian Schmiedeke,

Thomas SikoraAuthors Info & Claims

LBSN '12: Proceedings of the 5th ACM SIGSPATIAL International Workshop on Location-Based Social Networks

Pages 32 - 39

https://doi.org/10.1145/2442796.2442805

Published: 06 November 2012 Publication History

Abstract

These days the sharing of photographs and videos is very popular in social networks. Many of these social media websites such as Flickr, Facebook and Youtube allows the user to manually label their uploaded videos with geo-information using a interface for dragging them into the map. However, the manually labelling for a large set of social media is still borring and error-prone. For this reason we present a hierarchical, multi-modal approach for estimating the GPS information. Our approach makes use of external resources like gazetteers to extract toponyms in the metadata and of visual and textual features to identify similar content. First, the national borders detection recognizes the country and its dimension to speed up the estimation and to eliminate geographical ambiguity. Next, we use a database of more than 3.2 million Flickr images to group them together into geographical regions and to build a hierarchical model. A fusion of visual and textual methods for different granularities is used to classify the videos' location into possible regions. The Flickr videos are tagged with the geo-information of the most similar training image within the regions that is previously filtered by the probabilistic model for each test video. In comparison with existing GPS estimation and image retrieval approaches at the Placing Task 2011 we will show the effectiveness and high accuracy relative to the state-of-the art solutions.

References

[1]

http://translate.google.com.

[2]

http://www.geonames.org.

[3]

http://www.wikipedia.org.

[4]

http://code.google.com/apis/maps/index.html.

[5]

S. Agarwal, N. Snavely, I. Simon, S. Seitz, and R. Szeliski. Building rome in a day. In Computer Vision, 2009 IEEE 12th International Conference on.

[6]

E. Albuz, E. Kocalar, and A. Khokhar. Scalable color image indexing and retrieval using vector wavelets. Knowledge and Data Engineering, IEEE Transactions on, 13(5):851--861, 2001.

Digital Library

[7]

J. Baldridge. The OpenNLP Project. http://www.opennlp.com, 2005.

[8]

S. Chatzichristofis and Y. Boutalis. Cedd: Color and edge directivity descriptor: A compact descriptor for image indexing and retrieval. Computer Vision Systems, pages 312--322, 2008.

Digital Library

[9]

J. Choi, G. Friedland, V. Ekambaram, and K. Ramchandran. Multimodal location estimation of consumer media: Dealing with sparse training data. In proceedings of the IEEE International Conference on Multimedia and Expo (ICME 2012), Melbourne, Australia, 2012.

Digital Library

[10]

D. Crandall, L. Backstrom, D. Huttenlocher, and J. Kleinberg. Mapping the World's Photos. In Proceedings of the 18th international conference on World wide web, pages 761--770. ACM, 2009.

Digital Library

[11]

H. Feichtinger and T. Strohmer. Gabor analysis and algorithms: Theory and applications. Birkhauser, 1998.

Digital Library

[12]

J. Hays and A. Efros. Im2gps: estimating geographic information from a single image. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, pages 1--8. Ieee, 2008.

[13]

J. Huang, S. Kumar, M. Mitra, W. Zhu, and R. Zabih. Image indexing using color correlograms. In Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on, pages 762--768. IEEE, 1997.

Digital Library

[14]

P. Kelm, S. Schmiedeke, and T. Sikora. A hierarchical, multi-modal approach for placing videos on the map using millions of flickr photographs. In ACM Multimedia 2011 (Workshop on Social and Behavioral Networked Media Access - SBNMA). ACM, Nov. 2011.

Digital Library

[15]

P. Kelm, S. Schmiedeke, and T. Sikora. Multi-modal, multi-resource methods for placing Flickr videos on the map. In Proceedings of the 1st ACM International Conference on Multimedia Retrieval, ICMR '11, New York, NY, USA, 2011. ACM.

Digital Library

[16]

C. Keßler, K. Janowicz, and M. Bishr. An agenda for the next generation gazetteer: Geographic information contribution and retrieval. In Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pages 91--100. ACM, 2009.

Digital Library

[17]

M. Lux and S. Chatzichristofis. Lire: lucene image retrieval: an extensible java cbir library. In Proceeding of the 16th ACM international conference on Multimedia, pages 1085--1088. ACM, 2008.

Digital Library

[18]

B. Manjunath, J. Ohm, V. Vasudevan, and A. Yamada. Color and texture descriptors. Circuits and Systems for Video Technology, IEEE Transactions on, 11(6):703--715, 2001.

Digital Library

[19]

O. A. B. Penatti, L. T. Li, J. Almeida, and R. da S. Torres. A visual approach for video geocoding using bag-of-scenes. In Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, ICMR '12, pages 53:1--53:8, New York, NY, USA, 2012. ACM.

Digital Library

[20]

X. Sevillano, T. Piatrik, K. Chandramouli, Q. Zhang, and E. Izquierdoy. Geo-tagging online videos using semantic expansion and visual analysis. In Image Analysis for Multimedia Interactive Services (WIAMIS), 2012 13th International Workshop on, pages 1--4. IEEE, 2012.

[21]

I. Simon, N. Snavely, and S. Seitz. Scene summarization for online image collections. In Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on, pages 1--8. IEEE, 2007.

[22]

P. Smart, C. Jones, and F. Twaroch. Multi-source toponym data integration and mediation for a meta-gazetteer service. In Geographic Information Science, Lecture Notes in Computer Science.

Digital Library

[23]

H. Tamura, S. Mori, and T. Yamawaki. Textural features corresponding to visual perception. Systems, Man and Cybernetics, IEEE Transactions on, 8(6):460--473, 1978.

Cited By

Fallucchi FDi Stabile RPurificato EGiuliano RDe Luca E(2021)Enriching videos with automatic place recognition in google mapsMultimedia Tools and Applications10.1007/s11042-021-11253-981:16(23105-23121)Online publication date: 29-Jul-2021
https://doi.org/10.1007/s11042-021-11253-9
Yin YZhang LZimmermann RHauptmann ANgo CXue XJiang YSnoek CVasconcelos N(2015)Exploiting Spatial Relationship between Scenes for Hierarchical Video GeotaggingProceedings of the 5th ACM on International Conference on Multimedia Retrieval10.1145/2671188.2749354(363-370)Online publication date: 22-Jun-2015
https://dl.acm.org/doi/10.1145/2671188.2749354
Kelm PSchmiedeke SSchockaert SSikora TTrevisiol MVan Laere O(2014)Georeferencing Flickr Resources Based on Multimodal FeaturesMultimodal Location Estimation of Videos and Images10.1007/978-3-319-09861-6_8(127-152)Online publication date: 5-Oct-2014
https://doi.org/10.1007/978-3-319-09861-6_8
Show More Cited By

Index Terms

Multimodal geo-tagging in social media websites using hierarchical spatial segmentation
1. Information systems
  1. Information retrieval

Recommendations

Location Extraction from Social Media: Geoparsing, Location Disambiguation, and Geotagging

Location extraction, also called “toponym extraction,” is a field covering geoparsing, extracting spatial representations from location mentions in text, and geotagging, assigning spatial coordinates to content items. This article evaluates five “best-...
Retrieving geo-location of videos with a divide & conquer hierarchical multimodal approach
ICMR '13: Proceedings of the 3rd ACM conference on International conference on multimedia retrieval

This paper presents a strategy to identify the geographic location of videos. First, it relies on a multi-modal cascade pipeline that exploits the available sources of information, namely the user's upload history, his social network and a visual-based ...
Geo-Social Media Analytics
WWW '15 Companion: Proceedings of the 24th International Conference on World Wide Web

With the maturity of wireless communication techniques, GPS-equipped mobile devices become ubiquitous, and location-acquisition technologies and services are flourishing. These location applications as well as mobile devices, developed and combined with ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

LBSN '12: Proceedings of the 5th ACM SIGSPATIAL International Workshop on Location-Based Social Networks

November 2012

67 pages

ISBN:9781450316989

DOI:10.1145/2442796

Program Chairs:
Gabriel Ghinita
University of Massachusetts at Boston
,
Jennifer Neville
Purdue University
,
Shawn Newsam
University of California at Merced

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSPATIAL: ACM Special Interest Group on Spatial Information

In-Cooperation

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Seventh Framework Programme

Conference

SIGSPATIAL'12

Sponsor:

SIGSPATIAL

SIGSPATIAL'12: SIGSPATIAL 2012 International Conference on Advances in Geographic Information Systems

November 6, 2012

California, Redondo Beach

Acceptance Rates

Overall Acceptance Rate 8 of 15 submissions, 53%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
146
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fallucchi FDi Stabile RPurificato EGiuliano RDe Luca E(2021)Enriching videos with automatic place recognition in google mapsMultimedia Tools and Applications10.1007/s11042-021-11253-981:16(23105-23121)Online publication date: 29-Jul-2021
https://doi.org/10.1007/s11042-021-11253-9
Yin YZhang LZimmermann RHauptmann ANgo CXue XJiang YSnoek CVasconcelos N(2015)Exploiting Spatial Relationship between Scenes for Hierarchical Video GeotaggingProceedings of the 5th ACM on International Conference on Multimedia Retrieval10.1145/2671188.2749354(363-370)Online publication date: 22-Jun-2015
https://dl.acm.org/doi/10.1145/2671188.2749354
Kelm PSchmiedeke SSchockaert SSikora TTrevisiol MVan Laere O(2014)Georeferencing Flickr Resources Based on Multimodal FeaturesMultimodal Location Estimation of Videos and Images10.1007/978-3-319-09861-6_8(127-152)Online publication date: 5-Oct-2014
https://doi.org/10.1007/978-3-319-09861-6_8
Schmiedeke SKelm PSikora T(2013)DCT-based features for categorisation of social media in compressed domain2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP)10.1109/MMSP.2013.6659304(295-300)Online publication date: Sep-2013
https://doi.org/10.1109/MMSP.2013.6659304

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents