Article

How flickr helps us make sense of the world: context and content in community-contributed media collections

Authors:

Lyndon Kennedy,

Tye RattenburyAuthors Info & Claims

MM '07: Proceedings of the 15th ACM international conference on Multimedia

Pages 631 - 640

https://doi.org/10.1145/1291233.1291384

Published: 29 September 2007 Publication History

Abstract

The advent of media-sharing sites like Flickr and YouTube has drastically increased the volume of community-contributed multimedia resources available on the web. These collections have a previously unimagined depth and breadth, and have generated new opportunities - and new challenges - to multimedia research. How do we analyze, understand and extract patterns from these new collections? How can we use these unstructured, unrestricted community contributions of media (and annotation) to generate "knowledge".

As a test case, we study Flickr - a popular photo sharing website. Flickr supports photo, time and location metadata, as well as a light-weight annotation model. We extract information from this dataset using two different approaches. First, we employ a location-driven approach to generate aggregate knowledge in the form of "representative tags" for arbitrary areas in the world. Second, we use a tag-driven approach to automatically extract place and event semantics for Flickr tags, based on each tag's metadata patterns.

With the patterns we extract from tags and metadata, vision algorithms can be employed with greater precision. In particular, we demonstrate a location-tag-vision-based approach to retrieving images of geography-related landmarks and features from the Flickr dataset. The results suggest that community-contributed media and annotation can enhance and improve our access to multimedia resources - and our understanding of the world.

References

[1]

S. Ahern, M. Naaman, R. Nair, and J. Yang. World explorer: Visualizing aggregate data from unstructured text in geo-referenced collections. In Proceedings of the Seventh ACM/IEEE-CS Joint Conference on Digital Libraries. ACM Press, June 2007.

Digital Library

[2]

M. Ames and M. Naaman. Why we tag: Motivations for annotation in mobile and online media. In CHI '07: Proceedings of the SIGCHI conference on Human Factors in computing systems, New York, NY, USA, 2007. ACM Press.

Digital Library

[3]

T. L. Berg and D. A. Forsyth. Automatic ranking of iconic images. Technical report, U. C. Berkeley, January 2007.

[4]

C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/öcjlin/libsvm.

Digital Library

[5]

M. Davis, M. Smith, F. Stentiford, A. Bambidele, J. Canny, N. Good, S. King, and R. Janakiraman. Using context and similarity for face and location identification. In Proceedings of the IS&T/SPIE 18th Annual Symposium on Electronic Imaging Science and Technology, 2006.

[6]

M. Dubinko, R. Kumar, J. Magnani, J. Novak, P. Raghavan, and A. Tomkins. Visualizing tags over time. In WWW '06: Proceedings of the 15th international conference on World Wide Web, pages 193--202, New York, NY, USA, 2006. ACM Press.

Digital Library

[7]

R. Fergus, P. Perona, and A. Zisserman. A visual category filter for Google images. Proc. ECCV, pages 242--256, 2004.

[8]

W. Hsu, L. Kennedy, and S.-F. Chang. Video search reranking via information bottleneck principle. In ACM Multimedia, Santa Babara, CA, USA, 2006.

Digital Library

[9]

A. Jaffe, M. Naaman, T. Tassa, and M. Davis. Generating summaries and visualization for large collections of geo-referenced photographs. In MIR '06: Proceedings of the 8th ACM international workshop on Multimedia information retrieval, pages 89--98, New York, NY, USA, 2006. ACM Press.

Digital Library

[10]

L. Kennedy and S.-F. Chang. A reranking approach for context-based concept fusion in video indexing and retrieval. In Conference on Image and Video Retrieval, Amsterdam, 2007.

Digital Library

[11]

L. Kennedy, S.-F. Chang, and I. Kozintsev. To search or to label?: predicting the performance of search-based automatic image classifiers. Proceedings of the 8th ACM international workshop on Multimedia information retrieval, pages 249--258, 2006.

Digital Library

[12]

D. Lowe. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision, 60(2):91--110, 2004.

Digital Library

[13]

B. Manjunath and W. Ma. Texture features for browsing and retrieval of image data. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 18(8):837--842, 1996.

Digital Library

[14]

M. Naaman, A. Paepcke, and H. Garcia-Molina. From where to what: Metadata sharing for digital photographs with geographic coordinates. In 10th International Conference on Cooperative Information Systems (CoopIS), 2003.

[15]

M. Naaman, Y. J. Song, A. Paepcke, and H. Garcia-Molina. Automatic organization for digital photographs with geographic coordinates. In Proceedings of the Fourth ACM/IEEE-CS Joint Conference on Digital Libraries, 2004.

Digital Library

[16]

A. Natsev, M. Naphade, and J. Tešić. Learning the semantics of multimedia queries and concepts from a small number of examples. Proceedings of the 13th annual ACM international conference on Multimedia, pages 598--607, 2005.

Digital Library

[17]

N. O'Hare, C. Gurrin, G. J. Jones, and A. F. Smeaton. Combination of content analysis and context features for digital photograph retrieval. In 2nd IEE European Workshop on the Integration of Knowledge, Semantic and Digital Media Technologies, 2005.

[18]

A. Pigeau and M. Gelgon. Organizing a personal image collection with statistical model-based ICL clustering on spatio-temporal camera phone meta-data. Journal of Visual Communication and Image Representation, 15(3):425--445, September 2004.

Digital Library

[19]

T. Rattenbury, N. Good, and M. Naaman. Towards automatic extraction of event and place semantics from flickr tags. In Proceedings of the Thirtieth International ACM SIGIR Conference. ACM Press, July 2007.

Digital Library

[20]

R. Sarvas, E. Herrarte, A. Wilhelm, and M. Davis. Metadata creation system for mobile images. In Proceedings of the 2nd international conference on Mobile systems, applications, and services, pages 36--48. ACM Press, 2004.

Digital Library

[21]

A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell., 22(12):1349--1380, 2000.

Digital Library

[22]

N. Snavely, S. Seitz, and R. Szeliski. Photo tourism: exploring photo collections in 3D. ACM Transactions on Graphics (TOG), 25(3):835--846, 2006.

Digital Library

[23]

M. Stricker and M. Orengo. Similarity of color images. Proc. SPIE Storage and Retrieval for Image and Video Databases, 2420:381--392, 1995.

[24]

K. Toyama, R. Logan, and A. Roseway. Geographic location tags on digital images. In Proceedings of the 11th International Conference on Multimedia (MM2003), pages 156--166. ACM Press, 2003.

Digital Library

[25]

C.-M. Tsai, A. Qamra, and E. Chang. Extent: Inferring image metadata from context and content. In IEEE International Conference on Multimedia and Expo. IEEE, 2005.

[26]

V. Vapnik. The Nature of Statistical Learning Theory. Springer, 2000.

Digital Library

[27]

Y. Wu, E. Y. Chang, and B. L. Tseng. Multimodal metadata fusion using causal strength. In Proceedings of the 13th International Conference on Multimedia (MM2005), pages 872--881, New York, NY, USA, 2005. ACM Press.

Digital Library

Cited By

Wang JZhang HZhong YLiang YJi RCang Y(2024)Advanced Multimodal Deep Learning Architecture for Image-Text Matching2024 IEEE 4th International Conference on Electronic Technology, Communication and Information (ICETCI)10.1109/ICETCI61221.2024.10594167(1185-1191)Online publication date: 24-May-2024
https://doi.org/10.1109/ICETCI61221.2024.10594167
Fazal NFränti P(2024)Social media data for content creation in location-based gamesJournal of Location Based Services10.1080/17489725.2024.2414000(1-28)Online publication date: 30-Oct-2024
https://doi.org/10.1080/17489725.2024.2414000
Fazal NFränti P(2024)Relevant Tag Extraction Based on Image Visual ContentApplied Intelligence10.1007/978-981-97-0827-7_25(283-295)Online publication date: 1-Mar-2024
https://doi.org/10.1007/978-981-97-0827-7_25
Show More Cited By

Index Terms

How flickr helps us make sense of the world: context and content in community-contributed media collections
1. Information systems
  1. Information systems applications

Recommendations

Social distance, mobility and place: global and intimate genres in geo-tagged photographs of Guguletu, South Africa
DIS '10: Proceedings of the 8th ACM Conference on Designing Interactive Systems

This paper documents locative photographic practices on photo-sharing sites Flickr and The Grid and analyses how geo-tagged photographs of the Guguletu in South Africa represent interpersonal meanings and social distance. Distinct communicative genres ...
Generating summaries for large collections of geo-referenced photographs
WWW '06: Proceedings of the 15th international conference on World Wide Web

We describe a framework for automatically selecting a summary set of photographs from a large collection of geo-referenced photos. The summary algorithm is based on spatial patterns in photo sets, but can be expanded to support social, temporal, as well ...
Generating diverse and representative image search results for landmarks
WWW '08: Proceedings of the 17th international conference on World Wide Web

Can we leverage the community-contributed collections of rich media on the web to automatically generate representative and diverse views of the world's landmarks? We use a combination of context- and content-based tools to generate representative sets ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '07: Proceedings of the 15th ACM international conference on Multimedia

September 2007

1115 pages

ISBN:9781595937025

DOI:10.1145/1291233

General Chairs:
Rainer Lienhart
University of Augsburg, Germany
,
Anand R. Prasad
DoCoMo Euro-Labs,Germany
,
Program Chairs:
Alan Hanjalic
Delft University of Technology, The Netherlands
,
Sunghyun Choi
Seoul National University, South Korea
,
Brian Bailey
University of Illinois at Urbana-Champaign
,
Nicu Sebe
University of Amsterdam, The Netherlands

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 September 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM07

Sponsor:

MM07: The 15th ACM International Conference on Multimedia 2007

September 25 - 29, 2007

Augsburg, Germany

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

307
Total Citations
View Citations
2,834
Total Downloads

Downloads (Last 12 months)40
Downloads (Last 6 weeks)4

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang JZhang HZhong YLiang YJi RCang Y(2024)Advanced Multimodal Deep Learning Architecture for Image-Text Matching2024 IEEE 4th International Conference on Electronic Technology, Communication and Information (ICETCI)10.1109/ICETCI61221.2024.10594167(1185-1191)Online publication date: 24-May-2024
https://doi.org/10.1109/ICETCI61221.2024.10594167
Fazal NFränti P(2024)Social media data for content creation in location-based gamesJournal of Location Based Services10.1080/17489725.2024.2414000(1-28)Online publication date: 30-Oct-2024
https://doi.org/10.1080/17489725.2024.2414000
Fazal NFränti P(2024)Relevant Tag Extraction Based on Image Visual ContentApplied Intelligence10.1007/978-981-97-0827-7_25(283-295)Online publication date: 1-Mar-2024
https://doi.org/10.1007/978-981-97-0827-7_25
Mao WWen RCheng ZMei ZWang HWu BShi J(2024)A Location Recommendation Model Based on User Behavior and Sequence InfluenceInternet of Things – ICIOT 202310.1007/978-3-031-51734-1_2(18-30)Online publication date: 19-Jan-2024
https://doi.org/10.1007/978-3-031-51734-1_2
Kaur ARodrigues AHoogstraten SBlanco-Mora DMiranda BMorgado PMeshi D(2023)An Urban Image Stimulus Set Generated from Social MediaData10.3390/data81201848:12(184)Online publication date: 1-Dec-2023
https://doi.org/10.3390/data8120184
Gonzalez Bravo C(2023)Vocabulary in urban contexts using multimodalityWorking papers in Applied Linguistics and Linguistics at York10.25071/2564-2855.283Online publication date: 6-Nov-2023
https://doi.org/10.25071/2564-2855.28
Benevento A(2023)Identifying Hashtag Cultures to Study the Construction of Childhood Image and Parents’ AspirationsCulture & Psychology10.1177/1354067X231191480Online publication date: 28-Jul-2023
https://doi.org/10.1177/1354067X231191480
Karimi MMesgari M(2023)Extracting Place Functionality From Crowdsourced Textual Data Using Semantic Space ModelingIEEE Access10.1109/ACCESS.2023.333285411(129217-129229)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3332854
Bonacchi CJones SBroccoli EHiscock ARobson E(2023)Researching heritage values in social media environments: understanding variabilities and (in)visibilitiesInternational Journal of Heritage Studies10.1080/13527258.2023.223191929:10(1021-1040)Online publication date: 6-Jul-2023
https://doi.org/10.1080/13527258.2023.2231919
Deseure-Charron FDjebali SGuérard G(2022)Clustering Method for Touristic Photographic Spots RecommendationAdvanced Data Mining and Applications10.1007/978-3-031-22137-8_17(223-237)Online publication date: 24-Nov-2022
https://doi.org/10.1007/978-3-031-22137-8_17
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten