Article

Image labeling on a network: using social-network metadata for image classification

Authors:

Julian McAuley,

Jure LeskovecAuthors Info & Claims

ECCV'12: Proceedings of the 12th European conference on Computer Vision - Volume Part IV

Pages 828 - 841

https://doi.org/10.1007/978-3-642-33765-9_59

Published: 07 October 2012 Publication History

Abstract

Large-scale image retrieval benchmarks invariably consist of images from the Web. Many of these benchmarks are derived from online photo sharing networks, like Flickr, which in addition to hosting images also provide a highly interactive social community. Such communities generate rich metadata that can naturally be harnessed for image classification and retrieval. Here we study four popular benchmark datasets, extending them with social-network metadata, such as the groups to which each image belongs, the comment thread associated with the image, who uploaded it, their location, and their network of friends. Since these types of data are inherently relational, we propose a model that explicitly accounts for the interdependencies between images sharing common properties. We model the task as a binary labeling problem on a network, and use structured learning techniques to learn model parameters. We find that social-network metadata are useful in a variety of classification tasks, in many cases outperforming methods based on image content.

References

[1]

Everingham, M., Van Gool, L.J., Williams, C., Winn, J., Zisserman, A.: The PASCAL visual object classes (VOC) challenge. IJCV (2010).

[2]

Huiskes, M., Lew, M.: The MIR Flickr retrieval evaluation. In: CIVR (2008).

[3]

Nowak, S., Huiskes, M.: New strategies for image annotation: Overview of the photo annotation task at ImageCLEF 2010. In: CLEF (Notebook Papers/ LABs/Workshops) (2010).

[4]

Chua, T.S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.T.: NUS-WIDE: A real-world web image database from the National University of Singapore. In: CIVR (2009).

[5]

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A large-scale hierarchical image database. In: CVPR (2009).

[6]

Guillaumin, M., Verbeek, J., Schmid, C.: Multimodal semi-supervised learning for image classification. In: CVPR (2010).

[7]

Lindstaedt, S., Pammer, V., Mörzinger, R., Kern, R., Mülner, H., Wagner, C.: Recommending tags for pictures based on text, visual content and user context. In: Internet and Web Applications and Services (2008).

[8]

Sigurbjörnsson, B., van Zwol, R.: Flickr tag recommendation based on collective knowledge. In: WWW (2008).

[9]

Sawant, N., Datta, R., Li, J., Wang, J.: Quest for relevant tags using local interaction networks and visual content. In: MIR (2010).

[10]

Stone, Z., Zickler, T., Darrell, T.: Autotagging Facebook: Social network context improves photo annotation. In: CVPR Workshop on Internet Vision (2008).

[11]

Luo, J., Boutell, M., Brown, C.: Pictures are not taken in a vacuum - an overview of exploiting context for semantic scene content understanding. IEEE Signal Processing Magazine (2006).

[12]

Li, Y., Crandall, D., Huttenlocher, D.: Landmark classification in large-scale image collections. In: ICCV (2009).

[13]

Kalogerakis, E., Vesselova, O., Hays, J., Efros, A., Hertzmann, A.: Image sequence geolocation with human travel priors. In: ICCV (2009).

[14]

Joshi, D., Luo, J., Yu, J., Lei, P., Gallagher, A.: Using geotags to derive rich tag-clouds for image annotation. In: Social Media Modeling and Computing (2011).

[15]

Mensink, T., Verbeek, J., Csurka, G.: Trans media relevance feedback for image autoannotation. In: BMVC (2010).

[16]

Denoyer, L., Gallinari, P.: A ranking based model for automatic image annotation in a social network. In: ICWSM (2010).

[17]

Kolmogorov, V., Zabih, R.: What Energy Functions Can Be Minimized via Graph Cuts? In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part III. LNCS, vol. 2352, pp. 65-81. Springer, Heidelberg (2002).

[18]

Chapelle, O., Haffner, P., Vapnik, V.: Support vector machines for histogram-based image classification. IEEE Trans. on Neural Networks (1999).

[19]

Boros, E., Hammer, P.L.: Pseudo-boolean optimization. Discrete Applied Mathematics (2002).

[20]

Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. on PAMI (2001).

[21]

Strandmark, P., Kahl, F.: Parallel and distributed graph cuts by dual decomposition. In: CVPR (2010).

[22]

Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. JMLR (2005).

[23]

Teo, C.H., Smola, A., Vishwanathan, S., Le, Q.: A scalable modular convex solver for regularized risk minimization. In: KDD (2007).

[24]

Petterson, J., Caetano, T.: Submodular multi-label learning. In: NIPS (2011).

[25]

van de Sande, K., Gevers, T., Snoek, C.: Evaluating color descriptors for object and scene recognition. IEEE Trans. on PAMI (2010).

[26]

Huiskes, M., Thomee, B., Lew, M.: New trends and ideas in visual concept detection: the MIR Flickr retrieval evaluation initiative. In: CIVR (2010).

Cited By

Balasubramanian RLi JTadepalli PWang HWu QZhao HSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Adversarial attacks on combinatorial multi-armed banditsProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692170(2505-2526)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692170
Ghosh AMascolo CElkind E(2023)Modeling with homophily driven heterogeneous data in Gossip learningProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/416(3741-3749)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/416
Wang YTu ZXiang YZhou SChen XLi BZhang TSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Rapid Image Labeling via Neuro-Symbolic LearningProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599485(2467-2477)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599485
Show More Cited By

Recommendations

Object-aware image thumbnailing using image classification and enhanced detection of ROI

Thumbnail images are used to display a large collection of photos in various digital devices. It aims for people to browse and search the image collection effectively. The provided thumbnail images are expressed in a much lower resolution compared to ...
Does Haze Removal Help CNN-Based Image Classification?
Computer Vision – ECCV 2018
Abstract
Hazy images are common in real scenarios and many dehazing methods have been developed to automatically remove the haze from images. Typically, the goal of image dehazing is to produce clearer images from which human vision can better identify the ...
Blurred Image Recognition: A Joint Motion Deblurring and Classification Loss-Aware Approach
Artificial Neural Networks and Machine Learning – ICANN 2021
Abstract
Image motion blur can severely affect the performance of the image recognition model. Traditional methods to tackle this problem usually involve image motion deblurring to improve the image quality before its recognition. However, traditional ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

ECCV'12: Proceedings of the 12th European conference on Computer Vision - Volume Part IV

October 2012

884 pages

ISBN:9783642337642

Editors:
Andrew Fitzgibbon
Microsoft Research Ltd., Cambridge, UK
,
Svetlana Lazebnik
Dept. of Computer Science, University of North Carolina, Chapel Hill, NC, UK
,
Pietro Perona
Dept. of Computer Science, California Institute of Technology, Pasadena, CA, UK
,
Yoichi Sato
Institute of Industrial Science, The University of Tokyo, Tokyo, CA, Japan
,
Cordelia Schmid
Institute of Industrial Science, INRIA, Montbonnot, CA, France

Sponsors

Adobe
TOYOTA: TOYOTA
Google Inc.
IBMR: IBM Research
Microsoft Reasearch: Microsoft Reasearch

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 07 October 2012

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Balasubramanian RLi JTadepalli PWang HWu QZhao HSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Adversarial attacks on combinatorial multi-armed banditsProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692170(2505-2526)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692170
Ghosh AMascolo CElkind E(2023)Modeling with homophily driven heterogeneous data in Gossip learningProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/416(3741-3749)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/416
Wang YTu ZXiang YZhou SChen XLi BZhang TSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Rapid Image Labeling via Neuro-Symbolic LearningProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599485(2467-2477)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599485
Wang YWang WLiang YCai YHooi B(2021)Mixup for Node and Graph ClassificationProceedings of the Web Conference 202110.1145/3442381.3449796(3663-3674)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442381.3449796
Huang FLi CGao BLiu YAlotaibi SChen H(2021)Deep Attentive Multimodal Network Representation Learning for Social Media ImagesACM Transactions on Internet Technology10.1145/341729521:3(1-17)Online publication date: 16-Jun-2021
https://dl.acm.org/doi/10.1145/3417295
Ashrafi SShokouhi SAyatollahi A(2021)Action recognition in still images using a multi-attention guided network with weakly supervised saliency detectionMultimedia Tools and Applications10.1007/s11042-021-11215-180:21-23(32567-32593)Online publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1007/s11042-021-11215-1
Steininger MKobs KZehe ALautenschlager FBecker MHotho A(2020)MapLURACM Transactions on Spatial Algorithms and Systems10.1145/33809736:3(1-24)Online publication date: 15-Apr-2020
https://dl.acm.org/doi/10.1145/3380973
Wang YWang WLiang YCai YHooi B(2020)Progressive Supervision for Node ClassificationMachine Learning and Knowledge Discovery in Databases10.1007/978-3-030-67658-2_16(266-281)Online publication date: 14-Sep-2020
https://dl.acm.org/doi/10.1007/978-3-030-67658-2_16
Sharma URudinac SWorring MDemmers Jvan Dolen W(2020)Semantic Path-Based Learning for Review Volume PredictionAdvances in Information Retrieval10.1007/978-3-030-45439-5_54(821-835)Online publication date: 14-Apr-2020
https://dl.acm.org/doi/10.1007/978-3-030-45439-5_54
Zhang XÖzsu M(2019)Correlation constraint shortest path over large multi-relation graphsProceedings of the VLDB Endowment10.14778/3303753.330375612:5(488-501)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.14778/3303753.3303756
Show More Cited By

View Options

View options

Figures

Tables

Media

View Table of Conten