short-paper

Cross Modal Deep Model and Gaussian Process Based Model for MSR-Bing Challenge

Authors:

Chunhong PanAuthors Info & Claims

MM '14: Proceedings of the 22nd ACM international conference on Multimedia

Pages 225 - 228

https://doi.org/10.1145/2647868.2656400

Published: 03 November 2014 Publication History

Abstract

In the MSR-Bing Image Retrieval Challenge, the contestants are required to design a system that can score the query-image pairs based on the relevance between queries and images. To address this problem, we propose a regression based cross modal deep learning model and a Gaussian Process scoring model. The regression based cross modal deep learning model takes the image features and query features as inputs respectively and outputs the relevance scores directly. The Gaussian Process scoring model regards the challenge as a ranking problem and utilizes the click (or pseudo click) information from both the training set and the development set to predict the relevance scores. The proposed models are used in different situations: matched and miss-matched queries. Experiments on the development set show the effectiveness of the proposed models.

References

[1]

Y. Cheng. Mean shift, mode seeking, and clustering. IEEE Transactions on PAMI, 17(8):790--799, 1995.

Digital Library

[2]

J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell. DeCAF: A deep convolutional activation feature for generic visual recognition. arXiv preprint arXiv:1310.1531, 2013. Software available at https://github.com/UCB-ICSI-Vision-Group/decafrelease/wiki.

[3]

Q. Fang, H. Xu, R. Wang, S. Qian, T. Wang, J. Sang, and C. Xu. Towards MSR-Bing Challenge: Ensemble of diverse models for image retrieval. In MSR-Bing IRC 2013 Workshop, 2013.

[4]

G. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8):1771--1800, 2002.

Digital Library

[5]

X. Hua. Looking into "MSR-Bing Challenge on Image Retrieval". 2013.

[6]

X. Hua, L. Yang, J. Wang, J. Wang, M. Ye, K. Wang, Y. Rui, and J. Li. Clickage: Towards bridging semantic and intent gaps via mining click logs of search engines. In Proceedings of ACM International Conference on Multimedia, pages 243--252, 2013.

Digital Library

[7]

V. Jain and M. Varma. Learning to re-rank: query-dependent image re-ranking using click data. In Proceedings of International Conference on World Wide Web, pages 277--286, 2011.

Digital Library

[8]

A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in NIPS, pages 1106--1114, 2012.

[9]

T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.

[10]

J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Ng. Multimodal deep learning. In Proceedings of International Conference on Machine Learning, pages 689--696, 2011.

[11]

Y. Pan, T. Yao, H. Li, and C. Ngo. USTC-CityU at MSR-Bing IRC: Image search by graph-based label propagation. In MSR-Bing IRC 2013 Workshop, 2013.

[12]

Y. Rui, T. Huang, and S. Chang. Image retrieval: Current techniques, promising directions, and open issues. Journal of Visual Communication and Image Representation, 10(1):39--62, 1999.

Digital Library

[13]

A. Sayko and A. Slesarev. Report on a baseline approach to the 2nd MSR-Bing challenge on image retrieval. In MSR-Bing IRC 2013 Workshop, 2013.

[14]

N. Srivastava and R. Salakhutdinov. Multimodal learning with deep boltzmann machines. In Advances in Neural Information Processing Systems, pages 2222--2230, 2012.

Digital Library

[15]

L. Wang, S. Cen, H. Bai, C. Huang, N. Zhao, B. Liu, and Y. Feng. France telecom orange labs a MSR-Bing challenge on image retrieval 2013. In MSR-Bing IRC 2013 Workshop, 2013.

[16]

C. Wu, K. Chu, Y. Kuo, Y. Chen, W. Lee, and W. Hsu. Search-based relevance association with auxiliary contextual cues. In MSR-Bing IRC 2013 Workshop, 2013.

Digital Library

Cited By

Song QYu SLeng CWu JHu QCheng JZhou XSmeaton ATian QBulterman DShen HMayer-Patel KYan S(2015)Learning Deep Features For MSR-bing Information Retrieval ChallengeProceedings of the 23rd ACM international conference on Multimedia10.1145/2733373.2809928(169-172)Online publication date: 13-Oct-2015
https://dl.acm.org/doi/10.1145/2733373.2809928
Wu QZhang HLiu SCao X(2015)Multimedia Analysis with Deep LearningProceedings of the 2015 IEEE International Conference on Multimedia Big Data10.1109/BigMM.2015.27(20-23)Online publication date: 20-Apr-2015
https://dl.acm.org/doi/10.1109/BigMM.2015.27

Index Terms

Cross Modal Deep Model and Gaussian Process Based Model for MSR-Bing Challenge
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

Image Retrieval Using Fused Deep Convolutional Features

This paper proposes an image retrieval using fused deep convolutional features to solve the semantic gap between low-level features and high-level semantic features of traditional contend-based image retrieval method. Firstly, the improved network ...
Learning Deep Features For MSR-bing Information Retrieval Challenge
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

Two tasks have been put forward in the MSR-bing Grand Challenge 2015. To address the information retrieval task, we raise and integrate a series of methods with visual features obtained by convolution neural network (CNN) models. In our experiments, we ...
A statistical correlation model for image retrieval
MULTIMEDIA '01: Proceedings of the 2001 ACM workshops on Multimedia: multimedia information retrieval

A bigram correlation model for image retrieval is proposed, which captures the semantic relationship among images in a database from simple statistics of users' relevance feedback information. It is used in the post-processing of image retrieval results ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '14: Proceedings of the 22nd ACM international conference on Multimedia

November 2014

1310 pages

ISBN:9781450330633

DOI:10.1145/2647868

General Chairs:
Kien A. Hua
University of Central Florida, USA
,
Yong Rui
Microsoft Research, China
,
Ralf Steinmetz
Technische Universitt Darmstadt, Germany
,
Program Chairs:
Alan Hanjalic
Delft University of Technology, Netherlands
,
Apostol (Paul) Natsev
Google, USA
,
Wenwu Zhu
Tsinghua University, China

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Conference

MM '14

Sponsor:

SIGMM

MM '14: 2014 ACM Multimedia Conference

November 3 - 7, 2014

Florida, Orlando, USA

Acceptance Rates

MM '14 Paper Acceptance Rate 55 of 286 submissions, 19%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
289
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Song QYu SLeng CWu JHu QCheng JZhou XSmeaton ATian QBulterman DShen HMayer-Patel KYan S(2015)Learning Deep Features For MSR-bing Information Retrieval ChallengeProceedings of the 23rd ACM international conference on Multimedia10.1145/2733373.2809928(169-172)Online publication date: 13-Oct-2015
https://dl.acm.org/doi/10.1145/2733373.2809928
Wu QZhang HLiu SCao X(2015)Multimedia Analysis with Deep LearningProceedings of the 2015 IEEE International Conference on Multimedia Big Data10.1109/BigMM.2015.27(20-23)Online publication date: 20-Apr-2015
https://dl.acm.org/doi/10.1109/BigMM.2015.27

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten