Article

Semantic topic multimodal hashing for cross-media retrieval

Authors:

Lihuo HeAuthors Info & Claims

IJCAI'15: Proceedings of the 24th International Conference on Artificial Intelligence

Pages 3890 - 3896

Published: 25 July 2015 Publication History

Abstract

Multimodal hashing is essential to cross-media similarity search for its low storage cost and fast query speed. Most existing multimodal hashing methods embedded heterogeneous data into a common low-dimensional Hamming space, and then rounded the continuous embeddings to obtain the binary codes. Yet they usually neglect the inherent discrete nature of hashing for relaxing the discrete constraints, which will cause degraded retrieval performance especially for long codes. For this purpose, a novel Semantic Topic Multimodal Hashing (STMH) is developed by considering latent semantic information in coding procedure. It first discovers clustering patterns of texts and robust factorizes the matrix of images to obtain multiple semantic topics of texts and concepts of images. Then the learned multimodal semantic features are transformed into a common subspace by their correlations. Finally, each bit of unified hash code can be generated directly by figuring out whether a topic or concept is contained in a text or an image. Therefore, the obtained model by STMH is more suitable for hashing scheme as it directly learns discrete hash codes in the coding process. Experimental results demonstrate that the proposed method outperforms several state-of-the-art methods.

References

[1]

Punam Bedi, Harmeet Kaur, and Sudeep Marwaha. Trust based recommender system for semantic web. In Proceedings of the 20th International Joint Conference on Artificial Intelligence, pages 2677-2682, Hyderabad, India, January 2007. AAAI Press.

Digital Library

[2]

David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 3:993-1022, 2003.

Digital Library

[3]

David M. Blei. Probabilistic topic models. Communications of the ACM, 55(4):77-84, 2012.

Digital Library

[4]

Michael M. Bronstein, Alexander M. Bronstein, Fabrice Michel, and Nikos Paragios. Data fusion through cross-modality metric learning using simi-larity-sensitive hashing. In Proceedings of the 23rd IEEE International Conference on Computer Vision and Pattern Recognition, pages 3594-3601, San Francisco, CA, USA, June 2010. IEEE Computer Society.

[5]

Deng Cai, Xiaofei He, Jiawei Han, and T. S. Huang. Graph regularized nonnegative matrix factoriza-tion for data representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(8):1548-1560, 2011.

Digital Library

[6]

Chris Ding, Ding Zhou, Xiaofeng He, and Hongyuan Zha. R₁-PCA: rotational invariant L₁- norm principal component analysis for robust subspace fac-torization. In Proceedings of the 23rd International Conference on Machine Learning, pages 281-288, Pittsburgh, Pennsylvania, USA, June 2006. ACM.

Digital Library

[7]

Guiguang Ding, Yuchen Guo, and Jile Zhou. Collective matrix factorization hashing for multimodal data. In Proceedings of the 27th IEEE International Conference on Computer Vision and Pattern Recognition, pages 2083-2090, Columbus, OH, USA, June 2014. IEEE Computer Society.

Digital Library

[8]

Deguang Kong, Chris Ding, and Heng Huanga. Robust nonnegative matrix factorization using L_2,1-norm. In Proceedings of the 20th ACM International Conference on Information and knowledge management, pages 673-682, Glasgow, United Kingdom, October 2011. ACM.

Digital Library

[9]

Shaishav Kumar and Raghavendra Udupa. Learning hash functions for cross-view similarity search. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence, pages 1360-1367, Barcelona, Catalonia, Spain, July 2011. AAAI Press.

Digital Library

[10]

Daniel D. Lee and Sebastian H. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755):788-791, 1999.

[11]

Jingkuan Song, Yang Yang, Yi Yang, Zi Huang, and Hengtao Shen. Inter-media hashing for large-scale retrieval from heterogeneous data sources. In Proceedings of the 19th International Conference on Management of Data, pages 785-796, Ahmedabad, India, December 2013. Computer Society of India.

Digital Library

[12]

Yair Weiss, Antonio Torralba, and Rob Fergus. Spectral hashing. In Proceedings of the 23rd Annual Conference on Neural Information Processing Systems, pages 1753-1760, Vancouver, British Columbia, Canada, December 2009. Curran Associates.

[13]

Jason Weston, Samy Bengio, and Ni colas Usunier. Wsabie: scaling up to large vocabulary image annotation. In Proceedings of the 22nd Interna-tional Joint Conference on Artificial Intelligence, pages 2764-2770, Barcelona, Catalonia, Spain, July 2011. AAAI Press.

Digital Library

[14]

Deming Zhai, Hong Chang, Yi Zhen, Xianming Liu, Xilin Chen, and Wen Gao. Parametric local multimodal hashing for cross-view similarity search. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence, pages 2754-2760, Beijing, China, August 2013. AAAI Press.

Digital Library

[15]

Tianyi Zhou and Dacheng Tao. Godec: randomized low-rank & sparse matrix decomposition in noisy case. In Proceedings of the 28th International Conference on Machine Learning, pages 33-40, Bellevue, Washington, USA, June 2011. ACM.

[16]

Jile Zhou, Guiguang Ding, and Yuhen Guo. Latent semantic sparse hashing for cross-modal similarity search. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 415-424, Gold Coast, QLD, Australia, July 2014. ACM.

Digital Library

[17]

X. Zhu, Z. Huang, H. Shen, Z. Huang, and X. Zhao. Linear cross-modal hashing for efficient multimedia search. In Proceedings of the 21st ACM Multimedia Conference, pages 143-152, Barcelona, Spain, October 2013. ACM.

Digital Library

Cited By

Liu XZeng HShi YZhu JHsia CMa K(2023)Deep Cross-Modal Hashing Based on Semantic Consistent RankingIEEE Transactions on Multimedia10.1109/TMM.2023.325419925(9530-9542)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2023.3254199
Tu RMao XLin QJi WQin WWei WHuang H(2023)Unsupervised Cross-Modal Hashing via Semantic Text MiningIEEE Transactions on Multimedia10.1109/TMM.2023.324360825(8946-8957)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2023.3243608
Małkiński MMańdziuk J(2023)A review of emerging research directions in Abstract Visual ReasoningInformation Fusion10.1016/j.inffus.2022.11.01191:C(713-736)Online publication date: 9-Feb-2023
https://dl.acm.org/doi/10.1016/j.inffus.2022.11.011
Show More Cited By

Recommendations

Multi-modal Mutual Topic Reinforce Modeling for Cross-media Retrieval
MM '14: Proceedings of the 22nd ACM international conference on Multimedia

As an important and challenging problem in the multimedia area, multi-modal data understanding aims to explore the intrinsic semantic information across different modalities in a collaborative manner. To address this problem, a possible solution is to ...
Semantic Constraints Matrix Factorization Hashing for cross-modal retrieval
Abstract
Cross-modal hashing methods have attracted considerable attention due to their low memory usage and high query speed in large-scale cross-modal retrieval. During the encoding process, there still remains two crucial bottlenecks: how to ...
Latent semantic-enhanced discrete hashing for cross-modal retrieval
Abstract
Hashing methods have been proposed for the cross-modal retrieval tasks due to their flexibility and effectiveness. The main idea of cross-modal hashing is to embed heterogeneous multimedia data into common Hamming space. How to effectively exploit ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

IJCAI'15: Proceedings of the 24th International Conference on Artificial Intelligence

July 2015

4429 pages

ISBN:9781577357384

Editors:
Qiang Yang
Hong Kong University of Science and Technology
,
Michael Wooldridge
University of Oxford

Sponsors

The International Joint Conferences on Artificial Intelligence, Inc. (IJCAI)

Publisher

AAAI Press

Publication History

Published: 25 July 2015

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu XZeng HShi YZhu JHsia CMa K(2023)Deep Cross-Modal Hashing Based on Semantic Consistent RankingIEEE Transactions on Multimedia10.1109/TMM.2023.325419925(9530-9542)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2023.3254199
Tu RMao XLin QJi WQin WWei WHuang H(2023)Unsupervised Cross-Modal Hashing via Semantic Text MiningIEEE Transactions on Multimedia10.1109/TMM.2023.324360825(8946-8957)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2023.3243608
Małkiński MMańdziuk J(2023)A review of emerging research directions in Abstract Visual ReasoningInformation Fusion10.1016/j.inffus.2022.11.01191:C(713-736)Online publication date: 9-Feb-2023
https://dl.acm.org/doi/10.1016/j.inffus.2022.11.011
Li JYu EMa JChang XZhang HSun J(2022)Discrete Fusion Adversarial Hashing for cross-modal retrievalKnowledge-Based Systems10.1016/j.knosys.2022.109503253:COnline publication date: 11-Oct-2022
https://dl.acm.org/doi/10.1016/j.knosys.2022.109503
Liu XCao GLin QCao W(2022)Adaptive weight multi-channel center similar deep hashingJournal of Visual Communication and Image Representation10.1016/j.jvcir.2022.10364289:COnline publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.jvcir.2022.103642
Wang DHu QYang C(2022)Biometric recognition based on scalable end-to-end convolutional neural network using photoplethysmographyComputers in Biology and Medicine10.1016/j.compbiomed.2022.105654147:COnline publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1016/j.compbiomed.2022.105654
Kumari RAshok NGhosal TEkbal A(2021)Misinformation detection using multitask learning with mutual learning for novelty detection and emotion recognitionInformation Processing and Management: an International Journal10.1016/j.ipm.2021.10263158:5Online publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1016/j.ipm.2021.102631
Zhao GZhang MLi YLiu JZhang BWen J(2021)Pyramid regional graph representation learning for content-based video retrievalInformation Processing and Management: an International Journal10.1016/j.ipm.2020.10248858:3Online publication date: 1-May-2021
https://dl.acm.org/doi/10.1016/j.ipm.2020.102488
Ramesh RAnand VWang ZZhu TLyu WYuan SLin CTruong KHeylen DCzerwinski MBerthouze NChetouani MNakano M(2020)Kinetics and Scene Features for Intent DetectionCompanion Publication of the 2020 International Conference on Multimodal Interaction10.1145/3395035.3425641(135-139)Online publication date: 25-Oct-2020
https://dl.acm.org/doi/10.1145/3395035.3425641
Zhang JPeng Y(2020)Multi-Pathway Generative Adversarial Hashing for Unsupervised Cross-Modal RetrievalIEEE Transactions on Multimedia10.1109/TMM.2019.292212822:1(174-187)Online publication date: 3-Jan-2020
https://dl.acm.org/doi/10.1109/TMM.2019.2922128
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents