research-article

Multimedia Summarization for Social Events in Microblog Stream

Authors:

Tat-Seng ChuaAuthors Info & Claims

IEEE Transactions on Multimedia, Volume 17, Issue 2

Pages 216 - 228

https://doi.org/10.1109/TMM.2014.2384912

Published: 01 February 2015 Publication History

Abstract

Microblogging services have revolutionized the way people exchange information. Confronted with the ever-increasing numbers of social events and the corresponding microblogs with multimedia contents, it is desirable to provide visualized summaries to help users to quickly grasp the essence of these social events for better understanding. While existing approaches mostly focus only on text-based summary, microblog summarization with multiple media types (e.g., text, image, and video) is scarcely explored. In this paper, we propose a multimedia social event summarization framework to automatically generate visualized summaries from the microblog stream of multiple media types. Specifically, the proposed framework comprises three stages, as follows. 1) A noise removal approach is first devised to eliminate potentially noisy images. An effective spectral filtering model is exploited to estimate the probability that an image is relevant to a given event. 2) A novel cross-media probabilistic model, termed Cross-Media-LDA (CMLDA), is proposed to jointly discover subevents from microblogs of multiple media types. The intrinsic correlations among these different media types are well explored and exploited for reinforcing the cross-media subevent discovery process. 3) Finally, based on the cross-media knowledge of all the discovered subevents, a multimedia microblog summary generation process is designed to jointly identify both representative textual and visual samples, which are further aggregated to form a holistic visualized summary. We conduct extensive experiments on two real-world microblog datasets to demonstrate the superiority of the proposed framework as compared to the state-of-the-art approaches.

References

[1]

T.-S. Chua, H. Luan, M. Sun, and S. Yang, “Next: Nus-Tsinghua center for extreme search of user-generated content,” IEEE MultiMedia Mag., vol. 19, no. 3, pp. 81–87, Jul.-Sep. 2012.

Digital Library

[2]

T. Sakaki, M. Okazaki, and Y. Matsuo, “Earthquake shakes Twitter users: Real-time event detection by social sensors,” in Proc. WWW, 2010, pp. 851–860.

[3]

J. Weng and B.-S. Lee, “Event detection in Twitter,” in Proc. ICWSM, 2011.

[4]

Y. Chen, H. Amiri, Z. Li, and T.-S. Chua, “Emerging topic detection for organizations from microblogs,” in Proc. SIGIR, 2013, pp. 43–52.

[5]

K. Spärck Jones, “Automatic summarising: The state of the art,” Inf. Process. Manage., vol. 43, no. 6, pp. 1449–1481, 2007.

Digital Library

[6]

D. Inouye and J. K. Kalita, “Comparing Twitter summarization algorithms for multiple post summaries,” in Proc. SocialCom, 2011, pp. 298–306.

[7]

D. Chakrabarti and K. Punera, “Event summarization using tweets,” in Proc. ICWSM, 2011, pp. 66–73.

[8]

B. Sharifi, M.-A. Hutton, and J. Kalita, “Summarizing microblogs automatically,” in Proc. NAACL HLT, 2010, pp. 685–688.

[9]

Y. Yang, Z.-J. Zha, Y. Gao, X. Zhu, and T.-S. Chua, “Exploiting web images for semantic video indexing via robust sample-specific loss,” IEEE Trans. Multimedia, vol. 16, no. 6, pp. 1677–1689, Oct. 2014.

[10]

J. Bian, Y. Yang, and T.-S. Chua, “Predicting trending messages and diffusion participants in microblogging network,” in Proc. SIGIR, 2014, pp. 537–546.

[11]

J. Bian, Y. Yang, and T.-S. Chua, “Multimedia summarization for trending topics in microblogs,” in Proc. CIKM, 2013, pp. 1807–1812.

[12]

L. Vanderwende, H. Suzuki, C. Brockett, and A. Nenkova, “Beyond sumbasic: Task-focused summarization with sentence simplification and lexical expansion,” Inf. Process. Manage., vol. 43, no. 6, pp. 1606–1618, 2007.

Digital Library

[13]

D. R. Radev, H. Jing, M. Styś, and D. Tam, “Centroid-based summarization of multiple documents,” Inf. Process. Manage., vol. 40, no. 6, pp. 919–938, 2004.

Digital Library

[14]

J. Goldstein, M. Kantrowitz, V. Mittal, and J. Carbonell, “Summarizing text documents: Sentence selection and evaluation metrics,” in Proc. SIGIR, 1999, pp. 121–128.

[15]

R. Mihalcea and P. Tarau, “A language independent algorithm for single and multiple document summarization,” in Proc. IJCNLP, 2005.

[16]

G. Erkan and D. R. Radev, “Lexpagerank: Prestige in multi-document text summarization,” in Proc. EMNLP, 2004, pp. 365–371.

[17]

Y. Gong and X. Liu, “Generic text summarization using relevance measure and latent semantic analysis,” in Proc. SIGIR, 2001, pp. 19–25.

[18]

A. Haghighi and L. Vanderwende, “Exploring content models for multi-document summarization,” in Proc. NAACL HLT, 2009, pp. 362–370.

[19]

S. Park, J.-H. Lee, D.-H. Kim, and C.-M. Ahn, “Multi-document summarization based on cluster using non-negative matrix factorization,” in Proc. SOFSEM, 2007, pp. 761–770.

[20]

D. Shen, J.-T. Sun, H. Li, Q. Yang, and Z. Chen, “Document summarization using conditional random fields,” in Proc. ACL, 2007, vol. 7, pp. 2862–2867.

[21]

J. M. Conroy and D. P. O'leary, “Text summarization via hidden Markov models,” in Proc. SIGIR, 2001, pp. 406–407.

[22]

D. Wang, T. Li, S. Zhu, and C. Ding, “Multi-document summarization via sentence-level semantic analysis and symmetric matrix factorization,” in Proc. SIGIR, 2008, pp. 307–314.

[23]

C. Lin, C. Lin, J. Li, D. Wang, Y. Chen, and T. Li, “Generating event storylines from microblogs,” in Proc. CIKM, 2012, pp. 175–184.

[24]

Y. Yang, Y. Yang, and H. Shen, “Effective transfer tagging from image to video,” TOMCCAP, vol. 9, no. 2, 2013.

[25]

Y. Yang, Y. Yang, Z. Huang, H. Shen, and F. Nie, “Tag localization with spatial correlations and joint group sparsity,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2011, pp. 881–888.

[26]

R. Yan, X. Wan, M. Lapata, W. X. Zhao, P.-J. Cheng, and X. Li, “Visualizing timelines: Evolutionary summarization via iterative reinforcement between text and image streams,” in Proc. CIKM, 2012, pp. 275–284.

[27]

P. Li, J. Ma, and S. Gao, “Learning to summarize web image and text mutually,” in Proc. ICMR, 2012, p. 28.

[28]

W. Liu, Y.-G. Jiang, J. Luo, and S.-F. Chang, “Noise resistant graph ranking for improved web image search,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2011, pp. 849–856.

[29]

T. Shi, M. Belkin, and B. Yu, “Data spectroscopy: Eigenspaces of convolution operators and clustering,” Ann. Statist., vol. 37, no. 6B, pp. 3960–3984, 2009.

[30]

D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet allocation,” in Proc. JMLR, 2003, vol. 3, pp. 993–1022.

[31]

Y. Gao, F. Wang, H. Luan, and T.-S. Chua, “Brand data gathering from live social media streams,” in Proc. ICMR, 2014, p. 169.

[32]

C.-Y. Lin, “Rouge: A package for automatic evaluation of summaries,” in Text Summarization Branches Out: ACL-04 Workshop, 2004, pp. 74–81.

[33]

J. Shi and J. Malik, “Normalized cuts and image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888–905, Aug. 2000.

Digital Library

[34]

B. Sharifi, M.-A. Hutton, and J. K. Kalita, “Experiments in microblog summarization,” in Proc. SocialCom, 2010, pp. 49–56.

Cited By

Rong HChen ZLu ZXu FSheng V(2024)Multization: Multi-Modal Summarization Enhanced by Multi-Contextually Relevant and Irrelevant Attention AlignmentACM Transactions on Asian and Low-Resource Language Information Processing10.1145/365198323:5(1-29)Online publication date: 10-May-2024
https://dl.acm.org/doi/10.1145/3651983
Yuan MCui SZhang XWang SXu HLiu THui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Exploring the Trade-Off within Visual Information for MultiModal Sentence SummarizationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657753(2006-2017)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657753
Lu JWang GCai YWu X(2024)Towards Automated Infographic Authoring From Natural Language Statement With Multiple Proportional FactsIEEE Transactions on Multimedia10.1109/TMM.2024.336072226(7101-7113)Online publication date: 31-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3360722
Show More Cited By

Index Terms

Multimedia Summarization for Social Events in Microblog Stream
1. Computing methodologies
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Information extraction
  2. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Clustering to Improve Microblog Stream Summarization
SYNASC '12: Proceedings of the 2012 14th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing

Microblogging has shown a massive increase in use over the past couple of years. According to recent statistics, Twitter (the most popular microblogging platform) has over 340 million posts per day coming from its 140 million active users. In order to ...
Online Bursty Event Detection from Microblog
UCC '14: Proceedings of the 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing

Microblogs (e.g., Twitter and Weibo) have become a large social media platform for users to share contents, their interests and events with friends. A surge of the number of event related posts always reflects that some people's concern real-life events ...
On choosing an effective automatic evaluation metric for microblog summarisation
IIiX '14: Proceedings of the 5th Information Interaction in Context Symposium

Popular microblogging services, such as Twitter, are engaging millions of users who constantly post and share information about news and current events each day, resulting in millions of messages discussing what is happening in the world. To help users ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Multimedia

IEEE Transactions on Multimedia Volume 17, Issue 2

Feb. 2015

113 pages

ISSN:1520-9210

Issue’s Table of Contents

Copyright © 2014.

Publisher

IEEE Press

Publication History

Published: 01 February 2015

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Rong HChen ZLu ZXu FSheng V(2024)Multization: Multi-Modal Summarization Enhanced by Multi-Contextually Relevant and Irrelevant Attention AlignmentACM Transactions on Asian and Low-Resource Language Information Processing10.1145/365198323:5(1-29)Online publication date: 10-May-2024
https://dl.acm.org/doi/10.1145/3651983
Yuan MCui SZhang XWang SXu HLiu THui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Exploring the Trade-Off within Visual Information for MultiModal Sentence SummarizationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657753(2006-2017)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657753
Lu JWang GCai YWu X(2024)Towards Automated Infographic Authoring From Natural Language Statement With Multiple Proportional FactsIEEE Transactions on Multimedia10.1109/TMM.2024.336072226(7101-7113)Online publication date: 31-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3360722
Cui CLiang XWu SLi Z(2024)Align vision-language semantics by multi-task learning for multi-modal summarizationNeural Computing and Applications10.1007/s00521-024-09908-336:25(15653-15666)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s00521-024-09908-3
Xiao XDu MXu SLiu GZhang C(2024)Cross-media web video event mining based on multiple semantic-paths embeddingNeural Computing and Applications10.1007/s00521-023-09050-636:2(667-683)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s00521-023-09050-6
Jangra AMukherjee SJatowt ASaha SHasanuzzaman M(2023)A Survey on Multi-modal SummarizationACM Computing Surveys10.1145/358470055:13s(1-36)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3584700
Liu NSun XYu HYao FXu GFu K(2023)Abstractive Summarization for Video: A Revisit in Multistage Fusion Network With Forget GateIEEE Transactions on Multimedia10.1109/TMM.2022.315799325(3296-3310)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3157993
Zhang CLiu GXiao X(2023)Cross-media correlation learning for web video event mining with integrated text semantics and network structural informationNeural Computing and Applications10.1007/s00521-023-08323-435:16(11815-11831)Online publication date: 13-Feb-2023
https://dl.acm.org/doi/10.1007/s00521-023-08323-4
Zhang ZSun YSu S(2023)Multimodal Learning for Automatic Summarization: A SurveyAdvanced Data Mining and Applications10.1007/978-3-031-46664-9_25(362-376)Online publication date: 27-Aug-2023
https://dl.acm.org/doi/10.1007/978-3-031-46664-9_25
Papagiannopoulou AAngeli C(2022)Social Media Text Summarisation Techniques and Approaches: A Literature ReviewProceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial Intelligence10.1145/3579654.3579774(1-6)Online publication date: 23-Dec-2022
https://dl.acm.org/doi/10.1145/3579654.3579774
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents