research-article

Zero-Shot Stance Detection via Contrastive Learning

Authors:

Ruifeng XuAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 2738 - 2747

https://doi.org/10.1145/3485447.3511994

Published: 25 April 2022 Publication History

Abstract

Zero-shot stance detection (ZSSD) is challenging as it requires detecting the stance of previously unseen targets during the inference stage. Being able to detect the target-related transferable stance features from the training data is arguably an important step in ZSSD. Generally speaking, stance features can be grouped into target-invariant and target-specific categories. Target-invariant stance features carry the same stance regardless of the targets they are associated with. On the contrary, target-specific stance features only co-occur with certain targets. As such, it is important to distinguish these two types of stance features when learning stance features of unseen targets. To this end, in this paper, we revisit ZSSD from a novel perspective by developing an effective approach to distinguish the types (target-invariant/-specific) of stance features, so as to better learn transferable stance features. To be specific, inspired by self-supervised learning, we frame the stance-feature-type identification as a pretext task in ZSSD. Furthermore, we devise a novel hierarchical contrastive learning strategy to capture the correlation and difference between target-invariant and -specific features and further among different stance labels. This essentially allows the model to exploit transferable stance features more effectively for representing the stance of previously unseen targets. Extensive experiments on three benchmark datasets show that the proposed framework achieves the state-of-the-art performance in ZSSD.

References

[1]

Emily Allaway and Kathleen McKeown. 2020. Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 8913–8931.

[2]

Emily Allaway, Malavika Srikanth, and Kathleen McKeown. 2021. Adversarial Learning for Zero-Shot Stance Detection on Social Media. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4756–4767.

[3]

Isabelle Augenstein, Tim Rocktäschel, Andreas Vlachos, and Kalina Bontcheva. 2016. Stance Detection with Bidirectional Conditional Encoding. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 876–885.

[4]

David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. the Journal of machine Learning research 3 (2003), 993–1022.

[5]

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning. 1597–1607.

[6]

Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. 2020. Big Self-Supervised Models are Strong Semi-Supervised Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.). Vol. 33. Curran Associates, Inc., 22243–22255. https://proceedings.neurips.cc/paper/2020/file/fcbc95ccdd551da181207c0c1400c655-Paper.pdf

[7]

Ting Chen, Xiaohua Zhai, Marvin Ritter, Mario Lucic, and Neil Houlsby. 2019. Self-supervised gans via auxiliary rotation loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12154–12163.

[8]

Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, and Nigel Collier. 2020. Will-They-Won’t-They: A Very Large Dataset for Stance Detection on Twitter. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 1715–1724.

[9]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171–4186.

[10]

Carl Doersch, Abhinav Gupta, and Alexei A Efros. 2015. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE international conference on computer vision. 1422–1430.

Digital Library

[11]

Jiachen Du, Ruifeng Xu, Yulan He, and Lin Gui. 2017. Stance Classification with Target-specific Neural Attention. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17. 3988–3994.

[12]

Spyros Gidaris, Praveer Singh, and Nikos Komodakis. 2018. Unsupervised Representation Learning by Predicting Image Rotations. In International Conference on Learning Representations.

[13]

John Giorgi, Osvald Nitski, Bo Wang, and Gary Bader. 2021. DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 879–895. https://aclanthology.org/2021.acl-long.72

[14]

Thomas L Griffiths and Mark Steyvers. 2004. Finding scientific topics. Proceedings of the National academy of Sciences 101, suppl 1(2004), 5228–5235.

[15]

Beliz Gunel, Jingfei Du, Alexis Conneau, and Veselin Stoyanov. 2021. Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning. In International Conference on Learning Representations. https://openreview.net/forum?id=cu7IUiOhujH

[16]

Kazi Saidul Hasan and Vincent Ng. 2014. Why are You Taking this Stance? Identifying and Classifying Reasons in Ideological Debates. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 751–762.

[17]

Mohammad Kachuee, Hao Yuan, Young-Bum Kim, and Sungjin Lee. 2021. Self-Supervised Contrastive Learning for Efficient User Satisfaction Prediction in Conversational Agents. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4053–4064.

[18]

Ayush Kaushal, Avirup Saha, and Niloy Ganguly. 2021. tWT–WT: A Dataset to Assert the Role of Target Entities for Detecting Stance of Tweets. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 3879–3889.

[19]

Kornraphop Kawintiranon and Lisa Singh. 2021. Knowledge Enhanced Masked Language Model for Stance Detection. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4725–4735.

[20]

Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised Contrastive Learning. In Advances in Neural Information Processing Systems, Vol. 33. 18661–18673.

[21]

Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. 2016. Learning representations for automatic colorization. In European conference on computer vision. Springer, 577–593.

[22]

Yingjie Li and Cornelia Caragea. 2019. Multi-Task Stance Detection with Sentiment and Stance Lexicons. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 6299–6305.

[23]

Bin Liang, Yonghao Fu, Lin Gui, Min Yang, Jiachen Du, Yulan He, and Ruifeng Xu. 2021. Target-adaptive Graph for Cross-target Stance Detection. In the Web Conference 2021 (WWW ’21).

[24]

Rui Liu, Zheng Lin, Yutong Tan, and Weiping Wang. 2021. Enhancing Zero-shot and Few-shot Stance Detection with Commonsense Knowledge Graph. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 3152–3157.

[25]

Yixin Liu and Pengfei Liu. 2021. SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 1065–1072.

[26]

Ishan Misra and Laurens van der Maaten. 2020. Self-supervised learning of pretext-invariant representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6707–6717.

[27]

Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. SemEval-2016 Task 6: Detecting Stance in Tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016). 31–41.

[28]

Mehdi Noroozi and Paolo Favaro. 2016. Unsupervised learning of visual representations by solving jigsaw puzzles. In European conference on computer vision. Springer, 69–84.

[29]

Xiao Pan, Mingxuan Wang, Liwei Wu, and Lei Li. 2021. Contrastive Learning for Many-to-many Multilingual Neural Machine Translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 244–258.

[30]

Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, and Jie Zhou. 2021. ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 3350–3363.

[31]

Yao Qiu, Jinchao Zhang, and Jie Zhou. 2021. Improving Gradient-based Adversarial Training for Text Classification by Contrastive Learning and Auto-Encoder. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 1698–1707.

[32]

Umme Aymun Siddiqua, Abu Nowshed Chy, and Masaki Aono. 2019. Tweet Stance Detection Using an Attention based Neural Ensemble Model. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 1868–1873.

[33]

Nathaniel Simard and Guillaume Lagrange. 2021. Improving Few-Shot Learning with Auxiliary Self-Supervised Pretext Tasks. arXiv preprint arXiv:2101.09825(2021).

[34]

Swapna Somasundaran and Janyce Wiebe. 2010. Recognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 workshop on computational approaches to analysis and generation of emotion in text. 116–124.

Digital Library

[35]

Qingying Sun, Zhongqing Wang, Qiaoming Zhu, and Guodong Zhou. 2018. Stance Detection with Hierarchical Attention Network. In Proceedings of the 27th International Conference on Computational Linguistics. 2399–2409.

[36]

Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2020. Contrastive multiview coding. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16. Springer, 776–794.

[37]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research 9, 86 (2008), 2579–2605.

[38]

Feng Wang and Huaping Liu. 2021. Understanding the Behaviour of Contrastive Loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2495–2504.

[39]

Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning. PMLR, 9929–9939.

[40]

Penghui Wei and Wenji Mao. 2019. Modeling Transferable Topics for Cross-Target Stance Detection. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1173–1176.

Digital Library

[41]

Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. 2018. Unsupervised feature learning via non-parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3733–3742.

[42]

Chang Xu, Cécile Paris, Surya Nepal, and Ross Sparks. 2018. Cross-Target Stance Classification with Self-Attention Networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 778–783.

[43]

Nan Yang, Furu Wei, Binxing Jiao, Daxing Jiang, and Linjun Yang. 2021. xMoCo: Cross Momentum Contrastive Learning for Open-Domain Question Answering. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 6120–6129.

[44]

Bowen Zhang, Min Yang, Xutao Li, Yunming Ye, Xiaofei Xu, and Kuai Dai. 2020. Enhancing Cross-target Stance Detection with Transferable Semantic-Emotion Knowledge. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 3188–3197.

[45]

Dejiao Zhang, Feng Nan, Xiaokai Wei, Shang-Wen Li, Henghui Zhu, Kathleen McKeown, Ramesh Nallapati, Andrew O Arnold, and Bing Xiang. 2021. Supporting Clustering with Contrastive Learning. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 5419–5430.

[46]

Richard Zhang, Phillip Isola, and Alexei A Efros. 2016. Colorful image colorization. In European conference on computer vision. Springer, 649–666.

Cited By

Dong LSu ZFu XZhang BDai G(2024)Implicit Stance Detection with Hashtag Semantic EnrichmentMathematics10.3390/math1211166312:11(1663)Online publication date: 26-May-2024
https://doi.org/10.3390/math12111663
Ding DDai GPeng CPeng XZhang BHuang H(2024)Distantly Supervised Explainable Stance Detection via Chain-of-Thought SupervisionMathematics10.3390/math1207111912:7(1119)Online publication date: 8-Apr-2024
https://doi.org/10.3390/math12071119
Ding DFu XPeng XFan XHuang HZhang B(2024)Leveraging Chain-of-Thought to Enhance Stance Detection with Prompt-TuningMathematics10.3390/math1204056812:4(568)Online publication date: 13-Feb-2024
https://doi.org/10.3390/math12040568
Show More Cited By

Index Terms

Zero-Shot Stance Detection via Contrastive Learning
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning algorithms
2. Information systems
  1. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Enhancing Zero-Shot Stance Detection via Targeted Background Knowledge
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

Stance detection aims to identify the stance of the text towards a target. Different from conventional stance detection, Zero-Shot Stance Detection (ZSSD) needs to predict the stances of the unseen targets during the inference stage. For human beings, ...
Zero-shot stance detection based on multi-perspective transferable feature fusion
Abstract
Zero-shot stance detection involves predicting stances that have not previously been encountered by adapting models to learn transferable features by aligning the source and destination target spaces. The acquisition of transferable target-...
Highlights
- Introduces a fine-grained zero-shot stance model using unsupervised clustering.
- Utilizes contrastive learning to construct multi-perspective meta-features.
- Incorporates attention mechanism for effective feature fusion and transfer.
Zero-shot stance detection via multi-perspective contrastive learning with unlabeled data
Abstract
Stance detection is to distinguish whether the text’s author supports, opposes, or maintains a neutral stance towards a given target. In most real-world scenarios, stance detection needs to work in a zero-shot manner, i.e., predicting ...
Highlights
- We use unlabeled texts of unseen targets in training for zero-shot stance detection.

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Natural Science Foundation of Guangdong Province of China
Shenzhen Basic Research Foundation
National Natural Science Foundation of China
Shenzhen Science and Technology Innovation Program
UK Engineering and Physical Sciences Research Council
Turing AI Fellowship funded by the UK Research and Innovation
Shenzhen Foundational Research Funding

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
1,025
Total Downloads

Downloads (Last 12 months)351
Downloads (Last 6 weeks)34

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dong LSu ZFu XZhang BDai G(2024)Implicit Stance Detection with Hashtag Semantic EnrichmentMathematics10.3390/math1211166312:11(1663)Online publication date: 26-May-2024
https://doi.org/10.3390/math12111663
Ding DDai GPeng CPeng XZhang BHuang H(2024)Distantly Supervised Explainable Stance Detection via Chain-of-Thought SupervisionMathematics10.3390/math1207111912:7(1119)Online publication date: 8-Apr-2024
https://doi.org/10.3390/math12071119
Ding DFu XPeng XFan XHuang HZhang B(2024)Leveraging Chain-of-Thought to Enhance Stance Detection with Prompt-TuningMathematics10.3390/math1204056812:4(568)Online publication date: 13-Feb-2024
https://doi.org/10.3390/math12040568
Ding DChen RJing LZhang BHuang XDong LZhao XSong G(2024)Cross-Target Stance Detection by Exploiting Target Analytical PerspectivesICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10448397(10651-10655)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10448397
Zhao XTian LXie FZhou BWang HWu HGao L(2024)MSFR: Stance Detection Based on Multi-Aspect Semantic Feature Representation via Hierarchical Contrastive LearningICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10446704(11726-11730)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10446704
Xu RJiang KQi LZhao SZheng M(2024)DCPNet: Distribution Calibration Prototypical Network for Few-Shot Image ClassificationIEEE Access10.1109/ACCESS.2024.339813412(67036-67045)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3398134
Zhang HLi YZhu TLi C(2024)Commonsense-based adversarial learning framework for zero-shot stance detectionNeurocomputing10.1016/j.neucom.2023.126943563:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.neucom.2023.126943
Zhao XZou JMiao JTian LGao LZhou BPang S(2024)Zero-shot stance detection based on multi-perspective transferable feature fusionInformation Fusion10.1016/j.inffus.2024.102386108(102386)Online publication date: Aug-2024
https://doi.org/10.1016/j.inffus.2024.102386
Wang CZhang YWang S(2024)A meta-contrastive learning with data augmentation framework for zero-shot stance detectionExpert Systems with Applications10.1016/j.eswa.2024.123956250(123956)Online publication date: Sep-2024
https://doi.org/10.1016/j.eswa.2024.123956
Motyka DPiasecki M(2024)Target-Phrase Zero-Shot Stance Detection: Where Do We Stand?Computational Science – ICCS 202410.1007/978-3-031-63751-3_3(34-49)Online publication date: 27-Jun-2024
https://doi.org/10.1007/978-3-031-63751-3_3
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents