Abstract
This paper is a condensed report on the third year of the Touché lab on argument retrieval held at CLEF 2022. With the goal to foster and support the development of technologies for argument mining and argument analysis, we organized three shared tasks in the third edition of Touché: (a) argument retrieval for controversial topics, where participants retrieve a gist of arguments from a collection of online debates, (b) argument retrieval for comparative questions, where participants retrieve argumentative passages from a generic web crawl, and (c) image retrieval for arguments, where participants retrieve images from a focused web crawl that show support or opposition to some stance.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
‘touché’ is commonly “used to acknowledge a hit in fencing or the success or appropriateness of an argument” [https://merriam-webster.com/dictionary/touche]
- 2.
- 3.
- 4.
Three teams declined to proceed in the task after submitting the results
- 5.
The expected format of submissions was also described at https://touche.webis.de
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
Also available as a Python library: https://pypi.org/project/targer-api/
- 14.
- 15.
- 16.
- 17.
- 18.
Archived using https://github.com/webis-de/scriptor
- 19.
- 20.
- 21.
References
Aigrain, P., Zhang, H., Petkovic, D.: Content-based representation and retrieval of visual media: a state-of-the-art review. Multimed. Tools Appl. 3(3), 179–202 (1996). https://doi.org/10.1007/BF00393937
Ajjour, Y., Braslavski, P., Bondarenko, A., Stein, B.: Identifying argumentative questions in web search logs. In: 45th International ACM Conference on Research and Development in Information Retrieval (SIGIR 2022). ACM, July 2022. https://doi.org/10.1145/3477495.3531864
Ajjour, Y., Wachsmuth, H., Kiesel, J., Potthast, M., Hagen, M., Stein, B.: Data acquisition for argument search: the args.me corpus. In: Benzmüller, C., Stuckenschmidt, H. (eds.) KI 2019. LNCS (LNAI), vol. 11793, pp. 48–59. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30179-8_4
Alhamzeh, A., Bouhaouel, M., Egyed-Zsigmond, E., Mitrovic, J.: DistilBERT-based argumentation retrieval for answering comparative questions. In: Proceedings of the Working Notes of CLEF 2021 - Conference and Labs of the Evaluation Forum, CEUR Workshop Proceedings, vol. 2936, pp. 2319–2330, CEUR-WS.org (2021). http://ceur-ws.org/Vol-2936/paper-209.pdf
Alshomary, M., Düsterhus, N., Wachsmuth, H.: Extractive snippet generation for arguments. In: Proceedings of the 43nd International ACM Conference on Research and Development in Information Retrieval, SIGIR 2020, pp. 1969–1972. ACM (2020). https://doi.org/10.1145/3397271.3401186
Aristotle, Kennedy, G.A.: On Rhetoric: A Theory of Civic Discourse. Oxford University Press, Oxford (2006)
Bar-Haim, R., Kantor, Y., Venezian, E., Katz, Y., Slonim, N.: Project debater APIs: decomposing the AI grand challenge. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2021, Online and Punta Cana, Dominican Republic, 7–11 November 2021, pp. 267–274. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.emnlp-demo.31
Bevendorff, J., et al.: Webis at TREC 2020: Health Misinformation track. In: Voorhees, E., Ellis, A. (eds.) Proceedings of the 29th International Text Retrieval Conference, TREC 2020, NIST, November 2020
Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python. O’Reilly (2009). ISBN 978-0-596-51649-9. http://www.oreilly.de/catalog/9780596516499/index.html
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003). http://jmlr.org/papers/v3/blei03a.html
Bondarenko, A., Ajjour, Y., Dittmar, V., Homann, N., Braslavski, P., Hagen, M.: Towards understanding and answering comparative questions. In: Proceedings of the 15th ACM International Conference on Web Search and Data Mining, WSDM 2022. ACM (2022). https://doi.org/10.1145/3488560.3498534
Bondarenko, A., et al.: Comparative web search questions. In: Proceedings of the 13th ACM International Conference on Web Search and Data Mining, WSDM 2020, pp. 52–60. ACM (2020). https://dl.acm.org/doi/abs/10.1145/3336191.3371848
Bondarenko, A., et al.: Overview of Touché 2020: argument retrieval. In: Working Notes Papers of the CLEF 2020 Evaluation Labs. CEUR Workshop Proceedings, vol. 2696 (2020). http://ceur-ws.org/Vol-2696/
Bondarenko, A., Fröbe, M., Kasturia, V., Völske, M., Stein, B., Hagen, M.: Webis at TREC 2019: decision track. In: Voorhees, E., Ellis, A. (eds.) Proceedings of the 28th International Text Retrieval Conference, TREC 2019, NIST, November 2019
Bondarenko, A., et al.: Overview of touché 2022: argument retrieval. In: Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum. CEUR Workshop Proceedings. CEUR-WS.org, Berlin, Heidelberg, New York (2022, to appear)
Bondarenko, A., et al.: Overview of touché 2021: argument retrieval. In: Working Notes Papers of the CLEF 2021 Evaluation Labs. CEUR Workshop Proceedings, vol. 2936 (2021). http://ceur-ws.org/Vol-2936/
Chang, N., Fu, K.: Query-by-pictorial-example. IEEE Trans. Softw. Eng. 6(6), 519–524 (1980). https://doi.org/10.1109/TSE.1980.230801
Chekalina, V., Bondarenko, A., Biemann, C., Beloucif, M., Logacheva, V., Panchenko, A.: Which is better for deep learning: Python or MATLAB? Answering comparative questions in natural language. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, EACL 2021, pp. 302–311. Association for Computational Linguistics (2021). https://www.aclweb.org/anthology/2021.eacl-demos.36/
Chernodub, A., et al.: TARGER: neural argument mining at your fingertips. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, ACL 2019, pp. 195–200. ACL (2019). https://doi.org/10.18653/v1/p19-3031
Cormack, G.V., Clarke, C.L.A., Büttcher, S.: Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In: Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2009, pp. 758–759. ACM (2009). https://doi.org/10.1145/1571941.1572114
Cormack, G.V., Smucker, M.D., Clarke, C.L.A.: Efficient and effective spam filtering and re-ranking for large web datasets. Inf. Retr. 14(5), 441–465 (2011). https://doi.org/10.1007/s10791-011-9162-z
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, pp. 4171–4186. ACL (2019). https://doi.org/10.18653/v1/n19-1423
Dimitrov, D., et al.: SemEval-2021 task 6: detection of persuasion techniques in texts and images. In: 15th International Workshop on Semantic Evaluation (SemEval 2021), pp. 70–98. Association for Computational Linguistics, August 2021. https://doi.org/10.18653/v1/2021.semeval-1.7, https://aclanthology.org/2021.semeval-1.7
Dove, I.J.: On images as evidence and arguments. In: van Eemeren, F.H., Garssen, B. (eds.) Topical Themes in Argumentation Theory: Twenty Exploratory Studies. Argumentation Library, pp. 223–238. Springer, Dordrecht (2012). https://doi.org/10.1007/978-94-007-4041-9_15
Dumani, L., Neumann, P.J., Schenkel, R.: A framework for argument retrieval - ranking argument clusters by frequency and specificity. In: Jose, J.M., et al. (eds.) ECIR 2020. LNCS, vol. 12035, pp. 431–445. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45439-5_29
Dumani, L., Schenkel, R.: Quality aware ranking of arguments. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM 2020, pp. 335–344. Association for Computing Machinery (2020). https://doi.org/10.1007/978-3-030-45439-5_29
Dunaway, F.: Images, emotions, politics. Mod. Am. Hist. 1(3), 369–376 (2018). https://doi.org/10.1017/mah.2018.17. ISSN 2515-0456, 2397-1851
Erkan, G., Radev, D.R.: LexRank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. 22, 457–479 (2004). https://doi.org/10.1613/jair.1523
Fröbe, M., et al.: CopyCat: near-duplicates within and between the ClueWeb and the common crawl. In: Proceedings of the 44th International ACM Conference on Research and Development in Information Retrieval, SIGIR 2021, pp. 2398–2404. ACM (2021). https://dl.acm.org/doi/10.1145/3404835.3463246
Fröbe, M., Bevendorff, J., Reimer, J., Potthast, M., Hagen, M.: Sampling bias due to near-duplicates in learning to rank. In: Proceedings of the 43rd International ACM Conference on Research and Development in Information Retrieval, SIGIR 2020. ACM (2020). https://dl.acm.org/doi/10.1145/3397271.3401212
Fröbe, M., Bittner, J.P., Potthast, M., Hagen, M.: The effect of content-equivalent near-duplicates on the evaluation of search engines. In: Jose, J.M., et al. (eds.) ECIR 2020. LNCS, vol. 12036, pp. 12–19. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45442-5_2
Gienapp, L., Stein, B., Hagen, M., Potthast, M.: Efficient pairwise annotation of argument quality. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5772–5781. Association for Computational Linguistics, July 2020. https://doi.org/10.18653/v1/2020.acl-main.511, https://aclanthology.org/2020.acl-main.511
Gienapp, L., Stein, B., Hagen, M., Potthast, M.: Efficient pairwise annotation of argument quality. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, pp. 5772–5781. Association for Computational Linguistics (2020). https://www.aclweb.org/anthology/2020.acl-main.511/
Google: Google images best practices. Google Developers (2021). https://support.google.com/webmasters/answer/114016
Grancea, I.: Types of visual arguments. Argumentum. J. Seminar Discursive Log. Argument. Theory Rhetoric 15(2), 16–34 (2017)
Gretz, S., et al.: A large-scale dataset for argument quality ranking: construction and analysis. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, pp. 7805–7813. AAAI Press (2020). https://ojs.aaai.org/index.php/AAAI/article/view/6285
Ho, T.K.: Random decision forests. In: Third International Conference on Document Analysis and Recognition, ICDAR 1995, 14–15 August 1995, Montreal, Canada, vol. I, pp. 278–282. IEEE Computer Society (1995). https://doi.org/10.1109/ICDAR.1995.598994
Hovy, D., Berg-Kirkpatrick, T., Vaswani, A., Hovy, E.: Learning whom to trust with MACE. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HTL 2013), pp. 1120–1130. Association for Computational Linguistics, Atlanta, June 2013. https://aclanthology.org/N13-1132
Jindal, N., Liu, B.: Identifying comparative sentences in text documents. In: Proceedings of the 29th Annual International Conference on Research and Development in Information Retrieval, SIGIR 2006, pp. 244–251. ACM (2006). https://doi.org/10.1145/1148170.1148215
Jindal, N., Liu, B.: Mining comparative sentences and relations. In: Proceedings of the 21st National Conference on Artificial Intelligence and the 18th Innovative Applications of Artificial Intelligence Conference, AAAI 2006, pp. 1331–1336. AAAI Press (2006). http://www.aaai.org/Library/AAAI/2006/aaai06-209.php
Kaszkiel, M., Zobel, J.: Passage retrieval revisited. In: Belkin, N.J., Narasimhalu, A.D., Willett, P., Hersh, W.R., Can, F., Voorhees, E.M. (eds.) Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 1997, Philadelphia, PA, USA, 27–31 July 1997, pp. 178–185. ACM (1997). https://doi.org/10.1145/258525.258561
Kessler, W., Kuhn, J.: A corpus of comparisons in product reviews. In: Proceedings of the 9th International Conference on Language Resources and Evaluation, LREC 2014, pp. 2242–2248. European Language Resources Association (ELRA) (2014). http://www.lrec-conf.org/proceedings/lrec2014/summaries/1001.html
Khattab, O., Zaharia, M.: ColBERT: efficient and effective passage search via contextualized late interaction over BERT. In: Huang, J., et al. (eds.) Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, pp. 39–48. ACM (2020). https://doi.org/10.1145/3397271.3401075
Kiesel, J., Reichenbach, N., Stein, B., Potthast, M.: Image retrieval for arguments using stance-aware query expansion. In: Proceedings of the 8th Workshop on Argument Mining, ArgMining 2021 at EMNLP, pp. 36–45. ACL (2021)
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: ALBERT: a lite BERT for self-supervised learning of language representations. In: Proceedings of the 8th International Conference on Learning Representations, ICLR 2020. OpenReview.net (2020). https://openreview.net/forum?id=H1eA7AEtvS
Latif, A., et al.: Content-based image retrieval and feature extraction: A comprehensive review. Math. Probl. Eng. 2019, 21 (2019). https://doi.org/10.1155/2019/9658350
Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: International Conference on Machine Learning, pp. 1188–1196. PMLR (2014)
Levy, R., Bogin, B., Gretz, S., Aharonov, R., Slonim, N.: Towards an argumentative content search engine using weak supervision. In: Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, pp. 2066–2081. Association for Computational Linguistics (2018). https://www.aclweb.org/anthology/C18-1176/
Lin, J., Ma, X., Lin, S., Yang, J., Pradeep, R., Nogueira, R.: Pyserini: A python toolkit for reproducible information retrieval research with sparse and dense representations. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021, pp. 2356–2362. ACM (2021). https://doi.org/10.1145/3404835.3463238
Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019). http://arxiv.org/abs/1907.11692
Lovins, J.B.: Development of a stemming algorithm. Mech. Transl. Comput. Linguist. 11(1–2), 22–31 (1968). http://www.mt-archive.info/MT-1968-Lovins.pdf
Ma, N., Mazumder, S., Wang, H., Liu, B.: Entity-aware dependency-based deep graph attention network for comparative preference classification. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, pp. 5782–5788. Association for Computational Linguistics (2020). https://www.aclweb.org/anthology/2020.acl-main.512/
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of the 1st International Conference on Learning Representations, ICLR 2013 (2013). http://arxiv.org/abs/1301.3781
Miller, G.A.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)
Nadamoto, A., Tanaka, K.: A comparative web browser (CWB) for browsing and comparing web pages. In: Proceedings of the 12th International World Wide Web Conference, WWW 2003, pp. 727–735. ACM (2003). https://doi.org/10.1145/775152.775254
Nguyen, T., et al.: MS MARCO: a human generated MAchine reading COmprehension dataset. In: Proceedings of the Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches 2016 at NIPS, CEUR Workshop Proceedings, vol. 1773. CEUR-WS.org (2016). http://ceur-ws.org/Vol-1773/CoCoNIPS_2016_paper9.pdf
Nogueira, R., Lin, J., Epistemic, A.: From doc2query to docTTTTTquery. Online preprint (2019). https://cs.uwaterloo.ca/~jimmylin/publications/Nogueira_Lin_2019_docTTTTTquery-v2.pdf
Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank citation ranking: bringing order to the web. Technical report 1999–66, Stanford InfoLab (1999). http://ilpubs.stanford.edu:8090/422/
Palotti, J.R.M., Scells, H., Zuccon, G.: TrecTools: an open-source Python library for information retrieval practitioners involved in TREC-like campaigns. In: Proceedings of the 42nd International Conference on Research and Development in Information Retrieval, SIGIR 2019, pp. 1325–1328. ACM (2019). https://doi.org/10.1145/3331184.3331399
Panchenko, A., Bondarenko, A., Franzek, M., Hagen, M., Biemann, C.: Categorizing comparative sentences. In: Proceedings of the 6th Workshop on Argument Mining, ArgMining@ACL 2019, pp. 136–145. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/w19-4516
Porter, M.F.: An algorithm for suffix stripping. Program 14(3), 130–137 (1980). https://doi.org/10.1108/eb046814
Porter, M.F.: Snowball: a language for stemming algorithms (2001). http://snowball.tartarus.org/texts/introduction.html
Potthast, M., et al.: Argument search: assessing argument relevance. In: Proceedings of the 42nd International Conference on Research and Development in Information Retrieval, SIGIR 2019, pp. 1117–1120. ACM (2019). https://doi.org/10.1145/3331184.3331327
Potthast, M., Gollub, T., Wiegmann, M., Stein, B.: TIRA integrated research architecture. In: Information Retrieval Evaluation in a Changing World. TIRS, vol. 41, pp. 123–160. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22948-1_5
Pradeep, R., Nogueira, R., Lin, J.: The expando-mono-duo design pattern for text ranking with pretrained sequence-to-sequence models. CoRR abs/2101.05667 (2021). https://arxiv.org/abs/2101.05667
Radford, A., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21, 140:1–140:67 (2020). http://jmlr.org/papers/v21/20-074.html
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, pp. 3980–3990. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/D19-1410
Reimers, N., Schiller, B., Beck, T., Daxenberger, J., Stab, C., Gurevych, I.: Classification and clustering of arguments with contextualized word embeddings. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 567–578. Association for Computational Linguistics, Florence, July 2019. https://doi.org/10.18653/v1/P19-1054, https://aclanthology.org/P19-1054
Robertson, S.E., Walker, S., Jones, S., Hancock-Beaulieu, M., Gatford, M.: Okapi at TREC-3. In: Proceedings of The Third Text REtrieval Conference, TREC 1994, NIST Special Publication, vol. 500-225, pp. 109–126. NIST (1994). https://trec.nist.gov/pubs/trec3/papers/city.ps.gz
Robertson, S.E., Zaragoza, H., Taylor, M.J.: Simple BM25 extension to multiple weighted fields. In: Proceedings of the 13th International Conference on Information and Knowledge Management, CIKM 2004, pp. 42–49. ACM (2004). https://doi.org/10.1145/1031171.1031181
Rocchio, J.: Relevance feedback in information retrieval. In: The Smart Retrieval System-Experiments in Automatic Document Processing, pp. 313–323 (1971)
Roque, G.: Visual argumentation: a further reappraisal. In: van Eemeren, F.H., Garssen, B. (eds.) Topical Themes in Argumentation Theory. Argumentation Library, vol. 22, pp. 273–288. Springer, Dordrecht (2012). https://doi.org/10.1007/978-94-007-4041-9_18. ISBN 978-94-007-4040-2
Rose, S., Engel, D., Cramer, N., Cowley, W.: Automatic keyword extraction from individual documents. Text Min.: Appl. Theory 1(1–20), 10–1002 (2010)
Sanh, V., Debut, L., Chaumond, J., Wolf, T.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR abs/1910.01108 (2019). http://arxiv.org/abs/1910.01108
Sanh, V., et al.: Multitask prompted training enables zero-shot task generalization. CoRR abs/2110.08207 (2021). https://arxiv.org/abs/2110.08207
Schildwächter, M., Bondarenko, A., Zenker, J., Hagen, M., Biemann, C., Panchenko, A.: Answering comparative questions: better than ten-blue-links? In: Proceedings of the 2019 Conference on Human Information Interaction and Retrieval, CHIIR 2019, pp. 361–365. ACM (2019). https://doi.org/10.1145/3295750.3298916
Solli, M., Lenz, R.: Color emotions for multi-colored images. Color Res. Appl. 36(3), 210–221 (2011). https://doi.org/10.1002/col.20604
Stab, C., et al.: ArgumenText: searching for arguments in heterogeneous sources. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, NAACL 2018, pp. 21–25. Association for Computational Linguistics (2018). https://www.aclweb.org/anthology/N18-5005
Sun, J., Wang, X., Shen, D., Zeng, H., Chen, Z.: CWS: a comparative web search system. In: Proceedings of the 15th International Conference on World Wide Web, WWW 2006, pp. 467–476. ACM (2006). https://doi.org/10.1145/1135777.1135846
Wachsmuth, H., et al.: Argumentation quality assessment: theory vs. practice. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, pp. 250–255. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/P17-2039
Wachsmuth, H., et al.: Computational argumentation quality assessment in natural language. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, pp. 176–187 (2017). http://aclweb.org/anthology/E17-1017
Wachsmuth, H., et al.: Building an argument search engine for the web. In: Proceedings of the Fourth Workshop on Argument Mining (ArgMining), pp. 49–59. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/w17-5106
Wachsmuth, H., Stein, B., Ajjour, Y.: “PageRank” for argument relevance. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, pp. 1117–1127. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/e17-1105
Wachsmuth, H., Syed, S., Stein, B.: Retrieval of the best counterargument without prior topic knowledge. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, pp. 241–251. Association for Computational Linguistics (2018). https://www.aclweb.org/anthology/P18-1023/
Wang, W., He, Q.: A survey on emotional semantic image retrieval. In: International Conference on Image Processing (ICIP 2008), pp. 117–120. IEEE (2008). https://doi.org/10.1109/ICIP.2008.4711705
Wu, A.: Learn more about what you see on google images. Google Blog (2020). https://support.google.com/webmasters/answer/114016
Yanai, K.: Image collector: an image-gathering system from the world-wide web employing keyword-based search engines. In: International Conference on Multimedia and Expo (ICME 2001). IEEE (2001). https://doi.org/10.1109/ICME.2001.1237772
Acknowledgments
We are very grateful to the CLEF 2022 organizers and the Touché participants, who allowed this lab to happen. We also want to thank our volunteer annotators who helped to create the relevance and argument quality assessments and our reviewers for their valuable feedback on the participants’ notebooks.
This work was partially supported by the Deutsche Forschungsgemeinschaft (DFG) through the projects “ACQuA 2.0” (Answering Comparative Questions with Arguments; project number 376430233) and “OASiS: Objective Argument Summarization in Search” (grant WA 4591/3-1), all part of the priority program “RATIO: Robust Argumentation Machines” (SPP 1999), and the German Ministry for Science and Education (BMBF) through the project “Shared Tasks as an Innovative Approach to Implement AI and Big Data-based Applications within Universities (SharKI)” (grant FKZ 16DHB4021). We are also grateful to Jan Heinrich Reimer for developing the TARGER Python library and Erik Reuter for expanding a document collection for Task 2 with docT5query.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bondarenko, A. et al. (2022). Overview of Touché 2022: Argument Retrieval. In: Barrón-Cedeño, A., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2022. Lecture Notes in Computer Science, vol 13390. Springer, Cham. https://doi.org/10.1007/978-3-031-13643-6_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-13643-6_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13642-9
Online ISBN: 978-3-031-13643-6
eBook Packages: Computer ScienceComputer Science (R0)