The Impact of CHIIR Publications: A Study of Eight Years of CHIIR
Authors: 
Maria Gaede, Toine Bogers, Mark Michael Hall, Marijn Koolen, Vivien Petras, 
Birger LarsenAuthors Info & Claims
Maria Gaede, Toine Bogers, Mark Michael Hall, Marijn Koolen, Vivien Petras, Birger LarsenAuthors Info & ClaimsPages 34 - 44
Abstract
Across all scientific fields, there is an increased focus on the impact of scientific research: what academic and societal benefits does it provide? This question has spurred the development of a variety of different approaches to impact assessment, each appropriate in different circumstances. In this paper, we study the academic impact of the CHIIR community through a comprehensive analysis of the work published in the 2016-2023 CHIIR conference series. We collect citation counts, citing documents, and altmetrics scores for all CHIIR publications to determine their academic impact across a variety of different attributes of the CHIIR publications. In addition, we analyze a subset of citation contexts in the papers that have cited CHIIR publications to analyze how they are being used and what that means for their potential impact. Finally, we attempt to predict which properties of CHIIR publications are most predictive of future impact.
References
[1]
2008–2023. GROBID. https://github.com/kermitt2/grobid. swh:1:dir:dab86b296e3c3216e2241968f0d63b68e8209d3c
[2]
Amjad Abu-Jbara, Jefferson Ezra, and Dragomir Radev. 2013. Purpose and polarity of citation: Towards nlp-based bibliometrics. In Proceedings of the 2013 conference of the North American chapter of the association for computational linguistics: Human language technologies. 596–606.
[3]
Awais Athar and Simone Teufel. 2012. Detection of implicit citations for sentiment detection. In Proceedings of the workshop on detecting structure in scholarly discourse. 18–26.
[4]
Sumit Kumar Banshal, Vivek Kumar Singh, Pranab K Muhuri, and Philipp Mayr. 2019. Disciplinary variations in altmetric coverage of scholarly articles. arXiv preprint arXiv:1910.04205 (2019).
[5]
Toine Bogers, Maria Gäde, Mark Hall, Luanne Freund, Marijn Koolen, Vivien Petras, and Mette Skov. 2018. Report on the Workshop on Barriers to Interactive IR Resources Re-use (BIIRRR 2018). SIGIR Forum 52, 1 (Aug. 2018), 119–128.
[6]
Toine Bogers, Maria Gäde, Mark Michael Hall, Marijn Koolen, Vivien Petras, and Birger Larsen. 2023. How We Work, Share, and Re-Use at CHIIR. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval (Austin, TX, USA) (CHIIR ’23). Association for Computing Machinery, New York, NY, USA, 351–356. https://doi.org/10.1145/3576840.3578305
[7]
Toine Bogers, Birger Larsen, Marijn Koolen, Maria Gäde, Mark M. Hall, and Vivien Petras. 2023. Collaboration Patterns and Impact of Sharing at CHIIR. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval (Austin, TX, USA) (CHIIR ’23). Association for Computing Machinery, New York, NY, USA, 313–320. https://doi.org/10.1145/3576840.3578276
[8]
Arnab Chatterjee, Asim Ghosh, and Bikas K Chakrabarti. 2016. Universality of citation distributions for academic institutions and journals. PloS one 11, 1 (2016), e0146762.
[9]
Arman Cohan, Waleed Ammar, Madeleine van Zuylen, and Field Cady. 2019. Structural Scaffolds for Citation Intent Classification in Scientific Publications. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, 3586–3596. https://doi.org/10.18653/v1/N19-1361
[10]
Rodrigo Costas, Zohreh Zahedi, and Paul Wouters. 2015. Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology 66, 10 (2015), 2003–2019. https://doi.org/10.1002/asi.23309 arXiv:https://asistdl.onlinelibrary.wiley.com/doi/pdf/10.1002/asi.23309
[11]
Martin G Erikson and Peter Erlandson. 2014. A taxonomy of motives to cite. Social Studies of Science 44, 4 (2014), 625–637. https://doi.org/10.1177/0306312714522871 arXiv:https://doi.org/10.1177/0306312714522871PMID: 25272615.
[12]
Mark Garzone and Robert E Mercer. 2000. Towards an automated citation classifier. In Advances in Artificial Intelligence: 13th Biennial Conference of the Canadian Society for Computational Studies of Intelligence, AI 2000 Montéal, Quebec, Canada, May 14–17, 2000 Proceedings 13. Springer, 337–346.
[13]
Trisha Greenhalgh, James Raftery, Steve Hanney, and Matthew Glover. 2016. Research Impact: A Narrative Review. BMC Medicine 14, 1 (2016), 1–16.
[14]
Maria Gäde, Marijn Koolen, Mark Hall, Toine Bogers, and Vivien Petras. 2021. A Manifesto on Resource Re-Use in Interactive Information Retrieval. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval(CHIIR ’21). Association for Computing Machinery, New York, NY, USA, 141–149. https://doi.org/10.1145/3406522.3446056
[15]
Mark M. Hall. 2019. To Re-use is to Re-write: Experiences with Re-using IIR Experiment Software. In Proceedings of the CHIIR 2019 Workshop on Barriers to Interactive IR Resources Re-use co-located with the ACM SIGIR Conference on Human Information Interaction and Retrieval, BIIRRR@CHIIR 2019, Glasgow, UK, March 14, 2019(CEUR Workshop Proceedings, Vol. 2337), Toine Bogers, Samuel Dodson, Maria Gäde, Luanne Freund, Mark M. Hall, Marijn Koolen, Vivien Petras, Nils Pharo, and Mette Skov (Eds.). CEUR-WS.org, 19–23. http://ceur-ws.org/Vol-2337/paper3.pdf
[16]
Stefanie Haustein, Rodrigo Costas, and Vincent Larivière. 2015. Characterizing social media metrics of scholarly papers: the effect of document properties and collaboration patterns.PLoS ONE 10 (01 2015). https://doi.org/10.1371/journal.pone.0127830
[17]
Stefanie Haustein, Vincent Larivière, Mike Thelwall, Didier Amyot, and Isabella Peters. 2014. Tweets vs. Mendeley readers: How do these two social media metrics differ?it - Information Technology 56, 5 (2014), 207–215. https://doi.org/
[18]
Myriam Hernández-Alvarez and José M Gomez. 2016. Survey about citation context analysis: Tasks, techniques, and resources. Natural Language Engineering 22, 3 (2016), 327–349.
[19]
D. Hopf, S. Dellmann, C. Hauschke, and M. Tullney. 2022. Wirkungen von Open Access. Literaturstudie über empirische Arbeiten 2010-2021. Hannover : Technische Informationsbibliothek (TIB).
[20]
David Jurgens, Srijan Kumar, Raine Hoover, Dan McFarland, and Dan Jurafsky. 2018. Measuring the evolution of a scientific field through citation frames. Transactions of the Association for Computational Linguistics 6 (2018), 391–406.
[21]
Diane Kelly 2009. Methods for evaluating interactive information retrieval systems with users. Foundations and Trends® in Information Retrieval 3, 1–2 (2009), 1–224. https://doi.org/10.1561/1500000012
[22]
Suchetha N Kunnath, Drahomira Herrmannova, David Pride, and Petr Knoth. 2021. A meta-analysis of semantic classification of citations. Quantitative science studies 2, 4 (2021), 1170–1215.
[23]
Allison Langham-Putrow, Caitlin Bakker, and Amy Riegelman. 2021. Is the open access citation advantage real? A systematic review of the citation of open access and subscription-based articles. PloS one 16, 6 (2021), e0253129.
[24]
R. Merton. 1957. Priorities in scientific discovery: A chapter in the sociology of science.American Sociological Review 22 (1957), 635. https://api.semanticscholar.org/CorpusID:147253305
[25]
Heather Piwowar, Jason Priem, Vincent Larivière, Juan Pablo Alperin, Lisa Matthias, Bree Norlander, Ashley Farley, Jevin West, and Stefanie Haustein. 2018. The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles. PeerJ 6 (2018), e4375.
[26]
Henning Pohl and Aske Mottelson. 2019. How we Guide, Write, and Cite at CHI. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems(CHI EA ’19). ACM, New York, NY, USA, 1–11. https://doi.org/10.1145/3290607.3310429
[27]
David Pride and Petr Knoth. 2020. An Authoritative Approach to Citation Classification. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020 (Virtual Event, China) (JCDL ’20). Association for Computing Machinery, New York, NY, USA, 337–340. https://doi.org/10.1145/3383583.3398617
[28]
Cara Seitz, Marion Schmidt, Nathalie Schwichtenberg, and Theresa Velden. 2021. A Case Study of the Epistemic Function of Citations - Implications for Citation-based Science Mapping.
[29]
John A. Stewart. 1994. The Poisson-lognormal model for bibliometric/scientometric distributions. Information Processing & Management 30, 2 (1994), 239–251.
[30]
Pardeep Sud and Mike Thelwall. 2014. Evaluating altmetrics. scientometrics 98 (2014), 1131–1143.
[31]
Leho Tedersoo, Rainer Küngas, Ester Oras, Kajar Köster, Helen Eenmaa, Äli Leijen, Margus Pedaste, Marju Raju, Anastasiya Astapova, Heli Lukner, Karin Kogermann, and Tuul Sepp. 2021. Data sharing practices and data availability upon request differ across scientific disciplines. Scientific Data 8 (2021). https://api.semanticscholar.org/CorpusID:236461324
[32]
Simone Teufel, Advaith Siddharthan, and Dan Tidhar. 2006. An annotation scheme for citation function. In Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue. 80–87. https://aclanthology.org/W06-1312.pdf
[33]
Simone Teufel, Advaith Siddharthan, and Dan Tidhar. 2006. Automatic classification of citation function. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. 103–110. https://aclanthology.org/W06-1613.pdf
[34]
Mike Thelwall, Stefanie Haustein, Vincent Larivière, and Cassidy R Sugimoto. 2013. Do altmetrics work? Twitter and ten other social web services. PloS one 8, 5 (2013), e64841.
[35]
Marco Valenzuela, Vu Ha, and Oren Etzioni. 2015. Identifying Meaningful Citations. In AAAI workshop: Scholarly big data, Vol. 15. 13.
[36]
Ludo Waltman. 2016. A Review of the Literature on Citation Impact Indicators. Journal of Informetrics 10, 2 (2016), 365–391.
[37]
Howard D White. 2004. Citation analysis and discourse analysis revisited. Applied linguistics 25, 1 (2004), 89–116.
[38]
Lorna Wildgaard, Jesper W Schneider, and Birger Larsen. 2014. A Review of the Characteristics of 108 Author-level Bibliometric Indicators. Scientometrics 101 (2014), 125–158.
[39]
Peter Willett. 2013. Readers’ perceptions of authors’ citation behaviour. Journal of Documentation 69, 1 (2013), 145–156.
[40]
Han Xu, Eric Martin, and Ashesh Mahidadia. 2013. Using heterogeneous features for scientific citation classification. In Proceedings of the 13th conference of the Pacific Association for Computational Linguistics.
Index Terms
- The Impact of CHIIR Publications: A Study of Eight Years of CHIIR
Recommendations
Deconstructing the collaborative impact: article and author characteristics that influence citation count
ASIST '13: Proceedings of the 76th ASIS&T Annual Meeting: Beyond the Cloud: Rethinking Information BoundariesIt is well known that collaborative papers tend to receive more citations than solo-authored papers. Here we try to identify the subtle factors of this collaborative effect by analyzing metadata and citation counts for co-authored papers in the ...
Journal self-citation study for semiconductor literature: synchronous and diachronous approach
Special issue: InformetricsThe present study investigates the self-citations of the most productive semiconductor journals by synchronous (self-citing rate) and diachronous (self-cited rate) approaches. Journal's productivity of 100 most productive semiconductor journals was ...
Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators
AbstractThe h-index, introduced by Hirsch in 2005, was used by Schubert in 2009 to assess single publications. In 2011, Bornmann, Schier, Marx, and Daniel confirmed that the h-index is effective when assessing papers in chemistry. Quite a few Hirsch-type ...
Comments
Information & Contributors
Information
Published In
March 2024
481 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 10 March 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CHIIR '24
CHIIR '24: 2024 ACM SIGIR Conference on Human Information Interaction and Retrieval
March 10 - 14, 2024
Sheffield, United Kingdom
Acceptance Rates
Overall Acceptance Rate 55 of 163 submissions, 34%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 192Total Downloads
- Downloads (Last 12 months)192
- Downloads (Last 6 weeks)27
Reflects downloads up to 13 Jan 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in