research-article

Crème de la Crème: Lessons from Papers in Security Publications

Authors:

Luca Caviglione,

Steffen WendzelAuthors Info & Claims

ARES '21: Proceedings of the 16th International Conference on Availability, Reliability and Security

Article No.: 92, Pages 1 - 9

https://doi.org/10.1145/3465481.3470027

Published: 17 August 2021 Publication History

Abstract

The number of citations attracted by publications is a key criteria for measuring their success. To avoid discriminating newer research, such a metric is usually measured in average yearly citations. Understanding and characterizing how citations behave have been prime research topics, yet investigations targeting the cybersecurity domain seem to be particularly scarce. In this perspective, the paper aims at filling this gap by analyzing average yearly citations for 6,693 papers published in top-tier conferences and journals in cybersecurity. Results indicate the existence of three clusters, i.e., general security conferences, general security journals, and cryptography-centered publications. The analysis also suggests that the amount of conference-to-conference citations stands out compared to journal-to-journal and conference-to-journal citations. Besides, papers published at top conferences attract more citations although a direct comparison against other venues is not straightforward. To better quantify the impact of works dealing with cybersecurity aspects, the paper introduces two new metrics, namely the number of main words in the title, and the combined number of unique main words in title, abstract and keywords. Collected results show that they can be associated with average yearly citations (together with the number of cited references). Finally, the paper draws some ideas to take advantage from such findings.

References

[1]

Clarivate Analytics. 2021. Title Suppressions. http://help.prod-incites.com/incitesLiveJCR/JCRGroup/titleSuppressions/version/42. (2021). Accessed: 2021-04-11.

[2]

James Baldwin, Omar M. K. Alhawi, Simone Shaughnessy, Alex Akinbi, and Ali Dehghantanha. 2018. Emerging from the Cloud: A Bibliometric Analysis of Cloud Forensics Studies. Springer International Publishing, Cham, 311–331. https://doi.org/10.1007/978-3-319-73951-9_16

[3]

Nuria Bautista-Puig, Carmen Lopez-Illescas, Felix de Moya-Anegon, Vicente Guerrero-Bote, and Henk F Moed. 2020. Do journals flipping to gold open access show an OA citation or publication advantage?Scientometrics 124, 3 (2020), 2551–2575.

[4]

Maria Bras-Amorós, Josep Domingo-Ferrer, and Vicenç Torra. 2011. A bibliometric index based on the collaboration distance between cited and citing authors. Journal of Informetrics 5, 2 (2011), 248–264.

[5]

R.K. Buter and A.F.J. van Raan. 2011. Non-alphanumeric characters in titles of scientific publications: An analysis of their occurrence and correlation with citation impact. Journal of Informetrics 5, 4 (2011), 608–617. https://doi.org/10.1016/j.joi.2011.05.008

[6]

Tanmoy Chakraborty, Vihar Tammana, Niloy Ganguly, and Animesh Mukherjee. 2015. Understanding and modeling diverse scientific careers of researchers. Journal of Informetrics 9, 1 (2015), 69–78.

[7]

Ho Fai Chan, Sohel Md Bodiuzzman, and Benno Torgler. 2020. The power of social cues in the battle for attention: Evidence from an online platform for scholarly commentary. Journal of Informetrics 14, 4 (2020), 101077. https://doi.org/10.1016/j.joi.2020.101077

[8]

Guo Chen and Lu Xiao. 2016. Selecting publication keywords for domain analysis in bibliometrics: A comparison of three methods. Journal of Informetrics 10, 1 (2016), 212–223.

[9]

Bjorn De Sutter and Aäron Van Den Oord. 2012. To be or not to be cited in computer science. Commun. ACM 55, 8 (2012), 69–75.

Digital Library

[10]

Matthew E Falagas, Angeliki Zarkali, Drosos E Karageorgopoulos, Vangelis Bardakas, and Michael N Mavros. 2013. The impact of article length on the number of future citations: a bibliometric analysis of general medicine journals. PLoS One 8, 2 (2013), e49476.

[11]

Damjan Fujs, Anže Mihelič, and Simon L. R. Vrhovec. 2019. The power of interpretation: Qualitative methods in cybersecurity research. In 14th International Conference on Availability, Reliability and Security (ARES 2019). ACM, Canterbury, UK, 92:1–10. https://doi.org/10.1145/3339252.3341479

Digital Library

[12]

Damjan Fujs, Simon Vrhovec, and Damjan Vavpotič. 2020. Bibliometric Mapping of Research on User Training for Secure Use of Information Systems. Journal of Universal Computer Science 26, 7 (2020), 764–782.

[13]

Abby A Goodrum, Katherine W McCain, Steve Lawrence, and C Lee Giles. 2001. Scholarly publishing in the Internet age: a citation analysis of computer science literature. Information Processing & Management 37, 5 (2001), 661–675.

[14]

Robin Haunschild and Werner Marx. 2020. Discovering seminal works with marker papers. Scientometrics 125, 3 (2020), 2955–2969. https://doi.org/10.1007/s11192-020-03358-z

Digital Library

[15]

Jianhua Hou and Da Ma. 2020. How the high-impact papers formed? A study using data from social media and citation. Scientometrics 125, 3 (2020), 2597–2615. https://doi.org/10.1007/s11192-020-03703-2

Digital Library

[16]

Ya-Han Hu, Chun-Tien Tai, Kang Ernest Liu, and Cheng-Fang Cai. 2020. Identification of highly-cited papers using topic-model-based and bibliometric features: the consideration of keyword popularity. Journal of Informetrics 14, 1 (2020), 101004. https://doi.org/10.1016/j.joi.2019.101004

[17]

W. Iqbal, J. Qadir, G. Tyson, 2019. A bibliometric analysis of publications in computer networking research. Scientometrics 119(2019), 1121–1155.

Digital Library

[18]

Sergio Jimenez, Youlin Avila, George Dueñas, and Alexander Gelbukh. 2020. Automatic prediction of citability of scientific articles by stylometry of their titles and abstracts. Scientometrics 125, 3 (2020), 3187–3232. https://doi.org/10.1007/s11192-020-03526-1

Digital Library

[19]

T. Liskiewicz, G. Liskiewicz, and J. Paczesny. 2021. Factors affecting the citations of papers in tribology journals. Scientometrics 126, 4 (2021), 3321–3336. https://doi.org/10.1007/s11192-021-03870-w

Digital Library

[20]

Orestis-Stavros Loizides and Polychronis Koutsakis. 2017. On evaluating the quality of a computer science/computer engineering conference. Journal of Informetrics 11, 2 (2017), 541–552.

[21]

Wei Lu, Zhifeng Liu, Yong Huang, Yi Bu, Xin Li, and Qikai Cheng. 2020. How do authors select keywords? A preliminary study of author keyword selection behavior. Journal of Informetrics 14, 4 (2020), 101066. https://doi.org/10.1016/j.joi.2020.101066

[22]

Alina Lungeanu, Yun Huang, and Noshir S Contractor. 2014. Understanding the assembly of interdisciplinary teams and its impact on performance. Journal of informetrics 8, 1 (2014), 59–70.

[23]

Pooja R Makawana and Rutvij H Jhaveri. 2018. A bibliometric analysis of recent research on machine learning for cyber security. Intelligent communication and computational technologies (2018), 213–226.

[24]

Sharfah Ratibah Tuan Mat, Mohd Faizal Ab Razak, Mohd Nizam Mohmad Kahar, Juliza Mohamad Arif, Salwana Mohamad, and Ahmad Firdaus. 2021. Towards a systematic description of the field using bibliometric analysis: malware evolution. Scientometrics (2021), 1–43.

[25]

Clarivate Web of Science. 2020. Web of Science Journal Citation Reports: Suppression Policy. https://clarivate.com/webofsciencegroup/wp-content/uploads/sites/2/2020/06/jcr-suppression-policy-2020.pdf. (2020). Accessed: 2021-04-11.

[26]

William S. Pearson. 2020. Research article titles in written feedback on English as a second language writing. Scientometrics 123, 2 (2020), 997–1019. https://doi.org/10.1007/s11192-020-03388-7

Digital Library

[27]

William S. Pearson. 2021. Quoted speech in linguistics research article titles: patterns of use and effects on citations. Scientometrics 126, 4 (2021), 3421–3442. https://doi.org/10.1007/s11192-020-03827-5

Digital Library

[28]

Marta Serra-Garcia and Uri Gneezy. 2021. Nonreplicable publications are cited more than replicable ones. Science advances 7, 21 (2021), eabd1705.

[29]

Elizabeth S Vieira and José ANF Gomes. 2010. Citations to scientific articles: Its distribution and dependence on the article features. Journal of Informetrics 4, 1 (2010), 1–13.

[30]

VOSviewer. 2021. VOSviewer – Visualizing scientific landscapes. https://www.vosviewer.com/. (2021). Accessed: 2021-03-28.

[31]

Simon Vrhovec, Damjan Fujs, Luka Jelovčan, and Anže Mihelič. 2020. Evaluating case study and action research reports: Real-world research in cybersecurity. Journal of Universal Computer Science 26, 7 (2020), 827–853.

[32]

Steffen Wendzel. 2018. Get Me Cited, Scotty! Analysis of Citations in Covert Channel/Steganography Research. In Proc. 13th International Conference on Availability, Reliability and Security (ARES’18). ACM, New York, NY, USA, Article 13. https://doi.org/10.1145/3230833.3233265

Digital Library

[33]

Steffen Wendzel, Cédric Lévy-Bencheton, and Luca Caviglione. 2020. Not all areas are equal: Analysis of citations in information security research. Scientometrics 122 (January 2020), 267–286. https://doi.org/10.1007/s11192-019-03279-6

Digital Library

[34]

Zohreh Zahedi and Stefanie Haustein. 2018. On the relationships between bibliographic characteristics of scientific documents and citation and Mendeley readership counts: A large-scale analysis of Web of Science publications. Journal of Informetrics 12, 1 (2018), 191–202.

Cited By

Jedrzejewski FThode LFischbach JGorschek TMendez DLavesson N(2024)Adversarial Machine Learning in industry: A systematic literature reviewComputers & Security10.1016/j.cose.2024.103988(103988)Online publication date: Jul-2024
https://doi.org/10.1016/j.cose.2024.103988
Le Pochat VJoosen W(2023)Analyzing Cyber Security Research Practices through a Meta-Research FrameworkProceedings of the 16th Cyber Security Experimentation and Test Workshop10.1145/3607505.3607523(64-74)Online publication date: 7-Aug-2023
https://dl.acm.org/doi/10.1145/3607505.3607523

Recommendations

Sixty-four years of informetrics research: productivity, impact and collaboration

This paper analyses the information science research field of informetrics to identify publication strategies that have been important for its successful researchers. The study uses a micro-analysis of informetrics researchers from 5,417 informetrics ...
Scientometric analysis of Iraqi-Kurdistan universities' scientific productivity

Purpose - This purpose of this study is to examine research performance of Iraqi-Kurdistan universities, using the number of papers appearing in journals and proceedings, and the number of citations received by those papers as covered by Scopus, 1970-...
A novel method for depicting academic disciplines through Google Scholar Citations: The case of Bibliometrics

This article describes a procedure to generate a snapshot of the structure of a specific scientific community and their outputs based on the information available in Google Scholar Citations (GSC). We call this method multifaceted analysis of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ARES '21: Proceedings of the 16th International Conference on Availability, Reliability and Security

August 2021

1447 pages

ISBN:9781450390514

DOI:10.1145/3465481

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ARES 2021

ARES 2021: The 16th International Conference on Availability, Reliability and Security

August 17 - 20, 2021

Vienna, Austria

Acceptance Rates

Overall Acceptance Rate 228 of 451 submissions, 51%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
85
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jedrzejewski FThode LFischbach JGorschek TMendez DLavesson N(2024)Adversarial Machine Learning in industry: A systematic literature reviewComputers & Security10.1016/j.cose.2024.103988(103988)Online publication date: Jul-2024
https://doi.org/10.1016/j.cose.2024.103988
Le Pochat VJoosen W(2023)Analyzing Cyber Security Research Practices through a Meta-Research FrameworkProceedings of the 16th Cyber Security Experimentation and Test Workshop10.1145/3607505.3607523(64-74)Online publication date: 7-Aug-2023
https://dl.acm.org/doi/10.1145/3607505.3607523

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents