Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Scholarly Quality Measurements: A Systematic Literature Review

  • Conference paper
  • First Online:
Linking Theory and Practice of Digital Libraries (TPDL 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15177))

Included in the following conference series:

Abstract

The academic publishing landscape is rapidly evolving, making quality assessments and impact evaluations of scientific papers increasingly challenging. Understanding the respective methods is crucial for maintaining the integrity, quality, and relevance of academic publishing in such a changing environment. In this paper, we investigate existing quality-assessment methods for scientific papers, as well as their advantages and disadvantages. For this purpose, we conducted a systematic literature review to capture a comprehensive overview of existing methods, which led to 43 papers and 14 methods. Specifically, we analyze their usage, strengths, and weaknesses, in addition to potential avenues for enhancements. The results can support researchers by providing the knowledge to navigate through quality-assessment methods to make evaluations concerning the reliability and suitability of diverse methods within a specific scientific context.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.scopus.com/.

  2. 2.

    https://www.altmetric.com/.

  3. 3.

    https://plumanalytics.com/.

References

  1. Abdi, H., Williams, L.J.: Principal component analysis. Comput. Stat. 2(4), 433–459 (2010)

    Google Scholar 

  2. Abramo, G., Cicero, T., D’Angelo, C.A.: Individual research performance: a proposal for comparing apples to oranges. J. Informet. 7(2), 528–539 (2013)

    Google Scholar 

  3. Abramo, G., D’Angelo, C.A.: How do you define and measure research productivity? Scientometrics 101(2), 1129–1144 (2014)

    Google Scholar 

  4. Adie, E.: Gaming altmetrics. Altmetric blog (2013)

    Google Scholar 

  5. Agarwal, S., Agrawal, A.: Assessment of the impact of publication year and keywords in research manuscript quality using fuzzy inference system. In: Mahapatra, R.P., Panigrahi, B.K., Kaushik, B.K., Roy, S. (eds.) Proceedings of 6th International Conference on Recent Trends in Computing. LNNS, vol. 177, pp. 75–82. Springer, Singapore (2021). https://doi.org/10.1007/978-981-33-4501-0

  6. Ahlgren, P., Waltman, L.: The correlation between citation-based and expert-based assessments of publication channels: SNIP and SJR vs Norwegian quality assessments. J. Inform. 8(4), 985–996 (2014)

    Google Scholar 

  7. Aksnes, D.: Citations and their use as indicators in science policy : studies of validity and applicability issues with a particular focus on highly cited papers. Creativity Innovation Manag. (2005)

    Google Scholar 

  8. Aksnes, D.W., Langfeldt, L., Wouters, P.: Citations, citation indicators, and research quality: an overview of basic concepts and theories. Sage Open 9(1) (2019)

    Google Scholar 

  9. Alchokr, R., Krüger, J., Saake, G., Leich, T.: A comparative analysis of article recommendation platforms. In: 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL), pp. 1–10. IEEE (2021)

    Google Scholar 

  10. Alchokr, R., Krüger, J., Shakeel, Y., Saake, G., Leich, T.: Peer-reviewing and submission dynamics around top software-engineering venues: a juniors’ perspective. In: International Conference on Evaluation and Assessment in Software Engineering (EASE). ACM (2022)

    Google Scholar 

  11. Allik, J.: Factors affecting bibliometric indicators of scientific quality. Trames 17(3), 199–214 (2013)

    Google Scholar 

  12. Alonso, S., Cabrerizo, F.J., Herrera-Viedma, E., Herrera, F.: H-index: a review focused in its variants, computation and standardization for different scientific fields. J. Informet. 3(4), 273–289 (2009)

    Google Scholar 

  13. Bhatti, U.A., Huang, M., Wu, D., Zhang, Y., Mehmood, A., Han, H.: Recommendation system using feature extraction and pattern recognition in clinical care systems. Enterp. Inf. Syst. 13(3), 329–351 (2019)

    Google Scholar 

  14. Bhatti, U.A., et al.: Time series analysis and forecasting of air pollution particulate matter (PM 2.5): an SARIMA and factor analysis approach. IEEE Access 9, 41019–41031 (2021)

    Google Scholar 

  15. Birken, C.S., Parkin, P.C.: In which journals will pediatricians find the best evidence for clinical practice? Pediatrics 103(5), 941–947 (1999)

    Google Scholar 

  16. Bloom, B.S., Retbi, A., Dahan, S., Jonsson, E.: Evaluation of randomized controlled trials on complementary and alternative medicine. Int. J. Technol. Assess. Healthcar. 16(1), 13–21 (2000)

    Google Scholar 

  17. Bornmann, L.: Do altmetrics point to the broader impact of research? an overview of benefits and disadvantages of altmetrics. J. Informet. 8(4), 895–903 (2014)

    Google Scholar 

  18. Bornmann, L., Daniel, H.D.: Does the H-index for ranking of scientists really work? Scientometrics 65, 391–392 (2005)

    Google Scholar 

  19. Bornmann, L., Daniel, H.D.: The state of H-index research: is the H-index the ideal way to measure research performance? EMBO Rep. 10(1), 2–6 (2009)

    Google Scholar 

  20. Bornmann, L., Haunschild, R.: Do altmetrics correlate with the quality of papers? A large-scale empirical study based on f1000prime data. PloS one 13(5) (2018)

    Google Scholar 

  21. Bornmann, L., Marx, W.: Methods for the generation of normalized citation impact scores in bibliometrics: which method best reflects the judgements of experts? J. Inform. 9(2), 408–418 (2015)

    Google Scholar 

  22. Bornmann, L., Mutz, R., Daniel, H.D.: Are there better indices for evaluation purposes than the H-index? a comparison of nine different variants of the H-index using data from biomedicine. J. Am. Soc. Inform. Sci. Technol. 59(5), 830–837 (2008)

    Google Scholar 

  23. Bridges, D.: Research quality assessment in education: impossible science, possible art? Br. Edu. Res. J. 35(4), 497–517 (2009)

    Google Scholar 

  24. Cabanac, G., Oikonomidi, T., Boutron, I.: Day-to-day discovery of preprint-publication links. Scientometrics 126(6), 5285–5304 (2021)

    Google Scholar 

  25. Coelho, P., Antunes, C., Costa, H., Kroon, E., Lima, S., Linardi, P.: The use and misuse of the impact factor as a parameter for evaluation of scientific publication quality: a proposal to rationalize its application. Braz. J. Med. Biol. Res. 36, 1605–1612 (2003)

    Google Scholar 

  26. Corbyn, Z.: An easy way to boost a paper’s citations. Nature, 1476–4687 (2010)

    Google Scholar 

  27. Da Luz, M.P., et al.: Institutional H-index: the performance of a new metric in the evaluation of Brazilian psychiatric post-graduation programs. Scientometrics 77, 361–368 (2008)

    Google Scholar 

  28. DeSanto, D., Nichols, A.: Scholarly metrics baseline: a survey of faculty knowledge, use, and opinion about scholarly metrics. Coll. Res. Libr. 78(2) (2017)

    Google Scholar 

  29. Devos, P., Dufresne, E., Renard, J.M., Beuscart, R.: SIGAPS a prototype of bibliographic tool for medical research evaluation. In: The New Navigators: from Professionals to Patients, pp. 721–726. IOS Press (2003)

    Google Scholar 

  30. Dey, A.K.: Understanding and using context. Pers. Ubiquit. Comput. 5, 4–7 (2001)

    Google Scholar 

  31. Dinsmore, A., Allen, L., Dolby, K.: Alternative perspectives on impact: the potential of ALMs and altmetrics to inform funders about research impact. PLoS Biol. 12(11) (2014)

    Google Scholar 

  32. Dougherty, M.R., Horne, Z.: Citation counts and journal impact factors do not capture some indicators of research quality in the behavioural and brain sciences. Roy. Soc. Open Sci. 9(8) (2022)

    Google Scholar 

  33. Durieux, V., Gevenois, P.A.: Bibliometric indicators: quality measurements of scientific publication. Radiology 255(2), 342–351 (2010)

    Google Scholar 

  34. Ebadifar, A., et al.: How to assess quality of research in Iran, from input to impact? introduction of peer-based research evaluation model in Iran. Arch. Iran. Med. 20(11) (2017)

    Google Scholar 

  35. Egghe, L.: Theory and practise of the g-index. Scientometrics 69(131), 131–152 (2006). https://doi.org/10.1007/s11192-006-0144-7

    Article  Google Scholar 

  36. Erdt, M., Nagarajan, A., Sin, S.C.J., Theng, Y.L.: Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media. Scientometrics 109, 1117–1166 (2016)

    Google Scholar 

  37. Fahim, A., Tan, Q., Sahabuddin, M.: Sustainable research quality assessment model (SRQAM) for higher education using regression. In: 2021 IEEE International Conference on Educational Technology (ICET), pp. 268–272. IEEE (2021)

    Google Scholar 

  38. Franceschini, F., Maisano, D., Mastrogiacomo, L.: Research quality evaluation: comparing citation counts considering bibliometric database errors. Qual. Quant. 49, 155–165 (2015)

    Google Scholar 

  39. Garfield, E.: Citation analysis as a tool in journal evaluation: journals can be ranked by frequency and impact of citations for science policy studies. Science 178(4060), 471–479 (1972)

    Google Scholar 

  40. Garfield, E.: Citation indexes for science. a new dimension in documentation through association of ideas. Int. J. Epidemiol. 35(5), 1123–1127 (2006)

    Google Scholar 

  41. Gasparyan, A.Y., Nurmashev, B., Voronov, A.A., Gerasimov, A.N., Koroleva, A.M., Kitas, G.D.: The pressure to publish more and the scope of predatory publishing activities. J. Korean Med. Sci. 31(12), 1874–1878 (2016)

    Google Scholar 

  42. Glujovsky, D., Riestra, B., Coscia, A., Boggino, C., Comande, D., Ciapponi, A.: Assessment of research quality in major infertility journals. Fertil. Steril. 98(6), 1539–1543 (2012)

    Google Scholar 

  43. Gogolin, I.: European educational research quality indicators (EERQI): an experiment. In: Ochsner, M., Hug, S.E., Daniel, H.D. (eds.) Research Assessment in the Humanities, pp. 103–111. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-29016-4-9

  44. Goldstein, H., Maier, G.: The use and valuation of journals in planning scholarship: peer assessment versus impact factors. J. Plan. Educ. Res. 30(1), 66–75 (2010)

    Google Scholar 

  45. Haeffner-Cavaillon, N., Graillot-Gak, C.: The use of bibliometric indicators to help peer-review assessment. Arch. Immunol. Ther. Exp. 57, 33–38 (2009)

    Google Scholar 

  46. Hassanain, M., Anil, S., Abdo, A.: Institutional research evaluation model (IREM): a framework for measuring organizational research trends and impact and its application in medical academia in Saudi Arabia. J. Epidemiol. Glob. Health 6(4), 249–256 (2016)

    Google Scholar 

  47. Hirsch, J.E.: An index to quantify an individual’s scientific research output. Proc. Nat. Acad. Sci. 102(46), 16569–16572 (2005)

    Google Scholar 

  48. Jarwal, S.D., Brion, A.M., King, M.L.: Measuring research quality using the journal impact factor, citations and ranked journals: blunt instruments or inspired metrics? J. High. Educ. Policy Manag. 31(4), 289–300 (2009)

    Google Scholar 

  49. Jones, M.R.H.: Can research quality be measured quantitatively? on quality of scholarship, numerical research indicators and academic publishing-experiences from Norway. Fennia Int. J. Geogr. 195(2), 164–174 (2017)

    Google Scholar 

  50. Joshi, M.A.: Bibliometric indicators for evaluating the quality of scientific publications. J. Contemp. Dent. Pract. 15(2), 258 (2014)

    Google Scholar 

  51. Josiam, M., Lee, W., Johnson, T., Pee, C., Hall, J.: Beyond selecting a methodology: Discussing research quality, ethical, and equity considerations in qualitative engineering education research. In: 2022 ASEE Annual Conference and Exposition (2022)

    Google Scholar 

  52. Jost, J.T.: The existence of implicit bias is beyond reasonable doubt: a refutation of ideological and methodological objections and executive summary of ten studies that no manager should ignore. Res. Organ. Behav. 29, 39–69 (2009)

    Google Scholar 

  53. Khan, N.R., et al.: Part ii: should the h-index be modified? an analysis of the m-quotient, contemporary h-index, authorship value, and impact factor. World Neurosurg. 80(6), 766–774 (2013)

    Google Scholar 

  54. Kitchenham, B.A., Charters, S.: Guidelines for performing systematic literature reviews in software engineering, Technical report, EBSE-2007-01, Keele University and University of Durham (2007)

    Google Scholar 

  55. Kleijnen, J.P., Van Groenendaal, W.: Measuring the quality of publications: new methodology and case study. Inf. Process. Manag. 36(4), 551–570 (2000)

    Google Scholar 

  56. Konkiel, S.: Altmetrics: diversifying the understanding of influential scholarship. Palgrave Commun. 2(1), 1–7 (2016)

    Google Scholar 

  57. Konkiel, S.: Assessing the impact and quality of research data using altmetrics and other indicators. Sch. Assess. Rep. 2(1) (2020)

    Google Scholar 

  58. Konkiel, S., Scherer, D.: New opportunities for repositories in the age of altmetrics. Bull. Am. Soc. Inf. Sci. Technol. 39(4), 22–26 (2013)

    Google Scholar 

  59. Kosmulski, M., et al.: A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newslett. 2(3), 4–6 (2006)

    Google Scholar 

  60. Koya, K., Chowdhury, G.: A quality and popularity based ranking method for research datasets. In: Proceedings of the 2022 4th Asia Pacific Information Technology Conference, pp. 103–110 (2022)

    Google Scholar 

  61. Kratz, J.E., Strasser, C.: Making data count. Sci. Data 2(1), 1–5 (2015)

    Google Scholar 

  62. Kwan, M.M., Balasubramanian, P.: KnowledgeScope: managing knowledge in context. Decis. Support Syst. 35(4), 467–486 (2003)

    Google Scholar 

  63. Larsen, P., Von Ins, M.: The rate of growth in scientific publication and the decline in coverage provided by science citation index. Scientometrics 84(3), 575–603 (2010)

    Google Scholar 

  64. Lazaridis, T.: Ranking university departments using the mean h-index. Scientometrics 82(2), 211–216 (2010)

    Google Scholar 

  65. Leydesdorff, L., Wouters, P., Bornmann, L.: Professional and citizen bibliometrics: complementarities and ambivalences in the development and use of indicators-a state-of-the-art report. Scientometrics 109, 2129–2150 (2016)

    Google Scholar 

  66. Li, Z., Ho, Y.S.: Use of citation per publication as an indicator to evaluate contingent valuation research. Scientometrics 75, 97–110 (2008)

    Google Scholar 

  67. Locoro, A., David, J., Euzenat, J.: Context-based matching: design of a flexible framework and experiment. J. Data Seman. 3, 25–46 (2014)

    Google Scholar 

  68. Lowenberg, D., Chodacki, J., Fenner, M., Kemp, J., Jones, M.: Open data metrics: lighting the fire. Zenodo (2019)

    Google Scholar 

  69. Lowy, C.: Impact factor limits funding. Lancet 350(9083) (1997)

    Google Scholar 

  70. Margherita, A., Elia, G., Petti, C.: What is quality in research? building a framework of design, process and impact attributes and evaluation perspectives. Sustainability 14(5), 30–34 (2022)

    Google Scholar 

  71. Melkas, H., Harmaakorpi, V.: Data, information and knowledge in regional innovation networks: quality considerations and brokerage functions. Eur. J. Innov. Manag. 11(1), 103–124 (2008)

    Google Scholar 

  72. Memmi, D.: The Social Context of Knowledge, pp. 189–208 (2008)

    Google Scholar 

  73. Mingers, J., Leydesdorff, L.: A review of theory and practice in Scientometrics. Eur. J. Oper. Res. 246(1), 1–19 (2015)

    Google Scholar 

  74. Moed, H., De Bruin, R., Van Leeuwen, T.: New bibliometric tools for the assessment of national research performance: database description, overview of indicators and first applications. Scientometrics 33(3), 381–422 (1995)

    Google Scholar 

  75. Moed, H.F.: The impact-factors debate: the ISI’s uses and limits. Nature 415(6873), 731–732 (2002)

    Google Scholar 

  76. Moldoveanu, B., Cuciureanu, G.: Publishing as an indicator of scientific research quality and ethics: the case of law journals from Moldova. Sci. Eng. Ethics 26(2), 1039–1052 (2020)

    Google Scholar 

  77. Nagpaul, P., Roy, S.: Constructing a multi-objective measure of research performance. Scientometrics 56(3), 383–402 (2003)

    Google Scholar 

  78. Northcott, D., Linacre, S.: Producing spaces for academic discourse: the impact of research assessment exercises and journal quality rankings. Aust. Account. Rev. 20(1), 38–54 (2010)

    Google Scholar 

  79. Nuzzolese, A.G., Ciancarini, P., Gangemi, A., Peroni, S., Poggi, F., Presutti, V.: Do altmetrics work for assessing research quality? Scientometrics 118(2), 539–562 (2019)

    Google Scholar 

  80. Petersen, A.M., et al.: Reputation and impact in academic careers. Proc. Nat. Acad. Sci. 111(43), 15316–15321 (2014)

    Google Scholar 

  81. Pierce, E., Kahn, B., Melkas, H.: A comparison of quality issues for data, information, and knowledge. In: 17th Annual Information Resources Management Association International Conference (2006)

    Google Scholar 

  82. Polanyi, M.: The Republic of Science: Its Political and Economic Theory (1962). Knowing and Being, Essays (1969)

    Google Scholar 

  83. Praus, P.: Statistical evaluation of research performance of young university scholars: a case study. Transinformação 30, 167–177 (2018)

    Google Scholar 

  84. Praus, P.: High-ranked citations percentage as an indicator of publications quality. Scientometrics 120, 319–329 (2019)

    Google Scholar 

  85. Rao, L., Osei-Bryson, K.M.: Towards defining dimensions of knowledge systems quality. Expert Syst. Appl. 33(2), 368–378 (2007)

    Google Scholar 

  86. Rezaei, G.N., Azizi, F.: The impact factor-based quality assessment of biomedical research institutes in Iran: effect of impact factor normalization by subject (2007)

    Google Scholar 

  87. Roemer, R.C., Borchardt, R.: Issues, controversies, and opportunities for altmetrics. Libr. Technol. Rep. 51(5), 20–30 (2015)

    Google Scholar 

  88. Rudd, E.: The evaluation of the quality of research. Stud. High. Educ. 13(1), 45–57 (1988)

    Google Scholar 

  89. Sabetzadeh, F., Tsui, E., Lee, W.B.: Assessment of uncertainty in the quality of knowledge in the research publication review process. In: 2013 10th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), pp. 946–950. IEEE (2013)

    Google Scholar 

  90. Saha, S., Saint, S., Christakis, D.A.: Impact factor: a valid measure of journal quality? J. Med. Libr. Assoc. 91(1), 42 (2003)

    Google Scholar 

  91. Sándor, Á., Vorndran, A.: Enhancing relevance ranking of the EERQI search engine. In: Gogolin, I., Åström, F., Hansen, A. (eds.) Assessing Quality in European Educational Research, pp. 56–59. Springer, Wiesbaden (2014). https://doi.org/10.1007/978-3-658-05969-9

  92. Sándor, Á., Vorndran, A.: Highlighting salient sentences for reading assistance. In: Gogolin, I., Åström, F., Hansen, A. (eds.) Assessing Quality in European Educational Research, pp. 43–55. Springer, Wiesbaden (2014). https://doi.org/10.1007/978-3-658-05969-9

  93. Schreiber, M., Malesios, C., Psarakis, S.: Exploratory factor analysis for the Hirsch index, 17 h-type variants, and some traditional bibliometric indicators. J. Inform. 6(3), 347–358 (2012)

    Google Scholar 

  94. Schubert, A., Glänzel, W.: A systematic analysis of Hirsch-type indices for journals. J. Inform. 1(3), 179–184 (2007)

    Google Scholar 

  95. Seawright, K.W., Young, S.T.: A quality definition continuum. Interfaces 26(3), 107–113 (1996)

    Google Scholar 

  96. Seglen, P.O.: Why the impact factor of journals should not be used for evaluating research. BMJ 314(7079), 497 (1997)

    Google Scholar 

  97. Shakeel, Y., Alchokr, R., Krüger, J., Leich, T., Saake, G.: Are altmetrics useful for assessing scientific impact? A survey. In: International Conference on Management of Digital ecoSystems (MEDES). ACM (2022)

    Google Scholar 

  98. Shakeel, Y., Alchokr, R., Krüger, J., Leich, T., Saake, G.: Incorporating altmetrics to support selection and assessment of publications during literature analyses. In: International Conference on Evaluation and Assessment in Software Engineering (EASE), pp. 180–189. ACM (2022)

    Google Scholar 

  99. Shakeel, Y., Alchokr, R., Krüger, J., Leich, T., Saake, G.: Altmetrics and citation counts: an empirical analysis of the computer science domain. In: Joint Conference on Digital Libraries (JCDL), pp. 1–11. ACM (2022)

    Google Scholar 

  100. Shakeel, Y., Alchokr, R., Krüger, J., Saake, G., Leich, T.: Are altmetrics proxies or complements to citations for assessing impact in computer science? In: Joint Conference on Digital Libraries (JCDL), pp. 284–286. ACM/IEEE (2021)

    Google Scholar 

  101. Shakeel, Y., Krüger, J., Saake, G., Leich, T.: Indicating studies’ quality based on open data in digital libraries. In: Abramowicz, W., Paschke, A. (eds.) Business Information Systems, pp. 579–590. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-04849-5-50

  102. Sharma, O.P.: Quality indicators of scientific research. Indian J. Microbiol. 52(2) (2012)

    Google Scholar 

  103. Sombatsompop, N., Kositchaiyong, A., Markpin, T., Inrit, S.: Scientific evaluations of citation quality of international research articles in the sci database: Thailand case study. Scientometrics 66(3), 521–535 (2006)

    Google Scholar 

  104. Spezi, V., Wakeling, S., Pinfield, S., Fry, J., Creaser, C., Willett, P.: Let the community decide? The vision and reality of soundness-only peer review in open-access mega-journals. J. Doc. 74(1), 137–161 (2018)

    Google Scholar 

  105. Sugimoto, C.: Attention is Not Impact and Other Challenges for Altmetrics. Wiley, Discover the Future of Research (2015)

    Google Scholar 

  106. Sumner, J.Q., Vitale, C.H., McIntosh, L.D.: RipetaScore: measuring the quality, transparency, and trustworthiness of a scientific work. Front. Res. Metrics Anal. 6, 734–751 (2022)

    Google Scholar 

  107. Taylor, J.: The assessment of research quality in UK universities: peer review or metrics? Br. J. Manag. 22(2), 202–217 (2011)

    Google Scholar 

  108. Tetlock, P.E., Mitchell, G.: Implicit bias and accountability systems: what must organizations do to prevent discrimination? Res. Organ. Behav. 29, 3–38 (2009)

    Google Scholar 

  109. Thelwall, M., Haustein, S., Larivière, V., Sugimoto, C.R.: Do altmetrics work? Twitter and ten other social web services. PLoS one 8(5) (2013)

    Google Scholar 

  110. Tracy, S.J.: Qualitative quality: eight big-tent criteria for excellent qualitative research. Qual. Inq. 16(10), 837–851 (2010)

    Google Scholar 

  111. Ueda, R., et al.: Importance of quality assessment in clinical research in Japan. Front. Pharmacol. 10, 1228 (2019)

    Google Scholar 

  112. Vieira, E.S., Gomes, J.A.: Citations to scientific articles: its distribution and dependence on the article features. J. Informet. 4(1), 1–13 (2010)

    Google Scholar 

  113. Wallmark, J.T., Sedig, K.G.: Quality of research measured by citation method and by peer review, a comparison. IEEE Trans. Eng. Manage. 4, 218–222 (1986)

    Google Scholar 

  114. Ware, M., Mabe, M.: The STM Report: An Overview of Scientific and Scholarly Journal Publishing (2015)

    Google Scholar 

  115. Welljams-Dorof, A.: Quantitative citation data as indicators in science evaluations: a primer on their appropriate use. In: Frankel, M.S., Cave J. (eds.) Evaluating Science and Scientists: An East-West Dialogue on Research Evaluation in Post Communist Europe, pp. 202–211. Central European University Press (1997)

    Google Scholar 

  116. Wouters, P., Costas, R.: Users, narcissism and control: tracking the impact of scholarly publications in the 21st century (2012)

    Google Scholar 

  117. Yoo, D.K., Vonderembse, M.A., Ragu-Nathan, T.: Knowledge quality: antecedents and consequence in project teams. J. Knowl. Manag. 15(2), 329–343 (2011)

    Google Scholar 

  118. Zanon, B.: Research quality assessment and planning journals. Ital. J. Plan. Pract. 2(2), 96–123 (2014)

    Google Scholar 

  119. Zhang, C.T.: The e-index, complementing the h-index for excess citations. PLoS One 4(5) (2009)

    Google Scholar 

  120. Zhou, J., Zeng, A., Fan, Y., Di, Z.: The representative works of scientists. Scientometrics 117(3), 1721–1732 (2018)

    Google Scholar 

  121. Zwahlen, M., Junker, C., Egger, M.: Commentary ii-the journal impact factor in the evaluation of research quality: villain, scapegoat or innocent bystander? Soz. Präventivmed 49(1), 19–22 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Otto-von-Guericke University, Magdeburg, Germany

    Rand Alchokr, Abhishek Gopalrao & Gunter Saake

  2. Harz University & METOP GmbH, Wernigerode, Germany

    Thomas Leich

  3. Eindhoven University of Technology, Eindhoven, The Netherlands

    Jacob Krüger

Authors
  1. Rand Alchokr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abhishek Gopalrao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gunter Saake
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Leich
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jacob Krüger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rand Alchokr .

Editor information

Editors and Affiliations

  1. University of Salford, Salford, UK

    Apostolos Antonacopoulos

  2. University of Waikato, Hamilton, New Zealand

    Annika Hinze

  3. Sorbonne University (CNRS), Paris, France

    Benjamin Piwowarski

  4. University of La Rochelle (L3i Laboratory), La Rochelle, France

    Mickaël Coustaty

  5. University of Padova, Padua, Italy

    Giorgio Maria Di Nunzio

  6. University of Hamburg, Hamburg, Germany

    Francesco Gelati

  7. University of Waikato, Hamilton, New Zealand

    Nicholas Vanderschantz

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Alchokr, R., Gopalrao, A., Saake, G., Leich, T., Krüger, J. (2024). Scholarly Quality Measurements: A Systematic Literature Review. In: Antonacopoulos, A., et al. Linking Theory and Practice of Digital Libraries. TPDL 2024. Lecture Notes in Computer Science, vol 15177. Springer, Cham. https://doi.org/10.1007/978-3-031-72437-4_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72437-4_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72436-7

  • Online ISBN: 978-3-031-72437-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation