research-article

Do altmetrics follow the crowd or does the crowd follow altmetrics?

Authors:

Richard FurutaAuthors Info & Claims

JCDL '14: Proceedings of the 14th ACM/IEEE-CS Joint Conference on Digital Libraries

Pages 375 - 378

Published: 08 September 2014 Publication History

Abstract

Changes are occurring in scholarly communication as scientific discourse and research activities spread across various social media platforms. In this paper, we study altmetrics on the article and journal levels, investigating whether the online attention received by research articles is related to scholarly impact or may be due to other factors. We define a new metric, Journal Social Impact (JSI), based on eleven data sources: CiteULike, Mendeley, F1000, blogs, Twitter, Facebook, mainstream news outlets, Google Plus, Pinterest, Reddit, and sites running Stack Exchange (Q&A). We compare JSI against diverse citation-based metrics, and find that JSI significantly correlates with a number of them. These findings indicate that online attention of scholarly articles is related to traditional journal rankings and favors journals with a longer history of scholarly impact. We also find that journal-level altmetrics have strong significant correlations among themselves, compared with the weak correlations among article-level altmetrics. Another finding is that Mendeley and Twitter have the highest usage and coverage of scholarly activities. Among individual altmetrics, we find that the readership of academic social networks have the highest correlations with citation-based metrics. Our findings deepen the overall understanding of altmetrics and can assist in validating them.

References

[1]

Y. Ding, E. K. Jacob, Z. Zhang, S. Foo, E. Yan, N. L. George, and L. Guo, "Perspectives on social tagging," J. Am. Soc. Inf. Sci. Technol., vol. 60, no. 12, pp. 2388--2401, Dec. 2009.

Digital Library

[2]

S. Fausto, F. A. Machado, L. F. J. Bento, A. Iamarino, T. R. Nahas, and D. S. Munger, "Research blogging: indexing and registering the change in science 2.0.," PLoS One, vol. 7, no. 12, p. e50109, Dec. 2012.

[3]

E. Adie and W. Roe, "Altmetric: enriching scholarly content with article-level discussion and metrics," Learn. Publ., vol. 26, no. 1, pp. 11--17, Jan. 2013.

[4]

I. Rowlands, D. Nicholas, B. Russell, N. Canty, and A. Watkinson, "Social media use in the research workflow," Learn. Publ., vol. 24, no. 3, pp. 183--195, Jul. 2011.

[5]

J. C. Wallis, E. Rolando, and C. L. Borgman, "If we share data, will anyone use them? Data sharing and reuse in the long tail of science and technology," PLoS One, vol. 8, no. 7, p. e67332, Jul. 2013.

[6]

A. Gruzd, K. Staves, and A. Wilk, "Connected scholars: Examining the role of social media in research practices of faculty using the UTAUT model," Comput. Human Behav., vol. 28, no. 6, pp. 2340--2350, Nov. 2012.

Digital Library

[7]

H. Piwowar, "Altmetrics: Value all research products," Nature, vol. 493, no. 7431, p. 159, Jan. 2013.

[8]

"Higher Education Funding Council for England. Research Excellence Framework: Decisions on assessing research impact," 2011.

[9]

I. Viney, "Altmetrics: Research council responds," Nature, vol. 494, no. 7436, p. 176, Feb. 2013.

[10]

R. Smith, "Measuring the social impact of research," BMJ, vol. 323, no. 7312, p. 528, Sep. 2001.

[11]

L. Bornmann, "What is societal impact of research and how can it be assessed? a literature survey," J. Am. Soc. Inf. Sci. Technol., vol. 64, no. 2, pp. 217--233, Feb. 2013.

Digital Library

[12]

H. Lima, T. H. P. Silva, M. M. Moro, R. L. T. Santos, W. Meira, and A. H. F. Laender, "Aggregating productivity indices for ranking researchers across multiple areas," in Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries, 2013, pp. 97--106.

Digital Library

[13]

C. Neylon and S. Wu, "Article-level metrics and the evolution of scientific impact," PLoS Biol., vol. 7, no. 11, p. e1000242, Nov. 2009.

[14]

J. Bollen, H. Van de Sompel, A. Hagberg, and R. Chute, "A principal component analysis of 39 scientific impact measures," PLoS One, vol. 4, no. 6, p. e6022, Jun. 2009.

[15]

J. Priem, P. Groth, and D. Taraborelli, "The altmetrics collection," PLoS One, vol. 7, no. 11, p. e48753, Nov. 2012.

[16]

J. Priem, H. A. Piwowar, and B. M. Hemminger, "Altmetrics in the wild: Using social media to explore scholarly impact," arXiv:1203.4745, Mar. 2012.

[17]

M. Thelwall, S. Haustein, V. Larivière, and C. R. Sugimoto, "Do altmetrics work? Twitter and ten other social web services," PLoS One, vol. 8, no. 5, p. e64841, May 2013.

[18]

S. Haustein, I. Peters, C. R. Sugimoto, M. Thelwall, and V. Larivière, "Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature," J. Assoc. Inf. Sci. Technol., vol. 65, no. 4, pp. 656--669, Apr. 2014.

Digital Library

[19]

X. Shuai, Z. Jiang, X. Liu, and J. Bollen, "A comparative study of academic and Wikipedia ranking," in Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries, 2013, pp. 25--28.

Digital Library

[20]

L. Waltman and R. Costas, "F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations," J. Assoc. Inf. Sci. Technol., vol. 65, no. 3, pp. 433--445, Mar. 2014.

Digital Library

[21]

J. Bar-Ilan, S. Haustein, I. Peters, J. Priem, H. Shema, and J. Terliesner, "Beyond citations: Scholars' visibility on the social Web," in Proceedings of the 17th International Conference on Science and Technology Indicators, 2012, vol. 52900, pp. 98--109.

[22]

T. Brody, S. Harnad, and L. Carr, "Earlier Web usage statistics as predictors of later citation impact," J. Am. Soc. Inf. Sci. Technol., vol. 57, no. 8, pp. 1060--1072, Jun. 2006.

Digital Library

[23]

H. Alhoori and R. Furuta, "Can Social Reference Management Systems Predict a Ranking of Scholarly Venues?," in Proceedings of the 17th international conference on theory and practice of digital libraries, 2013, vol. 8092, pp. 138--143.

[24]

R. Costas, Z. Zahedi, and P. Wouters, "Do altmetrics correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective," arXiv:1401.4321, Jan. 2014.

[25]

J. Jiang, C. Ni, D. He, and W. Jeng, "Mendeley group as a new source of interdisciplinarity study," in Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries, 2013, pp. 135--138.

Digital Library

[26]

H. Alhoori, O. Alvarez, R. Furuta, N. Miguel Mu, and E. Urbina, "Supporting the Creation of Scholarly Bibliographies by Communities through Online Reputation Based Social Collaboration," in Proceedings of the 13th European conference on research and advanced technology for digital libraries, 2009, vol. 5714, pp. 180--191.

Digital Library

Cited By

Freeman CRoy MFattoruso MAlhoori HDownie SZhu QHahn J(2019)Shared feelingsProceedings of the 18th Joint Conference on Digital Libraries10.1109/JCDL.2019.00050(301-304)Online publication date: 2-Jun-2019
https://dl.acm.org/doi/10.1109/JCDL.2019.00050
Papachristopoulos LTsakonas GBoudourides MSfakakis MKleidis NLenis SPapatheodorou C(2019)Discovering the structure and impact of the digital library evaluation domainInternational Journal on Digital Libraries10.1007/s00799-017-0222-x20:2(125-141)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s00799-017-0222-x
Bailey CSiravuri HKale BAlhoori HWalker JPapka MMcDonald RWorby NJatowt AMarshall CMilligan I(2017)Exploring features for predicting policy citationsProceedings of the 17th ACM/IEEE Joint Conference on Digital Libraries10.5555/3200334.3200381(297-298)Online publication date: 19-Jun-2017
https://dl.acm.org/doi/10.5555/3200334.3200381
Show More Cited By

Index Terms

Do altmetrics follow the crowd or does the crowd follow altmetrics?
1. Applied computing
  1. Computers in other domains
    1. Digital libraries and archives
  2. Law, social and behavioral sciences
2. Information systems
  1. Information systems applications
    1. Digital libraries and archives

Recommendations

Using altmetrics for assessing research impact in the humanities

The prospects of altmetrics are especially encouraging for research fields in the humanities that currently are difficult to study using established bibliometric methods. Yet, little is known about the altmetric impact of research fields in the ...
Exploring prestigious citations sourced from top universities in bibliometrics and altmetrics: a case study in the computer science discipline

Citation count is an important indicator for measuring research outputs. There have been numerous studies that have investigated factors affecting citation counts from the perspectives of cited papers and citing papers. In this paper, we focused ...
Who reads international Egyptian academic articles? An altmetrics analysis of Mendeley readership categories
Abstract
Mendeley is a social network that allows researchers worldwide to discover, search and share resources and to cooperate with peer researchers. We can recognize a large amount of exhaustive information about who reads research articles and the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

JCDL '14: Proceedings of the 14th ACM/IEEE-CS Joint Conference on Digital Libraries

September 2014

498 pages

ISBN:9781479955695

General Chair:
George Buchanan,
Program Chairs:
Martin Klein,
Andreas Rauber,
Sally Jo Cunningham

Sponsors

Publisher

IEEE Press

Publication History

Published: 08 September 2014

Check for updates

Author Tags

Qualifiers

Research-article

Conference

JCDL '14

Sponsor:

JCDL '14: 14th ACM/IEEE-CS Joint Conference on Digital Libraries

September 8 - 12, 2014

London, United Kingdom

Acceptance Rates

Overall Acceptance Rate 415 of 1,482 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
189
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Freeman CRoy MFattoruso MAlhoori HDownie SZhu QHahn J(2019)Shared feelingsProceedings of the 18th Joint Conference on Digital Libraries10.1109/JCDL.2019.00050(301-304)Online publication date: 2-Jun-2019
https://dl.acm.org/doi/10.1109/JCDL.2019.00050
Papachristopoulos LTsakonas GBoudourides MSfakakis MKleidis NLenis SPapatheodorou C(2019)Discovering the structure and impact of the digital library evaluation domainInternational Journal on Digital Libraries10.1007/s00799-017-0222-x20:2(125-141)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s00799-017-0222-x
Bailey CSiravuri HKale BAlhoori HWalker JPapka MMcDonald RWorby NJatowt AMarshall CMilligan I(2017)Exploring features for predicting policy citationsProceedings of the 17th ACM/IEEE Joint Conference on Digital Libraries10.5555/3200334.3200381(297-298)Online publication date: 19-Jun-2017
https://dl.acm.org/doi/10.5555/3200334.3200381
Kale BSiravuri HAlhoori HPapka MFox PMcGuinness DPoirer LBoldi PKinder-Kurlanda K(2017)Predicting Research that will be Cited in Policy DocumentsProceedings of the 2017 ACM on Web Science Conference10.1145/3091478.3098865(389-390)Online publication date: 25-Jun-2017
https://dl.acm.org/doi/10.1145/3091478.3098865

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten