article

An assessment into the characteristics of award winning papers at CHI

Authors:

Dhaval Tejlavwala,

Mudassar Arsalan,

Simeon SimoffAuthors Info & Claims

Scientometrics, Volume 116, Issue 2

Pages 1181 - 1201

https://doi.org/10.1007/s11192-018-2778-7

Published: 01 August 2018 Publication History

Abstract

The overall readability of CHI publications is not known. In addition, little is understood about what lexical or demographic characteristics are unique to award winning papers at CHI and if they are significantly different from non award winning papers. We therefore carry out an exploration and assessment into the readability metrics as well as a meta analysis of 382 full papers and 54 notes from the 2014, 2015, 2016 and 2017 editions at CHI. Our results illustrate that notes did not have any significant trends whatsoever. On the other hand, award winning full papers were shown to have lower readability as compared to non award winning full papers. The type of research contribution played an important role; such that award winning full papers were significantly more likely to have a theoretical contribution as compared to non award winning full papers and full papers that presented an artifact as their contribution were more readable than other full papers. Our demographic analysis of authors indicated that the experience of authors nor their region of affiliation were not associated with the likelihood of their full paper being awarded. The experience of authors did not effect the overall readability of full papers however the region of affiliation did have a significant influence on the overall readability of full papers. In conclusion, we speculate on our obtained results through linkages with prior work in readability analysis.

References

[1]

Alluqmani, A., & Shamir, L. (2018). Writing styles in different scientific disciplines: A data science approach. Scientometrics, 115(2), 1-15.

Digital Library

[2]

Armstrong, J. S. (1989). Readability and prestige in scientific journals. Journal of Information Science, 15, 123-124. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=668145.

Digital Library

[3]

Bartneck, C., & Hu, J. (2009) Scientometric analysis of the chi proceedings. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 699-708). ACM.

Digital Library

[4]

Best of CHI 2011 Award Winners. (2011). Best of CHI 2011 award winners. http://www.chi2011.org/program/awards.html. Accessed 15 Feb 2018.

[5]

Bird, S., Klein, E., & Loper, E. (2009). Natural language processing with Python: Analyzing text with the natural language toolkit. Sebastopol, CA: O'Reilly Media Inc.

Digital Library

[6]

Brischoux, F., & Legagneux, P. (2009). Don't format manuscripts. The Scientist, 23(7), 24.

[7]

Costas, R., Leeuwen, T. N., & Bordons, M. (2012). Referencing patterns of individual researchers: Do top scientists rely on more extensive information sources? Journal of the Association for Information Science and Technology, 63(12), 2433-2450.

Digital Library

[8]

Coupé, T. (2013). Peer review versus citations--An analysis of best paper prizes. Research Policy, 42(1), 295-301.

[9]

Dolnicar, S., & Chapple, A. (2015). The readability of articles in tourism journals. Annals of Tourism Research, 52, 161-166.

[10]

DuBay, W. H. (2004). The principles of readability. Online Submission. https://files.eric.ed.gov/fulltext/ED490073.pdf.

[11]

Felt, A. P., Reeder, R. W., Almuhimedi, H., & Consolvo, S. (2014). Experimenting at scale with google chrome's SSL warning. In Proceedings of the 32nd annual ACM conference on human factors in computing systems (pp. 2667-2670). ACM.

Digital Library

[12]

Gazni, A. (2011). Are the abstracts of high impact articles more readable? Investigating the evidence from top research institutions in the world. Journal of Information Science, 37(3), 273-281.

Digital Library

[13]

Guerini, M., Pepe, A., & Lepri, B. (2012). Do linguistic style and readability of scientific abstracts affect their virality? In Proceedings of the Sixth International AAAI Conference on Weblogs and Social Media (pp. 475-478).

[14]

Hartley, J., & Cabanac, G. (2016). Are two authors better than one? Can writing in pairs affect the readability of academic blogs? Scientometrics, 109(3), 2119-2122.

Digital Library

[15]

Hartley, J., Pennebaker, J., & Fox, C. (2003). Abstracts, introductions and discussions: How far do they differ in style? Scientometrics, 57(3), 389-398.

[16]

Katan, S., Grierson, M., & Fiebrink, R. (2015). Using interactive machine learning to support interface development through workshops with disabled people. In Proceedings of the 33rd annual ACM conference on human factors in computing systems (pp. 251-254). ACM.

Digital Library

[17]

Kitchenham, B. A., Brereton, O. P., Owen, S., Butcher, J., & Jefferies, C. (2008). Length and readability of structured software engineering abstracts. IET software, 2(1), 37-45.

[18]

Knauff, M., & Nejasmic, J. (2014). An efficiency comparison of document preparation systems used in academic research and development. PloS One, 9(12), e115,069.

[19]

Landay, J. (2009). I give up on CHI/UIST. Blog entry. Retrieved November 10, 2009.

[20]

Lee, J., Vicente, K., Cassano, A., & Shearer, A. (2003). Can scientific impact be judged prospectively? A bibliometric test of simonton' s model of creative productivity. Scientometrics, 56(2), 223-232.

[21]

Lee, S., & French, N. (2011). The readability of academic papers in the journal of property investment & finance. Journal of Property Investment & Finance, 29(6), 693-704.

[22]

Lei, L. (2016). When science meets cluttered writing: adjectives and adverbs in academia revisited. Scientometrics, 107(3), 1361-1372.

Digital Library

[23]

Lei, L., & Yan, S. (2016). Readability and citations in information science: Evidence from abstracts and articles of four journals (2003-2012). Scientometrics, 108(3), 1155-1169.

Digital Library

[24]

Luo, Y., & Vogel, D. (2014). Crossing-based selection with direct touch input. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 2627-2636). ACM.

Digital Library

[25]

Mick, D. (2005). Inklings: From mind to page in consumer research. In Association of consumer research newsletter (pp. 1-3).

[26]

Mittal, H., & Gupta, P. (2011). Fate of award winning papers at annual conference of Indian academy of pediatrics: A 13 years experience. Indian Pediatrics, 48, 818-819. https://www.indianpediatrics.net/oct2011/oct-818-819.htm.

[27]

Mubin, O., Al Mahmud, A., & Ahmad, M. (2017). HCI down under: Reflecting on a decade of the OzCHI conference. Scientometrics, 112(1), 367-382.

Digital Library

[28]

Nacke, L. E. (2017). How to write and review chi papers. In Proceedings of the 2017 CHI conference extended abstracts on human factors in computing systems (pp. 1228-1231). ACM.

Digital Library

[29]

Okulicz-Kozaryn, A. (2013). Cluttered writing: Adjectives and adverbs in academia. Scientometrics, 96(3), 679-681.

Digital Library

[30]

Olsen, Jr D. R. (2007). Evaluating user interface systems research. In Proceedings of the 20th annual ACM symposium on user interface software and technology (pp 251-258). ACM.

Digital Library

[31]

Plavén-Sigray, P., Matheson, G. J., Schiffler, B. C., & Thompson, W. H. (2017). The readability of scientific texts is decreasing over time. bioRxiv 119370.

[32]

pypdf2. (2016). The PdfFileReader class. https://pythonhosted.org/PyPDF2/PdfFileReader.html. Accessed 15 Feb 2018.

[33]

Sawyer, A. G., Laran, J., & Xu, J. (2008). The readability of marketing journals: Are award-winning articles better written? Journal of Marketing, 72(1), 108-117.

[34]

Wainer, J., Eckmann, M., & Rocha, A. (2015). Peer-selected best papers are they really that good? PloS One, 10(3), e0118,446.

[35]

Wobbrock, J. O., & Kientz, J. A. (2016). Research contributions in human-computer interaction. Interactions, 23(3), 38-44.

Digital Library

[36]

Young, A. (2016). Reflections on the games & play virtual committee meeting. http://sigchi.tumblr.com/post/143440663545/reflections-on-the-games-play-virtual-committee. Accessed 15 Feb 2018.

[37]

Zhang, Y., & Hornof, A. J. (2014). Understanding multitasking through parallelized strategy exploration and individualized cognitive modeling. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (pp. 3885-3894). ACM.

Digital Library

[38]

Zimmerman, J. L. (1989). Improving a manuscript's readability and likelihood of publication. Issues in Accounting Education, 4(2), 458-466.

Cited By

Wang Y(2024)Comparison of citation impact between pre-and post-publication peer-selected best papers: the case of Best Paper Awards recipients at computer science conferencesScientometrics10.1007/s11192-023-04881-5129:1(641-662)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11192-023-04881-5
Wen JLei L(2022)Adjectives and adverbs in life sciences across 50 years: implications for emotions and readability in academic textsScientometrics10.1007/s11192-022-04453-z127:8(4731-4749)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1007/s11192-022-04453-z
McKay DBuchanan G(2021)…and That's What Gets Results: HCI Methods in OzCHI PublicationsProceedings of the 33rd Australian Conference on Human-Computer Interaction10.1145/3520495.3520497(144-152)Online publication date: 30-Nov-2021
https://dl.acm.org/doi/10.1145/3520495.3520497
Show More Cited By

Recommendations

SIGCSE top ten symposium papers of all time award winners

The SIGCSE Board created the SIGCSE Top Ten Symposium Papers of all Time Award to commemorate the 50th SIGCSE Technical Symposium held in Minneapolis, MN February 27-March 2, 2019. This award was created to recognize the outstanding papers published in ...
Outstanding Service Award
SIGGRAPH '18: ACM SIGGRAPH 2018 Awards

For his long-term, visionary, and dedicated service ACM SIGGRAPH recognizes Scott Owen with the 2018 Outstanding Service Award. He has been a Director on the Executive Committee, SIGGRAPH Conference Chair, SIGGRAPH Conference Advisory Group (CAG) Chair, ...
SIGCSE top ten symposium papers of all time award

Following approval by the ACM SIG Governing Board and then the ACM Awards Committee, the SIGCSE Board is pleased to announce the creation of an ACM approved SIG award to commemorate the 50th SIGCSE Technical Symposium to be held in Minneapolis, MN ...

Comments

Information & Contributors

Information

Published In

cover image Scientometrics

Scientometrics Volume 116, Issue 2

August 2018

724 pages

ISSN:0138-9130

Issue’s Table of Contents

Copyright © Copyright © 2018 Akadémiai Kiadó, Budapest, Hungary.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 August 2018

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang Y(2024)Comparison of citation impact between pre-and post-publication peer-selected best papers: the case of Best Paper Awards recipients at computer science conferencesScientometrics10.1007/s11192-023-04881-5129:1(641-662)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11192-023-04881-5
Wen JLei L(2022)Adjectives and adverbs in life sciences across 50 years: implications for emotions and readability in academic textsScientometrics10.1007/s11192-022-04453-z127:8(4731-4749)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1007/s11192-022-04453-z
McKay DBuchanan G(2021)…and That's What Gets Results: HCI Methods in OzCHI PublicationsProceedings of the 33rd Australian Conference on Human-Computer Interaction10.1145/3520495.3520497(144-152)Online publication date: 30-Nov-2021
https://dl.acm.org/doi/10.1145/3520495.3520497
Sandnes F(2021)A bibliometric study of human–computer interaction research activity in the Nordic-Baltic Eight countriesScientometrics10.1007/s11192-021-03940-z126:6(4733-4767)Online publication date: 1-Jun-2021
https://dl.acm.org/doi/10.1007/s11192-021-03940-z
Chen YWang SYang L(2021)Characterizing Award-Winning Papers in Library and Information Science (LIS): A Case Study of LIS Journals Published by Emerald PublishingDiversity, Divergence, Dialogue10.1007/978-3-030-71305-8_35(413-428)Online publication date: 17-Mar-2021
https://dl.acm.org/doi/10.1007/978-3-030-71305-8_35

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents