research-article

Characterizing user behaviors in open-source software user forums: an empirical study

Authors:

Jazlyn Hellman,

Md. Sami Uddin,

Jin L. C. GuoAuthors Info & Claims

CHASE '22: Proceedings of the 15th International Conference on Cooperative and Human Aspects of Software Engineering

Pages 46 - 55

https://doi.org/10.1145/3528579.3529178

Published: 19 July 2022 Publication History

Abstract

User forums of Open Source Software (OSS) enable end-users to collaboratively discuss problems concerning the OSS applications. Despite decades of research on OSS, we know very little about how end-users engage with OSS communities on these forums, in particular, the challenges that hinder their continuous and meaningful participation in the OSS community. Many previous works are developer-centric and overlook the importance of end-user forums. As a result, end-users' expectations are seldom reflected in OSS development. To better understand user behaviors in OSS user forums, we carried out an empirical study analyzing about 1.3 million posts from user forums of four popular OSS applications: Zotero, Audacity, VLC, and RStudio. Through analyzing the contribution patterns of three common user types (end-users, developers, and organizers), we observed that end-users not only initiated most of the threads (above 96% of threads in three projects, 86% in the other), but also acted as the significant contributors for responding to other users' posts, even though they tended to lack confidence in their activities as indicated by psycho-linguistic analyses. Moreover, we found end-users more open, reflecting a more positive emotion in communication than organizers and developers in the forums. Our work contributes new knowledge about end-users' activities and behaviors in OSS user forums that the vital OSS stakeholders can leverage to improve end-user engagement in the OSS development process.

References

[1]

Erin O'Carroll Bantum and Jason E Owen. 2009. Evaluating the validity of computerized content analysis programs for identification of emotional expression in cancer narratives. Psychological assessment 21, 1 (2009), 79.

[2]

Hind Benbya. 2016. Successful OSS Project Design and Implementation: Requirements, Tools, Social Designs and Reward Structures. CRC Press.

[3]

Jacopo Biggiogera, George Boateng, Peter Hilpert, Matthew Vowels, Guy Bodenmann, Mona Neysari, Fridtjof Nussbeck, and Tobias Kowatsch. 2021. BERT Meets LIWC: Exploring State-of-the-Art Language Models for Predicting Communication Behavior in Couples' Conflict Interactions. In Companion Publication of the 2021 International Conference on Multimodal Interaction (Montreal, QC, Canada) (ICMI '21 Companion). Association for Computing Machinery, New York, NY, USA, 385--389.

Digital Library

[4]

Fabio Calefato, Giuseppe Iaffaldano, Filippo Lanubile, and Bogdan Vasilescu. 2018. On Developers' Personality in Large-Scale Distributed Projects: The Case of the Apache Ecosystem. In Proceedings of the 13th International Conference on Global Software Engineering (Gothenburg, Sweden) (ICGSE '18). Association for Computing Machinery, New York, NY, USA, 92--101.

Digital Library

[5]

Jinghui Cheng and Jin L.C. Guo. 2018. How Do the Open Source Communities Address Usability and UX Issues? An Exploratory Study. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA '18). Association for Computing Machinery, New York, NY, USA, 1--6.

Digital Library

[6]

Michael A. Cohn, Matthias R. Mehl, and James W. Pennebaker. 2004. Linguistic Markers of Psychological Change Surrounding September 11, 2001. Psychological Science 15, 10 (2004), 687--693.

[7]

Cristian Danescu-Niculescu-Mizil, Michael Gamon, and Susan Dumais. 2011. Mark My Words! Linguistic Style Accommodation in Social Media. In Proceedings of the 20th International Conference on World Wide Web (Hyderabad, India) (WWW '11). Association for Computing Machinery, New York, NY, USA, 745--754.

Digital Library

[8]

Fabian Fagerholm, Alejandro S. Guinea, Jürgen Münch, and Jay Borenstein. 2014. The Role of Mentoring and Project Characteristics for Onboarding in Open Source Software Projects. In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (Torino, Italy) (ESEM '14). Association for Computing Machinery, New York, NY, USA, Article 55, 10 pages.

Digital Library

[9]

Jazlyn Hellman, Jinghui Cheng, and Jin L.C. Guo. 2021. Facilitating Asynchronous Participatory Design of Open Source Software: Bringing End Users into the Loop. Association for Computing Machinery, New York, NY, USA.

Digital Library

[10]

Michael Heron, Vicki L Hanson, and Ian Ricketts. 2013. Open source and accessibility: advantages and limitations. Journal of interaction Science 1, 1 (2013), 1--10.

[11]

Edward Ivanovic. 2005. Dialogue Act Tagging for Instant Messaging Chat Sessions. In Proceedings of the ACL Student Research Workshop (Ann Arbor, Michigan) (ACLstudent '05). Association for Computational Linguistics, USA, 79--84.

Digital Library

[12]

Minwoo Jeong, Chin-Yew Lin, and Gary Geunbae Lee. 2009. Semi-Supervised Speech Act Recognition in Emails and Forums. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume3 - Volume 3 (EMNLP '09). Association for Computational Linguistics, USA, 1250--1259.

Digital Library

[13]

Ewa Kacewicz, James W. Pennebaker, Matthew Davis, Moongee Jeon, and Arthur C. Graesser. 2014. Pronoun Use Reflects Standings in Social Hierarchies. Journal of Language and Social Psychology 33, 2 (2014), 125--143.

[14]

Georgi M. Kanchev, Pradeep K. Murukannaiah, Amit K. Chopra, and Pete Sawyer. 2017. Canary: Extracting Requirements-Related Information from Online Discussions. In 2017 IEEE 25th International Requirements Engineering Conference (RE). 31--40.

[15]

Su Nam Kim, Li Wang, and Timothy Baldwin. 2010. Tagging and Linking Web Forum Posts. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning (Uppsala, Sweden) (CoNLL '10). Association for Computational Linguistics, USA, 192--202.

Digital Library

[16]

Amy J. Ko and Parmit K. Chilana. 2010. How Power Users Help and Hinder Open Bug Reporting. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI'10). Association for Computing Machinery, New York, NY, USA, 1665--1674.

Digital Library

[17]

Cliff Lampe, Rebecca Gray, Andrew T. Fiore, and Nicole Ellison. 2014. Help is on the Way: Patterns of Responses to Resource Requests on Facebook. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (Baltimore, Maryland, USA) (CSCW '14). Association for Computing Machinery, New York, NY, USA, 3--15.

Digital Library

[18]

Georg J.P. Link. 2017. The Value of Engaging with Open Source Communities. In Proceedings of the 13th International Symposium on Open Collaboration Companion (Galway, Ireland) (OpenSym '17). Association for Computing Machinery, New York, NY, USA, Article 6, 5 pages.

Digital Library

[19]

Itzel Morales-Ramirez, Anna Perini, and Mariano Ceccato. 2015. Towards Supporting the Analysis of Online Discussions in OSS Communities: A Speech-Act Based Approach. In Information Systems Engineering in Complex Environments, Selmin Nurcan and Elias Pimenidis (Eds.). Springer International Publishing, Cham, 215--232.

[20]

Iraklis Moutidis and Hywel T. P. Williams. 2021. Community evolution on Stack Overflow. PLOS ONE 16, 6 (06 2021), 1--23.

[21]

Matthew L. Newman, James W. Pennebaker, Diane S. Berry, and Jane M. Richards. 2003. Lying Words: Predicting Deception from Linguistic Styles. Personality and Social Psychology Bulletin 29, 5 (2003), 665--675.

[22]

Jakob Nielsen. 1994. Usability Engineering. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

Digital Library

[23]

Yusuf Sulistyo Nugroho, Syful Islam, Keitaro Nakasai, Ifraz Rehman, Hideaki Hata, Raula Gaikovina Kula, Meiyappan Nagappan, and Kenichi Matsumoto. 2021. How are project-specific forums utilized? A study of participation, content, and sentiment in the Eclipse ecosystem. Empirical Software Engineering 26, 6 (2021), 132.

Digital Library

[24]

Shereen Oraby, Mansurul Bhuiyan, Pritam Gundecha, Jalal Mahmud, and Rama Akkiraju. 2019. Modeling and Computational Characterization of Twitter Customer Service Conversations. ACM Trans. Interact. Intell. Syst. 9, 2--3, Article 18 (mar 2019), 28 pages.

Digital Library

[25]

Shereen Oraby, Pritam Gundecha, Jalal Mahmud, Mansurul Bhuiyan, and Rama Akkiraju. 2017. "How May I Help You?": Modeling Twitter Customer Service-Conversations Using Fine-Grained Dialogue Acts. In Proceedings of the 22nd International Conference on Intelligent User Interfaces (Limassol, Cyprus) (IUI '17). Association for Computing Machinery, New York, NY, USA, 343--355.

Digital Library

[26]

James W. Pennebaker, Rojer J. Booth, Ryan L. Boyd, and Martha E. Francis. 2015. Linguistic Inquiry and Word Count: LIWC2015 Operator's Manual. https://s3-us-west-2.amazonaws.com/downloads.liwc.net/LIWC2015_OperatorManual.pdf

[27]

James W. Pennebaker, Ryan L. Boyd, Kayla Jordan, and Kate Blackburn. 2015. The Development and Psychometric Properties of LIWC2015. University of Texas at Austin.

[28]

James W. Pennebaker, Cindy K. Chung, Joey Frazee, Gary M. Lavergne, and David I. Beaver. 2015. When Small Words Foretell Academic Success: The Case of College Admissions Essays. PLOS ONE 9, 12 (12 2015), 1--10.

[29]

Huilian Sophie Qiu, Alexander Nolte, Anita Brown, Alexander Serebrenik, and Bogdan Vasilescu. 2019. Going Farther Together: The Impact of Social Capital on Sustained Participation in Open Source. In Proceedings of the 41st International Conference on Software Engineering (Montreal, Quebec, Canada) (ICSE '19). IEEE Press, 688--699.

Digital Library

[30]

Louis M Rea and Richard A Parker. 2014. Designing and conducting survey research: A comprehensive guide. John Wiley & Sons.

[31]

Peter C. Rigby and Ahmed E. Hassan. 2007. What Can OSS Mailing Lists Tell Us? A Preliminary Psychometric Text Analysis of the Apache Developer Mailing List. In Proceedings of the Fourth International Workshop on Mining Software Repositories (MSR '07). IEEE Computer Society, USA, 23.

Digital Library

[32]

Melanie Stade, Marc Oriol, Oscar Cabrera, Farnaz Fotrousi, Ronnie Schaniel, Norbert Seyff, and Oleg Schmidt. 2017. Providing a User Forum is not enough: First Experiences of a Software Company with CrowdRE. In 2017 IEEE 25th International Requirements Engineering Conference Workshops (REW). 164--169.

[33]

Yla R. Tausczik and James W. Pennebaker. 2009. The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods. Journal of Language and Social Psychology 29, 1 (Dec. 2009), 24--54.

[34]

Michael Terry, Matthew Kay, and Ben Lafreniere. 2010. Perceptions and Practices of Usability in the Free/Open Source Software (FoSS) Community. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI '10). Association for Computing Machinery, New York, NY, USA, 999--1008.

Digital Library

[35]

Frenk C.J. van Mil, Ayushi Rastogi, and Andy Zaidman. 2021. Promises and Perils of Inferring Personality on GitHub. In Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). Association for Computing Machinery, New York, NY, USA, Article 17, 11 pages.

Digital Library

[36]

Shaowei Wang, Tse-Hsun Chen, and Ahmed E. Hassan. 2018. Understanding the Factors for Fast Answers in Technical Q&A Websites. Empirical Softw. Engg. 23, 3 (jun 2018), 1552--1593.

Digital Library

[37]

Wenting Wang, Deeksha Arya, Nicole Novielli, Jinghui Cheng, and Jin L.C. Guo. 2020. ArguLens: Anatomy of Community Opinions On Usability Issues Using Argumentation Models. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--14.

Digital Library

[38]

Wenting Wang, Jinghui Cheng, and Jin L.C. Guo. 2022. How Do Open Source Software Contributors Perceive and Address Usability?: Valued Factors, Practices, and Challenges. IEEE Software 39, 1 (2022), 76--83.

Digital Library

[39]

Bei Yu. 2011. The Emotional World of Health Online Communities. In Proceedings of the 2011 IConference (Seattle, Washington, USA) (iConference '11). Association for Computing Machinery, New York, NY, USA, 806--807.

Digital Library

[40]

Amy Zhang, Bryan Culbertson, and Praveen Paritosh. 2017. Characterizing Online Discussion Using Coarse Discourse Sequences. Proceedings of the International AAAI Conference on Web and Social Media 11, 1 (May 2017), 357--366.

[41]

Thomas Zhang, Jason H. D. Cho, and Chengxiang Zhai. 2014. Understanding User Intents in Online Health Forums. In Proceedings of the 5th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics (BCB '14). Association for Computing Machinery, New York, NY, USA, 220--229.

Digital Library

Cited By

Kuutila MRantala LLi JHosio SMäntylä M(2024)What Makes Programmers Laugh? Exploring the Submissions of the Subreddit r/ProgrammerHumor.Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3686696(371-381)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3674805.3686696
Jahn LEngelbutzeder PRandall DBollmann YNtouros VMichel LWulf V(2024)In Between Users and Developers: Serendipitous Connections and Intermediaries in Volunteer-Driven Open-Source Software DevelopmentProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642541(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642541
Giwangkoro GNugroho Y(2024)Unveiling Research Trends in Stack Overflow: A Comprehensive Analysis of General Discussion Theme2024 International Conference on Smart Computing, IoT and Machine Learning (SIML)10.1109/SIML61815.2024.10578280(130-136)Online publication date: 6-Jun-2024
https://doi.org/10.1109/SIML61815.2024.10578280

Index Terms

Characterizing user behaviors in open-source software user forums: an empirical study

Recommendations

Characterizing women (not) contributing to open-source
GE '19: Proceedings of the 2nd International Workshop on Gender Equality in Software Engineering

Women are under-represented not only in software development, but even more so in the Open-Source Software (OSS) community. In this study we examine whether there are differences between women in OSS community and outside of it. Identifying these ...
User-Generated Open Source Products: Founder's Social Capital and Time to Product Release

Volunteer users employ collaborative Internet technologies to develop open source products, a form of user-generated content, where time to product release is a crucial measure of project success. The open source community features two separate but ...
Integrating user experience into free/libre open source software: CHI 2009 special interest group
CHI EA '09: CHI '09 Extended Abstracts on Human Factors in Computing Systems

The importance of software in daily life for casual and business purposes has led to a strong increase in the formal integration of usability in commercial software development processes. However, usability still appears to be largely an afterthought ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHASE '22: Proceedings of the 15th International Conference on Cooperative and Human Aspects of Software Engineering

May 2022

122 pages

ISBN:9781450393423

DOI:10.1145/3528579

General Chair:
Maria Teresa Baldassarre
University of Bari, Italy
,
Program Chairs:
Lutz Prechelt
Freie Universität Berlin, Germany
,
Gema Rodríguez-Pérez
University of British Columbia (UBC), Canada

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

ALFRED P. SLOAN FOUNDATION

Conference

ICSE '22

Sponsor:

SIGSOFT

ICSE '22: 44th International Conference on Software Engineering

May 21 - 29, 2022

Pennsylvania, Pittsburgh

Acceptance Rates

Overall Acceptance Rate 47 of 70 submissions, 67%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
167
Total Downloads

Downloads (Last 12 months)45
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kuutila MRantala LLi JHosio SMäntylä M(2024)What Makes Programmers Laugh? Exploring the Submissions of the Subreddit r/ProgrammerHumor.Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3686696(371-381)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3674805.3686696
Jahn LEngelbutzeder PRandall DBollmann YNtouros VMichel LWulf V(2024)In Between Users and Developers: Serendipitous Connections and Intermediaries in Volunteer-Driven Open-Source Software DevelopmentProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642541(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642541
Giwangkoro GNugroho Y(2024)Unveiling Research Trends in Stack Overflow: A Comprehensive Analysis of General Discussion Theme2024 International Conference on Smart Computing, IoT and Machine Learning (SIML)10.1109/SIML61815.2024.10578280(130-136)Online publication date: 6-Jun-2024
https://doi.org/10.1109/SIML61815.2024.10578280

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten