Abstract
To maintain development consciousness, simplify project coordination, and prevent misinterpretation, communication is essential for software development teams. Instant private messaging, group chats, and sharing code are just a few of the capabilities that chat rooms provide to assist and meet the communication demands of software development teams. All of this is capacitated to happen in real-time. Consequently, chat rooms have gained popularity among developers. Gitter is one of these platforms that has gained popularity, and the conversations it contains may be a treasure trove of data for academics researching open-source software systems. This research made use of the GitterCom dataset, The largest collection of Gitter developer messages that have been carefully labelled and curated and perform multi-label classification for the ’Purpose’ category in the dataset. An extensive empirical analysis is performed on 6 feature selection techniques, 14 machine learning classifiers, and BERT transformer layer architecture with layer-by-layer comparison. Consequently, we achieve proficient results through our research pipeline involving Extra Trees Classifier and Random Forest classifiers with AUC (OvR) median performance of 0.94 and 0.92 respectively. Furthermore, The research proposed research pipeline could be utilized for generic multi-label text classification on software developer forum text data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Parra, E., Ellis, A., Haiduc, S.: Gittercom: a dataset of open source developer communications in gitter. In: Proceedings of the 17th International Conference on Mining Software Repositories, pp. 563–567 (2020)
Sun, C., Qiu, X., Xu, Y., Huang, X.: How to fine-tune BERT for text classification? In: Sun, M., Huang, X., Ji, H., Liu, Z., Liu, Y. (eds.) CCL 2019. LNCS (LNAI), vol. 11856, pp. 194–206. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32381-3_16
Lu, Z., Du, P., Nie, J.-Y.: VGCN-BERT: augmenting BERT with graph embedding for text classification. In: Jose, J.M., et al. (eds.) ECIR 2020. LNCS, vol. 12035, pp. 369–382. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45439-5_25
Li, W., Gao, S., Zhou, H., Huang, Z., Zhang, K., Li, W.: The automatic text classification method based on bert and feature union. In: 2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS), pp. 774–777. IEEE (2019)
Shi, L., et al.: A first look at developers’ live chat on gitter. In: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 391–403 (2021)
Parra, E., Alahmadi, M., Ellis, A., Haiduc, S.: A comparative study and analysis of developer communications on slack and gitter. Empir. Softw. Eng. 27(2), 40 (2022)
El Mezouar, M., da Costa, D.A., German, D.M., Zou, Y.: Exploring the use of chatrooms by developers: an empirical study on slack and gitter. IEEE Trans. Softw. Eng. 48(10), 3988–4001 (2021)
Lin, B., Zagalsky, A., Storey, M.-A., Serebrenik, A.: Why developers are slacking off: understanding how software teams use slack. In: Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion, pp. 333–336 (2016)
Dash, M., Liu, H.: Feature selection for classification. Intell. Data Anal. 1(1–4), 131–156 (1997)
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Vaswani, A., et al.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)
Woolson, R.F.: Wilcoxon signed-rank test. In: Wiley Encyclopedia of Clinical Trials, pp. 1–3 (2007)
Zimmerman, D.W., Zumbo, B.D.: Relative power of the Wilcoxon test, the Friedman test, and repeated-measures Anova on ranks. J. Exp. Educ. 62(1), 75–86 (1993)
Al-Aidaroos, K.M., Bakar, A.A., Othman, Z.: Naive bayes variants in classification learning. In: 2010 International Conference on Information Retrieval and Knowledge Management (CAMP), pp. 276–281. IEEE (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Akash, B.S., Kumar, L., Singh, V., Patel, A.K., Krishna, A. (2024). Empirical Analysis of Multi-label Classification on GitterCom Using BERT and ML Classifiers. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14451. Springer, Singapore. https://doi.org/10.1007/978-981-99-8073-4_19
Download citation
DOI: https://doi.org/10.1007/978-981-99-8073-4_19
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8072-7
Online ISBN: 978-981-99-8073-4
eBook Packages: Computer ScienceComputer Science (R0)