research-article

Open access

Toward interactive bug reporting for (android app) end-users

Authors:

Junayed Mahmud,

Oscar Chaparro,

Andrian Marcus,

Denys PoshyvanykAuthors Info & Claims

ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Pages 344 - 356

https://doi.org/10.1145/3540250.3549131

Published: 09 November 2022 Publication History

Abstract

Many software bugs are reported manually, particularly bugs that manifest themselves visually in the user interface. End-users typically report these bugs via app reviewing websites, issue trackers, or in-app built-in bug reporting tools, if available. While these systems have various features that facilitate bug reporting (e.g., textual templates or forms), they often provide limited guidance, concrete feedback, or quality verification to end-users, who are often inexperienced at reporting bugs and submit low-quality bug reports that lead to excessive developer effort in bug report management tasks. We propose an interactive bug reporting system for end-users (Burt), implemented as a task-oriented chatbot. Unlike existing bug reporting systems, Burt provides guided reporting of essential bug report elements (i.e., the observed behavior, expected behavior, and steps to reproduce the bug), instant quality verification, and graphical suggestions for these elements. We implemented a version of Burt for Android and conducted an empirical evaluation study with end-users, who reported 12 bugs from six Android apps studied in prior work. The reporters found that Burt’s guidance and automated suggestions/clarifications are useful and Burt is easy to use. We found that Burt reports contain higher-quality information than reports collected via a template-based bug reporting system. Improvements to Burt, informed by the reporters, include support for various wordings to describe bug report elements and improved quality verification. Our work marks an important paradigm shift from static to interactive bug reporting for end-users.

References

[1]

2021. Android Token. https://f-droid.org/en/packages/uk.co.bitethebullet.android.token/

[2]

2021. Android’s Layout and Layout Validation. https://developer.android.com/studio/debug/layout-inspector

[3]

2021. AntennaPod. https://play.google.com/store/apps/details?id=de.danoeh.antennapod&hl=en_US&gl=US

[4]

2021. App Store: Ratings, Reviews, and Responses. https://developer.apple.com/app-store/ratings-and-reviews/

[5]

2021. Appetize.io. https://appetize.io/

[6]

2021. Droid Weight. https://fossdroid.com/a/droidweight.html

[7]

2021. GitHub Issue Tracker -. https://github.com/features

[8]

2021. GnuCash. https://play.google.com/store/apps/details?id=org.gnucash.android&hl=en_US&gl=US

[9]

2021. GrowTracker. https://f-droid.org/en/packages/me.anon.grow/

[10]

2021. JIRA Bug Reporting System -. https://www.atlassian.com/software/jira

[11]

2021. Microsoft Azure Chatbot Architecture. https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/ai/conversational-bot

[12]

2021. Mileage. https://fossdroid.com/a/mileage.html

[13]

2021. An open letter to GitHub from the maintainers of open source projects. https://github.com/dear-github/dear-github

[14]

2021. React Chatbot Kit. https://fredrikoseberg.github.io/react-chatbot-kit-docs/

[15]

2021. Report an issue or send feedback on Chrome. https://support.google.com/chrome/answer/95315

[16]

2021. Spring Boot. https://spring.io/projects/spring-boot

[17]

2021. Stanford Dependencies. https://nlp.stanford.edu/software/stanford-dependencies.html

[18]

2021. A Time Tracker. https://f-droid.org/en/packages/com.markuspage.android.atimetracker/

[19]

2021. https://www.bugsee.com

[20]

2021. Write a review on Google Play. https://support.google.com/googleplay/answer/4346705

[21]

2022. Online replication package. https://doi.org/10.5281/zenodo.6977413

Digital Library

[22]

2022. Qualtrics: Survey Software. https://www.qualtrics.com/core-xm/survey-software/

[23]

Carlos Bernal-Cárdenas, Nathan Cooper, Madeleine Havranek, Kevin Moran, Oscar Chaparro, Denys Poshyvanyk, and Andrian Marcus. 2022. Translating Video Recordings of Complex Mobile App UI Gestures Into Replayable Scenarios. IEEE Transactions on Software Engineering, to appear.

Digital Library

[24]

Carlos Bernal-Cárdenas, Nathan Cooper, Kevin Moran, Oscar Chaparro, Andrian Marcus, and Denys Poshyvanyk. 2020. Translating video recordings of mobile app usages into replayable scenarios. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (ICSE’20). 309–321.

Digital Library

[25]

Nicolas Bettenburg, Sascha Just, Adrian Schröter, Cathrin Weiss, Rahul Premraj, and Thomas Zimmermann. 2008. What Makes a Good Bug Report? In Proceedings of the 16th International Symposium on the Foundations of Software Engineering (FSE’08). 308–318.

Digital Library

[26]

Aaditya Bhatia, Shaowei Wang, Muhammad Asaduzzaman, and Ahmed E Hassan. 2022. A Study of Bug Management Using the Stack Exchange Question and Answering Platform. IEEE Transactions on Software Engineering, 48, 2 (2022), 502–518.

[27]

Silvia Breu, Rahul Premraj, Jonathan Sillito, and Thomas Zimmermann. 2010. Information Needs in Bug Reports: Improving Cooperation Between Developers and Users. In Proceedings of the Conference on Computer Supported Cooperative Work (CSCW’10). 301–310.

Digital Library

[28]

Oscar Chaparro, Carlos Bernal-Cárdenas, Jing Lu, Kevin Moran, Andrian Marcus, Massimiliano Di Penta, Denys Poshyvanyk, and Vincent Ng. 2019. Assessing the Quality of the Steps to Reproduce in Bug Reports. In Proceedings of the 27th ACM Joint Meeting on the Foundations of Software Engineering (ESEC/FSE’19). 86–96.

Digital Library

[29]

Oscar Chaparro, Jing Lu, Fiorella Zampetti, Laura Moreno, Massimiliano Di Penta, Andrian Marcus, Gabriele Bavota, and Vincent Ng. 2017. Detecting Missing Information in Bug Descriptions. In Proceedings of the 11th Joint Meeting on the Foundations of Software Engineering (ESEC/FSE’17). 396–407.

Digital Library

[30]

Norman Cliff. 2014. Ordinal methods for behavioral data analysis. Psychology Press.

[31]

Nathan Cooper, Carlos Bernal-Cárdenas, Oscar Chaparro, Kevin Moran, and Denys Poshyvanyk. 2021. It Takes Two to Tango: Combining Visual and Textual Information for Detecting Duplicate Video-Based Bug Reports. In Proceedings of the 43rd IEEE/ACM International Conference on Software Engineering (ICSE’21). 160–161.

[32]

Steven Davies and Marc Roper. 2014. What’s in a bug report? In Proceedings of the 8th International Symposium on Empirical Software Engineering and Measurement (ESEM’14). 26:1–26:10.

Digital Library

[33]

Mona Erfani Joorabchi, Mehdi Mirzaaghaei, and Ali Mesbah. 2014. Works for Me! Characterizing Non-reproducible Bug Reports. In Proceedings of the Working Conference on Mining Software Repositories (MSR’14). 62–71.

Digital Library

[34]

Mattia Fazzini, Kevin Patrick Moran, Carlos Bernal-Cardenas, Tyler Wendland, Alessandro Orso, and Denys Poshyvanyk. 2022. Enhancing Mobile App Bug Reporting via Real-time Understanding of Reproduction Steps. IEEE Transactions on Software Engineering, to appear.

Digital Library

[35]

Mattia Fazzini, Martin Prammer, Marcelo d’Amorim, and Alessandro Orso. 2018. Automatically translating bug reports into test cases for mobile apps. In Proceedings of the 27th International Symposium on Software Testing and Analysis (ISSTA’18). 141–152.

Digital Library

[36]

Jianfeng Gao, Michel Galley, and Lihong Li. 2019. Neural Approaches to Conversational AI. Foundations and Trends in Information Retrieval, 13, 2-3 (2019), 127–298. issn:1554-0669

Digital Library

[37]

Lorenzo Gomez, Iulian Neamtiu, Tanzirul Azim, and Todd Millstein. 2013. RERAN: Timing- and touch-sensitive record and replay for Android. In Proceedings of the 35th International Conference on Software Engineering (ICSE’13). 72–81.

[38]

Philip J. Guo, Thomas Zimmermann, Nachiappan Nagappan, and Brendan Murphy. 2010. Characterizing and predicting which bugs get fixed: an empirical study of Microsoft Windows. In Proceedings of the 32nd International Conference on Software Engineering (ICSE’10). 495–504.

Digital Library

[39]

Melita Hajdinjak and France Mihelič. 2006. The PARADISE evaluation framework: Issues and findings. Computational Linguistics, 32, 2 (2006), 263–272.

Digital Library

[40]

Madeleine Havranek, Carlos Bernal-Cárdenas, Nathan Cooper, Oscar Chaparro, Denys Poshyvanyk, and Kevin Moran. 2021. V2S: a tool for translating video recordings of mobile app usages into replayable scenarios. In Proceedings of the IEEE/ACM 43rd International Conference on Software Engineering (ICSE’21). 65–68.

Digital Library

[41]

Myles Hollander, Douglas A Wolfe, and Eric Chicken. 2013. Nonparametric statistical methods. John Wiley & Sons.

[42]

Pieter Hooimeijer and Westley Weimer. 2007. Modeling Bug Report Quality. In Proceedings of the 22nd International Conference on Automated Software Engineering (ASE’07). 34–43.

Digital Library

[43]

Yongjian Hu, Tanzirul Azim, and Iulian Neamtiu. 2015. Versatile yet lightweight record-and-replay for android. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA’15). 349–366.

Digital Library

[44]

Da Huo, Tao Ding, Collin McMillan, and Malcom Gethers. 2014. An empirical study of the effects of expert knowledge on bug reports. In Proceedings of the International Conference on Software Maintenance and Evolution (ICSME’14). 1–10.

Digital Library

[45]

Mia Mohammad Imran, Agnieszka Ciborowska, and Kostadin Damevski. 2021. Automatically Selecting Follow-up Questions for Deficient Bug Reports. In In proceedings of the 18th IEEE/ACM International Conference on Mining Software Repositories (MSR’18). 167–178.

[46]

Gün Karagöz and Hasan Sözer. 2017. Reproducing failures based on semiformal failure scenario descriptions. Software Quality Journal, 25, 1 (2017), 111–129.

Digital Library

[47]

Amy J. Ko and Brad A. Myers. 2008. Debugging Reinvented: Asking and Answering Why and Why Not Questions about Program Behavior. In Proceedings of the 30th International Conference on Software Engineering (ICSE’08). 301–310. isbn:9781605580791

[48]

Eero I. Laukkanen and Mika V. Mäntylä. 2011. Survey Reproduction of Defect Reporting in Industrial Software Development. In Proceedings of the International Symposium on Empirical Software Engineering and Measurement (ESEM’11). 197–206.

[49]

Hui Liu, Mingzhu Shen, Jiahao Jin, and Yanjie Jiang. 2020. Automated classification of actions in bug reports of mobile apps. In Proceedings of the 29th ACM International Symposium on Software Testing and Analysis (ISSTA’20). 128–140.

Digital Library

[50]

Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP Natural Language Processing Toolkit. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL’14). 55–60.

[51]

Kevin Moran, Richard Bonett, Carlos Bernal-Cárdenas, Brendan Otten, Daniel Park, and Denys Poshyvanyk. 2017. On-device bug reporting for android applications. In Proceedings of the IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft’17). 215–216.

Digital Library

[52]

Kevin Moran, Mario Linares-Váquez, Carlos Bernal-Cárdenas, Christopher Vendome, and Denys Poshyvanyk. 2016. Automatically Discovering, Reporting and Reproducing Android Application Crashes. In Proceedings of the International Conference on Software Testing, Verification and Validation (ICST’16). 33–44.

[53]

Kevin Moran, Mario Linares-Vásquez, Carlos Bernal-Cárdenas, and Denys Poshyvanyk. 2015. Auto-completing Bug Reports for Android Applications. In Proceedings of the Joint Meeting on Foundations of Software Engineering (FSE’15). 673–686.

Digital Library

[54]

Kevin Moran, Mario Linares-Vasquez, Carlos Bernal-Cardenas, Cristopher Vendome, and Denys Poshyvanyk. 2017. CrashScope: A Practical Tool for Automated Testing of Android Applications. In Proceedings of the 39th IEEE/ACM International Conference on Software Engineering (ICSE’17). 15–18.

Digital Library

[55]

Abraham Naftali Oppenheim. 1992. Questionnaire Design, Interviewing and Attitude Measurement. Pinter Publishers.

[56]

Zhengrui Qin, Yutao Tang, Ed Novak, and Qun Li. 2016. MobiPlay: A Remote Execution Based Record-and-replay Tool for Mobile Applications. In Proceedings of the 38th International Conference on Software Engineering (ICSE’16). 571–582.

Digital Library

[57]

Tommaso Dal Sasso, Andrea Mocci, and Michele Lanza. 2016. What Makes a Satisficing Bug Report? In Proceedings of the International Conference on Software Quality, Reliability and Security (QRS’16). 164–174.

[58]

Yang Song and Oscar Chaparro. 2020. BEE: a tool for structuring and analyzing bug reports. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE’21). 1551–1555.

Digital Library

[59]

Lin Tan, Chen Liu, Zhenmin Li, Xuanhui Wang, Yuanyuan Zhou, and Chengxiang Zhai. 2014. Bug characteristics in open source software. Empirical Software Engineering, 19, 6 (2014), 1665–1705. issn:1382-3256, 1573-7616

Digital Library

[60]

Marcelo Serrano Zanetti, Ingo Scholtes, Claudio Juan Tessone, and Frank Schweitzer. 2013. Categorizing Bugs with Social Networks: A Case Study on Four Open Source Software Communities. In Proceedings of the International Conference on Software Engineering (ICSE’13). 1032–1041.

Digital Library

[61]

Yu Zhao, Kye Miller, Tingting Yu, Wei Zheng, and Minchao Pu. 2019. Automatically extracting bug reproducing steps from android bug reports. In Proceedings of the International Conference on Software and Systems Reuse (ICSR’19). 100–111.

[62]

Yu Zhao, Ting Su, Yang Liu, Wei Zheng, Xiaoxue Wu, Ramakanth Kavuluru, William GJ Halfond, and Tingting Yu. 2022. ReCDroid+: Automated End-to-End Crash Reproduction from Bug Reports for Android Apps. ACM Transactions on Software Engineering and Methodology (TOSEM), 31, 3 (2022), 1–33.

Digital Library

[63]

Yu Zhao, Tingting Yu, Ting Su, Yang Liu, Wei Zheng, Jingzhi Zhang, and William G.J. Halfond. 2019. ReCDroid: Automatically Reproducing Android Application Crashes from Bug Reports. In Proceedings of the 41st IEEE/ACM International Conference on Software Engineering (ICSE’19). 128–139.

[64]

Thomas Zimmermann, Nachiappan Nagappan, Philip J. Guo, and Brendan Murphy. 2012. Characterizing and predicting which bugs get reopened. In Proceedings of the International Conference on Software Engineering (ICSE’12). 1074–1083.

Digital Library

[65]

Thomas Zimmermann, Rahul Premraj, Nicolas Bettenburg, Sascha Just, Adrian Schröter, and Cathrin Weiss. 2010. What Makes a Good Bug Report? IEEE Transactions on Software Engineering, 36, 5 (2010), 618–643. issn:0098-5589

Digital Library

[66]

Thomas Zimmermann, Rahul Premraj, Jonathan Sillito, and Silvia Breu. 2009. Improving bug tracking systems. In Proceedings of the 31st International Conference on Software Engineering (ICSE’09). 247–250.

Cited By

Chaves LOliveira FTiago Ld'Amorim M(2024)Automating Issue Reporting in Software Testing: Lessons Learned from Using the Template Generator ToolCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663847(278-282)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663847
Wang DZhao YFeng SZhang ZHalfond WChen CSun XShi JYu TChristakis MPradel M(2024)Feedback-Driven Automated Whole Bug Report Reproduction for Android AppsProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680341(1048-1060)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680341
Kallis RColavito GAl-Kaswan APascarella LChaparro ORani PIzadi MDi Sorbo APanichella S(2024)The NLBSE'24 Tool CompetitionProceedings of the Third ACM/IEEE International Workshop on NL-based Software Engineering10.1145/3643787.3648038(33-40)Online publication date: 20-Apr-2024
https://dl.acm.org/doi/10.1145/3643787.3648038
Show More Cited By

Index Terms

Toward interactive bug reporting for (android app) end-users
1. Human-centered computing
2. Software and its engineering
  1. Software creation and management
    1. Software post-development issues
    2. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software notations and tools
    1. Software maintenance tools

Index terms have been assigned to the content through auto-classification.

Recommendations

From Android Bug Reports to Android Bug Handling Process: An Empirical Study of Open-Source Development

Android is an operating system for mobile devices. Its development is led by Google and some other companies. Because of the open-source property of Android, anyone can report a bug through its online bug tracking system. In this paper, we analyze the ...
App review driven collaborative bug finding
Abstract
Software development teams generally welcome any effort to expose bugs in their code base. In this work, we build on the hypothesis that mobile apps from the same category (e.g., two web browser apps) may be affected by similar bugs in their ...
An Empirical Analysis of Bug Reports and Bug Fixing in Open Source Android Apps
CSMR '13: Proceedings of the 2013 17th European Conference on Software Maintenance and Reengineering

Smartphone platforms and applications (apps) have gained tremendous popularity recently. Due to the novelty of the smartphone platform and tools, and the low barrier to entry for app distribution, apps are prone to errors, which affects user experience ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 2022

1822 pages

ISBN:9781450394130

DOI:10.1145/3540250

General Chair:
Abhik Roychoudhury
National University of Singapore, Singapore
,
Program Chairs:
Cristian Cadar
Imperial College London, UK
,
Miryung Kim
University of California at Los Angeles, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tag

Software and its engineering → Software maintenance tools

Qualifiers

Research-article

Funding Sources

NSF (National Science Foundation)

Conference

ESEC/FSE '22

Sponsor:

ESEC/FSE '22: 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 14 - 18, 2022

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
644
Total Downloads

Downloads (Last 12 months)255
Downloads (Last 6 weeks)41

Reflects downloads up to 04 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chaves LOliveira FTiago Ld'Amorim M(2024)Automating Issue Reporting in Software Testing: Lessons Learned from Using the Template Generator ToolCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663847(278-282)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663847
Wang DZhao YFeng SZhang ZHalfond WChen CSun XShi JYu TChristakis MPradel M(2024)Feedback-Driven Automated Whole Bug Report Reproduction for Android AppsProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680341(1048-1060)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680341
Kallis RColavito GAl-Kaswan APascarella LChaparro ORani PIzadi MDi Sorbo APanichella S(2024)The NLBSE'24 Tool CompetitionProceedings of the Third ACM/IEEE International Workshop on NL-based Software Engineering10.1145/3643787.3648038(33-40)Online publication date: 20-Apr-2024
https://dl.acm.org/doi/10.1145/3643787.3648038
Mahmud JDe Silva NKhan SMostafavi SMansur SChaparro OMarcus AMoran KRoychoudhury APaiva AAbreu RStorey M(2024)On Using GUI Interaction Data to Improve Text Retrieval-based Bug LocalizationProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3608139(1-13)Online publication date: 20-May-2024
https://doi.org/10.1145/3597503.3608139
Khan SWang WRen YZhu BShi JMcGowan ALam WMoran K(2024)Aurora: Navigating UI Tarpits via Automated Neural Screen Understanding2024 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST60714.2024.00028(221-232)Online publication date: 27-May-2024
https://doi.org/10.1109/ICST60714.2024.00028
Kuramoto HWang DKondo MKashiwa YKamei YUbayashi N(2024)Understanding the characteristics and the role of visual issue reportsEmpirical Software Engineering10.1007/s10664-024-10459-329:4Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1007/s10664-024-10459-3
Aktas ECakmak EInan MYilmaz C(2024)Improving the quality of software issue report descriptions in Turkish: An industrial case study at SofttechEmpirical Software Engineering10.1007/s10664-023-10434-429:2Online publication date: 12-Feb-2024
https://dl.acm.org/doi/10.1007/s10664-023-10434-4
Matos GCosta DLins ABezerra EBarroso LAguiar CFerraz TTeixeira I(2023)Watson: Web Application Interface Data Collector for Feedback Reporting2023 IEEE 30th Annual Software Technology Conference (STC)10.1109/STC58598.2023.00007(3-6)Online publication date: 25-Sep-2023
https://doi.org/10.1109/STC58598.2023.00007
Kallis RIzadi MPascarella LChaparro ORani P(2023)The NLBSE'23 Tool Competition2023 IEEE/ACM 2nd International Workshop on Natural Language-Based Software Engineering (NLBSE)10.1109/NLBSE59153.2023.00007(1-8)Online publication date: May-2023
https://doi.org/10.1109/NLBSE59153.2023.00007
Li XYu SSun LLiu YFang C(2023)Towards Effective Bug Reproduction for Mobile Applications2023 10th International Conference on Dependable Systems and Their Applications (DSA)10.1109/DSA59317.2023.00024(114-125)Online publication date: 10-Aug-2023
https://doi.org/10.1109/DSA59317.2023.00024

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten