research-article

Automated bug reproduction from user reviews for Android applications

Authors:

Jian-Guang Lou,

Ting LiuAuthors Info & Claims

ICSE-SEIP '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Software Engineering in Practice

Pages 51 - 60

https://doi.org/10.1145/3377813.3381355

Published: 18 September 2020 Publication History

Abstract

Bug-related user reviews of mobile applications have negative influence on their reputation and competence, and thus these reviews are highly regarded by developers. Before bug fixing, developers need to manually reproduce the bugs reported in user reviews, which is an extremely time-consuming and tedious task. Hence, it is highly expected to automate this process. However, it is challenging to do so since user reviews are hard to understand and poorly informative for bug reproduction (especially lack of reproduction steps). In this paper, we propose RepRev to automatically Reproduce Android application bugs from user Reviews. Specifically, RepRev leverages natural language processing techniques to extract valuable information for bug reproduction. Then, it ranks GUI components by semantic similarity with the user review and dynamically searches on apps with a novel one-step exploration technique. To evaluate RepRev, we construct a benchmark including 63 crash-related user reviews from Google Play, which have been reproduced successfully by three graduate students. On this benchmark, RepRev presents comparable performance with humans, which successfully reproduces 44 user reviews in our benchmark (about 70%) with 432.2 seconds average time. We make the implementation of our approach publicly available, along with the artifacts and experimental data we used [4].

References

[1]

2018. gensim. https://radimrehurek.com/gensim/models/word2vec.html.

[2]

2019. Ankidroid. https://play.google.com/store/apps/details?id=com.ichi2.anki.

[3]

2019. apktool. https://ibotpeaches.github.io/Apktool/.

[4]

2019. RepRev. https://drive.google.com/open?id=116JTG1jqzZB3JUXI9GJCrMMjZKy2wwE.

[5]

2019. uiautomator2. https://github.com/openatx/uiautomator2.

[6]

Domenico Amalfitano, Anna Rita Fasolino, Porfirio Tramontana, Salvatore De Carmine, and Atif M Memon. 2012. Using GUI ripping for automated testing of Android applications. In Proceedings of the 27th International Conference on Automated Software Engineering. ACM, 258--261.

Digital Library

[7]

Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. 2012. Automated concolic testing of smartphone apps. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. ACM, 1--11.

Digital Library

[8]

Tanzirul Azim and Iulian Neamtiu. 2013. Targeted and depth-first exploration for systematic testing of android apps. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages and Applications. ACM, 641--660.

Digital Library

[9]

Ning Chen, Jialiu Lin, Steven C. H. Hoi, Xiaokui Xiao, and Boshen Zhang. 2014. AR-Miner: Mining Informative Reviews for Developers from Mobile App Marketplace. In Proceedings of the 36th International Conference on Software Engineering. ACM, 767--778.

Digital Library

[10]

Noei Ehsan, Zhang Feng, and Ying Zou. 2019. Too Many User-Reviews, What Should App Developers Look at First? IEEE Transactions on Software Engineering (2019), 1--1.

Digital Library

[11]

Mattia Fazzini, Martin Prammer, Marcelo d'Amorim, and Alessandro Orso. 2018. Automatically translating bug reports into test cases for mobile apps. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis. ACM, 141--152.

Digital Library

[12]

Google. 2018. googlenews. https://code.google.com/archive/p/word2vec/.

[13]

Google. 2019. androidmonkey. https://developer.android.com/studio/test/monkey.html.

[14]

Google. 2019. navigationdrawer. https://material.io/components/navigation-drawer/.

[15]

Giovanni Grano, Adelina Ciurumelea, Sebastiano Panichella, Fabio Palomba, and Harald C Gall. 2018. Exploring the integration of user feedback in automated testing of android applications. In Proceedings of the 25th International Conference on Software Analysis, Evolution and Reengineering. IEEE Press, 72--83.

[16]

Giovanni Grano, Andrea Di Sorbo, Francesco Mercaldo, Corrado A Visaggio, Gerardo Canfora, and Sebastiano Panichella. 2017. Android apps and user feedback: a dataset for software evolution and quality improvement. In Proceedings of the 2nd ACM SIGSOFT International Workshop on App Market Analytics. ACM, 8--11.

Digital Library

[17]

Tianxiao Gu, Chengnian Sun, Xiaoxing Ma, Chun Cao, Chang Xu, Yuan Yao, Qirun Zhang, Jian Lu, and Zhendong Su. 2019. Practical GUI testing of Android applications via model abstraction and refinement. In Proceedings of the 41st International Conference on Software Engineering. IEEE Press, 269--280.

Digital Library

[18]

Xiaodong Gu, Hongyu Zhang, and Sunghun Kim. 2018. Deep code search. In Proceedings of the 40th International Conference on Software Engineering. ACM, 933--944.

Digital Library

[19]

Shuai Hao, Bin Liu, Suman Nath, William GJ Halfond, and Ramesh Govindan. 2014. PUMA: programmable UI-automation for large-scale dynamic analysis of mobile apps. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 204âĂŞ217.

Digital Library

[20]

Gang Hu, Linjie Zhu, and Junfeng Yang. 2018. AppFlow: Using Machine Learning to Synthesize Robust, Reusable UI Tests. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, 269--282.

Digital Library

[21]

Hammad Khalid, Emad Shihab, Meiyappan Nagappan, and Ahmed E Hassan. 2014. What do mobile app users complain about? IEEE Software 32, 3 (2014), 70--77.

Digital Library

[22]

Aravind Machiry, Rohan Tahiliani, and Mayur Naik. 2013. Dynodroid: An input generation system for android apps. In Proceedings of the 9th Joint Meeting on Foundations of Software Engineering. ACM, 224--234.

Digital Library

[23]

Riyadh Mahmood, Nariman Mirzaei, and Sam Malek. 2014. Evodroid: Segmented evolutionary testing of android apps. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 599--609.

Digital Library

[24]

Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations. ACL, 55--60.

[25]

Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: Multi-objective automated testing for Android applications. In Proceedings of the 25th International Symposium on Software Testing and Analysis. ACM, 94--105.

Digital Library

[26]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems. Curran Associates Inc., 3111--3119.

Digital Library

[27]

Fabio Palomba, Pasquale Salza, Adelina Ciurumelea, Sebastiano Panichella, Harald Gall, Filomena Ferrucci, and Andrea De Lucia. 2017. Recommending and localizing change requests for mobile apps based on user reviews. In Proceedings of the 39th International Conference on Software Engineering. IEEE Press, 106--117.

Digital Library

[28]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. ACL, 1532--1543.

[29]

Juan Enrique Ramos. 2003. Using TF-IDF to determine word relevance in document queries. In Proceedings of the 1st Instructional Conference on Machine Learning. Piscataway, NJ, 133--142.

[30]

Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, and Zhendong Su. 2017. Guided, stochastic model-based GUI testing of Android apps. In Proceedings of the 11th Joint Meeting on Foundations of Software Engineering. ACM, 245--256.

Digital Library

[31]

Heila van der Merwe, Brink van der Merwe, and Willem Visser. 2014. Execution and property specifications for jpf-android. ACM SIGSOFT Software Engineering Notes 39, 1 (2014), 1--5.

Digital Library

[32]

Lili Wei, Yepang Liu, and Shing-Chi Cheung. 2017. OASIS: prioritizing static analysis warnings for Android apps based on app user reviews. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. ACM, 672--682.

Digital Library

[33]

Wei Yang, Mukul R Prasad, and Tao Xie. 2013. A grey-box approach for automated GUI-model generation of mobile applications. In Proceedings of the 16th International Conference on Fundamental Approaches to Software Engineering. Springer-Verlag, 250âĂŞ265.

Digital Library

[34]

Le Yu, Jiachi Chen, Hao Zhou, Xiapu Luo, and Kang Liu. 2018. Localizing Function Errors in Mobile Apps with User Reviews. In Proceedings of the 48th International Conference on Dependable Systems and Networks. IEEE Press, 418--429.

[35]

Yu Zhao, Tingting Yu, Ting Su, Yang Liu, Wei Zheng, Jingzhi Zhang, and William GJ Halfond. 2019. Recdroid: automatically reproducing android application crashes from bug reports. In Proceedings of the 41st International Conference on Software Engineering. IEEE Press, 128--139.

Digital Library

Cited By

Auer MDiner DFraser GLi XHandl J(2024)Search-based Crash Reproduction for Android AppsProceedings of the Genetic and Evolutionary Computation Conference10.1145/3638529.3654034(1426-1434)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638529.3654034
Fatima EKanwal HKhan JKhan N(2024)An exploratory and automated study of sarcasm detection and classification in app stores using fine-tuned deep learning classifiersAutomated Software Engineering10.1007/s10515-024-00468-331:2Online publication date: 27-Aug-2024
https://doi.org/10.1007/s10515-024-00468-3
Tan YChen JShang WZhang TFang SLuo XChen ZQi S(2023)STRE: An Automated Approach to Suggesting App Developers When to Stop Reading ReviewsIEEE Transactions on Software Engineering10.1109/TSE.2023.328574349:8(4135-4151)Online publication date: Aug-2023
https://doi.org/10.1109/TSE.2023.3285743
Show More Cited By

Index Terms

Automated bug reproduction from user reviews for Android applications
1. Software and its engineering
  1. Software notations and tools
    1. Software maintenance tools

Recommendations

ReCDroid+: Automated End-to-End Crash Reproduction from Bug Reports for Android Apps
The large demand of mobile devices creates significant concerns about the quality of mobile applications (apps). Developers heavily rely on bug reports in issue tracking systems to reproduce failures (e.g., crashes). However, the process of crash ...
Automatically Reproducing Android Bug Reports using Natural Language Processing and Reinforcement Learning
ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

As part of the process of resolving issues submitted by users via bug reports, Android developers attempt to reproduce and observe the crashes described by the bug reports. Due to the low-quality of bug reports and the complexity of modern apps, the ...
A systematic mapping study of bug reproduction and localization
Abstract Context:
Identifying the root cause of a software bug and fixing it is challenging. One reason for this is that many bugs are not reproducible during bug fixing.
Objective:
We aim to provide an overview of existing works on bug reproduction and ...
Highlights
- Systematic mapping of 134 primary studies.
- Bug localization and reproduction are mostly studied for Java and C programs.
- Most approaches were tested in experimental settings rather than active projects.

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE-SEIP '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Software Engineering in Practice

June 2020

258 pages

ISBN:9781450371230

DOI:10.1145/3377813

General Chairs:
Gregg Rothermel
North Carolina State University
,
Doo-Hwan Bae
KAIST, South Korea

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

KIISE: Korean Institute of Information Scientists and Engineers
IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Project of China Knowledge Centre for Engineering Science and Technology
Ministry of Education Innovation Research Team
National Key R&D Program of China
National Natural Science Foundation of China

Conference

ICSE '20

Sponsor:

SIGSOFT

ICSE '20: 42nd International Conference on Software Engineering

June 27 - July 19, 2020

Seoul, South Korea

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
206
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)2

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Auer MDiner DFraser GLi XHandl J(2024)Search-based Crash Reproduction for Android AppsProceedings of the Genetic and Evolutionary Computation Conference10.1145/3638529.3654034(1426-1434)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638529.3654034
Fatima EKanwal HKhan JKhan N(2024)An exploratory and automated study of sarcasm detection and classification in app stores using fine-tuned deep learning classifiersAutomated Software Engineering10.1007/s10515-024-00468-331:2Online publication date: 27-Aug-2024
https://doi.org/10.1007/s10515-024-00468-3
Tan YChen JShang WZhang TFang SLuo XChen ZQi S(2023)STRE: An Automated Approach to Suggesting App Developers When to Stop Reading ReviewsIEEE Transactions on Software Engineering10.1109/TSE.2023.328574349:8(4135-4151)Online publication date: Aug-2023
https://doi.org/10.1109/TSE.2023.3285743
Fazzini MMoran KBernal-Cardenas CWendland TOrso APoshyvanyk D(2023)Enhancing Mobile App Bug Reporting via Real-Time Understanding of Reproduction StepsIEEE Transactions on Software Engineering10.1109/TSE.2022.317402849:3(1246-1272)Online publication date: 1-Mar-2023
https://doi.org/10.1109/TSE.2022.3174028
Huang YWang JLiu ZWang SChen CLi MWang QGrundy JPollock LPenta M(2023)Context-Aware Bug Reproduction for Mobile AppsProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00196(2336-2348)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00196
Dąbrowski JLetier EPerini ASusi A(2022)Analysing app reviews for software engineering: a systematic literature reviewEmpirical Software Engineering10.1007/s10664-021-10065-727:2Online publication date: 20-Jan-2022
https://doi.org/10.1007/s10664-021-10065-7
Bergel AMunoz I(2021)Beacon: Automated Test Generation for Stack-Trace Reproduction using Genetic Algorithms2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing (SBST)10.1109/SBST52555.2021.00007(1-7)Online publication date: May-2021
https://doi.org/10.1109/SBST52555.2021.00007

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents